/
5. Setup: VC MIPI Module on a Raspberry Pi

5. Setup: VC MIPI Module on a Raspberry Pi

Table of Contents

  1. Overview

  2. Hardware Setup

2.1 Hardware Pre-Check: Install Raspberry Pi OS

2.2 Connect the MIPI module

3. Software Setup

3.1 Get the driver and demo code

3.2 Install necessary Raspberry Pi OS packages

3.3 Driver Installation

3.4 First Image Acquisition Test

3.5 Running the Demo

3.5.1 Compile the programs

3.5.2 Execute the demo

3.6 Switching Sensor Configuration

3.6.1 Sensor modes

3.6.2 Sensor modes description

3.6.3 IO configuration

3.6.4 Self-triggered mode

4. Troubleshooting and Background Information

4.1 Q/A

4.2 Driver Knowledge

4.2.1 Providing device tree overlays

4.2.2 Set up the I²C bus for driver-sensor communication

4.2.3 Providing the sensor driver as kernel module

4.2.4 Reserving Contiguous Memory for the Image Captures

 

1 Overview

Overview of relevant components excluding power supply, monitor, USB keyboard, cables

The power supply must at least provide 2.5A at 5V.

The hardware connections are described for the Raspberry Pi 3B+, but the driver is also compatible with the Raspberry Pi 4B, Raspberry Pi Zero, Raspberry Pi Compute Module. For the Raspberry Pi 5 driver please contact our support at patrik.drexel@notavis.com.

The following VC MIPI modules are supported at the time of writing:

  • VC MIPI OV7251

  • VC MIPI OV9281

  • VC MIPI IMX178

  • VC MIPI IMX183 / IMX183C

  • VC MIPI IMX226 / IMX226C

  • VC MIPI IMX250 / IMX250C

  • VC MIPI IMX252 / IMX252C

  • VC MIPI IMX264 / IMX264C

  • VC MIPI IMX265 / IMX265C

  • VC MIPI IMX273 / IMX273C

  • VC MIPI IMX290

  • VC MIPI IMX296 / IMX296C

  • VC MIPI IMX297

  • VC MIPI IMX327C

  • VC MIPI IMX335 / IMX335C

  • VC MIPI IMX392 / IMX392C

  • VC MIPI IMX412C

  • VC MIPI IMX415 / IMX415C

  • VC MIPI IMX462C

  • VC MIPI IMX565 / IMX565C

  • VC MIPI IMX566

  • VC MIPI IMX567

  • VC MIPI IMX568 / IMX568C

  • VC MIPI IMX585 / IMX585C

  • VC MIPI IMX900 / IMX900C

 

2 Hardware Setup

2.1 Hardware Pre-Check: Install Raspberry Pi OS

First step is to install Raspberry Pi OS from

https://www.raspberrypi.com/software/operating-systems/

The driver is compatible with kernel versions 5.4, 5.10, 5.15, 6.1 and 6.6 (32-bit and 64-bit), so download the appropriate Raspberry Pi OS version. Raspberry Pi OS Buster Lite is sufficient, and this guide expects this version to be installed not only for the framebuffer output handling.

For more installation instructions see the Raspberry Pi OS Installation Manual; the procedure depends on the platform type where the OS is going to be installed.

raspberrypi_os_66.png
(Original site may look different) Install Raspberry Pi OS by following the instructions provided there

The display shows a login prompt after successful installation. If this is not the case, you have to check your Raspberry Pi OS installation. The most relevant information to succeed can be found at the Raspberry Pi OS website or on the web.

Raspberry Pi OS showing login screen (user is usually pi with password raspberry)

 

2.2 Connect the MIPI module

Always disconnect other cables before connecting or disconnecting the MIPI module!

Always disconnect all cables before connecting or disconnecting the MIPI module!

The ends of the MIPI module connector cable is marked with the hardware to connect to. Open the socket connectors first by raising their lid, insert the cable and press their lid back when mounted correctly. You should then not be able to pull the cable out.

Open the MIPI module socket, put in the cable, close the MIPI module socket

The connection at this type of socket is not protected against bad alignment, so always check the orthogonality, and if it is bent, correct it! Also watch out for the right orientation of the cable! The MIPI module or the board connected on the other side can be irrevocably damaged if the cable is not inserted the right way, and warranty is lost!

Harmfully angled (left) and good (right) cable fixation

The socket type is also not protected against wrong orientation, so compare your setup to the figures below before switching the power on.

Watch the orientation of the cable (left: bad, right: good)

There may be a dust prevention sticker at the socket named CAMERA at the raspberryPi, remove it first. Like at the sensor module, open the lid first, insert the cable to be orthogonally fixed after shutting the lid. Also check the orthogonality here and correct it if the cable is angled!

Connect the cable to the CAMERA socket at the raspberry Pi equally (left: bad, right: good)

Do not connect other devices to the I²C bus named VC, since it can affect the communication between the camera sensor and the driver!

For example, running the touch screen of the Raspberry PI 7 inch display will lead to communication problems between driver and camera sensor. The display may work with the following line appended to the /boot/config.txt, but test first without connecting it to the Raspberry PI to be sure everything works so far:

disable_touchscreen=1

Don't connect the SDA/SCK of the 7 inch display since this would connect the I²C bus VC from the socket named DISPLAY with the I²C bus ARM at the pinout!

Reconnect the other peripherials to the Raspberry Pi.

Connection setup for the first image acquisition test.

You should have the login prompt back after switching the system on.

Raspberry Pi OS showing login screen (user is usually pi with password raspberry)

 

3 Software Setup

It is recommended to install this driver package on a fresh Raspberry Pi OS image. I you have older drivers from Vision Components (without Debian package) it may cause issues with the new installation.

3.1 Get the driver and demo code

You can download the driver and demo code from the following links.

Driver: vc-mipi-driver-bcm2835-dkms_0.2.8_all.zip

Demo code: vcmipidemo_0.7.0.zip

3.2 Install necessary Raspberry Pi OS packages

Before beginning with the installation, do the following steps first. This requires your Raspberry PI to already have an internet connection; otherwise you have to install the packages mentioned manually, search the web for the procedure needed.

  1. Update the raspberrypi-kernel package and your system by calling:

    sudo apt-get update && sudo apt-get upgrade

  2. Reboot.

  3. Install the raspberrypi-kernel-headers and device-tree-compiler package by using the following command:

    sudo apt-get install raspberrypi-kernel-headers device-tree-compiler

  4. Test if the version of the running kernel matches the version of the kernel headers, the following command should show the directory for compiling the sensor module kernel module driver:

    ls "/usr/src/linux-headers-$(uname -r)"

  5. Install the dkms package with:

    sudo apt-get install dkms

 

3.3 Driver Installation

If you already installed an older version of the driver, an update will not modify the device tree files and the configuration files, in case customers made their own changes. If you wish to update the device tree files and the configuration files together with the driver it is necessary to deinstall the driver first and to delete the device tree files with the following commands:

sudo apt-get purge vc-mipi-driver-bcm2835-dkms

sudo rm -rf /boot/config_vc*

sudo rm -rf /boot/overlays/vc-mipi*

It is important that the date and time of your Raspberry Pi are set correctly! You can set the date and time using NTP:

sudo apt-get install ntpdate

ntpdate ip_address_of_ntp_server

Or you can use the command "date":

sudo date -s "2024-07-24 09:41:00"

In both cases store the time and date in the hardware clock:

sudo hwclock -w

  1. Copy the driver debian package (vc-mipi-driver-bcm2835-dkms_x.x.x_all.deb) to the /tmp folder on the Raspberry Pi.

  2. Install the driver package by calling (replace x.x.x by the current version number):

    sudo dpkg -i /tmp/vc-mipi-driver-bcm2835-dkms_x.x.x_all.deb

  3. Edit the file /boot/config_vc-mipi-driver-bcm2835.txt (for example with nano with the command: sudo nano /boot/config_vc-mipi-driver-bcm2835.txt).

    • choose the correct platform by uncommenting the corresponding line

    • choose the correct overlay according to your MIPI module and platform by uncommenting the corresponding line (for the IMX565, IMX566, IMX567, IMX568 please activate the overlay for the IMX568.)

    • choose the desired sensor mode (see chapter Sensor modes below for mode description)

    • change the IO configuration if necessary (see chapter IO configuration below for IO description)

    • activate the self-triggered mode (see chapter Self-triggered mode below)

    All these settings can be done for cam0 and cam1 in case you are using a Raspberry Pi CMIO or a VC CMIO.

Part 2:

Example of configuration file for the OV9281 MIPI module on a Raspberry Pi 3B+ in sensor mode 0.

4. Reboot.

 

3.4 First Image Acquisition Test

A sensor device should be listed as Video input at the following command output (from the Video4Linux-Control):

v4l2-ctl --all

The following command dumps sensor data:

v4l2-ctl --stream-mmap --stream-count=-1 -d /dev/video0 --stream-to=/dev/null

It will output subsequent lines ending with a frames-per-second information (in the example named [number]) until pressing CTRL-C:

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< [number] fps

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< [number] fps

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< [number] fps

<<<<<<<<<<<<<<<<<<<<<<<^C

 

3.5 Running the Demo

The demo itself is a program named vcmipidemo and its source code is mainly in the file vcmipidemo.c. However more programs are provided, namely the vcimgnetsrv, a network image server, and its counterpart vcimgnetclient.py. The vcimgnetsrv is started as background service, and the vcmipidemo connects to it. Then you can use the vcimgnetclient.py on your PC to view live captured images.

But for the first run it is better to just run the vcmipidemo and check if it shows the ascii representation. This works without any network cable attached. You can then output the captured image to the framebuffer of the display by using the -f command line switch.

 

3.5.1 Compile the programs

  1. Unpack the previously downloaded archive vcmipidemo_x.x.x.zip and copy the folder vcmipidemo to the Raspberry Pi (for example to /home/pi/)

  2. Change to the subdirectory named vcmipidemo/src.

  3. The source directory contains a Makefile to compile the driver. Do so by calling:

make clean all

 

3.5.2 Execute the demo

Just run the demo itself:

./vcmipidemo

or with framebuffer output:

./vcmipidemo -f

or with live view over ethernet:

./vcimgnetsrv &

./vcmipidemo

For live view over ethernet, execute the vcimgnetclient.py at the client. This needs Python 2 and PyGTK. Install both following packages in this order (Windows):

You can change exposure and gain values by vcmipidemo command line arguments. To get a listing of possible parameters, just call it with a -?:

./vcmipidemo -?

 

3.6 Switching Sensor Configuration

3.6.1 Sensor modes

The sensor driver provides different modes which support several features. They can be switched by changing values of sensor driver parameters.

To list available parameters of the sensor driver kernel module, you can use the following command:

dmesg

You can also check the table below (Sensor modes description) for a complete list of available sensor modes.

Example of the dmesg command for an OV9281 MIPI module, showing the available sensor modes.

To set the desired mode, edit the file /boot/config_vc-mipi-driver-bcm2835.txt (for example with nano with the command: sudo nano /boot/config_vc-mipi-driver-bcm2835.txt). Change the sensor mode by modifing dtparam:

dtparam=cam0_sensor_mode_1

The number after the underscore is the sensor mode. For mode 0 the correct setting would be dtparam=cam0_sensor_0

Example of setting the sensor mode to 1 for cam0.

 

3.6.2 Sensor modes description

This table lists the available modes for all mipi modules.

VC MIPI OV7251

Mode

Image format (bits)

Lanes

Capture mode

Resolution

VC MIPI OV7251

Mode

Image format (bits)

Lanes

Capture mode

Resolution

 

0

10

2

Streaming

640x480

 

1

8

2

Streaming

640x480

 

2

10

2

External trigger

640x480

 

3

8

2

External trigger

640x480

VC MIPI OV9281

Mode

Image format (bits)

Lanes

Capture mode

Resolution

VC MIPI OV9281

Mode

Image format (bits)

Lanes

Capture mode

Resolution

 

0

10

2

Streaming

1280x800

 

1

8

2

Streaming

1280x800

 

2

10

2

External trigger

1280x800

 

3

8

2

External trigger

1280x800

VC MIPI IMX178

Mode

Image format (bits)

Lanes

Capture mode

Resolution

VC MIPI IMX178

Mode

Image format (bits)

Lanes

Capture mode

Resolution

 

0

8

2

Streaming

3104x2076

 

1

10

2

Streaming

3104x2076

 

2

12

2

Streaming

3104x2076

 

3

14

2

Streaming

3104x2076

 

4

8

2

External trigger

3104x2076

 

5

10

2

External trigger

3104x2076

 

6

12

2

External trigger

3104x2076

 

7

14

2

External trigger

3104x2076

 

8

8

4

Streaming

3104x2076

 

9

10

4

Streaming

3104x2076

 

10

12

4

Streaming

3104x2076

 

11

14

4

Streaming

3104x2076