5. Setup: VC MIPI Module on a Raspberry Pi
Table of Contents
3.5 Running the Demo
3.5.1 Compile the programs
3.5.2 Execute the demo
3.6.1 Sensor modes
3.6.2 Sensor modes description
3.6.3 IO configuration
3.6.4 Self-triggered mode
4. Troubleshooting and Background Information
4.1 Q/A
4.2 Driver Knowledge
1 Overview
Overview of relevant components excluding power supply, monitor, USB keyboard, cables
The power supply must at least provide 2.5A at 5V.
The hardware connections are described for the Raspberry Pi 3B+, but the driver is also compatible with the Raspberry Pi 4B, Raspberry Pi Zero, Raspberry Pi Compute Module. For the Raspberry Pi 5 driver please contact our support at patrik.drexel@notavis.com.
The following VC MIPI modules are supported at the time of writing:
VC MIPI OV7251
VC MIPI OV9281
VC MIPI IMX178
VC MIPI IMX183 / IMX183C
VC MIPI IMX226 / IMX226C
VC MIPI IMX250 / IMX250C
VC MIPI IMX252 / IMX252C
VC MIPI IMX264 / IMX264C
VC MIPI IMX265 / IMX265C
VC MIPI IMX273 / IMX273C
VC MIPI IMX290
VC MIPI IMX296 / IMX296C
VC MIPI IMX297
VC MIPI IMX327C
VC MIPI IMX335 / IMX335C
VC MIPI IMX392 / IMX392C
VC MIPI IMX412C
VC MIPI IMX415 / IMX415C
VC MIPI IMX462C
VC MIPI IMX565 / IMX565C
VC MIPI IMX566
VC MIPI IMX567
VC MIPI IMX568 / IMX568C
VC MIPI IMX585 / IMX585C
VC MIPI IMX900 / IMX900C
2 Hardware Setup
2.1 Hardware Pre-Check: Install Raspberry Pi OS
First step is to install Raspberry Pi OS from
https://www.raspberrypi.com/software/operating-systems/
The driver is compatible with kernel versions 5.4, 5.10, 5.15, 6.1 and 6.6 (32-bit and 64-bit), so download the appropriate Raspberry Pi OS version. Raspberry Pi OS Buster Lite is sufficient, and this guide expects this version to be installed not only for the framebuffer output handling.
For more installation instructions see the Raspberry Pi OS Installation Manual; the procedure depends on the platform type where the OS is going to be installed.
The display shows a login prompt after successful installation. If this is not the case, you have to check your Raspberry Pi OS installation. The most relevant information to succeed can be found at the Raspberry Pi OS website or on the web.
2.2 Connect the MIPI module
Always disconnect all cables before connecting or disconnecting the MIPI module!
The ends of the MIPI module connector cable is marked with the hardware to connect to. Open the socket connectors first by raising their lid, insert the cable and press their lid back when mounted correctly. You should then not be able to pull the cable out.
The socket type is also not protected against wrong orientation, so compare your setup to the figures below before switching the power on.
There may be a dust prevention sticker at the socket named CAMERA at the raspberryPi, remove it first. Like at the sensor module, open the lid first, insert the cable to be orthogonally fixed after shutting the lid. Also check the orthogonality here and correct it if the cable is angled!
Reconnect the other peripherials to the Raspberry Pi.
You should have the login prompt back after switching the system on.
3 Software Setup
3.1 Get the driver and demo code
You can download the driver and demo code from the following links.
Driver: vc-mipi-driver-bcm2835-dkms_0.2.8_all.zip
Demo code: vcmipidemo_0.7.0.zip
3.2 Install necessary Raspberry Pi OS packages
Before beginning with the installation, do the following steps first. This requires your Raspberry PI to already have an internet connection; otherwise you have to install the packages mentioned manually, search the web for the procedure needed.
Update the raspberrypi-kernel package and your system by calling:
sudo apt-get update && sudo apt-get upgrade
Reboot.
Install the raspberrypi-kernel-headers and device-tree-compiler package by using the following command:
sudo apt-get install raspberrypi-kernel-headers device-tree-compiler
Test if the version of the running kernel matches the version of the kernel headers, the following command should show the directory for compiling the sensor module kernel module driver:
ls "/usr/src/linux-headers-$(uname -r)"
Install the dkms package with:
sudo apt-get install dkms
3.3 Driver Installation
Copy the driver debian package (vc-mipi-driver-bcm2835-dkms_x.x.x_all.deb) to the /tmp folder on the Raspberry Pi.
Install the driver package by calling (replace x.x.x by the current version number):
sudo dpkg -i /tmp/vc-mipi-driver-bcm2835-dkms_x.x.x_all.deb
Edit the file /boot/config_vc-mipi-driver-bcm2835.txt (for example with nano with the command: sudo nano /boot/config_vc-mipi-driver-bcm2835.txt).
choose the correct platform by uncommenting the corresponding line
choose the correct overlay according to your MIPI module and platform by uncommenting the corresponding line (for the IMX565, IMX566, IMX567, IMX568 please activate the overlay for the IMX568.)
choose the desired sensor mode (see chapter Sensor modes below for mode description)
change the IO configuration if necessary (see chapter IO configuration below for IO description)
activate the self-triggered mode (see chapter Self-triggered mode below)
All these settings can be done for cam0 and cam1 in case you are using a Raspberry Pi CMIO or a VC CMIO.
Part 2:
4. Reboot.
3.4 First Image Acquisition Test
A sensor device should be listed as Video input at the following command output (from the Video4Linux-Control):
v4l2-ctl --all
The following command dumps sensor data:
v4l2-ctl --stream-mmap --stream-count=-1 -d /dev/video0 --stream-to=/dev/null
It will output subsequent lines ending with a frames-per-second information (in the example named [number]) until pressing CTRL-C:
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< [number] fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< [number] fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< [number] fps
<<<<<<<<<<<<<<<<<<<<<<<^C
3.5 Running the Demo
The demo itself is a program named vcmipidemo and its source code is mainly in the file vcmipidemo.c. However more programs are provided, namely the vcimgnetsrv, a network image server, and its counterpart vcimgnetclient.py. The vcimgnetsrv is started as background service, and the vcmipidemo connects to it. Then you can use the vcimgnetclient.py on your PC to view live captured images.
But for the first run it is better to just run the vcmipidemo and check if it shows the ascii representation. This works without any network cable attached. You can then output the captured image to the framebuffer of the display by using the -f command line switch.
3.5.1 Compile the programs
Unpack the previously downloaded archive vcmipidemo_x.x.x.zip and copy the folder vcmipidemo to the Raspberry Pi (for example to /home/pi/)
Change to the subdirectory named vcmipidemo/src.
The source directory contains a Makefile to compile the driver. Do so by calling:
make clean all
3.5.2 Execute the demo
Just run the demo itself:
./vcmipidemo
or with framebuffer output:
./vcmipidemo -f
or with live view over ethernet:
./vcimgnetsrv &
./vcmipidemo
For live view over ethernet, execute the vcimgnetclient.py at the client. This needs Python 2 and PyGTK. Install both following packages in this order (Windows):
Python 2: https://files.vision-components.com/ImageTransfer/python-2.7.11.msi
PyGTK: https://files.vision-components.com/ImageTransfer/pygtk-all-in-one-2.24.2.win32-py2.7.msi
3.6 Switching Sensor Configuration
3.6.1 Sensor modes
The sensor driver provides different modes which support several features. They can be switched by changing values of sensor driver parameters.
To list available parameters of the sensor driver kernel module, you can use the following command:
dmesg
You can also check the table below (Sensor modes description) for a complete list of available sensor modes.
To set the desired mode, edit the file /boot/config_vc-mipi-driver-bcm2835.txt (for example with nano with the command: sudo nano /boot/config_vc-mipi-driver-bcm2835.txt). Change the sensor mode by modifing dtparam:
dtparam=cam0_sensor_mode_1
The number after the underscore is the sensor mode. For mode 0 the correct setting would be dtparam=cam0_sensor_0
3.6.2 Sensor modes description
This table lists the available modes for all mipi modules.
VC MIPI OV7251 | Mode | Image format (bits) | Lanes | Capture mode | Resolution |
---|---|---|---|---|---|
| 0 | 10 | 2 | Streaming | 640x480 |
| 1 | 8 | 2 | Streaming | 640x480 |
| 2 | 10 | 2 | External trigger | 640x480 |
| 3 | 8 | 2 | External trigger | 640x480 |
VC MIPI OV9281 | Mode | Image format (bits) | Lanes | Capture mode | Resolution |
---|---|---|---|---|---|
| 0 | 10 | 2 | Streaming | 1280x800 |
| 1 | 8 | 2 | Streaming | 1280x800 |
| 2 | 10 | 2 | External trigger | 1280x800 |
| 3 | 8 | 2 | External trigger | 1280x800 |
VC MIPI IMX178 | Mode | Image format (bits) | Lanes | Capture mode | Resolution |
---|---|---|---|---|---|
| 0 | 8 | 2 | Streaming | 3104x2076 |
| 1 | 10 | 2 | Streaming | 3104x2076 |
| 2 | 12 | 2 | Streaming | 3104x2076 |
| 3 | 14 | 2 | Streaming | 3104x2076 |
| 4 | 8 | 2 | External trigger | 3104x2076 |
| 5 | 10 | 2 | External trigger | 3104x2076 |
| 6 | 12 | 2 | External trigger | 3104x2076 |
| 7 | 14 | 2 | External trigger | 3104x2076 |
| 8 | 8 | 4 | Streaming | 3104x2076 |
| 9 | 10 | 4 | Streaming | 3104x2076 |
| 10 | 12 | 4 | Streaming | 3104x2076 |
| 11 | 14 | 4 | Streaming | 3104x2076 |
| 12 | 8 | 4 | External trigger | 3104x2076 |
| 13 | 10 | 4 | External trigger | 3104x2076 |
| 14 | 12 | 4 | External trigger | 3104x2076 |
| 15 | 14 | 4 | External trigger | 3104x2076 |
VC MIPI IMX183 | Mode | Image format (bits) | Lanes | Capture mode | Resolution |
---|---|---|---|---|---|
| 0 | 8 | 2 | Streaming | 5440x3648 |
| 1 | 10 | 2 | Streaming | 5440x3648 |
| 2 | 12 | 2 | Streaming | 5440x3648 |
| 3 | 8 | 2 | External trigger | 5440x3648 |
| 4 | 10 | 2 | External trigger | 5440x3648 |
| 5 | 12 | 2 | External trigger | 5440x3648 |
| 6 | 8 | 4 | Streaming | 5440x3648 |
| 7 | 10 | 4 | Streaming | 5440x3648 |
| 8 | 12 | 4 | Streaming | 5440x3648 |
| 9 | 8 | 4 | External trigger | 5440x3648 |
| 10 | 10 | 4 | External trigger | 5440x3648 |
| 11 | 12 | 4 | External trigger | 5440x3648 |
VC MIPI IMX226 | Mode | Image format (bits) | Lanes | Capture mode | Resolution |
---|---|---|---|---|---|
| 0 | 8 | 2 | Streaming | 3840x3046 |
| 1 | 10 | 2 | Streaming | 3840x3046 |
| 2 | 12 | 2 | Streaming | 3840x3046 |
| 3 | 8 | 2 | External trigger | 3840x3046 |
| 4 | 10 | 2 | External trigger | 3840x3046 |
| 5 | 12 | 2 | External trigger | 3840x3046 |
| 6 | 8 | 4 | Streaming | 3840x3046 |
| 7 | 10 | 4 | Streaming | 3840x3046 |
| 8 | 12 | 4 | Streaming | 3840x3046 |
| 9 | 8 | 4 | External trigger | 3840x3046 |
| 10 | 10 | 4 | External trigger | 3840x3046 |
| 11 | 12 | 4 | External trigger | 3840x3046 |
VC MIPI IMX250 | Mode | Image format (bits) | Lanes | Capture mode | Resolution |
---|---|---|---|---|---|
| 0 | 8 | 2 | Streaming | 2432x2048 |
| 1 | 10 | 2 | Streaming | 2432x2048 |
| 2 | 12 | 2 | Streaming | 2432x2048 |
| 3 | 8 | 2 | External trigger | 2432x2048 |
| 4 | 10 | 2 | External trigger | 2432x2048 |
| 5 | 12 | 2 | External trigger | 2432x2048 |
| 6 | 8 | 4 | Streaming | 2432x2048 |
| 7 | 10 | 4 | Streaming | 2432x2048 |
| 8 | 12 | 4 | Streaming | 2432x2048 |
| 9 | 8 | 4 | External trigger | 2432x2048 |
| 10 | 10 | 4 | External trigger | 2432x2048 |
| 11 | 12 | 4 | External trigger | 2432x2048 |
VC MIPI IMX252 | Mode | Image format (bits) | Lanes | Capture mode | Resolution |
---|---|---|---|---|---|
| 0 | 8 | 2 | Streaming | 2048x1536 |
| 1 | 10 | 2 | Streaming | 2048x1536 |
| 2 | 12 | 2 | Streaming | 2048x1536 |
| 3 | 8 | 2 | External trigger | 2048x1536 |
| 4 | 10 | 2 | External trigger | 2048x1536 |
| 5 | 12 | 2 | External trigger | 2048x1536 |
| 6 | 8 | 4 | Streaming | 2048x1536 |
| 7 | 10 | 4 | Streaming | 2048x1536 |
| 8 | 12 | 4 | Streaming | 2048x1536 |
| 9 | 8 | 4 | External trigger | 2048x1536 |
| 10 | 10 | 4 | External trigger | 2048x1536 |
| 11 | 12 | 4 | External trigger | 2048x1536 |
VC MIPI IMX264 | Mode | Image format (bits) | Lanes | Capture mode | Resolution |
---|---|---|---|---|---|
| 0 | 8 | 2 | Streaming | 2432x2048 |
| 1 | 10 | 2 | Streaming | 2432x2048 |
| 2 | 12 | 2 | Streaming | 2432x2048 |
| 3 | 8 | 2 | External trigger | 2432x2048 |
| 4 | 10 | 2 | External trigger | 2432x2048 |
| 5 | 12 | 2 | External trigger | 2432x2048 |
VC MIPI IMX265 | Mode | Image format (bits) | Lanes | Capture mode | Resolution |
---|---|---|---|---|---|
| 0 | 8 | 2 | Streaming | 2048x1536 |
| 1 | 10 | 2 | Streaming | 2048x1536 |
| 2 | 12 | 2 | Streaming | 2048x1536 |
| 3 | 8 | 2 | External trigger | 2048x1536 |
| 4 | 10 | 2 | External trigger | 2048x1536 |
| 5 | 12 | 2 | External trigger | 2048x1536 |
VC MIPI IMX273 | Mode | Image format (bits) | Lanes | Capture mode | Resolution |
---|---|---|---|---|---|
| 0 | 8 | 2 | Streaming | 1440x1080 |
| 1 | 10 | 2 | Streaming | 1440x1080 |
| 2 | 12 | 2 | Streaming | 1440x1080 |
| 3 | 8 | 2 | External trigger | 1440x1080 |
| 4 | 10 | 2 | External trigger | 1440x1080 |
| 5 | 12 | 2 | External trigger | 1440x1080 |
| 6 | 8 | 4 | Streaming | 1440x1080 |
| 7 | 10 | 4 | Streaming | 1440x1080 |
| 8 | 12 | 4 | Streaming | 1440x1080 |
| 9 | 8 | 4 | External trigger | 1440x1080 |
| 10 | 10 | 4 | External trigger | 1440x1080 |
| 11 | 12 | 4 | External trigger | 1440x1080 |
| 12 | 8 | 2 | Streaming | 720x540 (binning) |
| 13 | 10 | 2 | Streaming | 720x540 (binning) |
| 14 | 12 | 2 | Streaming | 720x540 (binning) |
| 15 | 8 | 2 | External trigger | 720x540 (binning) |
| 16 | 10 | 2 | External trigger | 720x540 (binning) |
| 17 | 12 | 2 | External trigger | 720x540 (binning) |
| 18 | 8 | 4 | Streaming | 720x540 (binning) |
| 19 | 10 | 4 | Streaming | 720x540 (binning) |
| 20 | 12 | 4 | Streaming | 720x540 (binning) |
| 21 | 8 | 4 | External trigger | 720x540 (binning) |
| 22 | 10 | 4 | External trigger | 720x540 (binning) |
| 23 | 12 | 4 | External trigger | 720x540 (binning) |
VC MIPI IMX290 | Mode | Image format (bits) | Lanes | Capture mode | Resolution |
---|---|---|---|---|---|
| 0 | 10 | 2 | Streaming | 1920x1080 |
| 1 | 10 | 4 | Streaming | 1920x1080 |
VC MIPI IMX296 | Mode | Image format (bits) | Lanes | Capture mode | Resolution |
---|---|---|---|---|---|
|