Machine Vision: Part 5 Install IMX219-160 Camera

Never could get the Arducam IMX477 to work. So let’s try an IMX219:

Hardware connection

  1. Connect the camera to CSI interface of Jetson Nano.Set the metal side of FFC into Heat-sink
  2. Connect a HDMI LCD to Jetson Nano

Software setting

  1. Power on Jetson Nano and open the Terminal (Ctrl+ALT+T)
  2. Test camera with command:
    DISPLAY=:0.0 gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM), width=3280, height=2464, format=(string)NV12, framerate=(fraction)20/1' ! nvoverlaysink -e

    If you find that the image captured is red. You can try to download .isp file and installed:

    tar zxvf Camera_overrides.tar.gz 
    sudo cp camera_overrides.isp /var/nvidia/nvcam/settings/
    sudo chmod 664 /var/nvidia/nvcam/settings/camera_overrides.isp
    sudo chown root:root /var/nvidia/nvcam/settings/camera_overrides.isp

Theoretically it should now work, but I found I had to perform the following additional steps:

  1. Install the video 4 linux utils
    sudo apt-get install v4l-utils
  2. Run the command v4l2-ctl to check the device. It should show this output:
    $ v4l2-ctl --list-devices
    vi-output, imx219 6-0010 (
  3. Further check using v4l2-ctl. This command must be run on a local terminal (not a remote terminal via SSH). Something like this should output
    $ v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=300
    <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.00 fps
    <<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.00 fps
    <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.03 fps
    <<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.02 fps
    <<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.01 fps
    <<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.01 fps
    <<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.01 fps
    <<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.01 fps
    <<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.01 fps
  4. Finally, run this command and the live stream window should pop up:
    gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM),width=3820, height=2464, framerate=21/1, format=NV12' ! nvvidconv flip-method=0 ! 'video/x-raw,width=960, height=616' ! nvvidconv ! nvegltransform ! nveglglessink -e



Machine Vision, Part 3: Install Arducam IMX477 Camera

I purchased an Arducam IMX477.

  1. The ribbon cable attaches with the bare contacts facing towards the board.
  2. Make note of your kernel version:
    $ uname -a

    Mine is currently

    Linux nvidia 4.9.140-tegra #1 SMP PREEMPT Thu Jun 25 21:22:12 PDT 2020 aarch64 aarch64 aarch64 GNU/Linux
  3. Download the driver .deb files from here. The kernel version of the drivers had better match the kernel version of your OS.
  4. Install them:
    $ sudo apt-get install --reinstall ./nvidia-l4t-kernel_4.9.140-tegra-32.4.3-20200625213407_arm64.deb
    $ sudo apt-get install --reinstall ./nvidia-l4t-kernel-dtbs_4.9.140-tegra-32.4.3-20200625213407_arm64.deb
  5. Enable the dtb changes by modifying the /boot/extlinux/extlinux.conf file:
    $ sudo echo "FDT /boot/dtb/tegra194-p3668-all-p3509-0000.dtb" | sudo tee -a /boot/extlinux/extlinux.conf
  6. Reboot
  7. Upon reboot, your video camera should appear in the /dev folder:
    $ ls /dev/video*


  8. Install some video utilities so we can really test it:
    sudo apt-get install v4l-utils
  9. Now we can display the available video formats:
    v4l2-ctl --list-formats-ext


    ioctl: VIDIOC_ENUM_FMT
    	Index       : 0
    	Type        : Video Capture
    	Pixel Format: 'RG10'
    	Name        : 10-bit Bayer RGRG/GBGB
    		Size: Discrete 4032x3040
    			Interval: Discrete 0.033s (30.000 fps)
    		Size: Discrete 1920x1080
    			Interval: Discrete 0.017s (60.000 fps)
  10. Arducam has a test program you can build here — hand-patching python is required to get it to work! Instead, let’s get OpenCV up and running.
  11. Never could get any test of the camera (to display an image) to work.

Machine Vision, Part 2: SSD Drive

Set Up SSD Drive

With so many disk intensive activities on the horizon, it makes sense to do as many of those as possible on a true SSD.

After you have physically mounted/installed the NVMe stick, perform the following steps:

  1. Figure out the device: “ls /dev/nvmen*”. In my case (and probably yours), “/dev/nvme0”. We don’t need to partition our SSD drive into separate “sub-drives”: one big drive is what we want. So:
    $ sudo mkfs.ext4 /dev/nvme0n1
  2. Create an empty folder where we’ll mount our new drive. I’m using /mnt/ssd — use what makes sense to you:
    $ mkdir -p /mnt/ssd
  3. Edit /etc/fstab to add a line (so your OS can auto-mount the new drive on boot):
    /dev/nvme0n1 /mnt/ssd ext4 defaults 0 0
  4. Test your work:
    $ sudo mount -a
    This manually mounts your drive. You should get no error messages.
    $ df -h
    Produces a list. You should see a line listing your new drive and mount folder.


Let’s also set up a swap file on our SSD. Opinions vary how large it should be. Here is a good article. For Ubuntu systems with more than 1G of memory, the recommended swap size is the square root of RAM. If you’re going to use hibernation, add the size of RAM. I’m not interested in hibernation, so I’m going with 3G of swap space.

  1. Create a file that will be used for swap:
    $ sudo fallocate -l 3G /mnt/ssd/swapfile
  2. Only root should be able to write and read the swap file:
    $ sudo chmod 600 /mnt/ssd/swapfile
  3. Use mkswap to set up the file as Linux swap area:
    $ sudo mkswap /mnt/ssd/swapfile
  4. Enable the swap:
    $ sudo swapon /mnt/ssd/swapfile
  5. To make the change permanent open /etc/fstab file and append the following line:
    /mnt/ssd/swapfile swap swap defaults 0 0
  6. o verify that the swap is active:
    sudo swapon --show

Machine Vision, Part 1: Nvidia Jetson Xavier NX Initial setup

I purchased the following:

  • Nvidia Jetson Xavier NX. I want to be able to experiment with both Cuda and Tensor cores — this is their smallest model that meets that requirement. At Amazon
  • 128G MicroSD — the one that NVidia recommends. At Amazon
  • 250G NVMe SSD Drive — I’ll be building of lots of libraries. Doing that on an SSD drive as much as possible should move things along considerably. 250G SSD is at a good price/size tradeoff. At Amazon.
  • Case — I’m a big believer in protective cases! At Amazon

Initial setup of the Nvidia Jetson Xavier NX is described here and works fine.

Samba Share Issues

I have a Samba drive shared amongst my various Linux and Windows machines, so I wanted the Jetson to be able to access it as well. Getting this to work on the Jetson required some fiddling I haven’t had to do elsewhere.

On all my other Linux, in /etc/fstab I specified iocharset=utf8 for sharing the Samba drive (because all of computer-dom is moving to UTF8). However, I had to omit this (giving me only ASCII file names — I’ll get over it). With iocharset=utf8 omitted, the Samba share mounted just fine.

The Samba share also failed to automount on boot (although ‘mount -a’ worked fine after boot.) Simply adding ‘_netdev’ to the /etc/fstab line didn’t work. What did work was:

[1] Create a new file at: /etc/network/if-up.d/fstab

[2] Add this content:

mount -a

[3] Make the file executable:

sudo chmod +x /etc/network/if-up.d/fstab

Any script placed in /etc/network/if-up.d will run only after the network is up and operational so this fixed the timing issue, that /etc/fstab was apparently being read too early in the boot process. (Thanks to this post.)

Parasoft Insure++

Recently my employer purchased a license for Parasoft’s Insure++ for my use.

The idea of Insure++ is that you build your C/C++ application with their Insure++ tool, and it instruments your application to identify buffer overruns, etc.

NOTE BENE: The Insure++ instrumented build will only run on the machine on which Insure++ is licensed. You can’t build your application with Insure++ on your development machine and execute it on any other machine (e.g. a ‘lab rat’ test machine) without purchasing an additional license.

NOTE BENE: The license only covers one OS. It’s $4000 for the Linux version. And another $4000 for the Windows version. Etc.

Installing Insure++

I’m currently running Ubuntu 16.04. There were dependencies not covered in the installation documentation: specifically you will need to install ‘motif’ and another ‘.so’ (an X Printing Extension: Xprint) client library. Perform the following to get Parasoft Insure++ to work:

apt install libmotif-dev

And, append this line to /etc/apt/sources.list:
deb precise-security main

apt update
apt install libxp6

A Simple Example

Parasoft includes an example makefile that compiles numerous example projects. It was not immediately obvious to me how to apply the given example(s) to my particular project. So, for an initial example, I still think it’s hard to beat K&R’s “Hello world”. Here’s how to instrument that with Insure++:


int main(void) {
printf("Hello, world\n");
return 0;

You will need the following voodoo (not covered in the documentation that I could find) to successfully build this trivial program:

insure gcc -Wl,--no-as-needed -g main.c -o main

Insure++ and CMake

Insure++ assumes you are using old-school make files. Being an old-school developer I’m quite comfortable with those. Meanwhile, today I also use the JetBrains line of IDEs, including CLion, which depends on CMake for its underlying make engine.

Enabling Insure++ requires you to modify your ‘makefile’, basically replacing ‘gcc’ with ‘insure gcc’. Even after a Google search it was far from obvious how to accomplish that with CMake. So I called tech support to find out how to implement Insure++ with CMake. Mind you, theJetBrains/Clion IDE (depending as it does on mainstream technologies like gcc and CMake) is a dominant player in Linux C/C++ development. So imagine my surprise when the support tech’s questions made it clear that he had no idea what CMake was about.

In the end I ended up creating a make file (in parallel with our existing CMake CMakeLists.txt file) just so I could implement Insure++.

Insure++ and a Real C++ Program

Armed with the above, I was still unable to build a real C++ application. The following additional voodoo was required:

OPTIONS="-O0 -g"
insure gcc $OPTIONS $SOURCE -Zvm -Zvm -Zvm -Zvm -o myapp

(That is not a typo: tech support said I needed ‘-Zvm -Zvm -Zvm -Zvm’ in my command line. Which resulted in successfully compiling the program. So far no useful/actionable results.)

Tentative Conclusion

Purchasing Insure++ was motivated by tracking down a segmentation fault that occurs occasionally. (A seg fault that Valgrind wasn’t finding either.) In parallel with implementing Insure++ I did deeper research on existing capabilities of gcc (and other open source resources). To my pleasant surprise there is much more debug instrumentation available with gcc than I knew. (See my blog C/C++ Stack & Heap Testing with gcc). So far this additional gcc instrumentation has flushed out several issues, where Insure++ has yet to flush out even one. And I personally have five ‘labrat’ test machines at my disposal, and they can all run gcc-instrumentation-enhanced debug builds of our app, whereas I can’t apply Insure++ to ANY of those testing resources without a considerable additional fee. Which might be OK if it were delivering results.

Even when it’s my own personal wallet, I have no problem paying for software that is useful. So far Insure++ has yet to deliver anything close to the results of [free] gcc capabilities. I’ll revise this blog if that assessment changes.

C/C++ Stack & Heap Checking with gcc

Stack, heap, and memory issues in general are a chronic problem with C/C++.

For Linux, there’s Valgrind, which is a truly useful tool. But it requires that one explicitly run the application under development under Valgrind. And we were experiencing a segmentation fault issue that Valgrind wasn’t identifying. (Nobody’s perfect!)

It turns out that gcc itself includes debugging support for this class of issues. One huge advantage of these sorts of ‘built in tests’ (also true of assert()s) is that you get ‘free’ testing every time you run a debug build. It’s practically free testing! Who doesn’t want THAT?

Here’s a list of these options (not entirely mutually exclusive). To use, say, -fmydebugoption you would compile with

gcc -fmydebugoption main.c -o main


If the two macros associated with this feature, namely STACK_CHECK_BUILTIN and STACK_CHECK_STATIC_BUILTIN are left at the default 0, this option inserts a NULL byte every 4kb (page) when the stack grows. By default only one 4K page, but when the stack can grow more than one page (the most dangerous case), every 4KB. See for details.


A family of checks involving the same concept, with different degrees of thoroughness/performance-cost. The idea is to put a ‘canary’ — a randomly generated number/signature at the end of each function call, and check that this canary is uncorrupted when the function exits. That is, if a function overwrites the end of a local array (of any type), it will also overwrite the ‘canary’ and when the function exits this can be checked, and the program aborted if necessary.

  • fstack-protector: Add a ‘canary’ (see ‘-fstack-protector’) to functions that call malloc/calloc/etc, and functions with buffers larger than 8 bytes.
  • fstack-protector-all: Add a ‘canary’ to every function (whether it has local arrays or memory allocations).
  • fstack-protector-string: Add a canary under any of the following circumstances:
    • Local variable’s address used as part of the right hand side of an assignment or function argument
    • Local variable is an array (or union containing an array), regardless of array type or length
    • Uses register local variables


    Generate instrumentation calls for entry and exit to functions. Just after function entry and just before function exit, the following profiling functions are called with the address of the current function and its call site. (On some platforms, __builtin_return_address does not work beyond the current function, so the call site information may not be available to the profiling functions otherwise.)

    void __cyg_profile_func_enter (void *this_fn,
    void *call_site);
    void __cyg_profile_func_exit (void *this_fn,
    void *call_site);

    The first argument is the address of the start of the current function, which may be looked up exactly in the symbol table.


    Enable AddressSanitizer, a fast memory error detector. Memory access instructions are instrumented to detect out-of-bounds and use-after-free bugs. The option enables -fsanitize-address-use-after-scope. See for more details. The option cannot be combined with -fsanitize=thread.


    Enable LeakSanitizer, a memory leak detector. This option only matters for linking of executables and the executable is linked against a library that overrides malloc and other allocator functions. See for more details. The run-time behavior can be influenced using the LSAN_OPTIONS environment variable. The option cannot be combined with -fsanitize=thread.


    A family of functions enabled by ‘sanitize=undefined’. Options include (not a complete list):

    • -fsanitize=integer-divide-by-zero:
      Detect integer division by zero as well as INT_MIN / -1 division.
    • -fsanitize=vla-bound
      This option instructs the compiler to check that the size of a variable length array is positive.

      For example:

      void fun(int n)
      int arr[n];
      // ......
      int main()

    • -fsanitize=null
      This option enables pointer checking. Particularly, the application built with this option turned on will issue an error message when it tries to dereference a NULL pointer, or if a reference (possibly an rvalue reference) is bound to a NULL pointer, or if a method is invoked on an object pointed by a NULL pointer.
    • -fsanitize=return
      This option enables return statement checking. Programs built with this option turned on will issue an error message when the end of a non-void function is reached without actually returning a value. This option works in C++ only.
    • -fsanitize=signed-integer-overflow
      This option enables signed integer overflow checking. We check that the result of +, *, and both unary and binary – does not overflow in the signed arithmetics. Note, integer promotion rules must be taken into account. That is, the following is not an overflow:

      signed char a = SCHAR_MAX;
    • -fsanitize=bounds
      This option enables instrumentation of array bounds. Various out of bounds accesses are detected. Flexible array members, flexible array member-like arrays, and initializers of variables with static storage are not instrumented.
    • -fsanitize=bounds-strict
      This option enables strict instrumentation of array bounds. Most out of bounds accesses are detected, including flexible array members and flexible array member-like arrays. Initializers of variables with static storage are not instrumented.
    • -fsanitize=object-size
      This option enables instrumentation of memory references using the __builtin_object_size function. Various out of bounds pointer accesses are detected.
    • -fsanitize=float-divide-by-zero
      Detect floating-point division by zero. Unlike other similar options, -fsanitize=float-divide-by-zero is not enabled by -fsanitize=undefined, since floating-point division by zero can be a legitimate way of obtaining infinities and NaNs.
    • -fsanitize=float-cast-overflow
      This option enables floating-point type to integer conversion checking. Check that the result of the conversion does not overflow. Unlike other similar options, -fsanitize=float-cast-overflow is not enabled by -fsanitize=undefined. This option does not work well with FE_INVALID exceptions enabled.
    • fsanitize=bool
      This option enables instrumentation of loads from bool. If a value other than 0/1 is loaded, a run-time error is issued.

    • -fsanitize=enum
      This option enables instrumentation of loads from an enum type. If a value outside the range of values for the enum type is loaded, a run-time error is issued.
    • -fsanitize=vptr
      This option enables instrumentation of C++ member function calls, member accesses and some conversions between pointers to base and derived classes, to verify the referenced object has the correct dynamic type.
    • -fsanitize=pointer-overflow
      This option enables instrumentation of pointer arithmetics. If the pointer arithmetics overflows, a run-time error is issued.


    Generally speaking makes life easier for debuggers by putting frame pointers in a CPU register.

    Based on the above, I’m currently using

    gcc ... -fstack-check -stack-protector-strong -finstrument-functions -fsanitize=address -fsanitize=leak -fsanitize-undefined ...