Sensor Specific Functions

Some of the sensors supported in the qtec C-series platform have some specific functionality which is listed and explained below.

Sony IMX

Refers to the IMX42x/25x series used in the RGB AI Cameras and IMX99x series used in the Hypervision 1700 HSI Camera.

Light Sensitivity

The Sony IMX42x/25x/99x families have 3 available bit modes (8, 10 and 12 bit). The internal Vsat is adjusted to 1/4 when in 8 bit mode compared to the 10/12 bit modes. Therefore the light sensitivity is 4x higher in 8 bit mode when compared to the other modes, which makes the 8 bit mode the recommended mode for applications where the available light intensity is limited. For example fast moving objects or hyperspectral imaging.

Gpixel GSENSE

Refers to the GSENSE2020 used in the Hypervision 1000 HSI Camera.

Shutter modes

The Gpixel GSENSE2020 is a quite unique sensor which offers the possibility of choosing between rolling shutter and global shutter. Each of the shutter modes offers some specific features and advantages: for example it is possible to adjust the exposure time for each individual line when using the rolling shutter mode and to select individually which lines should be captured.

Rolling shutter vs Global shutter

The shutter type refers to the image capture process in the sensor: cameras can either have a rolling shutter or a global shutter.

In a rolling shutter setup each sensor line is exposed and read-out sequentially (in waves), while with a global shutter all pixels are exposed simultaneously. Global shutter sensors require therefore more internal memory (to store all the pixel values until they can be read out). Rolling shutter sensors are normally simpler and therefore cheaper and they also tend to have less noise and bigger sensitivity. But because of the sequential nature of the rolling shutter it is not recommended for fast moving objects since it can result in severe distortions in the image (see below).

Rolling shutter effect (cmglee, CC BY-SA 3.0, via Wikimedia Commons)

Qtec focuses on industrial cameras which normally involve fast moving objects, therefore most of the sensors that qtec works with use global shutter, with the exception of the GSENSE2020 which is a scientific CMOS with the option to select the shutter mode.

Shutter mode selection

The shutter modes can be selected by choosing the appropriate bitstream from the qtec-c-paris series:

• pisces-paris: for rolling shutter mode
  • allows adjusting the exposure time individually per line using the line exposure sequence v4l2 control
  • allows selecting individually which lines should be captured using the line order v4l2 control
• capricorn-paris: for global shutter mode
  • uses a global exposure time like all other sensors, except that it returns two "images" per frame (where the 1st is a reference frame that should be subtracted from the 2nd in order to generate the actual image)

info

pisces-paris is the default bitstream for the GSENSE2020 and is what is used in the Hypervision 1000 HSI Camera because of it's ability to control the exposure individually for each line.

Rolling shutter mode

The rolling shutter on the GSENSE2020 is fully programmable. This means that the exposure time of lines can be selected individually. This can be achieved by programming the sensor via the Line Exposure Sequence V4L2 control, which is effectively an array of $2048$ values, one for each line.

While global shutter sensors normally allow for a single continuous cropping area (or in some cases up to 8 areas), the fully programmable rolling shutter makes it possible to select individually which lines should be captured (as well in what order). In this way the frame capture process and speed can be greatly optimized (the maximum framerate is directly proportional to the amount of lines captured).

If a single cropping area is desired the standard V4L2 crop/selection interface can be used, just like for all other sensors supported by qtec. However if selection of individual lines is desired, then the Line Order V4L2 control should be used. The Line Order control is an array with the same size as the height of the cropping area (set via the standard V4L2 crop/selection interface). It's contents reflect the position of the lines to be captured, in the desired capture order.

Setting the exposure per line

Bitstream specific

Only available in the pisces-paris bitstream

This is a small example showing how to change the exposure per line in rolling shutter mode, with qamlib.

import qamlib

cam = qamlib.Camera()

# Get the current exposure sequence
exposures = cam.get_ext_control("line exposure sequence").value

# Set the exposure for all lines to 5000
exposures[:] = 5000

# Set a different exposure on the bottom half of the lines 
n = int(len(exposures)/2)
exposures[n:] = 2000

# Set the new exposure sequence
try:
        cam.set_ext_control("line exposure sequence", qamlib.ArrayControlValue(exposures))
except V4L2Exception as e:
        print(e)

tip

quark-ctl can also be used as a quick way to set all lines to the same exposure value:

quark-ctl exposure-sequence --exposure 1000

caution

Because of limited RAM in the FPGA, we have a limit on how complex/long the exposures can be. E.g. for a full frame the max exposure is $\approx 68000 us$

The requested change in the exposure sequence will be rejected if the requested parameters don't fit within the available memory. In this case qamlib will throw an V4L2Exception and an error will be visible in dmesg:

V4L2Exception: Failed to set extended control : (22) Invalid argument

dmesg: gsense-0: Line exposure sequence is too long or complex

framerate

Changing Line Exposure Sequence will adjust the framerate if necessary (if the current fps is higher than the new maximum fps)

Changing the selected lines

Bitstream specific

Only available in the pisces-paris bitstream

This is a small example showing how to change the line order in rolling shutter mode, with qamlib.

import qamlib

cam = qamlib.Camera()

# Start by adjusting the crop region to the desired size
cam.set_crop(left=0, top=0, width=2048, height=100)

# Get the current line order array
line_order = cam.get_ext_control("line order").value

# Adjust the desired lines:
# For example use every 5th line starting from 50
line = 50
for i, pos in enumerate(line_order):
        line_order[i] = line
        line += 5

# Set the new line order
try:
        cam.set_ext_control("line order", qamlib.ArrayControlValue(line_order))
except V4L2Exception as e:
        print(e)

framerate

Changing Line Order will adjust the framerate if necessary (if the current fps is higher than the new max fps)

Global Shutter mode

With the GSENSE2020 in global shutter mode, it acts a lot more like all the other supported sensors, except that for each image two frames are generated. The first is a reference frame that needs to be subtracted from the second one in order to generate the actual image.

Frame read-out in global shutter mode

This is a small example (using qamlib) of how to do the frame read-out from the GSENSE2020 in global shutter mode and to combine the 2 frames into the resulting image.

import qamlib

try:
        # throw exception on dropped frames
        cam = qamlib.Camera(overflow_exception=True)
except Exception as e:
        print(e)
        exit(-1)

# Set to global shutter (default is rolling shutter)
cam.set_control("global shutter", 1)

# Frame capture
i2 = 0
try:
        with cam:
                while True:
                        # Using 'buffered=True' will return the next frame in the queue
                        # and throw a 'DroppedFrameException' if the queue gets filled up
                        # set 'overflow_exception=False' in the Camera constructor
                        # if you want to disable this exception, but then it will be necessary
                        # to check for dropped frames using the sequence nr from the metadata

                        ref_meta, ref_frame = cam.get_frame(timeout=1, buffered=True)
                        meta, frame = cam.get_frame(timeout=1, buffered=True)

                        # the GSENSE2020 produces 2 frames "per frame" when on global shutter
                        # the 1st is a reference frame that should be subtracted from the 2nd
                        # in order to create the resulting image
                        res_frame = frame - ref_frame

                        # all this sequence nr checks shouldn't be necessary
                        # since we will get a 'DroppedFrameException' if frames are dropped
                        # but are present to illustrate how to check for dropped frames under other circumstances
                        
                        # reference frames must have even seq nr
                        if ref_meta.sequence % 2 != 0:
                                print(f"cam: reference frame out of sequence")
                                break
                                
                        # secondary frames must have odd seq nr
                        if meta.sequence % 2 == 0:
                                print(f"cam: secondary frame out of sequence")
                                break
                                
                        # both frames seq nrs must be sequential
                        n3 = meta.sequence - ref_meta.sequence
                        if n3 != 1:
                                print(f"cam skipped {n3} frames")
                                break

                        n2 = ref_meta.sequence - i2
                        if n2 > 0:
                                print(f"cam skipped {n2} frames")
                                break
                        i2 = meta.sequence + 1
except Exception as e:
        template = "An exception of type {0} occurred. Arguments:\n{1!r}"
        message = template.format(type(e).__name__, e.args)
        print(message)