Image Operations#

The HV SDK provides a lazy and memory-efficient interface for working with the hsi.HSImage type. This allows the user to delay calculations to when it is appropriate and to better control the amount of memory used. The interface is streaming-based and therefore allows you to do some operations without loading in the entire image file at once.

The basic approach for the interface is to do as many operations as possible lazily and only convert to in-memory representations or NumPy whenever (a) the result has to be used many times or (b) the result has to be used in NumPy. The Recipes demonstrates how to implement some common HSI preprocessing operations.

Indexing#

You can index hsi.HSImage images using the Python slicing syntax.

img[:, :, :] # The full image
img[:20, :20, :] # The first 20 lines and samples
img[:, :, ::4] # Take only every 4th band

# Demonstration of how lines/samples/bands map to the index dimensions
lines = slice(start=0, end=10)
samples = slice(end=None)
bands = slice(end=None, step=2)
img[lines, samples, bands]

The syntax is much more restrictive than NumPy. You always need exactly three arguments and all arguments must be slices, i.e. you can’t use integer indexes. The reason for this is that a hsi.HSImage is always three-dimensional.

The slice indices are always in BIP format (lines, samples, bands) as shown in the example above. This can cause confusion when used together with the hsi.HSImage.to_numpy() method.

img = hsi.HSImage.from_numpy(np.ones((10, 10, 10)), interleave=hsi.bil)

# Note how the index order differs between the two types. This is because `img` is in BIL format but the index
# method always accepts BIP format.
assert img[:, :, :5].to_numpy().shape == (10, 5, 10)

Elementwise operations#

You can use common arithmetic elementwise operations using the regular operators ‘+’, ‘-’, ‘*’, ‘/’. Currently, only these four are supported. Broadcasting works between HSImage objects but the operators don’t yet work on scalar values.

# Subtract two bands
diff = img[:, :, 500:501] - img[:, :, 810:811]

It is possible to simulate scalar operations by creating a 1x1x1 image with the desired value.

scalar = 5
scalar_img = hsi.HSImage.from_python(np.ones((1, 1, 1), dtype=np.uint8) * scalar, interleave=hsi.bil)

img / scalar_img # Effectively scalar division due to the broadcasting rules

Reduction operations#

The library supports a limited set of reduction operations for calculating the mean, standard deviation, variance, and sum. The operations always work on a single axis only. Because the images are always three-dimensional, the operations reduce the specified axis to length 1, similar to how the keepdims argument works in NumPy.

img.mean_axis(hsi.bands) # Mean spatial image

# Workaround for performing multiple reductions. The result is the mean spectrum.
img.mean_axis(hsi.lines).mean_axis(hsi.samples)

Resolving calculations#

The purpose of the library’s design is to minimise memory usage. The consequence is that calculation efficiency is somewhat reduced. For operations that run only once, this shouldn’t be a big issue and may in fact improve performance due to cache locality (hypothetically). However, for operations that run multiple times, the lazy design means that all intermediate calculations are performed each time the result is accessed.

A good example is the reflectance calibration preprocessing step in HSI. The white and dark references are typically calculated as the mean of multiple images to minimise noise and material irregularities. These are then used to calibrate a main image. The white and dark reference mean operations have to be recalculated each time the calibrated image is accessed. For large hyperspectral images, this is a necessary tradeoff if limited memory is available, but since the references contain only a single band, it could easily fit in memory.

The solution to the problem can be found in the hsi.HSImage.resolve() method. The method resolves the underlying calculations and stores the result directly in memory as an array.

# Since mean_axis reduces the bands dimension to 1, the result only takes up *lines * samples * bytes_per_sample*
# bytes in memory.
img.mean_axis(hsi.bands).resolve()