A spatial shift observed in the cubert data (.cu3 ) when plotting the spectrum and the RGB equivalent Image

[JobinFrancisGithub]

I'm utilising cubert data and tried to extract the.cu3 file, which is done using dark current and white current subtraction preprocessing. Then I tried showing the RGB equivalent to gain a sense of spatial detail. But now I'm experiencing a shift between the bands; how can I handle it? I can provide you with the code I tested and the spectrum I acquired.

cu3_file = r"D:\Strawberry_Data\Strawberry_data_2024\20240620\Cubert\session_001\session_001_078.cu3"
dark_file = r"D:\Strawberry_Data\Strawberry_data_2024\20240620\Cubert\Calibration\dark__session_001_005_snapshot17188827878374578.cu3"
white_file = r"D:\Strawberry_Data\Strawberry_data_2024\20240620\Cubert\Calibration\white__session_001_003_snapshot17188827396517688.cu3"

# cu3_file = r"D:\Strawberry_Data\Strawberry_data_2024\20241107\Cubert\session_000_001k\session_000_001k_079.cu3"
# dark_file = r"D:\Strawberry_Data\Strawberry_data_2024\20241107\Cubert\Calibration\dark__session_000_007_snapshot17309741521169685.cu3"
# white_file = r"D:\Strawberry_Data\Strawberry_data_2024\20241107\Cubert\Calibration\white__session_000_005_snapshot17309741151846849.cu3"


measurement = cuvis.Measurement(cu3_file)
dark_measurement = cuvis.Measurement(dark_file)
white_measurement = cuvis.Measurement(white_file)

dark = dark_measurement.data['cube'].array.astype(np.float32)
white = white_measurement.data['cube'].array.astype(np.float32)
raw = measurement.data['cube'].array.astype(np.float32)

white_minus_dark = np.clip((white - dark), 1e-6, None)

reflectance_cube = (raw - dark) / white_minus_dark

[birkholz-cubert]

Hello,

im assuming you need to additionally need to apply a distance correction onto the data.
Using the SDK (Python Wrapper) you can do this in two ways:

    proc = cuvis.ProcessingContext(...)
    distance_mesu = cuvis.Measurement(...)

   # Either set a distance Reference
    proc.set_reference(distance_mesu, cuvis.ReferenceType.Distance)
    
    # or set the distance as a parameter
    proc.calc_distance(distance_in_mm)
    
    # Then apply the distance correction onto your measurements...
    
    proc.apply(measurement)
    proc.apply(white_measurement)
    ....
    

If you have more question regarding the distance calibration for your camera,
the best way would be to open an issue on our service desk: http://support.cubert-hyperspectral.com/

[JobinFrancisGithub]

Oh, thank you so much! I’ve updated the code based on the hint you provided. I’ve incorporated the distance calibration .cu3 file and normalized the reflectance values by dividing the cube by its maximum value, bringing them into the [0, 1] range.

As a result, the spatial shift has been significantly reduced — I’d say by about 98%. I can share a few output images and their corresponding spectra with you.

However, I do have a question: I noticed a dip in the spectrum after 900 nm. Could you kindly check the overall processing pipeline — particularly the use of the dark reference, white reference, and distance calibration — as well as the method I’ve used to extract the spectrum? I just want to ensure everything is implemented correctly.

And the snippet is :

import cuvis
import numpy as np
import matplotlib.pyplot as plt

# Define your input paths
cu3_file = r"D:\Strawberry_Data\Strawberry_data_2024\20241001\Cubert\session_000\session_000_150.cu3"
dark_file = r"D:\Strawberry_Data\Strawberry_data_2024\20241001\Cubert\Calibration\dark__session_000_003_snapshot17277709108509428.cu3"
white_file = r"D:\Strawberry_Data\Strawberry_data_2024\20241001\Cubert\Calibration\white__session_000_001_snapshot17277708628432653.cu3"
distance_file = r"D:\Strawberry_Data\Strawberry_data_2024\20241001\Cubert\Calibration\distanceCalib__session_000_005_snapshot17277709676208773.cu3"

# Load measurement and calibration images
meas = cuvis.Measurement(cu3_file)
white = cuvis.Measurement(white_file)
dark = cuvis.Measurement(dark_file)
distance = cuvis.Measurement(distance_file)

# Create processing context
proc_context = cuvis.ProcessingContext(meas)

# Set white and dark references using ReferenceType

proc_context.set_reference(distance, cuvis.ReferenceType.Distance)
proc_context.set_reference(white, cuvis.ReferenceType.White)
proc_context.set_reference(dark, cuvis.ReferenceType.Dark)


# Set reflectance processing mode
#proc_context.set_processing_mode(cuvis.ProcessingMode.Reflectance)
proc_context.processing_mode = cuvis.ProcessingMode.Reflectance

# Apply processing
processed = proc_context.apply(meas)

# Extract the reflectance cube and wavelengths
cube = processed.data['cube'].array
#wavelengths = processed.channel_desc.Center

wavelengths = np.array(processed.data['cube'].wavelength)
print(wavelengths)

#reflectance_cube=cube

reflectance_cube = cube / np.max(cube)

# Find the indices corresponding to the visible spectrum (approx 400-700 nm)
# You can adjust this range as needed based on your wavelength values
blue_idx = np.argmin(np.abs(wavelengths - 450))  # Closest to 450 nm for Blue
green_idx = np.argmin(np.abs(wavelengths - 550))  # Closest to 550 nm for Green
red_idx = np.argmin(np.abs(wavelengths - 650))  # Closest to 650 nm for Red

# Extract the 3 bands
blue = reflectance_cube[:, :, blue_idx]
green = reflectance_cube[:, :, green_idx]
red = reflectance_cube[:, :, red_idx]

# Normalize the bands to [0, 1] range for better visualization
blue = np.clip(blue, 0, np.max(blue)) / np.max(blue)
green = np.clip(green, 0, np.max(green)) / np.max(green)
red = np.clip(red, 0, np.max(red)) / np.max(red)

# Create the RGB composite image
rgb_image = np.stack((red, green, blue), axis=-1)

# Display the RGB image
plt.imshow(rgb_image)
plt.title("RGB Composite of Hyperspectral Image")
plt.axis('off')  # Hide axes for clarity
plt.show()

def extract_roi(arr, x, y, w, h, intensity, line):
    roi = arr[y:y+h, x:x+w, :]

    bounding_box = arr
    bounding_box[y-line:y, x-line:x+w+line, :] = intensity # top line
    bounding_box[y:y+h, x-line:x, :] = intensity # left line
    bounding_box[y+h:y+h+line, x-line:x+w+line, :] = intensity # bottom line
    bounding_box[y:y+h, x+w:x+w+line, :] = intensity # right line

    return (roi, bounding_box)

#coordinates = [(127, 18), (160, 16), (124, 110), (216, 107), (121, 216), (195, 207), (127, 278), (225, 330) , (121,394), (225, 394) ]
coordinates = [(160, 320) ]
coordinates=np.array(coordinates)

rois = [] # returned ROIs
length1 = 15 # width and height
length2 = 15 # width and height
line = 2 # bounding box line width
intensity = 2 # bounding box line intensity
image_bboxed = None


#twisted_cube = np.transpose(reflectance_cube, (1, 2, 0))

print("Original shape:", reflectance_cube.shape)   # (410, 410, 164)
#print("Twisted shape:", twisted_cube.shape)         # (410, 410, 164)


for coordinate in coordinates:
    (x, y) = coordinate
    (roi, image_bboxed) = extract_roi(
        reflectance_cube, x, y, length1, length2, intensity, line)
    rois.append(roi)


#roi = np.clip(roi, 0, np.percentile(roi, 99))

plt.imshow(image_bboxed[:, :, 100], cmap='gray')  # Displaying band 100 as an example
plt.title("ROI with Bounding Box (Band 100)")
plt.axis('off')
plt.show()

# for i in range(len(rois)):
#     roi = rois[i]
#     intensity = []
#     for b in range(roi.shape[2]):
#         intensity.append(np.mean(roi[:,:,b]))
#     plt.plot(bands, intensity, label='ROI {}'.format(i+1))


for i in range(len(rois)):
    roi = rois[i]  # Shape: (H, W, Bands)
    h, w, d = roi.shape
    plt.figure(figsize=(10, 6))
    for y in range(h):
        for x in range(w):
            spectrum = roi[y, x, :]  # Extract spectrum at pixel (y, x)
            plt.plot(wavelengths, spectrum, alpha=0.3, linewidth=0.8)

    plt.title(f'Spectral Footprints - Individual Pixels in ROI {i+1}')
    plt.xlabel('Wavelength (nm)')
    plt.ylabel('Intensity')
    plt.grid(True)
    plt.tight_layout()
    plt.show()


# plt.legend(loc='upper left')
# plt.title(' Spectral Footprint\n Mean in ROI Area')
# plt.xlabel('Wavelength (nm)')
# plt.ylabel('Raw Data')
# plt.show()

#Plot for the mean spectra of the ROI
for i in range(len(rois)):
    roi = rois[i]
    intensity = []
    for b in range(roi.shape[2]):
        intensity.append(np.mean(roi[:,:,b]))
    plt.plot(wavelengths, intensity, label='ROI {}'.format(i+1))

plt.legend(loc='upper left')
plt.title(' Spectral Footprint\n Mean in ROI Area')
plt.xlabel('Wavelength (nm)')
plt.ylabel('Reflectance')
plt.show()

#np.save(r"D:\Python_Codes_N_Results_Folder\Results\Spectrum_Strawberry_Folder\rois_trail.npy", rois)

[birkholz-cubert]

Hey,

so i don't have the time to go through your code in detail.
If you experience data qualtiy problems with your camera i would recommend you to open an issue at the service desk http://support.cubert-hyperspectral.com/.

However some thing that i noticed:

when you use the sdk to reprocess a measurmement to reflectance mode, the unit for this cube is Percentage / 100 (meaning it describes up to two decimal places of the reflectance percentage)
To now get the correct reflectance value in the range of [0,1] you would divide by 10000

reflectance_cube = cube / 10_000

The rest of the usage of the sdk looks good to me.
If you encounter any other issues with the sdk, feel free to ask.

[JobinFrancisGithub]

Thank you so much.