Frame drops and data loss occur when using realsense d455 in Jetson nano

[didwldnd03-hash]

Camera model : D455
Firmware version : 5.13.00.5
Ros2 wrapper: v 4.51.1
Librealsense SDK: v 2.51.1
USB Type: 3.2
Jetpack :6.0
Ubuntu: 22.04

I published topics using the command ros2 launch realsense2_cameras_launch.py. Use the instruction (rosbag record) to collect camera data. As shown in the picture, I used python visualize to view the data packet.

I also record data to .mcap and visualize in foxglove studio, but similar problem occur. (103.5 ~ 104)

Also, Frame drop problem occur in realsense-viewer

[MartyG-RealSense]

Hi @didwldnd03-hash The CPU might be being over-burdened by the number of simultaneous streams being published. You could check whether this is the case by using the Ubuntu system monitoring tool htop whilst the programs are running and see how high the CPU usage percentage value is.

[didwldnd03-hash]

@MartyG-RealSense I check it by using Jtop while recording but there was no problem...

[MartyG-RealSense]

In the RealSense Viewer, the Frame Drops Per Second chart is not a reliable indicator of performance. It is better to use the 'i' button to activate the info overlay and then look at the real-time FPS value on the overlay to see how close it is to the FPS value that you set in the Viewer's options side-panel before starting the stream.

[didwldnd03-hash]

@MartyG-RealSense Thanks, I'll try it. But I think that would not solve the loss of data. So, are there any other way that I could try to solve the problem?

[MartyG-RealSense]

Looking at the graph that you kindly provided at the beginning of this issue, it looks as though 30 frames per second are being provided for depth and color in the recording if each vertical dash represents 1 recorded frame (I checked the frame count for a 1 second period inbetween 255 and 256 seconds, and the default FPS speed of depth and color is 30 frames per second).

---

It looks though as though the Infra1 and Infra2 infrared streams are only producing 15 frames per second instead of 30.

Let's try setting all streams (depth, color and infrared) to 640x480 and 15 FPS to see if that makes the frame production more equal.

ros2 launch realsense2_camera rs_launch.py depth_module.depth_profile:=640x480x15 depth_module.infra_profile:=640x480x15 rgb_camera.color_profile:=640x480x15

[didwldnd03-hash]

@MartyG-RealSense I tried it before. But the data loss hasn't gone away...

[MartyG-RealSense]

Does your project require the infrared topics, please? Having the infrared streams enabled is not required in order to publish the depth stream.

[didwldnd03-hash]

@MartyG-RealSense Yes, my project required them... And data loss hasn't gone when I reduce the recording topics.

[MartyG-RealSense]

Given that RealSense Viewer also appears to struggle, do you have another computer to try the camera on to establish whether the frame loss only occurs on Nano?

Are you using the original 1 meter official USB cable supplied with the D455 or a longer cable of your own choice?

May I also confirm whether you have the original Jetson Nano or the newer Jetson Orin Nano? On the original Jetson Nano, Jetpack 4.6 and Ubuntu 18.04 is recommended to be used as the maximum JP and Ubuntu versions, whilst Orin Nano can support Ubuntu 22.04 and JetPack 6.

[didwldnd03-hash]

@MartyG-RealSense I don't have other computer..
I think my USB cable is shorter than 1 meter. And I use Jetson Orin Nano. Is there a possibility that it's a problem with the firmware version or the sdk version?

[MartyG-RealSense]

With RealSense cameras, USB cable quality is important for stability. It should be a high quality cable designed for data transfer and not one that was intended only for charging. And if the cable is a C to C type instead of the official cable's Type-C (a full size connector on one end and the micro-sized connector on the other) then use of a C to C cable can also negatively impact performance compared to an A to C cable.

Official JetPack 6 support was not introduced until an SDK version much newer than the one you are using. So if you need to use JP6 because of Ubuntu 22.04 then it is highly recommended to build the SDK from source code with the libuvc backend installation method if you have not done so already, as the libuvc backend install method bypasses the kernel and so works around problems caused by JP6.