Unable to Publish raw mjpeg stream directly via compressed image topic

[wendwosenbb]

@flynneva

I was trying to use the feature you merged in pull request #270, to save on cpu usage by directly publishing via the compressed image topic.
scenarios I tested

1. I set the pixel_format to raw_mjpeg and av_device_format MJPG (I was expecting this work)
   crashed without any log.
2. I set the pixel_format to raw_mjpeg and av_device_format YUV422P, I see the green screen

3. 1. I set the pixel_format to mjpeg2rgb and av_device_format YUV422P

this produces compressed image as desired but with significant CPU usage.

would you direct me in how to levarage the feature described in #270. Thank you!

this is my camera_config file:

ros__parameters: video_device: "/dev/windshield_camera" framerate: 15.0 io_method: "mmap" frame_id: "windshield_camera" pixel_format: "raw_mjpeg" av_device_format: "MJPG" #YUV422P, MJPEG image_width: 640 image_height: 480 camera_name: "windshield_camera" camera_info_url: "package://usb_cam/config/camera_info.yaml" brightness: -1 contrast: -1 saturation: -1 sharpness: -1 gain: -1 auto_white_balance: true white_balance: 4000 autoexposure: true exposure: 100 autofocus: false focus: -1

The device formats supports:

`v4l2-ctl --list-formats-ex --device /dev/windshield_camera

ioctl: VIDIOC_ENUM_FMT
Type: Video Capture

    [0]: 'MJPG' (Motion-JPEG, compressed)
            Size: Discrete 1920x1080
                    Interval: Discrete 0.033s (30.000 fps)
                    Interval: Discrete 0.040s (25.000 fps)
                    Interval: Discrete 0.050s (20.000 fps)
                    Interval: Discrete 0.067s (15.000 fps)
                    Interval: Discrete 0.100s (10.000 fps)
            Size: Discrete 1280x720
                    Interval: Discrete 0.033s (30.000 fps)
                    Interval: Discrete 0.040s (25.000 fps)
                    Interval: Discrete 0.050s (20.000 fps)
                    Interval: Discrete 0.067s (15.000 fps)
                    Interval: Discrete 0.100s (10.000 fps)
            Size: Discrete 800x600
                    Interval: Discrete 0.033s (30.000 fps)
                    Interval: Discrete 0.050s (20.000 fps)
                    Interval: Discrete 0.067s (15.000 fps)
                    Interval: Discrete 0.100s (10.000 fps)
                    Interval: Discrete 0.200s (5.000 fps)
            Size: Discrete 640x480
                    Interval: Discrete 0.033s (30.000 fps)
                    Interval: Discrete 0.040s (25.000 fps)
                    Interval: Discrete 0.050s (20.000 fps)
                    Interval: Discrete 0.067s (15.000 fps)
                    Interval: Discrete 0.100s (10.000 fps)
            Size: Discrete 320x240
                    Interval: Discrete 0.033s (30.000 fps)
                    Interval: Discrete 0.040s (25.000 fps)
                    Interval: Discrete 0.050s (20.000 fps)
                    Interval: Discrete 0.067s (15.000 fps)
                    Interval: Discrete 0.100s (10.000 fps)
    [1]: 'YUYV' (YUYV 4:2:2)
            Size: Discrete 1280x720
                    Interval: Discrete 0.100s (10.000 fps)
            Size: Discrete 1920x1080
                    Interval: Discrete 0.200s (5.000 fps)
            Size: Discrete 800x600
                    Interval: Discrete 0.100s (10.000 fps)
                    Interval: Discrete 0.200s (5.000 fps)
            Size: Discrete 640x480
                    Interval: Discrete 0.033s (30.000 fps)
                    Interval: Discrete 0.040s (25.000 fps)
                    Interval: Discrete 0.050s (20.000 fps)
                    Interval: Discrete 0.067s (15.000 fps)
                    Interval: Discrete 0.100s (10.000 fps)
            Size: Discrete 320x240
                    Interval: Discrete 0.033s (30.000 fps)
                    Interval: Discrete 0.040s (25.000 fps)
                    Interval: Discrete 0.050s (20.000 fps)
                    Interval: Discrete 0.067s (15.000 fps)
                    Interval: Discrete 0.100s (10.000 fps)`

The usb_cam driver is showing me pixel formats supported

[usb_cam_node_exe-2] This driver supports the following formats:
[usb_cam_node_exe-2] rgb8
[usb_cam_node_exe-2] yuyv
[usb_cam_node_exe-2] yuyv2rgb
[usb_cam_node_exe-2] uyvy
[usb_cam_node_exe-2] uyvy2rgb
[usb_cam_node_exe-2] mono8
[usb_cam_node_exe-2] mono16
[usb_cam_node_exe-2] y102mono8
[usb_cam_node_exe-2] raw_mjpeg
[usb_cam_node_exe-2] mjpeg2rgb
[usb_cam_node_exe-2] m4202rgb

[boitumeloruf]

Hi @wendwosenbb,

in your second experiment, in which you go the green screen, to which image topic have you subscribed?

[wendwosenbb]

Hi @boitumeloruf
there are three topics and I was subscribing to /windshield_camera/image_raw and /windshield_camera/image_raw/compressed. The compressed image topic gives black image while raw_image topic gives green image.

1. /windshield_camera/image_raw/compressed: this topic is remapped from 'image_raw/compressed'
   It gives the following image

2. /windshield_camera/image_raw: this topic is remapped from '/image_raw'
   It gives the following image

[wendwosenbb]

@boitumeloruf
I did some more test by commenting out both pixel_format and av_device_format (using their default values, i.e, pixel_format: yuyv, and av_device_format: YUV422P)

the compressed image topic (/windshield_camera/image_raw/compressed) gives a gray scale image:

the raw image topic (/windshield_camera/image_raw) gives rgb image as expected

[boitumeloruf]

Ok, maybe something broke through further development since the merge of my feature request and. Maybe try using the older version still available in my fork (branch 'ros2'): https://github.com/boitumeloruf/usb_cam/tree/ros2

[wendwosenbb]

@boitumeloruf

Thank you for following up on my questions.
The available pixel formats list doesn't include "mjpeg" but here and here in the usb_cam_node is checking if the pixel format is "mjpeg". Changing "mjpeg" to "raw_mjpeg" in those lines solved the issue. I now have the raw image published on a compressed image topic with reduced CPU usage.

[flynneva]

@wendwosenbb apologies for being absent here in the replies - I am about to become a dad in the next few hours/days/weeks so I've been busy the last few weeks/months supporting my wife and preparing our house.

There are two "available format" lists that are generated - one that should list the available formats this driver supports (which should include mjpeg in it) and the other list should include the available formats your selected hardware supports.

Both of these should automatically be printed if you've misconfigured something...but only one at a time depending on the configuration error.

There is a high probability I won't be that active here once my son is born so apologies in advanced for not replying here that quickly 😅

[wendwosenbb]

@flynneva
Congratulations to you and your wife!. No worries on replies, my issue has been resolved. Sending good vibes for a smooth and joyful experience as you become a dad!

[boitumeloruf]

@boitumeloruf

Thank you for following up on my questions. The available pixel formats list doesn't include "mjpeg" but here and here in the usb_cam_node is checking if the pixel format is "mjpeg". Changing "mjpeg" to "raw_mjpeg" in those lines solved the issue. I now have the raw image published on a compressed image topic with reduced CPU usage.

Great that you have found the issue. This problem also seemed to have been fixed in this PR: #336

[roboticvedant]

Its still not working for me:

My camera:

My config: