Skip to content

Conversation

@Barry-Xu-2018
Copy link
Contributor

@fujitatomoya
Copy link
Contributor

@Barry-Xu-2018 thanks 👍 please let us know once this is ready to review.

@fujitatomoya
Copy link
Contributor

CC: @MichaelOrlov

@Barry-Xu-2018 Barry-Xu-2018 force-pushed the reivew/topic-support-action-record-and-info branch from 3b18d88 to 83b0d07 Compare March 16, 2025 03:04
@Barry-Xu-2018
Copy link
Contributor Author

please let us know once this is ready to review.

Okay. I will notify you once it's ready.

@Barry-Xu-2018 Barry-Xu-2018 marked this pull request as ready for review March 17, 2025 06:17
@Barry-Xu-2018
Copy link
Contributor Author

@fujitatomoya @MichaelOrlov

Although the design (#1928) is still being finalized, there are no objections regarding the record and info sections, so I have implemented these parts according to the design.

If you have time, please help review it.

Copy link
Contributor

@MichaelOrlov MichaelOrlov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Barry-Xu-2018 Thank you for your contribution and hard work. The PR is huge. I've made a first round of review which is not thorough yet. However, let's address first my finding from the first round then will do one more round.

Among my findings and suggestions mentioned inline in the code, I have one major concern in regards to the design of the API changes in the RecordOptions structure.
In the current implementation a newly added fields as

std::vector<std::string> actions;  // action internal topics and service event topics

and

std::vector<std::string> exclude_actions;  // action internal topics and service event topics

imply to keep a list of extended "full" action's topic names and events.
i.e. we need to expand each action to a list of action topics list by calling for the rosbag2_cpp::action_name_to_action_topic_name(action); in a multiple places.
However, we are using this new parameters for actions and exclude_actions from RecordOptions struct only in the TopicFilter class.
It would be a more elegant and cleaner implementation if we would use non-expaned action names in the RecordOptions struct (the same as in the CLI options) and expand them to the real action topics inside TopicFilter constructor and keep them as a private member variables of the TopicFilter class.
This will also simplify the cases when we need to serialize-deserialize the RecordOptions structure and store it in config files or in the metadata.

@Barry-Xu-2018
Copy link
Contributor Author

@MichaelOrlov

Thank you for your review of the first round. I will handle your review comments.

Copy link
Contributor

@fujitatomoya fujitatomoya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Barry-Xu-2018 this one is big... 😅 thank you very much for your effort. i got several comments, most of them are minor. please have them checked out!

@Barry-Xu-2018
Copy link
Contributor Author

@MichaelOrlov

It would be a more elegant and cleaner implementation if we would use non-expaned action names in the RecordOptions struct (the same as in the CLI options) and expand them to the real action topics inside TopicFilter constructor and keep them as a private member variables of the TopicFilter class.
This will also simplify the cases when we need to serialize-deserialize the RecordOptions structure and store it in config files or in the metadata.

Thanks. I agree with you.
I have updated code for the first round of review comment. 4946f5f

@Barry-Xu-2018
Copy link
Contributor Author

@fujitatomoya

Thanks for your comments. I will handle them.

@Barry-Xu-2018
Copy link
Contributor Author

Barry-Xu-2018 commented Mar 26, 2025

@fujitatomoya

i think this one is not addressed yet?

During your review, my fix (4946f5f) hadn't been submitted yet. However, these issues were already fixed in my fix, so I marked it as resolved.

@Barry-Xu-2018
Copy link
Contributor Author

@fujitatomoya

I submitted below commit to handle your review comments.
8e6f827

Copy link
Contributor

@fujitatomoya fujitatomoya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i confirmed all the comments are addressed, lgtm with green CI.

@fujitatomoya
Copy link
Contributor

@Barry-Xu-2018 github action is not haapy, https://github.com/ros2/rosbag2/actions/runs/14080698263/job/39432789706?pr=1939 can you check this?

@fujitatomoya
Copy link
Contributor

@Barry-Xu-2018 github action is happy now, thanks!

@MichaelOrlov could you check the your comments? in the meantime, i will start the CI.

@fujitatomoya
Copy link
Contributor

Pulls: #1939
Gist: https://gist.githubusercontent.com/fujitatomoya/01ae3c5effe622750ff6f312bb94a203/raw/290701278e198437896adf90b5a853a4aad99cdd/ros2.repos
BUILD args: --packages-above-and-dependencies ros2bag rosbag2_cpp rosbag2_py rosbsg2_test_common rosbag2_tests rosbag2_transport
TEST args: --packages-above ros2bag rosbag2_cpp rosbag2_py rosbsg2_test_common rosbag2_tests rosbag2_transport
ROS Distro: rolling
Job: ci_launcher
ci_launcher ran: https://ci.ros2.org/job/ci_launcher/15499

  • Linux Build Status
  • Linux-aarch64 Build Status
  • Linux-rhel Build Status
  • Windows Build Status

@Barry-Xu-2018
Copy link
Contributor Author

  • Linux Build Status
  • Linux-rhel Build Status

The error log is

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of '[email protected]:osrf/rticonnextdds-bins.git' into submodule path '/home/jenkins-agent/workspace/ci_linux/linux_docker_resources/rticonnextdds-src' failed
Failed to clone 'linux_docker_resources/rticonnextdds-src'. Retry scheduled
Cloning into '/home/jenkins-agent/workspace/ci_linux/linux_docker_resources/rticonnextdds-src'...

I have no idea about this issue.
But I find the following action also included the same error.
https://ci.ros2.org/job/ci_linux/22910/console

  • Linux-aarch64 Build Status
  • Windows Build Status

The cause is related to an error in Gist.
@fujitatomoya Please help to fix an error in Gist

  Barry-Xu-2018/rosbag2:  <== ros2/rosbag2
    type: git
    url: https://github.com/Barry-Xu-2018/rosbag2.git

@MichaelOrlov
Copy link
Contributor

MichaelOrlov commented Mar 28, 2025

@Barry-Xu-2018 I see that there is existent /fibonacci/_action/feedback topic in actions. But we are not handling it.
I am curious why. Do we miss it?

@MichaelOrlov
Copy link
Contributor

@Barry-Xu-2018 I see that there is existent /fibonacci/_action/feedback topic in actions. But we are not handling it. I am curious why. Do we miss it?

My bad, I messed up and was wrong. We do handle feedback messages.
What I can't find is the handling of the GoalInfo.msg.

record_options.topic_types = args.topic_types
# Convert service name to service event topic name
record_options.services = convert_service_to_service_event_topic(args.services)
record_options.actions = args.actions if args.actions else []
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need to explicitly assign an empty list []?
It seems it could be simplified as a simple assignment

record_options.actions = args.actions

The same as topics

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I change to this, running the ros2bag test will report this error.

[ros2bag-cli-7]   File "/root/ros2_src_rolling/build/ros2bag/ros2bag/verb/record.py", line 361, in main
[ros2bag-cli-7]     record_options.actions = args.actions
[ros2bag-cli-7]     ^^^^^^^^^^^^^^^^^^^^^^
[ros2bag-cli-7] TypeError: (): incompatible function arguments. The following argument types are supported:
[ros2bag-cli-7]     1. (self: rosbag2_py._transport.RecordOptions, arg0: List[str]) -> None

I compared the code with the record_options.topics and did not find any differences in the types. Do you have any suggestion about this issue ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting.. I will take a look

record_options.exclude_topics = args.exclude_topics if args.exclude_topics else []
record_options.exclude_service_events = \
convert_service_to_service_event_topic(args.exclude_services)
record_options.exclude_actions = args.exclude_actions if args.exclude_actions else []
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same here it seems we don;t need conversion here.

Copy link
Contributor Author

@Barry-Xu-2018 Barry-Xu-2018 Mar 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have the same problem mentioned in #1939 (comment)

If I change record_options.exclude_topics, it also has the same problem.

@fujitatomoya
Copy link
Contributor

  • Linux Build Status
  • Linux-rhel Build Status

The error log is

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of '[email protected]:osrf/rticonnextdds-bins.git' into submodule path '/home/jenkins-agent/workspace/ci_linux/linux_docker_resources/rticonnextdds-src' failed
Failed to clone 'linux_docker_resources/rticonnextdds-src'. Retry scheduled
Cloning into '/home/jenkins-agent/workspace/ci_linux/linux_docker_resources/rticonnextdds-src'...

I have no idea about this issue. But I find the following action also included the same error. https://ci.ros2.org/job/ci_linux/22910/console

  • Linux-aarch64 Build Status
  • Windows Build Status

The cause is related to an error in Gist. @fujitatomoya Please help to fix an error in Gist

  Barry-Xu-2018/rosbag2:  <== ros2/rosbag2
    type: git
    url: https://github.com/Barry-Xu-2018/rosbag2.git

hmmm, this is weird. i just use https://github.com/ros-tooling/ros-github-scripts, probably recent PR introduce some regression... anyway, for now, i just fixed the https://gist.githubusercontent.com/fujitatomoya/01ae3c5effe622750ff6f312bb94a203/raw/b2829009be047691533f0103136e64b5c27efbfa/ros2.repos, i will restart the CI

@fujitatomoya
Copy link
Contributor

fujitatomoya commented Mar 28, 2025

CI:

  • Linux Build Status
  • Linux-aarch64 Build Status
  • Linux-rhel Build Status
  • Windows Build Status

@fujitatomoya
Copy link
Contributor

@Barry-Xu-2018

The cause is related to an error in Gist. @fujitatomoya Please help to fix an error in Gist

i had a typo for the test target package. the new CI is started.

@Barry-Xu-2018
Copy link
Contributor Author

@MichaelOrlov

What I can't find is the handling of the GoalInfo.msg.

Why do you think we need to handle GoalInfo.msg for recording or info command ?
For showing action info, we only display the following info. We don't care each Goal. We do not count the number of Goal operations.

Actions:           2
Action information: 
  Action: /test_action1 | Type: example_interfaces/action/Fibonacci | Topics: 2 | Service: 3 | Size Contribution: 584 B | Serialization Format: cdr
    Topic: feedback | Count: 5
    Topic: status | Count: 2
    Service: send_goal | Request Count: 1 | Response Count: 1
    Service: cancel_goal | Request Count: 0 | Response Count: 0
    Service: get_result | Request Count: 1 | Response Count: 1
  Action: /test_action2 | Type: example_interfaces/action/Fibonacci | Topics: 2 | Service: 3 | Size Contribution: 708 B | Serialization Format: cdr
    Topic: feedback | Count: 7
    Topic: status | Count: 2
    Service: send_goal | Request Count: 1 | Response Count: 1
    Service: cancel_goal | Request Count: 0 | Response Count: 0
    Service: get_result | Request Count: 1 | Response Count: 1

Barry-Xu-2018 and others added 17 commits March 31, 2025 11:28
Signed-off-by: Barry Xu <[email protected]>
Signed-off-by: Barry Xu <[email protected]>
Signed-off-by: Barry Xu <[email protected]>
- Rationale: We shall store the original message definition for recorded
 topics by design. This is also true for service events.

Signed-off-by: Michael Orlov <[email protected]>
@Barry-Xu-2018 Barry-Xu-2018 force-pushed the reivew/topic-support-action-record-and-info branch from da8fcd6 to 6bb2be1 Compare March 31, 2025 04:44
@Barry-Xu-2018
Copy link
Contributor Author

Rebase was done.

Gist: https://gist.githubusercontent.com/fujitatomoya/01ae3c5effe622750ff6f312bb94a203/raw/b2829009be047691533f0103136e64b5c27efbfa/ros2.repos
BUILD args: --packages-above-and-dependencies ros2bag rosbag2_cpp rosbag2_py rosbag2_test_common rosbag2_tests rosbag2_transport
TEST args: --packages-above ros2bag rosbag2_cpp rosbag2_py rosbag2_test_common rosbag2_tests rosbag2_transport
ROS Distro: rolling
Job: ci_launcher
ci_launcher ran: https://ci.ros2.org/job/ci_launcher/15530

  • Linux Build Status
  • Linux-aarch64 Build Status
  • Linux-rhel Build Status
  • Windows Build Status

@MichaelOrlov
Copy link
Contributor

@Barry-Xu-2018 The Windows CI Job fails with error messages:

Syntax error at index 90 in simple regular expression "Topic: /test_topic1 | Type: std_msgs/msg/String | Count: //d+ | Size Contribution: //+d B | Serialization Format: cdr
": '|' is unsupported.


C:\ci\ws\src\ros2\rosbag2\rosbag2_tests\test\rosbag2_tests\test_rosbag2_info_end_to_end.cpp:133
Value of: output
Expected: contains regular expression "Topic: /test_topic1 | Type: std_msgs/msg/String | Count: //d+ | Size Contribution: //+d B | Serialization Format: cdr\n"
  Actual: "\nFiles:             bag_with_topics_and_service_events_and_action.db3\nBag size:          103.8 KiB\nStorage id:        sqlite3\nROS Distro:        rolling\nDuration:          52.559897611s\nStart:             Mar 14 2025 10:51:19.681027429 (1741949479.681027429)\nEnd:               Mar 14 2025 10:52:12.240925040 (1741949532.240925040)\nMessages:          159\nTopic information: Topic: /events/write_split | Type: rosbag2_interfaces/msg/WriteSplitEvent | Count: 0 | Size Contribution: 0 B | Serialization Format: cdr\n                   Topic: /parameter_events | Type: rcl_interfaces/msg/ParameterEvent | Count: 0 | Size Contribution: 0 B | Serialization Format: cdr\n                   Topic: /rosout | Type: rcl_interfaces/msg/Log | Count: 100 | Size Contribution: 17.5 KiB | Serialization Format: cdr\n                   Topic: /test_topic1 | Type: std_msgs/msg/String | Count: 32 | Size Contribution: 768 B | Serialization Format: cdr\n                   Topic: /test_topic2 | Type: std_msgs/msg/String | Count: 27 | Size Contribution: 648 B | Serialization Format: cdr\nServices:          2\nService information: Service: /test_service1 | Type: example_interfaces/srv/AddTwoInts | Request Count: 1 | Response Count: 1 | Size Contribution: 132 B | Serialization Format: cdr\n                     Service: /test_service2 | Type: example_interfaces/srv/AddTwoInts | Request Count: 1 | Response Count: 1 | Size Contribution: 132 B | Serialization Format: cdr\nActions:           2\nAction information: \n  Action: /test_action1 | Type: example_interfaces/action/Fibonacci | Topics: 2 | Service: 3 | Size Contribution: 584 B | Serialization Format: cdr\n    Topic: feedback | Count: 5\n    Topic: status | Count: 2\n    Service: send_goal | Request Count: 1 | Response Count: 1\n    Service: cancel_goal | Request Count: 0 | Response Count: 0\n    Service: get_result | Request Count: 1 | Response Count: 1\n  Action: /test_action2 | Type: example_interfaces/action/Fibonacci | Topics: 2 | Service: 3 | Size Contribution: 708 B | Serialization Format: cdr\n    Topic: feedback | Count: 7\n    Topic: status | Count: 2\n    Service: send_goal | Request Count: 1 | Response Count: 1\n    Service: cancel_goal | Request Count: 0 | Response Count: 0\n    Service: get_result | Request Count: 1 | Response Count: 1\n"

@Barry-Xu-2018
Copy link
Contributor Author

@MichaelOrlov

The Windows CI Job fails with error messages:
...

Thank you. I’ll investigate it.
I might need to spend some time setting up a Windows environment for testing.

@Barry-Xu-2018
Copy link
Contributor Author

Rerun CI

  • Linux Build Status
  • Linux-aarch64 Build Status
  • Linux-rhel Build Status
  • Windows Build Status

@Barry-Xu-2018
Copy link
Contributor Author

@MichaelOrlov

Syntax error at index 90 in simple regular expression "Topic: /test_topic1 | Type: std_msgs/msg/String | Count: //d+ | Size Contribution: //+d B | Serialization Format: cdr
": '|' is unsupported.


C:\ci\ws\src\ros2\rosbag2\rosbag2_tests\test\rosbag2_tests\test_rosbag2_info_end_to_end.cpp:133
Value of: output
Expected: contains regular expression "Topic: /test_topic1 | Type: std_msgs/msg/String | Count: //d+ | Size Contribution: //+d B | Serialization Format: cdr\n"
  Actual: "\nFiles:             bag_with_topics_and_service_events_and_action.db3\nBag size:          103.8 KiB\nStorage id:        sqlite3\nROS Distro:        rolling\nDuration:          52.559897611s\nStart:             Mar 14 2025 10:51:19.681027429 (1741949479.681027429)\nEnd:               Mar 14 2025 10:52:12.240925040 (1741949532.240925040)\nMessages:          159\nTopic information: Topic: /events/write_split | Type: rosbag2_interfaces/msg/WriteSplitEvent | Count: 0 | Size Contribution: 0 B | Serialization Format: cdr\n                   Topic: /parameter_events | Type: rcl_interfaces/msg/ParameterEvent | Count: 0 | Size Contribution: 0 B | Serialization Format: cdr\n                   Topic: /rosout | Type: rcl_interfaces/msg/Log | Count: 100 | Size Contribution: 17.5 KiB | Serialization Format: cdr\n                   Topic: /test_topic1 | Type: std_msgs/msg/String | Count: 32 | Size Contribution: 768 B | Serialization Format: cdr\n                   Topic: /test_topic2 | Type: std_msgs/msg/String | Count: 27 | Size Contribution: 648 B | Serialization Format: cdr\nServices:          2\nService information: Service: /test_service1 | Type: example_interfaces/srv/AddTwoInts | Request Count: 1 | Response Count: 1 | Size Contribution: 132 B | Serialization Format: cdr\n                     Service: /test_service2 | Type: example_interfaces/srv/AddTwoInts | Request Count: 1 | Response Count: 1 | Size Contribution: 132 B | Serialization Format: cdr\nActions:           2\nAction information: \n  Action: /test_action1 | Type: example_interfaces/action/Fibonacci | Topics: 2 | Service: 3 | Size Contribution: 584 B | Serialization Format: cdr\n    Topic: feedback | Count: 5\n    Topic: status | Count: 2\n    Service: send_goal | Request Count: 1 | Response Count: 1\n    Service: cancel_goal | Request Count: 0 | Response Count: 0\n    Service: get_result | Request Count: 1 | Response Count: 1\n  Action: /test_action2 | Type: example_interfaces/action/Fibonacci | Topics: 2 | Service: 3 | Size Contribution: 708 B | Serialization Format: cdr\n    Topic: feedback | Count: 7\n    Topic: status | Count: 2\n    Service: send_goal | Request Count: 1 | Response Count: 1\n    Service: cancel_goal | Request Count: 0 | Response Count: 0\n    Service: get_result | Request Count: 1 | Response Count: 1\n"

On the Windows platform, I couldn't find a successful way to make ContainsRegex support the "|" character ("\\|" doesn't make ContainsRegex work as expected. But this can avoid above error.). I had to switch to using std::regex for testing. e196b67

@fujitatomoya
Copy link
Contributor

@MichaelOrlov CI is green, if you are good to go, can you merge this?
@Barry-Xu-2018 thanks for your effort!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants