Add experimental `child_frame` & `parent_frame` to `Transform3D` by Wumpf · Pull Request #11730 · rerun-io/rerun

Wumpf · 2025-10-30T17:58:50Z

What

EDIT: fields got renamed in #11770

Another crucial stepping stone towards ROS style transform support!

source & target are singular right now in the API (since all of Transform3D is), but the underlying datastructures are already unit-tested against multiple sources (alas, not targets though; sources is where all the complexity is though anyways).

The doc of these new fields explains what's going on here:

  /// ⚠️ Experimental ⚠️: The frame this transform transforms from.
  ///
  /// The entity at which the transform relationship of any given source frame is specified musn't change over time.
  /// E.g. if you specified the source "robot_arm" on an entity named "my_transforms", you may not log transforms
  /// with the source "robot_arm" on any other entity than "my_transforms".
  /// An exception to this rule is static time - you may first mention a source on one entity statically and later on
  /// another one temporally.
  ///
  /// ⚠️ This also affects the source frame of [archetype.Pinhole] & [archetype.PoseTransforms3D].
  ///
  /// If not specified, this is set to the implicit transform frame of the current entity path.
  /// This means that if a [archetypes.Transform3D] is set on an entity called `/my/entity/path` then this will default to `tf#/my/entity/path`.
  ///
  /// To set the frame an entity is part of see [archetypes.CoordinateFrame].
  // TODO(RR-2777): Mark as stable when we're ready for it.
  source_frame: rerun.components.TransformFrameId ("attr.rerun.component_optional", nullable, order: 2000);

  /// ⚠️ Experimental ⚠️: The frame this transform transforms to.
  ///
  ///
  /// ⚠️ This also affects the target frame of [archetype.Pinhole].
  ///
  /// If not specified, this is set to the implicit transform frame of the current entity path's parent.
  /// This means that if a [archetypes.Transform3D] is set on an entity called `/my/entity/path` then this will default to `tf#/my/entity`.
  ///
  /// To set the frame an entity is part of see [archetypes.CoordinateFrame].
  // TODO(RR-2777): Mark as stable when we're ready for it.
  target_frame: rerun.components.TransformFrameId ("attr.rerun.component_optional", nullable, order: 2100);

⚠️ TransformForest does not yet walk arbitrary frames and is still bound to implicit frames. Addressing this is the next step after.
As a consequence the added example is unfinished and a little bit "weird": we can define Transform3D outside of the entity hierarchy, but we can not yet collapse said entity hierarchy!

This PR unfortunately makes queries a bit slower again as captured by existing benchmarks. Datastructure build-up is roughtly the same (I had fairly big swings on that one without doing anything)

Windows Ryzen 9 7950X3D comparing against a previous run on main (dab04a6) via pixi run cargo bench -p re_tf

Benchmarking build_from_entity_db: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 35.9s, or reduce sample count to 10.
build_from_entity_db    time:   [347.14 ms 353.18 ms 359.98 ms]
                        change: [-1.5899% +0.4380% +2.7028%] (p = 0.70 > 0.05)
                        No change in performance detected.
Found 10 outliers among 100 measurements (10.00%)
  3 (3.00%) high mild
  7 (7.00%) high severe

Benchmarking query_uncached_frame: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 37.5s, or reduce sample count to 10.
query_uncached_frame    time:   [26.557 ms 27.016 ms 27.504 ms]
                        change: [+36.134% +39.507% +42.906%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
  4 (4.00%) high mild

query_cached_frame      time:   [30.762 ns 30.990 ns 31.222 ns]
                        change: [-2.1408% -0.4344% +1.0960%] (p = 0.62 > 0.05)
                        No change in performance detected.
Found 5 outliers among 100 measurements (5.00%)
  2 (2.00%) low mild
  2 (2.00%) high mild
  1 (1.00%) high severe

Still a far cry from how slow it was before #11655

github-actions · 2025-10-30T17:59:15Z

Web viewer built successfully.

Result	Commit	Link	Manifest
✅	`8878168`	https://rerun.io/viewer/pr/11730	`+nightly` `+main`

View image diff on kitdiff.

^{Note: This comment is updated whenever you push a commit.}

github-actions · 2025-10-30T17:59:20Z

Latest documentation preview deployed successfully.

Result	Commit	Link
✅	`8878168`	https://landing-cvt9myh4i-rerun.vercel.app/docs

^{Note: This comment is updated whenever you push a commit.}

Wumpf · 2025-10-30T22:23:18Z

@rerun-bot full-check

github-actions · 2025-10-30T22:23:57Z

Started a full build: https://github.com/rerun-io/rerun/actions/runs/18956703515

Copilot

Pull Request Overview

This PR adds experimental support for explicit source and target frames to the Transform3D archetype, enabling ROS-style transform frame relationships. The new source_frame and target_frame fields allow transforms to explicitly specify which frames they connect, moving beyond the implicit entity hierarchy-based transform system.

Key changes:

Added source_frame and target_frame optional components to Transform3D
Enhanced transform resolution cache to track frame relationships across entities and time
Added fallback providers for the new frame components that derive from entity paths

Reviewed Changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
scripts/ci/check_large_files.py	Added transform resolution cache file to exception list
rerun_py/rerun_sdk/rerun/archetypes/transform3d_ext.py	Added source_frame and target_frame parameters to Transform3D constructor
rerun_py/rerun_sdk/rerun/archetypes/transform3d.py	Updated Transform3D archetype with new frame components and documentation
rerun_cpp/src/rerun/archetypes/transform3d.hpp	Added C++ support for source_frame and target_frame components
rerun_cpp/src/rerun/archetypes/transform3d.cpp	Updated C++ implementation for new frame components
docs/snippets/snippets.toml	Added experimental transform3d_hierarchy_frames example exclusions
docs/snippets/all/archetypes/transform3d_hierarchy_frames.py	Added Python example demonstrating explicit frame relationships
docs/snippets/INDEX.md	Updated snippet index with new transform frame example
docs/content/reference/types/components/transform_frame_id.md	Updated documentation to reflect Transform3D usage
docs/content/reference/types/archetypes/transform3d.md	Added source_frame and target_frame to component list
crates/viewer/re_view_spatial/src/caches/transform_database_store.rs	Fixed doc link reference
crates/viewer/re_component_fallbacks/tests/snapshots/all_component_fallbacks__arch_fallback_rerun.archetypes.Transform3D.snap	Updated test snapshot with new fallback components
crates/viewer/re_component_fallbacks/src/component_fallbacks.rs	Added fallback providers for source_frame and target_frame
crates/store/re_types/src/transform_frame_id_hash.rs	Added from_str method for creating TransformFrameIdHash from strings
crates/store/re_types/src/reflection/mod.rs	Added reflection metadata for new frame components
crates/store/re_types/src/archetypes/transform3d_ext.rs	Updated IDENTITY constant with new frame fields
crates/store/re_types/src/archetypes/transform3d.rs	Added Rust implementation for source_frame and target_frame
crates/store/re_types/definitions/rerun/archetypes/transform3d.fbs	Added FlatBuffer definitions for new frame components
crates/store/re_tf/src/transform_resolution_cache.rs	Major rewrite to support arbitrary frame tracking and source-target relationships
crates/store/re_tf/src/lib.rs	Added entity_to_source_frame_tracking module
crates/store/re_tf/src/entity_to_source_frame_tracking.rs	New module for tracking frame relationships over time
crates/store/re_tf/benches/transform_resolution_cache_bench.rs	Fixed typo in benchmark function name

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

rerun_py/rerun_sdk/rerun/archetypes/transform3d_ext.py

rerun_py/rerun_sdk/rerun/archetypes/transform3d.py

rerun_cpp/src/rerun/archetypes/transform3d.hpp

crates/store/re_types/src/reflection/mod.rs

crates/store/re_types/src/archetypes/transform3d.rs

crates/store/re_types/definitions/rerun/archetypes/transform3d.fbs

… relevant

…ource relation

Wumpf · 2025-11-03T08:24:46Z

crates/store/re_types/definitions/rerun/archetypes/transform3d.fbs

+  /// This means that if a [archetypes.Transform3D] is set on an entity called `/my/entity/path` then this will default to `tf#/my/entity/path`.
+  ///
+  /// To set the frame an entity is part of see [archetypes.CoordinateFrame].
+  // TODO(RR-2777): Mark as stable when we're ready for it.


does the attr.rerun.state: unstable actuall work on fields? Use that?

oxkitsune

Few comments w.r.t. docs/comments to make it easier to follow, other than that LGTM!

oxkitsune · 2025-11-03T11:43:14Z

crates/store/re_tf/src/entity_to_source_frame_tracking.rs

+use std::ops::Range;
+use vec1::smallvec_v1::SmallVec1;
+
+/// Datastructures for tracking which transform relationships are specified by any moment in time for a given entity.


Suggested change

/// Datastructures for tracking which transform relationships are specified by any moment in time for a given entity.

/// Datastructures for tracking which transform relationships are specified at any moment in time for a given entity.

oxkitsune · 2025-11-03T11:47:04Z

crates/store/re_tf/src/entity_to_source_frame_tracking.rs

+    /// Insert a new range-start for a set of sources.
+    ///
+    /// Every time the start of a range is inserted, an existing range gets split in two!
+    /// The return value is the range previously not using `new_sources` but now is.


Suggested change

/// The return value is the range previously not using `new_sources` but now is.

/// The return value is the range that was not previously using `new_sources`, but is now.

IIUC, the retrun value is the range that wasn't previously using the new_sources, but now is.
Wording it like this is a bit easier to understand.

crates/store/re_tf/src/transform_resolution_cache.rs

oxkitsune · 2025-11-03T12:06:26Z

crates/store/re_tf/src/transform_resolution_cache.rs

+/// If any of the components yields an invalid transform, returns `None`.
 // TODO(#3849): There's no way to discover invalid transforms right now (they can be intentional but often aren't).
 fn query_and_resolve_tree_transform_at_entity(


I feel like this is a bit scary, similar to #8952 people could be depending on Affine3A::ZERO for invalid transforms.

I think this is the right thing to do, but might be worth to make a note of this somewhere in the changelog.

agreed, the whole "what is an invalid transform" thing was a bit vague before still is :/

Wumpf added 📺 re_viewer affects re_viewer itself 🍏 primitives Relating to Rerun primitives include in changelog labels Oct 30, 2025

Wumpf marked this pull request as ready for review October 30, 2025 18:11

Wumpf requested a review from oxkitsune October 30, 2025 18:11

Wumpf requested a review from Copilot October 30, 2025 22:25

Copilot AI reviewed Oct 30, 2025

View reviewed changes

Wumpf and others added 19 commits October 31, 2025 11:14

add source_frame & target_frame to Transform3D

c867496

fixup: multiple frames & sources

e5985f9

add from_str for TranformFrameIdHash

5f0f89a

fixup source_frame

77b8260

add utility for querying time ranges in which which set of sources is…

2094c36

… relevant

simpler, more conservative source discovery

410da28

Add query_source_frames_in_static_chunk query

3956ba8

fix-up various callsites

afb52ea

Add some more docs to transform3d.fbs

1222e3c

Read out target frame of Transform3D

8f2099b

fixup benchmarks, some formatting

e2e698d

make source/target frame singular for the moment

9bdef6b

test wip

0eb877d

wip fixes & tests

cd063ce

overhaul transform resolution cache to deal with tracking of entity s…

10c2182

…ource relation

improve EntityToAffectedSources::iter_ranges

60a35fb

Implement source range updating

ba23cd7

Note on target_frame/source_frame affecting other transform archetypes!

8130670

Fix issues with static transforms

a2544e7

Wumpf and others added 9 commits October 31, 2025 11:14

Add experimental sample for using source/target.

3ba2a8f

undo toml formatting messup

4e0f54c

fix missing source/target frames in python

2dab939

add large file exception

34b00eb

fix rerun lints

6b893de

More codegen & formatting

0d51427

Fix fbs doc issues

361fca2

Add fallback provider for target/source

989e65a

doc string fixups

d67679d

Wumpf force-pushed the andreas/tf/arbitrary-source-target branch from ab97148 to d67679d Compare October 31, 2025 10:15

Wumpf commented Nov 3, 2025

View reviewed changes

oxkitsune approved these changes Nov 3, 2025

View reviewed changes

Wumpf added 6 commits November 3, 2025 16:47

comment improvements

a52dafe

use rerun.state unstable on source/target frame

36535ce

fix some clippy issues

c2807b5

comment fix

72d4a39

fix snippet index issues

db7f5e3

fix py test

8878168

Wumpf merged commit d9674fc into main Nov 3, 2025
51 checks passed

Wumpf deleted the andreas/tf/arbitrary-source-target branch November 3, 2025 17:55

This was referenced Nov 3, 2025

Improve recently regressed common case transform ingestion #11762

Merged

Update transform related rrd files #11764

Merged

Rename source_frame/target_frame to child_frame/parent_frame #11770

Merged

Wumpf changed the title ~~Add experimental source_frame & target_frame to Transform3D~~ Add experimental child_frame & parent_frame to Transform3D Nov 4, 2025

	/// Datastructures for tracking which transform relationships are specified by any moment in time for a given entity.
	/// Datastructures for tracking which transform relationships are specified at any moment in time for a given entity.

	/// The return value is the range previously not using `new_sources` but now is.
	/// The return value is the range that was not previously using `new_sources`, but is now.

Conversation

Wumpf commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Related

What

Uh oh!

github-actions bot commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Wumpf commented Oct 30, 2025

Uh oh!

github-actions bot commented Oct 30, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Wumpf Nov 3, 2025

Choose a reason for hiding this comment

Uh oh!

oxkitsune left a comment

Choose a reason for hiding this comment

Uh oh!

oxkitsune Nov 3, 2025

Choose a reason for hiding this comment

Uh oh!

oxkitsune Nov 3, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

oxkitsune Nov 3, 2025

Choose a reason for hiding this comment

Uh oh!

Wumpf Nov 3, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Wumpf commented Oct 30, 2025 •

edited

Loading

github-actions bot commented Oct 30, 2025 •

edited

Loading

github-actions bot commented Oct 30, 2025 •

edited

Loading