Add support for arbitrary/third party glTF Extension processing via GltfExtensionHandler

[ChristopherBiscardi]

Objective

Currently Bevy doesn't support arbitrary glTF extensions. The ones it does support are hardcoded.

We should support glTF extensions, as this is a primary mechanism for sharing behavior via data exported from applications like Blender.

I personally have found usecases in exporting component data, lightmap textures/information, and processing other kinds of data (AnimationGraph, 3d meshes into 2d, etc).

Solution

This PR introduces a new GltfExtensionHandler trait that users can implement and add to the glTF loader processing via inserting into a Resource.

There are two example processors currently added, with a third that I'd like to add after this PR.

• examples/gltf/gltf_extension_animation_graph.rs duplicates the functionality of animation_mesh, constructing AnimationGraphs via extension processing and applying them to be played on the relevant nodes.
• examples/gltf/gltf_extension_mesh_2d.rs duplicates the functionality of the custom_gltf_vertex_attribute example, showing how the extension processing could be used to convert 3d meshes to 2d meshes alongside custom materials.

Both of these examples re-use existing assets and thus don't actually use extension data, but show how one could access the relevant data to say, only convert specifically labelled Mesh3ds to 2d, or process many animations into multiple graphs based on extension-data based labelling introduced in Blender.

A third example I want to introduce after this PR is the same core functionality Skein requires: an example that uses reflected component data stored in glTF extensions and inserts that data onto the relevant entities, resulting in scenes that are "ready to go".

Comparison to Extras

In comparison to extensions: data placed in glTF extras is well supported through the GltfExtras category of components.

Extras only support adding an additional extras field to any object.

Data stored in extras is application-specific. It should be usable by Bevy developers to implement their own, application-specific, data transfer. This is supported by applications like Blender through the application of Custom Properties.

Once data is used by more than one application, it belongs in a glTF extension.

What is a glTF Extension?

Extensions are named with a prefix like KHR or EXT. Bevy has already reserved the BEVY namespace for this, which is listed in the official prefix list.

For a glTF file, an extension must be listed in extensionsUsed and optionally extensionsRequired.

{
    "extensionsRequired": [
        "KHR_texture_transform"
    ],
    "extensionsUsed": [
        "KHR_texture_transform"
    ]
}

Extension data is allowed in any place extras are also allowed, but also allow much more flexibility.

Extensions are also allowed to define global data, add additional binary chunks, and more.

For meshes, extensions can add additional attribute names, accessor types, and/or component types

KHR_lights_punctual is a contained and understandable example of an extension: https://github.com/KhronosGroup/glTF/blob/7bbd90978cad06389eee3a36882c5ef2f2039faf/extensions/2.0/Khronos/KHR_lights_punctual/README.md . This one happens to be already hardcoded into Bevy's handling, so it doesn't benefit from arbitrary extension processing, but there are additional ratified and in-progress extensions, as well as vendor and other arbitrary extensions that would benefit from userland support.

Implementation

This initial implementation is reasonably minimal: enabling extension processing for objects/etc as they're loaded which may also define extension data, including the scene world. This may leave out useful functionality; as detailed in the next section: "What's not implemented".

Extension handlers are defined by implementing a trait which can optionally define hooks and data.
Extension handler data is cloned to start with a fresh slate for each glTF load, which limits scope to "one glTF load".
So while state can be maintained across hooks during a single load, users who want to combine or handle multiple glTF assets should do so in the main app, not in an extension handler.
Following this, because the extensions are stored as dyn GltfExtension and we want to clone them to isolate state to a single load, dyn_clone must be included as a workaround to enable this cloning.

An extension handler has to be added to the list of handler by accessing a Resource and pushing an instantiated handler into it.
This Resource keeps the list of extension handlers so that a new glTF loader can bootstrap them.

The design of the hooks is such that:

• If no extensions handlers are registered, none are called for processing
• If an extension handler is defined, it receives all "events"
  • handlers are defined by a trait, and default implementations are called if an override is not specified.
    • default implementations are no-ops

It is important that extensions receive all events because certain information is not embedded in extension data.
For example, processing animation data into an animation graph could require both processing animations with extension data, tracking the animation roots through hooks like on_node, and applying those graphs in the on_scene_completed hook.

• Extension data is passed to hooks as Option<&serde_json::Value> which is only passing references around as the data has already been converted to Value by the gltf crate.
• LoadContext is required for creating any new additional assets, like AnimationGraphs.
  • scene World access is provided in hooks like on_scene_completed, which allows calculating data over the course of a glTF load and applying it to a Scene.

What's not implemented

This PR chooses to not implement some features that it could. Instead the approach in this PR is to offer up the data that Bevy has already processed to extensions to do more with that data.

• Overriding load_image/process_loaded_texture
  • This could allow projects like bevy_web_codecs, which currently forks the entire gltf loader. Associated issue. However I believe this needs some design work dedicated to what exactly happens here to support that use case.
• This PR doesn't include any refactoring of the glTF loader, which I feel is important for a first merge.
• There is some benefit to passing in the relevant gltf::* object to every hook. For example, I believe this is the only way to access extension data for KHR_lights_punctual, and KHR_materials_variants or other extensions with "built-in" support. I haven't done this in all places. (edit: after external implementation I decided this was a good idea and added it to more places)

Testing

cargo run --example gltf_extension_animation_graph
cargo run --example gltf_extension_mesh_2d

---

Showcase

Both examples running:

screenshot-2025-12-13-at-05.17.59-converted.mp4

screenshot-2025-12-13-at-05.18.54-converted.mp4

An example that showcases converting Mesh3d to Mesh2d

#[derive(Default, Clone)]
struct GltfExtensionProcessorToMesh2d;

impl GltfExtensionProcessor for GltfExtensionProcessorToMesh2d {
    fn extension_ids(&self) -> &'static [&'static str] {
        &[""]
    }

    fn dyn_clone(&self) -> Box<dyn GltfExtensionHandler> {
        Box::new((*self).clone())
    }

    fn on_spawn_mesh_and_material(
        &mut self,
        load_context: &mut LoadContext<'_>,
        _gltf_node: &gltf::Node,
        entity: &mut EntityWorldMut,
    ) {
        if let Some(mesh3d) = entity.get::<Mesh3d>()
            && let Some(_) = entity.get::<MeshMaterial3d<StandardMaterial>>()
        {
            let material_handle =
                load_context.add_loaded_labeled_asset("AColorMaterial", (CustomMaterial {}).into());
            let mesh_handle = mesh3d.0.clone();
            entity
                .remove::<(Mesh3d, MeshMaterial3d<StandardMaterial>)>()
                .insert((Mesh2d(mesh_handle), MeshMaterial2d(material_handle.clone())));
        }
    }
}

[viridia]

Approved, although I realized that this isn't actually a gltf extensions processor, it's a general gltf post-processor that may or may not have anything to do with extensions. For example, I could probably use this to add outline materials based on gltf extras.

[ChristopherBiscardi]

@viridia you could, yes. The generality of extensions basically makes these the same things. All of the processing needs to be in similar places whether you're actually processing the data or not.

Even a simpler case of a not-built-in KHR_lights_punctual would require having access to the global extension data while processing individual light nodes. The KHR_physics_rigid_bodies proposal requires access to new node types stored in the global data from two separate extensions (itself and a completely separate KHR_implicit_shapes). and that's without getting into things like KHR_texture_procedurals which defines entire node graphs and such. (also without addressing what you actually do with that data once you've accessed it).

[ChristopherBiscardi]

I have also used this PR to implement extension-based handling for Skein, which stores the TypeRegistry in the extension handler and reflected component data in gltf extensions. While loading a gltf, the component data is inserted into the appropriate locations using the underlying infrastructure for insert_reflect.

    {
      "extensions": {
        "BEVY_skein": {
          "components": [
            {
              "test_components::Player": {
                "name": "Is the Mesh Object",
                "power": 90.93000030517578,
                "test": -234
              }
            }
          ]
        }
      },
      "mesh": 0,
      "name": "Cube"
    },

https://github.com/rust-adventure/skein/pull/89/changes#diff-b1a35a68f14e696205874893c07fd24fdb88882b47c23cc0e0c80a30c7d53759R377

[janhohenheim]

Wow, excellent documentation! Looks all good to me :)

[andriyDev]

Overall LGTM. A couple minor details here but nothing big.

The only thing that is weird is that we're calling it GltfExtensionHandler which as @viridia mentioned is not quite accurate. It would be nice to have a name that indicates it can be used for any sort of manipulation. But that's bikeshedding a little too much.

[ChristopherBiscardi]

The only thing that is weird is that we're calling it GltfExtensionHandler

Yeah I suppose. We could name it GltfLoaderHelper or GltfPostProcess or something, but each has their own problems (its not a post-process, and helper isn't a useful descriptor). The feature is really intended to support the processing of glTF extension data as a step up from Bevy's current support for glTF extras.

I'm not sure the fact that you can ignore the extension data entirely if you want to is worth renaming the type. Naming it GltfExtensionHandler gives it a firm use case, and future additions to the hooks can be evaluated against that. Extension handling is generic enough that the set of "useful glTF object processing that should happen in a loader" and "extension use cases" is pretty close to a circle.

I'm open to other names but I think its also fine to rename it in a future release if a better name comes up.