Add support for recording dynamic fields

[LukeMathWalker]

Feature Request

Crates

tracing

Motivation

Currently all macros (e.g. tracing::info!) require upfront declaration of the fields you want to capture. They are populated either when the span is created or using .record afterwards.
This makes composition quite challenging - e.g. in tracing-actix-web we'd like to allow the library user to add custom fields to the request root span on top of the default ones (see LukeMathWalker/tracing-actix-web#13 for example) but it's unclear if this is even possible without exposing a macro to users.

The other scenario where I longed for this feature is using the Baggage API from OpenTelemetry: it would be ideal to get the baggage out of an incoming request and then add the retrieved field to the request root span. (cc @jtescher @tl-alex-chilcott)

Proposal

Add a way to specify dynamic fields, documenting explicitly the performance tradeoff compared to the current optimized solution.

Previous art

I tried to scan the issue tracker for previous discussions on the topic but I haven't found much - I suspect I wasn't looking for the right terms. Happy to read old issues to understand if we can move blockers you previously identified.

[robo-corg]

Running into a similar problem of trying to get various fields to always show up at the top of a json log entry (I think pretty similar to OT's baggage concept?). There does not seem to be a good way to do this with either the subscriber or layer apis to do this. Probably going to fork the json formatter.

[jtescher]

This would indeed be nice to have! I'm not sure of a way of adding this in a performant way currently. Perhaps others have put some though into this one already?

[LukeMathWalker]

This would indeed be nice to have! I'm not sure of a way of adding this in a performant way currently. Perhaps others have put some though into this one already?

The naive approach in my mind is more or less along these lines:

• Preserve the current performance profile when working/manipulating pre-declared fields;
• Add a slower path to handle dynamic fields, relying on some kind of heap-allocated storage.

E.g. in Span::record instead of silently ignoring the new field, add it to some kind of HashMap (created only if needed).

I'd be happy to build a prototype of what this might look like in a PR - if we agree it's something desirable/it goes in the direction the maintainers envision for tracing. I am conscious it's likely to cause changes in multiple places for the current API.

[purkhusid]

I have the exact same usecase as @LukeMathWalker. I want to add fields to the trace created by my gRPC server tracing and they are not known beforehand. Since this feature request is pretty old, are there any new workarounds for this?

[BrynCooke]

I've also hit this issue, and came up with something that seems to work for spans and events: https://crates.io/crates/tracing_dynamic
It's really just a proof of concept until someone tries it in a real application though.