Add tracing support in bbr

[nirrozenbaum]

recently tracing support was added to EPP. for allowing this we have InitTracing function that initializes the tracing bits which then enables tracing spans across different places in the code.
This is useful both in IGW and also in other schedulers that extends IGW like llm-d.

BBR is going in the same direction, currently there is a WIP to make it pluggable.
conceptually, we should have tracing support also in BBR.

the tracing initialization functions were extracted to a common pkg as a prep step.
we should reuse the same to init tracing, and if needed to generalize it further to make sure it's reusable.

cc: @sallyom

[AtharvaPakade]

/assign

[AtharvaPakade]

Hi @nirrozenbaum, @sallyom
I'd like to take a shot at this. I've been looking at the architecture and have a high-level direction in mind sharing it here to get thoughts before diving in.

High-level tracing proposal for the project:

• Plugin-scoped sub-spans — since we're moving toward pluggable BBR and EPP, each plugin execution would get its own child span. This way, when someone plugs in a new scheduling configuration, they automatically get a span scoped to that plugin, without any manual instrumentation at the plugin level. Mabybe we can also have a scheduler level spans.

• Per-plugin attribute injection — a generic mechanism for each plugin to declare span attributes based on its own config/context. Keeps the tracing layer decoupled from plugin internals. Additionally, this can help compare behavior across different plugin versions or configurations.

• Conditional root span + cross-boundary propagation — if BBR is present, it owns the root span and injects traceparent into the ext-proc response headers so EPP can extract and continue the same trace. If BBR is absent, EPP creates the root span. This gives a complete end-to-end trace.

I'm happy to pick up the EPP-specific parts (plugin sub-spans + attribute injection) in a separate issue or maybe draft a propose and request for comment.

Question for this issue: Given that BBR is moving toward a pluggable model, would you like to see the plugin-scoped sub-spans and attribute injection included as part of the BBR tracing work here as well, or keep this scoped to just the InitTracing reuse for now?

[nirrozenbaum]

tagging our tracing experts:
@JeffLuoo @sallyom

[JeffLuoo]

Plugin-scoped sub-spans — since we're moving toward pluggable BBR and EPP, each plugin execution would get its own child span. This way, when someone plugs in a new scheduling configuration, they automatically get a span scoped to that plugin, without any manual instrumentation at the plugin level. Mabybe we can also have a scheduler level spans.

Can you elaborate on scheduler level span? IIUC, the scheduler happens after the server receives the request. Is the scheduler you are referring to here a scheduler before the IGW receives the request? That's being said, BBR and EPP should have separate root span (from the server of each) with different span name.

Per-plugin attribute injection — a generic mechanism for each plugin to declare span attributes based on its own config/context. Keeps the tracing layer decoupled from plugin internals. Additionally, this can help compare behavior across different plugin versions or configurations.

+1 to it.

Conditional root span + cross-boundary propagation — if BBR is present, it owns the root span and injects traceparent into the ext-proc response headers so EPP can extract and continue the same trace. If BBR is absent, EPP creates the root span. This gives a complete end-to-end trace.

I am not sure if I understand. Can both EPP and BBR be present in the same stack? @nirrozenbaum I am not closely following up on the architecture change in the IGW. What document can help me understand the motivation of making BBR pluggable and how?

[nirrozenbaum]

@JeffLuoo bbr and epp are separate ext processes,
the gateway (e.g., gke/istio) is calling bbr first. bbr should execute its plugins and might optionally mutate the request headers/body. then the request is back to the gateway and from there to epp. epp can also mutate headers/body/metadata (e.g., updates field with the selected endpoint) and back to the gateway. then finally gw sends it to the selected endpoint according to field epp set.