Skip to content

fix(tracing): separate request and response LLM models#4018

Open
oWretch wants to merge 2 commits intopipecat-ai:mainfrom
oWretch:fix/llm-response-model
Open

fix(tracing): separate request and response LLM models#4018
oWretch wants to merge 2 commits intopipecat-ai:mainfrom
oWretch:fix/llm-response-model

Conversation

@oWretch
Copy link

@oWretch oWretch commented Mar 13, 2026

Summary

  • keep gen_ai.request.model stable to configured/requested model values
  • add gen_ai.response.model support in shared LLM span attributes
  • update OpenAI-compatible streaming paths to set response model on the active span when response data arrives
  • add focused tests for request/response model attribution behavior
  • add changelog fragment in changelog/4018.fixed.md

Problem This Fix Addresses

Today, multi-turn conversations can show inconsistent model attribution in traces:

  • first LLM call: gen_ai.request.model is typically the configured deployment/model name (for example gpt-4o)
  • subsequent LLM calls: gen_ai.request.model can drift to the resolved response model version from a prior call (for example gpt-4o-2024-11-20)
  • gen_ai.response.model may be missing entirely

This makes trace analysis confusing because the request model appears to change across turns even though the configured request model is the same.

Behavior After This Change

  • gen_ai.request.model remains consistent across calls and reflects the requested/configured model
  • gen_ai.response.model is populated when provider response data arrives, so resolved model versions are captured in the correct attribute

In other words, request-model attribution is now stable across the first and subsequent calls, and response-model attribution is now explicit and accurate.

Validation

  • uv run ruff check src/pipecat/utils/tracing/service_decorators.py src/pipecat/utils/tracing/service_attributes.py src/pipecat/services/openai/base_llm.py src/pipecat/services/sambanova/llm.py tests/test_llm_tracing_models.py
  • uv run pytest tests/test_llm_tracing_models.py

@oWretch oWretch marked this pull request as ready for review March 13, 2026 03:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant