Add AgentHandoff Chat Item

[toubatbrian]

Implement AgentHandoffItem into chat context

[changeset-bot]

⚠️ No Changeset found

Latest commit: d57c817

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[simllll]

What's the idea of adding it to the chat context and then filtering out again? Just curious about the reason behind this feature? 🤔

[toubatbrian]

Hey @simllll, those events will be recorded and sent to the observability for traces. It's been filtered before passing to LLM since LLM does not support a "handoff" message type.

[Shubhrakanti]

why is this optional?

[toubatbrian]

This also follows python agents:

class Agent:
    def __init__(
        self,
        *,
        instructions: str,
        id: str | None = None,
        chat_ctx: NotGivenOr[llm.ChatContext | None] = NOT_GIVEN,
        tools: list[llm.FunctionTool | llm.RawFunctionTool] | None = None,
        turn_detection: NotGivenOr[TurnDetectionMode | None] = NOT_GIVEN,
        stt: NotGivenOr[stt.STT | STTModels | str | None] = NOT_GIVEN,
        vad: NotGivenOr[vad.VAD | None] = NOT_GIVEN,
        llm: NotGivenOr[llm.LLM | llm.RealtimeModel | LLMModels | str | None] = NOT_GIVEN,
        tts: NotGivenOr[tts.TTS | TTSModels | str | None] = NOT_GIVEN,
        mcp_servers: NotGivenOr[list[mcp.MCPServer] | None] = NOT_GIVEN,
        allow_interruptions: NotGivenOr[bool] = NOT_GIVEN,
        min_consecutive_speech_delay: NotGivenOr[float] = NOT_GIVEN,
        use_tts_aligned_transcript: NotGivenOr[bool] = NOT_GIVEN,
        min_endpointing_delay: NotGivenOr[float] = NOT_GIVEN,
        max_endpointing_delay: NotGivenOr[float] = NOT_GIVEN,
    ) -> None:
        tools = tools or []
        if type(self) is Agent:
            self._id = "default_agent"
        else:
            self._id = id or misc.camel_to_snake_case(type(self).__name__)

[simllll]

@toubatbrian i see, thanks for the insights and the quick reply.

My thoughts: But if we are talking about opentelemetry support or similar, is it really necessary to put it in the chat context at all? Or would it be enough to "fake" this call or find some other way to make this traceable. (E.g. if we observe a special function for tracing, we could think about a flag or a second function just for logging purposes?)
The chat context right now is only for the llm, if this is not true anymore it opens the world for more "non related" llm output.
Not saying this doesn't make sense, but it's a different approach than there is right now.

[toubatbrian]

Hey @simllll, I see your thoughts, which totally make sense! The main goal is to trying to achieve as much as parity as possible with the python agent framework. Adding handoff object into chat context is what currently been implemented in python side.

The other reason tie-ing the handoff to chat context is that, we'll add support for export the context json once a session is finished, so developer can do certain things on it such as eval. Having the agent handoff info as part of the chat context would be useful for that case.