LangSmith Threads tracing wrong attribute as a chat message for custom input types

[valentalent]

I want to trace messages for specific "thread_id" using LangSmith Threads, consequently I use same attribute naming for getting chat_history from db, but that's not relevant for current discussion.

I successfully modified config dynamically in the runtime with desired thread_id, but on LangSmith Threads wrong attribute is traced as a chat message, as seen from AgentInput class below there are "input" and "thread_id" attributes, "input" should be the one traced on LangSmith Threads but "thread_id" is constantly being traced.

I'm not sure how the attribute is depicted internally, I assume it has something to do with naming and type of attribute, I've also included my workaround which helped solve the problem, but It's kinda hacky and I don't want to do it that way.
Is there an intended way to explicitly set which attribute should be traced when using custom data types?

This is the code where I had problems:

class AgentInput(BaseModel):
    input: str
    chat_history: List[Tuple[str, str]] = Field(
        default=[],
        extra={"widget": {"type": "chat", "input": "input", "output": "output"}},
    )
    thread_id: str
    
 async def fetch_thread_id(config: Dict[str, Any], req: Request) -> Dict[str, Any]:
    # await json
    json = await req.json()
    # modify config
    config["metadata"] = {"thread_id": json["input"].get("thread_id", None)}
    return config

agent_executor = AgentExecutor(
    agent=agent, tools=tools, handle_parsing_errors=True, verbose=True
).with_types(input_type=AgentInput)

add_routes(
    app,
    agent_executor ,
    path="/chat_agent_executor",
    per_req_config_modifier=fetch_thread_id,  # dinamically modify config for langsmith threads purposes
)

This is the hacky workaround:

class AgentInput(BaseModel):
    input: str
    chat_history: List[Tuple[str, str]] = Field(
        default=[],
        extra={"widget": {"type": "chat", "input": "input", "output": "output"}},
    )
    thread_id: List[str] # workaround

    @validator("thread_id")
    def check_thread_id(cls, v):
        if len(v) > 1:
            raise ValueError("Only one thread_id is allowed")
        return v

 async def fetch_thread_id(config: Dict[str, Any], req: Request) -> Dict[str, Any]:
    # await json
    json = await req.json()
    # modify config
    config["metadata"] = {"thread_id": json["input"].get("thread_id", None)[0]} # workaround with list
    return config

agent_executor = AgentExecutor(
    agent=agent, tools=tools, handle_parsing_errors=True, verbose=True
).with_types(input_type=AgentInput)

add_routes(
    app,
    agent_executor ,
    path="/chat_agent_executor",
    per_req_config_modifier=fetch_thread_id,  # dinamically modify config for langsmith threads purposes
)

[hinthornw]

Would it be possible to share an example failing trace to help us debug this?

[valentalent]

Here it is:
https://smith.langchain.com/public/a54f4711-75a1-43bb-879e-bc5dc98de04a/r

[hinthornw]

Is this example from the workaround you made or before? The metadata looks fine to me, so if it's still not being included in the threads view then that's something we ought to fix on our end.

[valentalent]

Yeah, metadata is fine, my point was that it would be useful to have a way to specify what attribute will be displayed in the threads view as a chat message.
As for the problem, my workaround will do.

[AliE02]

I second this !