Response Type of "unknown" almost never emits a /response event for GPT-5 and other models

[kevin-m-kent]

Does this issue occur when all extensions are disabled?: Yes/No

• VS Code Version: Does not appear to be version specific
• OS Version: n/a

When examining our data for GPT-5, the chat/agent mode events that have a Properties.type or Properties.responsetype of "unknown" only emit /panel.request events and never /response events. This event appears to correspond to the 'Sorry, no response was returned' message. For other models it ranges from 10-40% of turns not having /response events for this error case.

I'm assuming this a bug, but please let me know if this is by design. We use the /response events for counting turn failures, so this is a blind spot at the moment (I plan to update our logic to use /panel.request events for 'unknown' cases to account for this in the meantime).

FYI @isidorn / @roblourens

[vs-code-engineering]

Please look at the following meta issue: #253126. Please refer to that issue for updates and discussions. Feel free to open a new issue if you think this is a different problem.

[isidorn]

@roblourens assigning to you first. Also cc @lramos15 for thoughts

@Yoyokrazy this is an example of a bad bot classification. Any idea how to improve this? And I noticed this happened with some other issues Kevin created.

[Yoyokrazy]

@isidorn I'll do a pass over our classification prompting, and restrict more towards empty issues that just say "no response returned" etc. In general we are seeing less of that type of issue spam, so tightening the classification should be good regardless. Thanks for the ping 👍

[roblourens]

I see the same thing in telemetry, but I don't see any way that we could get out from https://github.com/microsoft/vscode-copilot-chat/blob/cf071d3f79704836421564dc5cbbb96c7888f04b/src/extension/prompt/node/chatMLFetcher.ts#L937-L942 without sending a response.error. I've seen this often enough that I shoudl be able to debug it next time I see it. The fetcher code is a maze.

[kevin-m-kent]

I see the same thing in telemetry, but I don't see any way that we could get out from https://github.com/microsoft/vscode-copilot-chat/blob/cf071d3f79704836421564dc5cbbb96c7888f04b/src/extension/prompt/node/chatMLFetcher.ts#L937-L942 without sending a response.error. I've seen this often enough that I shoudl be able to debug it next time I see it. The fetcher code is a maze.

Thanks @roblourens - I ended up coalescing across the two event types by turn to solve the tracking/visibility issue. But if there ends up being a way to emit /response and /panel.request events for type='unknown' that would be fantastic.