What would you like to be added:
I propose updating the ResponseComplete plugin interface signature to include an error parameter (or a structured status object).
Currently, the signature is:
ResponseComplete(ctx context.Context, request *types.LLMRequest, response *Response, targetPod *backend.Pod)
It should be updated to something resembling:
ResponseComplete(..., err error)
// OR
ResponseComplete(..., status TerminationStatus)
// OR
ResponseComplete(..., reason string)
Why is this needed:
With the recent merge of PR #2064, the ResponseComplete hook is now guaranteed to execute symmetrically with PreRequest, covering all termination states:
- Successful completion.
- Internal errors (e.g., JSON marshaling failures).
- Client disconnects / Context cancellation.
The Problem:
Currently, a plugin implementing this hook has no way to distinguish between a successful request and a cancelled/failed one. This limitation prevents plugins from accurately recording observability data (e.g., incrementing an error_count metric vs. a success_count, or logging the cause of a disconnect).
Notes:
- This is a breaking change for existing plugins.
What would you like to be added:
I propose updating the
ResponseCompleteplugin interface signature to include anerrorparameter (or a structured status object).Currently, the signature is:
It should be updated to something resembling:
Why is this needed:
With the recent merge of PR #2064, the
ResponseCompletehook is now guaranteed to execute symmetrically withPreRequest, covering all termination states:The Problem:
Currently, a plugin implementing this hook has no way to distinguish between a successful request and a cancelled/failed one. This limitation prevents plugins from accurately recording observability data (e.g., incrementing an
error_countmetric vs. asuccess_count, or logging the cause of a disconnect).Notes: