fix: ensure ResponseComplete hook always executes

[LukeAVanDrie]

What type of PR is this?
/kind bug

What this PR does / why we need it:
This PR ensures that the ResponseComplete plugin hook is always executed if a request was scheduled to a pod, regardless of how the request terminates (success, error, or client disconnect).

The Problem:
Stateful plugins (like the upcoming Concurrency Saturation Detector) rely on strict symmetry between PreRequest (increment) and ResponseComplete (decrement). Previously, several edge cases could break this symmetry, causing capacity leaks where the system believed it was saturated when it was actually empty:

1. Errors & Disconnects: If json.Marshal failed or the client context was canceled after scheduling but before response generation, the function would return early, skipping the completion hook.
2. Split Streaming Chunks: The streaming logic relied on string parsing (strings.Contains(..., "[DONE]")) to detect the end of a stream. If the TCP/Envoy chunk boundary split this message across two reads, the completion signal was missed.

The Fix:

1. Safety Defer: Added a defer block in server.Process that checks if a TargetPod was assigned but ResponseComplete was never called. If so, it forces the completion hook to run.
2. Authoritative Streaming Signal: Moved the streaming completion trigger out of the content-parsing utility and into the main loop, relying on the gRPC EndOfStream boolean. This makes the logic robust against split chunks.

Which issue(s) this PR fixes:

Prerequisite for #1793

Does this PR introduce a user-facing change?:

Fixed a bug where client disconnects, internal errors, or split streaming chunks could cause request capacity to leak, potentially leading to false saturation signals in the Flow Control layer.

[netlify]

✅ Deploy Preview for gateway-api-inference-extension ready!

Name	Link
🔨 Latest commit	3800fe3
🔍 Latest deploy log	https://app.netlify.com/projects/gateway-api-inference-extension/deploys/695c4a9331b48600082891f3
😎 Deploy Preview	https://deploy-preview-2064--gateway-api-inference-extension.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[k8s-ci-robot]

Hi @LukeAVanDrie. Thanks for your PR.

I'm waiting for a github.com member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[LukeAVanDrie]

/assign @kfswain

[kfswain]

Good call, lets make an issue for this.

[kfswain]

/lgtm
/approve

Looks good.

I think we can make this change safely. I checked to make sure this plugin was run on streaming and nonstreaming, which it is. Other plugins may need to handle the failure case. But I think thats okay. Thanks!

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kfswain, LukeAVanDrie

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [kfswain]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[kfswain]

/ok-to-test

[nirrozenbaum]

getting back to my comments on the original PR that added the response hooks - naming selection is really not clear IMO.
I think we should make the names self explained to avoid confusion, or more concretely, I think all response hooks should be renamed.

[LukeAVanDrie]

getting back to my comments on the original PR that added the response hooks - naming selection is really not clear IMO. I think we should make the names self explained to avoid confusion, or more concretely, I think all response hooks should be renamed.

I can create an issue for this tomorrow morning and link it to this PR as a discussion starter. The semantics of ResponseComplete are a bit unclear. I do think making this symmetric with PreRequest is the right choice for a lot of different extension point implementations though - specifically to handle closing resources or cleaning up memory references. For the concurrency-limit saturation detector the use case is even more evident.

[nirrozenbaum]

getting back to my comments on the original PR that added the response hooks - naming selection is really not clear IMO. I think we should make the names self explained to avoid confusion, or more concretely, I think all response hooks should be renamed.

I can create an issue for this tomorrow morning and link it to this PR as a discussion starter. The semantics of ResponseComplete are a bit unclear. I do think making this symmetric with PreRequest is the right choice for a lot of different extension point implementations though - specifically to handle closing resources or cleaning up memory references. For the concurrency-limit saturation detector the use case is even more evident.

@LukeAVanDrie agreed, definitely should be symmetric. I have no comments on the behavior, only on the names selection :).
would be great if you can create an issue and we can come up with names that best describe what is done by the extension points.