Break PostResponse requestcontrol plugin into 3 separate plugins to add streamed request functionality

[BenjaminBraunDev]

/kind bug
/kind feature

What this PR does / why we need it:
PostResponse runs at the first chunk received instead of at the end of the request, however some PostResponse plugins would like to run only when a request is fully complete. This PR allows that flexibility by breaking out PostResponse into 3 separate requestcontrol plugin hooks for when the response is received, each chunk streamed, and fully complete respectively. The requestcontrol plugins are now the following:

PreRequest
PostResponseReceived
PostResponseStreamed
PostResponseComplete

Which issue(s) this PR fixes:
Fixes #1483

Does this PR introduce a user-facing change?:

PostResponse plugin has been renamed to PostResponseReceived

[netlify]

✅ Deploy Preview for gateway-api-inference-extension ready!

Name	Link
🔨 Latest commit	34a17a9
🔍 Latest deploy log	https://app.netlify.com/projects/gateway-api-inference-extension/deploys/68f0110d739f6c00092c0f37
😎 Deploy Preview	https://deploy-preview-1661--gateway-api-inference-extension.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[k8s-ci-robot]

Hi @BenjaminBraunDev. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[ahg-g]

/ok-to-test

[kfswain]

Some smaller comments, overall looks good. Thanks!

/approve

[nirrozenbaum]

please remove logger from the args, we don't use logr as argument to functions, we use contextual logging.
for example

log.FromContext(ctx).V(logutil.TRACE).Info(....)

see for example here:

gateway-api-inference-extension/pkg/epp/requestcontrol/director.go

Line 171 in 75b2144

loggerTrace := log.FromContext(ctx).V(logutil.TRACE)

[kfswain]

I asked him to remove that, in the streaming case the high volume of instantiations of the logger will cause allocation/gc pressure. Luke mentioned this as an optimization during flow control benchmarking, and i think the same applies here b/c there will be multiple streaming calls per request.

[BenjaminBraunDev]

Does log.FromContext(ctx).V(logutil.TRACE).Info(....) actually make another logger? I would think it takes the logger "from the context" given ctx has a logger. Does it actually allocate more memory? I would think that pretty inefficient.

From the FromContext() in go-logr package:

// FromContext returns a Logger from ctx or an error if no Logger is found.

[BenjaminBraunDev]

@nirrozenbaum @kfswain What is the desired state for this? I can revert it back or keep the changes

[nirrozenbaum]

I don't think it makes another logger.
that wouldn't make sense.

the desired state (at least in my opinion) is to remove logger from the args and use contextual logging if/when necessary.
we don't pass around logger in any part of the code. we pass ctx (which includes inside it the logger).

[BenjaminBraunDev]

I'm slightly confused here, the FromContext in log.go always calls log.WithValues(keysAndValues...), it's the only way the function can return. Should I keep this logging regardless?

[nirrozenbaum]

I was checking now again. I see your point.
I’m still not sure about that, but anyway this looks to me like a system wide issue cause we’re using contextual logging in all places.

I suggest to move forward with current PR using Log.FromContext, open a dedicated issue to check about this issue, and if we decide to change, we need to update everywhere, not only in this function.

let me know if it makes sense.

[kfswain]

This PR has been open for a while, so I'm fine with whatever we do in the short term to keep this moving. This PR also needs a rebase, so when that is rebased we can update this to whatever. (Personally I think we leave it out here since this extension point is potentially called significantly more than any other). But don't feel strongly enough to hold up this PR further.

@BenjaminBraunDev pls ping me when this PR is rebased.

[ahg-g]

+1; please revert and lets move forward with the PR and resolve the logging issue separately

[BenjaminBraunDev]

@kfswain Reverted and rebased! Should be ready for merge after tests finish.

[nirrozenbaum]

@BenjaminBraunDev made another iteration on the PR.
overall it looks good. the main point I was highlighting in the review is mainly around keeping consistency and use contextual logging.
otherwise the PR lgtm.

[kfswain]

/approve

Thanks Ben! Will let Nir give final stamp

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: BenjaminBraunDev, kfswain

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [kfswain]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[kfswain]

/lgtm

Thanks!

[nirrozenbaum]

Thanks @BenjaminBraunDev!
for follow up work, can you please open an issue describing the logger problem that was raised?