feat: Allow request control plugins to return ext_proc dynamic metadata by fcfort · Pull Request #2156 · kubernetes-sigs/gateway-api-inference-extension

fcfort · 2026-01-15T17:02:58Z

What type of PR is this?

/kind feature

What this PR does / why we need it:

Here we create a mechanism to pass in the usage struct to the ResponseBodyComplete callback and also a mechanism for returning dynamic_metadata out of the callback and into the ext_proc's ProcessingResponse object. This will be used by the new plugin introduced in #2114.

See #2019 for more details.

Which issue(s) this PR fixes:

This contributes to #2019, but does not fix it.

Does this PR introduce a user-facing change?:

No user-facing change.

netlify · 2026-01-15T17:03:08Z

✅ Deploy Preview for gateway-api-inference-extension ready!

Name	Link
🔨 Latest commit	`484ffc0`
🔍 Latest deploy log	https://app.netlify.com/projects/gateway-api-inference-extension/deploys/6970ef88e40f10000800f6c0
😎 Deploy Preview	https://deploy-preview-2156--gateway-api-inference-extension.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

k8s-ci-robot · 2026-01-15T17:03:11Z

Hi @fcfort. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

linux-foundation-easycla · 2026-01-15T17:03:18Z

The committers listed above are authorized under a signed CLA.

✅ login: fcfort / name: Frank (484ffc0)

ahg-g

/ok-to-test

ahg-g · 2026-01-15T21:20:48Z

 	github.com/cespare/xxhash/v2 v2.3.0
 	github.com/elastic/crd-ref-docs v0.2.0
-	github.com/envoyproxy/go-control-plane/envoy v1.36.0
+	github.com/envoyproxy/go-control-plane/envoy v1.35.0


it seems lots of pkgs are being downgraded, why?

I don't actually know how these files get changed. Should I revert the changes to go.mod / go.sum?

It should be automatic, can you revert the changes and then run go build again?

ahg-g · 2026-01-16T18:27:09Z

 	// Currently, this is only used by conformance test.
 	ReqMetadata map[string]any
+	// Token usage counts parsed from the response body.
+	Usage handlerstypes.Usage


This creates a dependency on the handlers pkg for the plugins, it doesn't seem ideal.

@kfswain @nirrozenbaum I feel the interfaces and types for the framework should be grouped in one place under epp/framework (renaming epp/plugins) with a separate pkg for each subsystem under it. With this, a plugin should only have a dependency on this epp/framework pkg. wdyt?

generally I agree.
I have tried that a while ago and got to a circular dependency, cause scheduler needs to reference plugins, and profile handler plugin references scheduler profiles (its return value).
we can reorganize the scheduler package to get it fixed

I love that idea. If we can make it work in practice that would be great.

yes, we can.
that would still include some awkward references but it would work.

e.g.,
epp/framework would reference epp/scheduler/profile
and
epp/scheduler would reference epp/framework.

that’s not ideal but doable.
LMKWYT.

One way to avoid the circular dependency is to move pkg/epp/requestcontrol/types.go to pkg/epp/requestcontrol/types/types.go so that it is in a separate package. This does end up touching another dozen or so files. Is that what you'd like?

It will also impact out of tree plugins if we move the existing types. I plan to re-organize all of that as discussed above, and so I don't want to force people to change twice.

Can we define two types and copy? once we do the refactor, we will converge on one.

Created #2190 to track the work, I will take a first stab at doing that for the scheduler and requestcontrol pkgs.

@fcfort we can merge your PR as is and we will move the types as part of the above suggested refactor.

Sounds good!

First PR focused on the scheduling layer: #2192

ahg-g · 2026-01-21T21:35:08Z

/lgtm
/approve

k8s-ci-robot · 2026-01-21T21:35:18Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ahg-g, fcfort

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [ahg-g]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

…bernetes-sigs/gateway-api-inference-extension#2156)

k8s-ci-robot added the kind/feature Categorizes issue or PR as related to a new feature. label Jan 15, 2026

k8s-ci-robot requested review from danehans and nirrozenbaum January 15, 2026 17:03

k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Jan 15, 2026

k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jan 15, 2026

fcfort mentioned this pull request Jan 15, 2026

feat: New cost reporting plugin for returning cost in ext_proc dynamic metadata response to proxy #2114

Merged

fcfort force-pushed the push-ztoqlzylpvrw branch from 5f93208 to efea3a8 Compare January 15, 2026 21:16

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 15, 2026

ahg-g reviewed Jan 15, 2026

View reviewed changes

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jan 15, 2026

fcfort force-pushed the push-ztoqlzylpvrw branch from efea3a8 to af17094 Compare January 16, 2026 15:45

ahg-g reviewed Jan 16, 2026

View reviewed changes

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 16, 2026

ahg-g mentioned this pull request Jan 21, 2026

Refactor all pluggable framework interfaces and types into a separate pkg #2190

Closed

Allow request control plugins to return ext_proc dynamic metadata

484ffc0

fcfort force-pushed the push-ztoqlzylpvrw branch from af17094 to 484ffc0 Compare January 21, 2026 15:23

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 21, 2026

k8s-ci-robot assigned ahg-g Jan 21, 2026

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 21, 2026

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 21, 2026

k8s-ci-robot merged commit 1fecaf9 into kubernetes-sigs:main Jan 21, 2026
9 checks passed

elevran pushed a commit to llm-d/llm-d-inference-scheduler that referenced this pull request Apr 23, 2026

Allow request control plugins to return ext_proc dynamic metadata (ku…

a390556

…bernetes-sigs/gateway-api-inference-extension#2156)

Conversation

fcfort commented Jan 15, 2026

Uh oh!

netlify Bot commented Jan 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for gateway-api-inference-extension ready!

Uh oh!

k8s-ci-robot commented Jan 15, 2026

Uh oh!

linux-foundation-easycla Bot commented Jan 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ahg-g left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ahg-g Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ahg-g commented Jan 21, 2026

Uh oh!

k8s-ci-robot commented Jan 21, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

netlify Bot commented Jan 15, 2026 •

edited

Loading

linux-foundation-easycla Bot commented Jan 15, 2026 •

edited

Loading

ahg-g Jan 21, 2026 •

edited

Loading