BBR pluggable framework proposal by davidbreitgand · Pull Request #1964 · kubernetes-sigs/gateway-api-inference-extension

davidbreitgand · 2025-12-07T12:44:10Z

What type of PR is this?

/kind feature

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #1963

This is a design proposal PR, it does not introduce code changes.

Does this PR introduce a user-facing change?:

NONE

netlify · 2025-12-07T12:44:16Z

✅ Deploy Preview for gateway-api-inference-extension ready!

Name	Link
🔨 Latest commit	`3e004a7`
🔍 Latest deploy log	https://app.netlify.com/projects/gateway-api-inference-extension/deploys/695d7f484dccd70008af0486
😎 Deploy Preview	https://deploy-preview-1964--gateway-api-inference-extension.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

k8s-ci-robot · 2025-12-07T12:44:19Z

Hi @davidbreitgand. Thanks for your PR.

I'm waiting for a github.com member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

davidbreitgand · 2025-12-12T14:39:31Z

@srampal , @elevran , @nirrozenbaum could you please review?

Initial implementation: PR 1981

will be refactored into a series of smaller PRs to fit the phased scope of the proposal.

elevran · 2025-12-15T10:36:18Z

+const (
+    //A deafult plugin implementation of this plugin type will always be configured for request plugins chain
+    //Even though BBRPlugin type is not (yet) a K8s resource, it's logically akin to `kind`
+    //MUST start wit an upper case letter, use CamelNotation, only aplhanumericals after the first letter


Suggested change

//MUST start wit an upper case letter, use CamelNotation, only aplhanumericals after the first letter

// MUST start with an upper case letter, use CamelNotation, only aplhanumericals after the first letter

elevran · 2025-12-15T11:56:54Z

+The pluggable framework will be implemented iteratively over several phases.
+
+1. Introduce `BBRPlugin` `MetadataExtractor`, interface, registry, plugins chain, sample plugin implementation (`SimpleModelExtraction`) and its factory. Plugin configuration will be implemented via environment variables set in helm chart
+1. Introduce a second plugin interface, `ModelSelector` and sample plugin implementation


Q: for the initial task (various ways to do body based routing), a ModelSelector interface would have been sufficient. Why do we need multiple interfaces and what's the delta between interface in (1) and in (2)?

elevran · 2025-12-15T11:57:04Z

+
+1. Introduce `BBRPlugin` `MetadataExtractor`, interface, registry, plugins chain, sample plugin implementation (`SimpleModelExtraction`) and its factory. Plugin configuration will be implemented via environment variables set in helm chart
+1. Introduce a second plugin interface, `ModelSelector` and sample plugin implementation
+1. Introduce shared struct (shared among the plugins of a plugins chain)


that does what?

ModelSelector is aimed at semantic model selection rather than simple metadata extraction and putting it on the headers.

The question was about the introduction of a shared struct. What's its use?

srampal

Some initial comments for now. Review is ongoing so maybe more to come.

davidbreitgand · 2026-01-06T17:05:03Z

Hi @nirrozenbaum 👋
Could you please review this PR when you have a chance?
All reviewers' comments and feedback have been addressed, commits have been squashed into one, and the branch is up to date with main.
If everything looks good, feel free to merge. Thanks!

…ng data

kfswain · 2026-01-08T04:14:11Z

/ok-to-test

kfswain · 2026-03-06T03:59:47Z

+    // A plugin implementation of "semantic-model-selector" sets X-Gateway-Model-Name to any valid
+    // model name from the inventory of the backend models and also mutates the body accordingly
+
+    Execute(requestBodyBytes []byte) (headers map[string]string, mutatedBodyBytes []byte, err error)


This plugin essentially encapsulates any logic the BBR would do right?

ahg-g · 2026-03-14T14:26:14Z

What is the status of this proposal?

ahg-g · 2026-04-01T16:57:23Z

@nirrozenbaum what is the status of this proposal?

nirrozenbaum · 2026-04-01T21:45:36Z

@nirrozenbaum what is the status of this proposal?

I think we can close it, bbr is now pluggable. the goals stated in the proposal were achieved but the implementation is different than what’s described here (some principles are aligned partially)
I don’t see clear value in updating the proposal retroactively. instead we have an open issue to document pluggable bbr which we should do.

nirrozenbaum · 2026-04-05T15:49:25Z

@ahg-g on second thought, I'm going to re-open and merge this PR. and later to update the proposal with its current state. I think it would be useful to have the proposal documented moving forward.

nirrozenbaum · 2026-04-05T15:50:14Z

/lgtm
/approve

k8s-ci-robot · 2026-04-05T15:50:22Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: davidbreitgand, nirrozenbaum

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [nirrozenbaum]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot added the kind/feature Categorizes issue or PR as related to a new feature. label Dec 7, 2025

k8s-ci-robot requested review from kfswain and shmuelk December 7, 2025 12:44

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Dec 7, 2025

k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Dec 7, 2025

davidbreitgand mentioned this pull request Dec 7, 2025

Extensible BBR Framework #1963

Closed

davidbreitgand force-pushed the bbr-pluggable-proposal branch from 4ffe090 to af49267 Compare December 7, 2025 13:18

davidbreitgand changed the title ~~Bbr pluggable framework proposal~~ [WIP] Bbr pluggable framework proposal Dec 7, 2025

k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 7, 2025

davidbreitgand marked this pull request as draft December 7, 2025 19:20

davidbreitgand force-pushed the bbr-pluggable-proposal branch from af49267 to 377259c Compare December 7, 2025 23:03

davidbreitgand changed the title ~~[WIP] Bbr pluggable framework proposal~~ BBR pluggable framework proposal Dec 12, 2025

davidbreitgand force-pushed the bbr-pluggable-proposal branch 2 times, most recently from cd109fd to 9b24a69 Compare December 12, 2025 14:33

davidbreitgand marked this pull request as ready for review December 12, 2025 14:35

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 12, 2025

k8s-ci-robot requested review from ahg-g and liu-cong December 12, 2025 14:35

davidbreitgand mentioned this pull request Dec 14, 2025

Pluggable bbr framework initial PR #1981

Closed

elevran reviewed Dec 15, 2025

View reviewed changes

srampal reviewed Dec 17, 2025

View reviewed changes

davidbreitgand force-pushed the bbr-pluggable-proposal branch from 1395c8c to c4f9775 Compare January 6, 2026 17:00

davidbreitgand force-pushed the bbr-pluggable-proposal branch from c4f9775 to 663ccb4 Compare January 6, 2026 21:23

Adressing reviewers comments, added missing figures, added benchmarki…

3e004a7

…ng data

davidbreitgand force-pushed the bbr-pluggable-proposal branch from 663ccb4 to 3e004a7 Compare January 6, 2026 21:31

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jan 8, 2026

davidbreitgand mentioned this pull request Jan 11, 2026

Interfaces towards pluggable BBR framework (initial PR) #2121

Merged

kfswain reviewed Mar 6, 2026

View reviewed changes

kfswain added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Mar 26, 2026

ahg-g closed this Apr 2, 2026

nirrozenbaum reopened this Apr 5, 2026

k8s-ci-robot assigned nirrozenbaum Apr 5, 2026

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Apr 5, 2026

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 5, 2026

k8s-ci-robot merged commit 7eabe43 into kubernetes-sigs:main Apr 5, 2026
13 checks passed

	//MUST start wit an upper case letter, use CamelNotation, only aplhanumericals after the first letter
	// MUST start with an upper case letter, use CamelNotation, only aplhanumericals after the first letter

Conversation

davidbreitgand commented Dec 7, 2025

Uh oh!

netlify Bot commented Dec 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for gateway-api-inference-extension ready!

Uh oh!

k8s-ci-robot commented Dec 7, 2025

Uh oh!

davidbreitgand commented Dec 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

elevran Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

elevran Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

elevran Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

davidbreitgand Dec 16, 2025

Choose a reason for hiding this comment

Uh oh!

elevran Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

srampal left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

davidbreitgand commented Jan 6, 2026

Uh oh!

kfswain commented Jan 8, 2026

Uh oh!

kfswain Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

ahg-g commented Mar 14, 2026

Uh oh!

ahg-g commented Apr 1, 2026

Uh oh!

nirrozenbaum commented Apr 1, 2026

Uh oh!

nirrozenbaum commented Apr 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nirrozenbaum commented Apr 5, 2026

Uh oh!

k8s-ci-robot commented Apr 5, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

netlify Bot commented Dec 7, 2025 •

edited

Loading

davidbreitgand commented Dec 12, 2025 •

edited

Loading

nirrozenbaum commented Apr 5, 2026 •

edited

Loading