docs: add Kgateway BBR documentation#1908
Conversation
k8s-ci-robot
added
kind/documentation
k8s-ci-robot
added
the
size/S
✅ Deploy Preview for gateway-api-inference-extension ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
cc @danehans |
|
|
||
| === "Kgateway" | ||
|
|
||
| Kgateway does not require the Body-Based Routing Extension, and instead natively implements Body-Based Routing. |
There was a problem hiding this comment.
@howardjohn thanks for the PR.
we’re currently in the process of extending BBR to more than just extracting the model from the body to a header.
more details in #1812.
is it possible to add to kgateway support to invoke additional ext proc (before epp) as it is with the other gateways?
current PR is very tailored to the current BBR but that will get outdated very quickly.
There was a problem hiding this comment.
@nirrozenbaum kgateway can do ext_proc but we believe its best to do processing natively. We can do what is described in #1812 natively as well and am happy to update the docs with examples once it lands.
There was a problem hiding this comment.
we believe its best to do processing natively
I agree with you here 100%. there is a different (not directly related) effort to standardize exactly this as part of k8s AI-Gateway WG, and ideally after we standardize it we should replace the existing ext proc based bbr with native one. but that’s a different story.
+1 on proceeding and take the “fix it forward” approach.
/lgtm
/approve
Thanks! 🙏🏼
k8s-ci-robot
added
the
lgtm
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: howardjohn, nirrozenbaum The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
k8s-ci-robot
added
the
approved
3 participants
What type of PR is this?
/kind documentation
What this PR does / why we need it:
This adds documentation for performing BBR with Kgateway. Kgateway is listed on the getting started (https://gateway-api-inference-extension.sigs.k8s.io/guides/#__tabbed_2_3) but not on the BBR (https://gateway-api-inference-extension.sigs.k8s.io/guides/serve-multiple-genai-models) docs.
Which issue(s) this PR fixes:
Fixes #
Does this PR introduce a user-facing change?: