[bbr] Skip body re-serialization when unchanged by asaadbalum · Pull Request #2556 · kubernetes-sigs/gateway-api-inference-extension

asaadbalum · 2026-03-12T11:35:57Z

Summary

Add body mutation tracking to InferenceMessage via SetBody, SetBodyField, RemoveBodyField, and BodyMutated so the request and response handlers skip json.Marshal and Content-Length update when no plugin modified the body
Apply the optimization to both request (request.go) and response (response.go) paths: for unary, omit BodyMutation when unchanged; for streaming, forward original bytes when unchanged
Add integration tests that verify body mutation tracking over real gRPC for both unary and streaming paths

This is a follow-up to #2551, which added request body mutation support but always re-marshaled regardless of whether plugins modified the body.

What type of PR is this?

/kind feature

What's changed:

Path	Change
`pkg/bbr/framework/types.go`	Add `bodyMutated` flag; `SetBody` replaces entire body; `SetBodyField`/`RemoveBodyField` for field-level mutations; `BodyMutated` accessor
`pkg/bbr/framework/types_test.go`	Unit tests for body mutation tracking API (`SetBody`, `SetBodyField`, `RemoveBodyField`, `BodyMutated`)
`pkg/bbr/handlers/request.go`	Conditionally marshal/include `BodyMutation` only when body changed
`pkg/bbr/handlers/response.go`	Same conditional logic for response path
`pkg/bbr/handlers/request_test.go`	Body mutation tests (unary + streaming)
`pkg/bbr/handlers/response_test.go`	Body mutation + no-mutation tests (unary + streaming)
`pkg/bbr/handlers/server_test.go`	Remove stale `Content-Length`/`BodyMutation` expectations
`test/integration/bbr/harness.go`	Add `NewBBRHarnessWithPlugins` for custom plugin injection
`test/integration/bbr/body_mutation_test.go`	Integration tests: body-mutating plugin over real gRPC (unary + streaming)
`test/integration/bbr/util.go`	Remove stale expectations, clean unused `prompt` parameter
`test/integration/bbr/hermetic_test.go`	Update call sites for cleaned function signatures

Does this PR introduce a user-facing change?

Users writing BBR plugins that do NOT modify the request/response body will see reduced overhead — no unnecessary JSON marshaling or Content-Length rewriting. Plugins that modify the body should use SetBody (full replacement), SetBodyField, or RemoveBodyField to ensure the dirty flag is set.

netlify · 2026-03-12T11:36:05Z

✅ Deploy Preview for gateway-api-inference-extension ready!

Name	Link
🔨 Latest commit	`b60e5b8`
🔍 Latest deploy log	https://app.netlify.com/projects/gateway-api-inference-extension/deploys/69b2e9f52ca9b80008181df8
😎 Deploy Preview	https://deploy-preview-2556--gateway-api-inference-extension.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

k8s-ci-robot · 2026-03-12T11:36:08Z

Hi @asaadbalum. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Tip

We noticed you've done this a few times! Consider joining the org to skip this step and gain /lgtm and other bot rights. We recommend asking approvers on your previous PRs to sponsor you.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

nirrozenbaum

overall lgtm
left few minor nits

nirrozenbaum · 2026-03-12T12:12:03Z

+	}
+}
+
+func (r *InferenceMessage) BodyWasMutated() bool {


nit: rename to BodyMutated()

Suggested change

func (r *InferenceMessage) BodyWasMutated() bool {

func (r *InferenceMessage) BodyMutated() bool {

nirrozenbaum · 2026-03-12T13:25:08Z

 	}

+	// Necessary so that the new headers are used in the routing decision.
+	resp := &eppb.CommonResponse{


nit: can we call response?

nirrozenbaum · 2026-03-12T13:34:35Z

+	bodyMutated := reqCtx.Request.BodyWasMutated()
+	var mutatedBodyBytes []byte
+	if bodyMutated {
+		b, err := json.Marshal(reqCtx.Request.Body)


why do we need b? can't we marshal directly to mutatedBodyBytes?

nirrozenbaum · 2026-03-12T13:34:45Z

+	bodyMutated := reqCtx.Response.BodyWasMutated()
+	var mutatedBytes []byte
+	if bodyMutated {
+		b, err := json.Marshal(reqCtx.Response.Body)


nirrozenbaum · 2026-03-12T13:34:56Z

 		return ret, nil
 	}

+	resp := &eppb.CommonResponse{


nirrozenbaum · 2026-03-12T16:19:47Z

/ok-to-test

nirrozenbaum · 2026-03-12T16:22:33Z

+		r.bodyMutated = true
+	}
+}
+


I think we need here additional function to set the whole body.
something like:

func (r *InferenceMessage) SetBody(body map[string]any) { r.Body = body r.bodyMutated = true }

this might be useful for plugins that transform the body in some way (e.g., api translation from OpenAI to Anthropic) and want to set the whole body at once.

Agree, sending a patch in a min

Add body mutation tracking to InferenceMessage via SetBody, SetBodyField, RemoveBodyField, and BodyMutated so the request and response handlers skip json.Marshal and Content-Length update when no plugin modified the body. SetBody replaces the entire body map for plugins that transform the full payload (e.g., API translation). SetBodyField and RemoveBodyField handle granular field-level mutations. For unary mode, omit BodyMutation from the ext_proc response when unchanged. For streaming mode, forward original bytes instead of re-marshaling. Add integration tests that verify body mutation tracking over real gRPC for both unary and streaming paths. Signed-off-by: Asaad Balum <asaad.balum@gmail.com>

nirrozenbaum

/lgtm
/approve

Thanks!

k8s-ci-robot · 2026-03-12T22:27:22Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: asaadbalum, nirrozenbaum

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [nirrozenbaum]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

asaadbalum · 2026-03-12T22:55:23Z

/test pull-gateway-api-inference-extension-test-e2e-main

…tes-sigs#2556) Add body mutation tracking to InferenceMessage via SetBody, SetBodyField, RemoveBodyField, and BodyMutated so the request and response handlers skip json.Marshal and Content-Length update when no plugin modified the body. SetBody replaces the entire body map for plugins that transform the full payload (e.g., API translation). SetBodyField and RemoveBodyField handle granular field-level mutations. For unary mode, omit BodyMutation from the ext_proc response when unchanged. For streaming mode, forward original bytes instead of re-marshaling. Add integration tests that verify body mutation tracking over real gRPC for both unary and streaming paths. Signed-off-by: Asaad Balum <asaad.balum@gmail.com>

…tes-sigs/gateway-api-inference-extension#2556) Add body mutation tracking to InferenceMessage via SetBody, SetBodyField, RemoveBodyField, and BodyMutated so the request and response handlers skip json.Marshal and Content-Length update when no plugin modified the body. SetBody replaces the entire body map for plugins that transform the full payload (e.g., API translation). SetBodyField and RemoveBodyField handle granular field-level mutations. For unary mode, omit BodyMutation from the ext_proc response when unchanged. For streaming mode, forward original bytes instead of re-marshaling. Add integration tests that verify body mutation tracking over real gRPC for both unary and streaming paths. Signed-off-by: Asaad Balum <asaad.balum@gmail.com>

k8s-ci-robot added the kind/feature Categorizes issue or PR as related to a new feature. label Mar 12, 2026

k8s-ci-robot requested review from nirrozenbaum and robscott March 12, 2026 11:36

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Mar 12, 2026

k8s-ci-robot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Mar 12, 2026

asaadbalum marked this pull request as draft March 12, 2026 11:46

k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 12, 2026

asaadbalum force-pushed the skip-request-body-remarshal branch 2 times, most recently from 5ae56cc to bea9ce8 Compare March 12, 2026 12:06

asaadbalum marked this pull request as ready for review March 12, 2026 12:06

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 12, 2026

k8s-ci-robot requested a review from kfswain March 12, 2026 12:07

asaadbalum force-pushed the skip-request-body-remarshal branch from bea9ce8 to 0be4cf4 Compare March 12, 2026 12:09

nirrozenbaum reviewed Mar 12, 2026

View reviewed changes

asaadbalum force-pushed the skip-request-body-remarshal branch from 0be4cf4 to b106e2b Compare March 12, 2026 14:06

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Mar 12, 2026

nirrozenbaum reviewed Mar 12, 2026

View reviewed changes

asaadbalum force-pushed the skip-request-body-remarshal branch from b106e2b to b60e5b8 Compare March 12, 2026 16:29

nirrozenbaum approved these changes Mar 12, 2026

View reviewed changes

k8s-ci-robot assigned nirrozenbaum Mar 12, 2026

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 12, 2026

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 12, 2026

k8s-ci-robot merged commit 0ff1d13 into kubernetes-sigs:main Mar 12, 2026
11 checks passed

asaadbalum deleted the skip-request-body-remarshal branch March 12, 2026 23:08

	func (r *InferenceMessage) BodyWasMutated() bool {
	func (r *InferenceMessage) BodyMutated() bool {

Conversation

asaadbalum commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

netlify Bot commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for gateway-api-inference-extension ready!

Uh oh!

k8s-ci-robot commented Mar 12, 2026

Uh oh!

nirrozenbaum left a comment

Choose a reason for hiding this comment

Uh oh!

nirrozenbaum Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

nirrozenbaum Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

nirrozenbaum Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

nirrozenbaum Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

nirrozenbaum Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

nirrozenbaum commented Mar 12, 2026

Uh oh!

nirrozenbaum Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

asaadbalum Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

nirrozenbaum left a comment

Choose a reason for hiding this comment

Uh oh!

k8s-ci-robot commented Mar 12, 2026

Uh oh!

asaadbalum commented Mar 12, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

asaadbalum commented Mar 12, 2026 •

edited

Loading

netlify Bot commented Mar 12, 2026 •

edited

Loading