Add flowcontrol queue length in bytes metric by RyanRosario · Pull Request #2044 · kubernetes-sigs/gateway-api-inference-extension

RyanRosario · 2025-12-30T06:43:48Z

What type of PR is this?
/kind documentation
/kind feature

What this PR does / why we need it:

It adds new observability metrics for flow control.

Which issue(s) this PR fixes:
Related to #1708

Does this PR introduce a user-facing change?:

Added `inference_extension_flow_control_queue_bytes` metric to track the total size (in bytes) of requests currently buffered in the Flow Control layer.

netlify · 2025-12-30T06:43:54Z

✅ Deploy Preview for gateway-api-inference-extension ready!

Name	Link
🔨 Latest commit	`04c958d`
🔍 Latest deploy log	https://app.netlify.com/projects/gateway-api-inference-extension/deploys/69716124a7513300089389d8
😎 Deploy Preview	https://deploy-preview-2044--gateway-api-inference-extension.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

k8s-ci-robot · 2025-12-30T06:43:58Z

Hi @RyanRosario. Thanks for your PR.

I'm waiting for a github.com member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

kfswain · 2026-01-12T20:31:40Z

/ok-to-test

ahg-g · 2026-01-14T15:18:06Z

/assign @LukeAVanDrie

k8s-ci-robot · 2026-01-14T15:18:10Z

@ahg-g: GitHub didn't allow me to assign the following users: LukeAVanDrie.

Note that only kubernetes-sigs members with read permissions, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time.
For more information please see the contributor guide

Details

In response to this:

/assign @LukeAVanDrie

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

LukeAVanDrie

Thanks, Ryan! I have left a few minor inline comments, but I have no blocking concerns.
This LGTM!

/assign @ahg-g

LukeAVanDrie · 2026-01-15T22:46:08Z

Oh, @RyanRosario, since we are adding a new public metric that operators will use, this is a user-facing change. Please update the release note section in your PR description (on this and the other metrics PRs):

E.g.,

Added `inference_extension_flow_control_queue_bytes` metric to track the total size (in bytes) of requests currently buffered in the Flow Control layer.

ahg-g · 2026-01-20T15:18:40Z

+		prometheus.GaugeOpts{
+			Subsystem: InferenceExtension,
+			Name:      "flow_control_queue_bytes",
+			Help:      metricsutil.HelpMsgWithStability("Current number of bytes associated with requests actively managed by the EPP flow control layer, from the start of the EnqueueAndWait call until a final outcome is reached.", compbasemetrics.ALPHA),


please remove the phrase EnqueueAndWait as it is an internal detail and describe the metric in abstract terms.

Ryan is mirroring my description for the flow_control_queue_size (len) metric I already added. Ryan, do you mind updating it there as well?

Will do. Thank you both for catching that.

this is not addressed yet.

Ah, I looked through the last commit and saw that two were updated (queue_size and queue_duration). We still need queue_bytes.

Sorry about that. This is addressed.

RyanRosario · 2026-01-20T20:33:45Z

All feedback addressed. Ready for final review.

LukeAVanDrie · 2026-01-20T20:45:18Z

/approve
/lgtm

@ahg-g

RyanRosario · 2026-01-21T23:44:07Z

/assign @LukeAVanDrie

ahg-g · 2026-01-22T09:04:45Z

/lgtm
/approve

k8s-ci-robot · 2026-01-22T09:04:55Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ahg-g, LukeAVanDrie, RyanRosario

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [ahg-g]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

…-api-inference-extension#2044) * Add flow_control_queue_bytes metric * Add flow_control_queue_bytes metric and documentation * Address reviewer feedback * Update comment to remove internal function name --------- Co-authored-by: Ryan Rosario <6713180+RyanRosario@users.noreply.github.com>

k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. kind/documentation Categorizes issue or PR as related to documentation. kind/feature Categorizes issue or PR as related to a new feature. labels Dec 30, 2025

k8s-ci-robot requested review from ahg-g and liu-cong December 30, 2025 06:43

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Dec 30, 2025

k8s-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Dec 30, 2025

RyanRosario changed the title ~~[WIP] Add additional observability metrics for flow control~~ Add flowcontrol queue length in bytes metric Jan 9, 2026

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 9, 2026

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jan 12, 2026

LukeAVanDrie approved these changes Jan 15, 2026

View reviewed changes

Comment thread pkg/epp/flowcontrol/controller/controller.go Outdated

Comment thread pkg/epp/metrics/metrics_test.go Outdated

k8s-ci-robot assigned ahg-g Jan 15, 2026

LukeAVanDrie mentioned this pull request Jan 16, 2026

[Flow Control] Observability (Prom Metrics, not Tracing) #1708

Closed

ahg-g reviewed Jan 20, 2026

View reviewed changes

RyanRosario added 3 commits January 20, 2026 20:32

Add flow_control_queue_bytes metric

e9c732f

Add flow_control_queue_bytes metric and documentation

96c3cf2

Address reviewer feedback

055ad5d

RyanRosario force-pushed the bytemetric branch from fa18396 to 055ad5d Compare January 20, 2026 20:33

k8s-ci-robot assigned LukeAVanDrie Jan 20, 2026

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 20, 2026

Update comment to remove internal function name

04c958d

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 21, 2026

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 22, 2026

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 22, 2026

k8s-ci-robot merged commit c5cf00c into kubernetes-sigs:main Jan 22, 2026
11 checks passed

Conversation

RyanRosario commented Dec 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

netlify Bot commented Dec 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for gateway-api-inference-extension ready!

Uh oh!

k8s-ci-robot commented Dec 30, 2025

Uh oh!

kfswain commented Jan 12, 2026

Uh oh!

ahg-g commented Jan 14, 2026

Uh oh!

k8s-ci-robot commented Jan 14, 2026

Uh oh!

LukeAVanDrie left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

LukeAVanDrie commented Jan 15, 2026

Uh oh!

ahg-g Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

LukeAVanDrie Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

RyanRosario Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

ahg-g Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

LukeAVanDrie Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

RyanRosario Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

RyanRosario commented Jan 20, 2026

Uh oh!

LukeAVanDrie commented Jan 20, 2026

Uh oh!

RyanRosario commented Jan 21, 2026

Uh oh!

ahg-g commented Jan 22, 2026

Uh oh!

k8s-ci-robot commented Jan 22, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

RyanRosario commented Dec 30, 2025 •

edited

Loading

netlify Bot commented Dec 30, 2025 •

edited

Loading