What would you like to be added:
Support specifying port roles (serving, metrics, health) in InferencePool spec, so that EPP and other consumers can use the correct port per function.
spec:
ports:
serving: 8080
metrics: 9090 # optional, defaults to serving port if not specified
health: 8080 # optionalWhy is this needed:
This continues the discussion from #1396 and PR #2762.
In Istio mTLS STRICT environments, EPP scrapes pod metrics via direct pod-IP access, bypassing the Service layer. The receiving sidecar rejects this with 503 as it expects mTLS but receives plain HTTP.
PR #2762 adds metricsPort to the metrics-data-source plugin as a workaround, but this requires per-plugin config and doesn't address the structural gap at the InferencePool spec level. A spec-level solution would make port roles a first-class concept, benefiting all consumers consistently.
Happy to contribute to a design doc or prototype implementation.
What would you like to be added:
Support specifying port roles (serving, metrics, health) in InferencePool spec, so that EPP and other consumers can use the correct port per function.
Why is this needed:
This continues the discussion from #1396 and PR #2762.
In Istio mTLS STRICT environments, EPP scrapes pod metrics via direct pod-IP access, bypassing the Service layer. The receiving sidecar rejects this with 503 as it expects mTLS but receives plain HTTP.
PR #2762 adds
metricsPortto themetrics-data-sourceplugin as a workaround, but this requires per-plugin config and doesn't address the structural gap at the InferencePool spec level. A spec-level solution would make port roles a first-class concept, benefiting all consumers consistently.Happy to contribute to a design doc or prototype implementation.