Request Patch Release 0.5.x for InferenceObjective and InferenceModelRewrite Functionality

[yetisun]

Problem Summary

The reconcilers for InferenceObjective and InferenceModelRewrite types fail to start during EPP (Endpoint Picker) initialization, rendering these CRD types non-functional.

The issue exists in

1. Gateway API Inference Extension v1.3.0
2. llm-d-inference-scheduler v0.5.0

Problem Description

Impact

• InferenceObjective reconciler does not start
• InferenceModelRewrite reconciler does not start
• User-configured InferenceObjective and InferenceModelRewrite resources are not processed correctly by EPP

Root Cause

1. PR #1967 introduced ControllerConfig and PopulateControllerConfig to dynamically detect whether InferenceObjective and InferenceModelRewrite CRDs exist on the cluster via Discovery API, and decide whether to start the corresponding reconcilers.

2. However, PR #1967 missed a critical step: after creating controllerCfg in runner.go, it did not call PopulateControllerConfig(cfg).

3. As a result, the hasInferenceObjective and hasInferenceModelRewrites fields remained at their default value false, causing the reconcilers for InferenceObjective and InferenceModelRewrite to never be registered with the controller manager.

Affected Code Location

• cmd/epp/runner/runner.go: Missing call to PopulateControllerConfig(cfg) after creating controllerCfg and before creating the manager.

Fix

PR #2300 has fixed this issue:

• Added PopulateControllerConfig(cfg) call after NewDefaultManager in cmd/epp/runner/runner.go
• Ensures controller configuration is properly populated and validated before startup
• Allows InferenceObjective and InferenceModelRewrite CRDs to be correctly read during EPP startup

Request

Since this bug affects the functional completeness of a released version, we kindly ask the community to consider releasing a Patch Release so users can obtain the fixed stable version.

1. Gateway API Inference Extension v1.3.x
2. lm-d-inference-scheduler v0.5.x

Related Links

• Bug introduced by: PR #1967
• Fix PR: PR #2300

[kfswain]

Reading this issue it sounds like Inference Objective and InferenceModelRewrite just don't work at all, is that correct?

[yetisun]

Reading this issue it sounds like Inference Objective and InferenceModelRewrite just don't work at all, is that correct?

Hi @kfswain correct. Both are not working now in the latest release.

[nirrozenbaum]

@kfswain sounds like a fix that deserves a patch release.

[elevran]

waiting on IGW patch release to update v0.5

[yetisun]

Hi @elevran, do I need to open an issue in IGW, or can I just wait for its patch release? 🙂

[elevran]

no need for an issue. I think the maintainers are aware and will cut a patch release in IGW and we'll follow afterwards here.

[elevran]

IGW has cut an RC1 for patch

[elevran]

fixed in release v0.5.1