[Flow Control] Implement Dynamic Priority Band Configuration

[LukeAVanDrie]

What would you like to be added: The Flow Control layer should dynamically configure its recognized priority bands based on the InferenceObjective resources registered to the pool. This will remove the current need for a static, at-startup configuration of the priority domain.

Why is this needed: Currently, only priority 0 is configured by default, and any other priority (including negative values for load shedding) must be hardcoded. This change will allow users to define and use any priority value in their InferenceObjective resources without needing to fork, modify, and redeploy the EPP. This is a significant usability improvement and makes the priority system far more flexible.

[LukeAVanDrie]

/assign

[LukeAVanDrie]

Here is a quick summary of the problem shape.

• The Hot Path (Stats Propagation):

  • Frequency: Executes on every single Request Enqueue and Dequeue.
  • Concurrency: High (called by every active request goroutine).
  • Constraint: Must not block.
  • Characteristics: Read-Only 99.999% of the time.

• The Cold Path (Dynamic Provisioning):

  • Frequency: Extremely Rare. Happens once per unique Priority Level process-lifetime (e.g., we have 10 priority bands, this runs 10 times total).
  • Concurrency: Low.
  • Constraint: Must be thread-safe (prevent double-creation). Latency is irrelevant.
  • Characteristics: Write-Heavy (modifies the topology map).

---

	sync.RWMutex	sync.Map	atomic.Pointer (COW)
Code Style	Idiomatic Go.	Standard Lib. Common for caches, but requires casting.	Custom implementation of lock-free logic.
Type Safety	Strong. Compiler-enforced.	Weak. Stores any. Requires runtime casting (val.(*band)).	Strong. Uses Go Generics (atomic.Pointer[T]).
Hot Path Cost	Updates a shared atomic counter.	Interface boxing + internal atomic checks.	Single pointer dereference.
Maintenance	Hard to break.	Risk of panic if types drift in future refactors, but practically mitigated by unit testing.	High. Easy to break the "Copy" logic in the write path.

[LukeAVanDrie]

I am leaning toward sync.Map. Will send out a draft PR with this approach.

[LukeAVanDrie]

To start, any un-configured band will get the default config. Later we can consider if we want to match the config of the nearest configured floor or ceiling. (e.g., if 10, 20, and 30 are configured and 31 comes in, use band 30 config).