Discovered as part of: llm-d/llm-d#550
the approximate prefix cache plugin assumes only the PrimaryPluginName will be populated with the prefix cache.
In order to work with llm-d's P/D setup, this plugin needs to add awareness of the prefill profile result: https://github.com/llm-d/llm-d/blob/03f2b9c6e630da73c2ff6b38714042e855309cbd/guides/pd-disaggregation/gaie-pd/values.yaml#L34
Discovered as part of: llm-d/llm-d#550
the approximate prefix cache plugin assumes only the PrimaryPluginName will be populated with the prefix cache.
In order to work with llm-d's P/D setup, this plugin needs to add awareness of the
prefillprofile result: https://github.com/llm-d/llm-d/blob/03f2b9c6e630da73c2ff6b38714042e855309cbd/guides/pd-disaggregation/gaie-pd/values.yaml#L34