feat: improve golden token injection

[maxdebayser]

A rebase of @wallashss's PR #536 without the LogitsProcessor refactors.

Improvements to the GoldenTokenInjector (GTI):

• update GTI args to support parsing from JSON string, eg. passed through the vLLM OpenAI vllm_xargs extension
• support always injecting the golden token regardless of expected logprobs / divergence checks
• GTI is now part of the vllm_spyre package; it can be enabled with: --logits-processors vllm_spyre.v1.sample.golden_token_injector:GoldenTokenInjector

Example usage:

VLLM_SPYRE_USE_CB=1 vllm serve ibm-ai-platform/micro-g3.3-8b-instruct-1b --max-num-seqs 16 --max-model-len 1024 --logits-processors vllm_spyre.v1.sample.golden_token_injector:GoldenTokenInjector

curl http://localhost:8000/v1/completions \
    -H "Content-Type: application/json" \
    -d '{
  "model": "ibm-ai-platform/micro-g3.3-8b-instruct-1b",
  "prompt": "a squared plus b squared equals ",
  "vllm_xargs": {
     "golden_token_injector": "{\"expected_token_ids\": [85, 39560]}"
  },
  "logprobs": 0,
  "return_tokens_as_token_ids": true,
  "max_tokens": 2,
  "temperature": 0
}'

to generate "c squared" instead of "16".

[github-actions]

👋 Hi! Thank you for contributing to vLLM support on Spyre.
Just a reminder: Make sure that your code passes all the linting checks, otherwise your PR won't be able to be merged. To do so, first install the linting requirements, then run format.sh and commit the changes. This can be done with uv directly:

uv sync --frozen --group lint --active --inexact

Or this can be done with pip:

uv pip compile --group lint > requirements-lint.txt
pip install -r requirements-lint.txt
bash format.sh

Now you are good to go 🚀

[tjohnson31415]

bot:test

[tjohnson31415]

This change is needed now that all tests use GTI by default (before this PR, only tests using get_engine would use GTI). Having a copied reference instead of a .clone() meant that all sequences had the same GTI config (even with different prompts).

[wallashss]

bot:test
MARKERS="spyre and cb and not quantized and not multi"

[waleedqk]

bot:test
MARKERS="spyre and cb and not quantized and not multi"

[waleedqk]

bot:test
MARKERS="spyre and cb and not quantized and not multi"

[wallashss]

@tjohnson31415 tests included!

[tjohnson31415]

LGTM lots of improvements and some tests too!!

[wallashss]

bot:test
MARKERS="spyre and cb and not multi"

[wallashss]

bot:test

[wallashss]

bot:test
TEST_FILE=tests/v1/worker/test_spyre_input_batch.py

[wallashss]

bot:test
TEST_FILE=tests/v1/worker/test_spyre_input_batch.py MARKERS=cpu