Simplify vllm batch

[sahel-sh]

Mostly agentic coding, both vLLM inference modes are tested via demos, no impact on Gemini and GPT rankers.
I will add tests for sglang and tensorrt in a follow up PR if we decide to keep them.

Before this change we would create the prompts for all requests, then process the sliding window for all requests in parallel. This meant every single request had to complete the ranking of the current window before sending the next batch. For thinking models, only a single harder query is enough to keep everyone waiting, lots of idle time.

This cl does the prompt creation, llm inference, permutation update and enqueing of the next window for each request individually, the batch_size controls the number of on the fly requests to vllm inference handlers.

Pull Request Checklist

Reference Issue

Please provide the reference to issue this PR is addressing (# followed by the issue number). If there is no associated issue, write "N/A".

ref:

Checklist Items

Before submitting your pull request, please review these items:

• Have you followed the contributing guidelines?
• Have you verified that there are no existing Pull Requests for the same update/change?
• Have you updated any relevant documentation or added new tests where needed?

PR Type

What kind of change does this PR introduce?

• Bugfix
• Feature
• Code style update (formatting, local variables)
• Refactoring (no functional changes, no API changes)
• Documentation content changes
• Reproduction logs
• Other...
  • Description: