[Bug]: IndexError: list index out of range when priority length not matching length of prompts in LLM class

[junpuf]

Your current environment

The output of python collect_env.py

==============================
     Environment Variables
==============================
VLLM_CACHE_ROOT=/home/ec2-user/.cache/vllm
LD_LIBRARY_PATH=/opt/amazon/openmpi/lib64:/opt/amazon/efa/lib64:/opt/amazon/ofi-nccl/lib64:/usr/local/cuda/lib:/usr/local/cuda:/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64:/usr/local/cuda/targets/x86_64-linux/lib:/usr/local/lib:/usr/lib:/lib:/opt/amazon/openmpi/lib64:/opt/amazon/efa/lib64:/opt/amazon/ofi-nccl/lib64:/usr/local/cuda/lib:/usr/local/cuda:/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64:/usr/local/cuda/targets/x86_64-linux/lib:/usr/local/lib:/usr/lib:/lib:/opt/amazon/openmpi/lib64:/opt/amazon/efa/lib64:/opt/amazon/ofi-nccl/lib64:/usr/local/cuda/lib:/usr/local/cuda:/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64:/usr/local/cuda/targets/x86_64-linux/lib:/usr/local/lib:/usr/lib:/lib
PYTORCH_NVML_BASED_CUDA_CHECK=1
TORCHINDUCTOR_COMPILE_THREADS=1

🐛 Describe the bug

Checked with chatbot that there is no issue or PR related to this.

In the code at llm.py#L1591, there is no explicit validation to ensure that the priority list matches the length of prompts. This could lead to unsafe behavior if their lengths differ. The code only checks lengths for params and lora_request, not for priority.

from vllm import LLM, SamplingParams

prompts = [
    "Hello, my name is",
    "The president of the United States is",
    "The capital of France is",
    "The future of AI is",
]
sampling_params = SamplingParams(temperature=0.8, top_p=0.95)

llm = LLM(model="facebook/opt-125m")

outputs = llm.generate(prompts, sampling_params, priority=[0, 0, 0])

# Print the outputs.
for output in outputs:
    prompt = output.prompt
    generated_text = output.outputs[0].text
    print(f"Prompt: {prompt!r}, Generated text: {generated_text!r}")

Output

Traceback (most recent call last):
  File "/home/ec2-user/inference_playground/vllm/bug.py", line 10, in <module>
    outputs = llm.generate(promtps, priority=[0, 0, 0])
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ec2-user/inference_playground/vllm/vllm/entrypoints/llm.py", line 432, in generate
    self._validate_and_add_requests(
  File "/home/ec2-user/inference_playground/vllm/vllm/entrypoints/llm.py", line 1591, in _validate_and_add_requests
    priority=priority[i] if priority else 0,
             ~~~~~~~~^^^
IndexError: list index out of range

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.