-
-
Notifications
You must be signed in to change notification settings - Fork 12k
[Bugfix]: Fix final_res_batch list index out of range error #21055
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: chaunceyjiang <[email protected]>
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add 🚀 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request correctly fixes a list index out of range error that occurs when an empty list of prompts is provided. The change prevents accessing final_res_batch[0] when the list is empty. I've added one critical comment pointing out a related NameError that can occur under the same conditions, with a suggestion on how to resolve it to make the function fully robust.
Signed-off-by: chaunceyjiang <[email protected]>
Signed-off-by: chaunceyjiang <[email protected]>
|
@DarkLight1337 PTAL. |
| total_tokens=num_prompt_tokens + num_generated_tokens, | ||
| ) | ||
|
|
||
| if self.enable_prompt_tokens_details and final_res.num_cached_tokens: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note: final_res is an internal variable from the for-loop above. If final_res_batch is an empty list ([]), then final_res will not be initialized.
Signed-off-by: chaunceyjiang <[email protected]>
Signed-off-by: chaunceyjiang <[email protected]>
Head branch was pushed to by a user without write access
…ject#21055) Signed-off-by: chaunceyjiang <[email protected]> Signed-off-by: x22x22 <[email protected]>
…ject#21055) Signed-off-by: chaunceyjiang <[email protected]>
…ject#21055) Signed-off-by: chaunceyjiang <[email protected]>
…ject#21055) Signed-off-by: chaunceyjiang <[email protected]> Signed-off-by: Jinzhen Lin <[email protected]>
…ject#21055) Signed-off-by: chaunceyjiang <[email protected]> Signed-off-by: Paul Pak <[email protected]>
…ject#21055) Signed-off-by: chaunceyjiang <[email protected]> Signed-off-by: Diego-Castan <[email protected]>
…ject#21055) Signed-off-by: chaunceyjiang <[email protected]>
Purpose
Fix https://buildkite.com/vllm/ci/builds/24047/steps/canvas?jid=01980fe5-71cd-4800-b119-932abd62ee98
Test Plan
Test Result
(Optional) Documentation Update