Skip to content

Conversation

@hl475
Copy link
Contributor

@hl475 hl475 commented Oct 31, 2025

Purpose

This PR fixes a startup crash in the v1 runtime for attention‑free models (e.g., Terratorch) introduced after #27809. The engine unconditionally computed the batch reorder threshold even when no attention backends were created, leading to:

TypeError: reduce() of empty iterable with no initial value

from the nightly run (https://buildkite.com/vllm/ci/builds/37041/steps/canvas?sid=019a386d-1b25-4c07-9a9b-085c1e07ea05, https://buildkite.com/vllm/ci/builds/37041/steps/canvas?sid=019a386d-1b26-4f42-b55f-f0125da20368)

This PR (1) skip the calculation when there are no attention groups, and (2) make calculate_reorder_batch_threshold() defensive by resolving an empty list to None.

Test Plan

CI

Test Result

Basic Models Tests (Extra Initialization) 1 + Basic Models Tests (Extra Initialization) 2
https://buildkite.com/vllm/ci/builds/37052/steps/canvas?sid=019a3901-5ee6-45ba-bace-2ccb858b53a1
Basic Models Tests (Initialization)
https://buildkite.com/vllm/ci/builds/37052/steps/canvas?sid=019a3901-5ee5-48b7-ac1d-0cf8db1b054d


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

@mergify mergify bot added the v1 label Oct 31, 2025
@hl475 hl475 changed the title fix_attention_free_models v1: Don’t compute reorder threshold when there are no attention groups Oct 31, 2025
@hl475 hl475 marked this pull request as ready for review October 31, 2025 07:59
@LucasWilkinson LucasWilkinson changed the title v1: Don’t compute reorder threshold when there are no attention groups [BugFix] Don’t compute reorder threshold when there are no attention groups Oct 31, 2025
Copy link
Collaborator

@LucasWilkinson LucasWilkinson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution! Overall LGTM, left a couple nits

Signed-off-by: Huamin Li <[email protected]>
@hl475 hl475 force-pushed the fix_attention_free_models branch from a92ec36 to 2e0dc6a Compare October 31, 2025 08:36
@hl475
Copy link
Contributor Author

hl475 commented Oct 31, 2025

Thanks @LucasWilkinson for reviewing!

I just updated this PR to address the comments, please take another look!

@LucasWilkinson LucasWilkinson enabled auto-merge (squash) October 31, 2025 09:22
@github-actions github-actions bot added the ready ONLY add when PR is ready to merge/full CI is needed label Oct 31, 2025
@LucasWilkinson LucasWilkinson merged commit 933cdea into vllm-project:main Oct 31, 2025
47 checks passed
ZhengHongming888 pushed a commit to ZhengHongming888/vllm that referenced this pull request Nov 8, 2025
rtourgeman pushed a commit to rtourgeman/vllm that referenced this pull request Nov 10, 2025
devpatelio pushed a commit to SumanthRH/vllm that referenced this pull request Nov 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready ONLY add when PR is ready to merge/full CI is needed v1

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants