[BugFix] Ray with multiple nodes

[juliendenize]

Purpose

This is a hotfix to launch vllm with multiple nodes and Ray.

To reproduce on a 2x8 (nodes x gpus) Ray cluster:

vllm serve mistralai/Mistral-Small-3.2-24B-Instruct-2506   --tokenizer_mode mistral --config_format mistral   --load_format mistral --tool-call-parser mistral   --enable-auto-tool-choice --limit-mm-per-prompt '{"image":10}'   --tensor-parallel-size 16   --max_model_len 65536   --max_num_seqs 128   --distributed-executor-backend ray --enforce_eager

Error:

(EngineCore_DP0 pid=570282)   File "/home/julien.denize/workspace/vllm/vllm/v1/worker/gpu_worker.py", line 212, in init_device
(EngineCore_DP0 pid=570282)     assert self.parallel_config.local_world_size <= visible_device_count, (
(EngineCore_DP0 pid=570282)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=570282) AssertionError: local_world_size (16) must be less than or equal to the number of visible devices (8).

Test Plan

Test Result

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[gemini-code-assist]

Code Review

This pull request aims to fix a crash when running vLLM with Ray on multiple nodes. The crash is caused by an assertion that checks if the number of workers on a node exceeds the number of available GPUs. The fix in this PR is to move this assertion inside an if block that is skipped for Ray, which resolves the immediate issue. However, my review finds that this change is too broad and incorrectly disables this important sanity check for other valid configurations, which could lead to other crashes. I've provided a critical comment explaining the issue and suggesting a more targeted fix.

[gemini-code-assist]

This change moves the assertion for local_world_size inside the if block that handles a specific single-node data parallelism setup. While this fixes the issue for multi-node Ray where local_world_size might be miscalculated, it incorrectly disables this important sanity check for other valid configurations, such as multi-node setups without Ray or single-node setups without data parallelism.

The assertion self.parallel_config.local_world_size <= visible_device_count is a general check to ensure that the number of workers on a node does not exceed the number of available GPUs. It should not be confined to the specific data parallelism case.

A more targeted fix would be to skip this check only for Ray, or to fix the underlying issue with the calculation of local_world_size for Ray environments. Disabling this check for all other configurations could hide potential resource allocation issues and lead to crashes in other scenarios.

[njhill]

Thanks @juliendenize... looks like this was introduced by #23691? cc @luccafong

[luccafong]

lgtm, thanks for the fix!

[ortegaalfredo]

Still not working for me:

(APIServer pid=3167402) Value error, Tensor parallel size (10) cannot be larger than the number of available GPUs (8). [type=value_error, input_value=ArgsKwargs((), {'pipeline...'_api_process_rank': 0}), input_type=ArgsKwargs]
(APIServer pid=3167402) For further information visit https://errors.pydantic.dev/2.12/v/value_error

However, this is the output of ray status showing 12 GPUs available:

ray status
======== Autoscaler status: 2025-11-28 07:01:38.452724 ========
Node status

Active:
1 node_5bc3651f81e58183465936ce830e1197646fbeca8a72b261d79a0b17
1 node_6e0cdeabd00bec7a6039f1745ddbeb6aa2a6592fcd4feb7e971b2bdb
Pending:
(no pending nodes)
Recent failures:
(no failures)

Resources

Total Usage:
0.0/40.0 CPU
0.0/12.0 GPU

[devonthomas35]

Still not working for me either