Skip to content

[BugFix] Fix DP headless mode arg validation #20398

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 3, 2025
Merged

[BugFix] Fix DP headless mode arg validation #20398

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions vllm/entrypoints/cli/serve.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -119,17 +119,17 @@ def run_headless(args: argparse.Namespace):
if not envs.VLLM_USE_V1:
raise ValueError("Headless mode is only supported for V1")

if engine_args.data_parallel_rank is not None:
raise ValueError("data_parallel_rank is not applicable in "
"headless mode")
Comment on lines +122 to +124
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Jul 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

Raising a ValueError here prevents the program from running with an invalid configuration. This is a critical check to ensure that the headless mode is used correctly and to avoid unexpected behavior or errors during runtime. Consider adding a more descriptive error message to help users understand the issue and how to resolve it.

raise ValueError("The argument 'data_parallel_rank' is not applicable in headless mode. Please ensure it is not set when running in headless mode.")


parallel_config = vllm_config.parallel_config
local_engine_count = parallel_config.data_parallel_size_local

if local_engine_count <= 0:
raise ValueError("data_parallel_size_local must be > 0 in "
"headless mode")

if parallel_config.data_parallel_rank is not None:
raise ValueError("data_parallel_rank is not applicable in "
"headless mode")

host = parallel_config.data_parallel_master_ip
port = engine_args.data_parallel_rpc_port # add to config too
handshake_address = get_tcp_uri(host, port)
Expand Down