Skip to content

[Misc] Enable tensor_parallel_size argument with online serving cmd#761

Merged
Gaohan123 merged 2 commits intovllm-project:mainfrom
JustQJ:online-tp-cmd
Jan 13, 2026
Merged

[Misc] Enable tensor_parallel_size argument with online serving cmd#761
Gaohan123 merged 2 commits intovllm-project:mainfrom
JustQJ:online-tp-cmd

Conversation

@JustQJ
Copy link
Contributor

@JustQJ JustQJ commented Jan 13, 2026

PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS (AT THE BOTTOM) HAVE BEEN CONSIDERED.

Purpose

Enable tensor-parallel-size argument for online serving when running Z-Image with TP=2

MODEL="${MODEL:-Tongyi-MAI/Z-Image-Turbo}"
PORT="${PORT:-8091}"
vllm serve "$MODEL" --omni \
    --port "$PORT" \
    --tensor-parallel-size 2

Test Plan

Test Result


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft.

BEFORE SUBMITTING, PLEASE READ https://github.com/vllm-project/vllm-omni/blob/main/CONTRIBUTING.md (anything written below this line will be removed by GitHub Actions)

@JustQJ JustQJ requested a review from hsliuustc0106 as a code owner January 13, 2026 04:41
@hsliuustc0106 hsliuustc0106 requested a review from ZJY0516 January 13, 2026 04:43
@JustQJ JustQJ changed the title Enable tensor_parallel_size argument with online serving cmd [Misc] Enable tensor_parallel_size argument with online serving cmd Jan 13, 2026
@ZJY0516
Copy link
Collaborator

ZJY0516 commented Jan 13, 2026

thanks for contribution. could you please also add enforce-eager too? it was introduced in #684 and added to online serving cmd

@JustQJ
Copy link
Contributor Author

JustQJ commented Jan 13, 2026

thanks for contribution. could you please also add enforce-eager too? it was introduced in #684 and added to online serving cmd

done

@ZJY0516 ZJY0516 added the ready label to trigger buildkite CI label Jan 13, 2026
Copy link
Collaborator

@Gaohan123 Gaohan123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks

@Gaohan123 Gaohan123 merged commit 9cdf592 into vllm-project:main Jan 13, 2026
7 checks passed
sniper35 pushed a commit to sniper35/vllm-omni that referenced this pull request Jan 14, 2026
with1015 pushed a commit to with1015/vllm-omni that referenced this pull request Jan 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready label to trigger buildkite CI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants