[Misc] Add stage config for Qwen3-Omni-30B-A3B-Thinking by linyueqian · Pull Request #172 · vllm-project/vllm-omni

linyueqian · 2025-12-03T03:44:32Z

Purpose

Add a single-stage configuration example for Qwen3-Omni-MoE-Thinking models (e.g., Qwen3-Omni-30B-A3B-Thinking) that only
have the thinker component and produce text-only output (no audio synthesis).

Test Plan

N/A (config file only)

Test Result

Verified on 2x H200 GPUs with tensor_parallel_size=2.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2025-12-03T03:48:11Z

vllm_omni/model_executor/stage_configs/qwen3_omni_moe_thinking.yaml

+      engine_output_type: text
+      distributed_executor_backend: "mp"
+      enable_prefix_caching: false
+      hf_config_name: thinker_config


Avoid nested thinker_config for Thinking checkpoints

This YAML sets hf_config_name: thinker_config, which makes OmniModelConfig.draw_hf_text_config (vllm_omni/config/model.py:79-85) dereference hf_config.thinker_config before building the model. The Qwen3-Omni-*Thinking checkpoints you are targeting only ship the thinker config itself (Qwen3OmniMoeThinkerConfig) and do not wrap it in a thinker_config attribute, so loading this stage file against those models will raise AttributeError and the config cannot be used. Drop the hf_config_name indirection (and use the thinker architecture) so thinker-only checkpoints load successfully.

Useful? React with 👍 / 👎.

ywang96

Actually I do have a question - looks like right now we're using model_type for huggingface to identify the stage config yaml.

vllm-omni/vllm_omni/entrypoints/utils.py

Lines 41 to 47 in 574e1fb

    
           # Fall back to default config 
        
           stage_config_file = f"vllm_omni/model_executor/stage_configs/{model_type}.yaml" 
        
           stage_config_path = PROJECT_ROOT / stage_config_file 
        
           if not os.path.exists(stage_config_path): 
        
               raise FileNotFoundError(f"Stage config file {stage_config_path} not found") 
        
           stage_configs = load_stage_configs_from_yaml(config_path=str(stage_config_path)) 
        
           return stage_configs

How does this work for this model? qwen3_omni_moe_thinking isn't a valid model_type right? https://huggingface.co/Qwen/Qwen3-Omni-30B-A3B-Thinking/blob/main/config.json#L10

linyueqian · 2025-12-03T06:28:38Z

Actually I do have a question - looks like right now we're using model_type for huggingface to identify the stage config yaml.

vllm-omni/vllm_omni/entrypoints/utils.py

Lines 41 to 47 in 574e1fb

# Fall back to default config

stage_config_file = f"vllm_omni/model_executor/stage_configs/{model_type}.yaml"

stage_config_path = PROJECT_ROOT / stage_config_file

if not os.path.exists(stage_config_path):

raise FileNotFoundError(f"Stage config file {stage_config_path} not found")

stage_configs = load_stage_configs_from_yaml(config_path=str(stage_config_path))

return stage_configs

How does this work for this model? qwen3_omni_moe_thinking isn't a valid model_type right? https://huggingface.co/Qwen/Qwen3-Omni-30B-A3B-Thinking/blob/main/config.json#L10

i add a small check in the utils.py. would that work?

ywang96

I think this is a reasonable change for now! Please fix the pre-commit though

Gaohan123

Is it possible to use think mode to take end2end generation for audio?

Gaohan123 · 2025-12-03T14:40:35Z

vllm_omni/entrypoints/utils.py

+    # (no talker/code2wav configs) but reuse the base qwen3_omni_moe model_type.
+    # Detect this using multiple hints so users don't need to manually rewrite
+    # the stage config path.
+    is_qwen3_omni_moe_thinking = (


Is it possible to set up just in stage config? Here it is a little bit model specific in general utils.

If we only add the YAML without this routing logic, vLLM will automatically pick qwen3_omni_moe.yaml due to the shared model_type. The user would then be forced to explicitly pass --stage-config vllm_omni/.../qwen3_omni_moe_thinking.yaml every time.

I understand your concern about polluting utils.py with model-specific code. Could you point me to a better place to insert this auto-detection?"

I think it is totally ok to add a custom config file in examples. After all, the folder stage_configs is just for default setting.

I have moved it to examples folder.

Gaohan123

I think it is good. Please use git commit -s to pass the DCO check. Then I will help to merge. Thanks!

linyueqian · 2025-12-04T15:35:48Z

@Gaohan123 I have added DCO sign-offs. Thanks!

Add a single-stage configuration example for Qwen3-Omni-MoE-Thinking models that only have the thinker component (text-only output, no audio synthesis). Signed-off-by: linyueqian <[email protected]>

Signed-off-by: linyueqian <[email protected]>

…#172) Signed-off-by: linyueqian <[email protected]> Signed-off-by: Prajwal A <[email protected]>

…#172) Signed-off-by: linyueqian <[email protected]> Signed-off-by: Fanli Lin <[email protected]>

…#172) Signed-off-by: linyueqian <[email protected]>

linyueqian requested a review from hsliuustc0106 as a code owner December 3, 2025 03:44

chatgpt-codex-connector bot reviewed Dec 3, 2025

View reviewed changes

ywang96 approved these changes Dec 3, 2025

View reviewed changes

ywang96 reviewed Dec 3, 2025

View reviewed changes

ywang96 approved these changes Dec 3, 2025

View reviewed changes

Gaohan123 reviewed Dec 3, 2025

View reviewed changes

Gaohan123 reviewed Dec 4, 2025

View reviewed changes

linyueqian added 5 commits December 4, 2025 11:33

[Misc] Add stage config for Qwen3-Omni-30B-A3B-Thinking

c4680d5

Add a single-stage configuration example for Qwen3-Omni-MoE-Thinking models that only have the thinker component (text-only output, no audio synthesis). Signed-off-by: linyueqian <[email protected]>

Route thinking-only checkpoints to stage config

7eb0a9b

Signed-off-by: linyueqian <[email protected]>

Handle Qwen3 Omni thinking config routing

ad650c9

Signed-off-by: linyueqian <[email protected]>

Move Qwen3 thinking example config and drop routing override

729b5d7

Signed-off-by: linyueqian <[email protected]>

Move Qwen3 thinking example config and drop routing override

0f87094

Signed-off-by: linyueqian <[email protected]>

linyueqian force-pushed the add-qwen3-omni-moe-thinking-config branch from 6543b74 to 0f87094 Compare December 4, 2025 16:34

Merge branch 'main' into add-qwen3-omni-moe-thinking-config

ed81aae

ywang96 enabled auto-merge (squash) December 4, 2025 16:41

ywang96 disabled auto-merge December 4, 2025 16:41

ywang96 merged commit 1406c6e into vllm-project:main Dec 4, 2025
2 of 4 checks passed

LawJarp-A pushed a commit to LawJarp-A/vllm-omni that referenced this pull request Dec 12, 2025

[Misc] Add stage config for Qwen3-Omni-30B-A3B-Thinking (vllm-project…

cba2ea3

…#172) Signed-off-by: linyueqian <[email protected]> Signed-off-by: Prajwal A <[email protected]>

LawJarp-A pushed a commit to LawJarp-A/vllm-omni that referenced this pull request Dec 12, 2025

[Misc] Add stage config for Qwen3-Omni-30B-A3B-Thinking (vllm-project…

29a4b34

…#172) Signed-off-by: linyueqian <[email protected]> Signed-off-by: Prajwal A <[email protected]>

linyueqian deleted the add-qwen3-omni-moe-thinking-config branch December 16, 2025 01:50

faaany pushed a commit to faaany/vllm-omni that referenced this pull request Dec 19, 2025

[Misc] Add stage config for Qwen3-Omni-30B-A3B-Thinking (vllm-project…

2f19846

…#172) Signed-off-by: linyueqian <[email protected]> Signed-off-by: Fanli Lin <[email protected]>

princepride pushed a commit to princepride/vllm-omni that referenced this pull request Jan 10, 2026

[Misc] Add stage config for Qwen3-Omni-30B-A3B-Thinking (vllm-project…

608b51e

…#172) Signed-off-by: linyueqian <[email protected]>

	# Fall back to default config
	stage_config_file = f"vllm_omni/model_executor/stage_configs/{model_type}.yaml"
	stage_config_path = PROJECT_ROOT / stage_config_file
	if not os.path.exists(stage_config_path):
	raise FileNotFoundError(f"Stage config file {stage_config_path} not found")
	stage_configs = load_stage_configs_from_yaml(config_path=str(stage_config_path))
	return stage_configs

Comments

Conversation

linyueqian commented Dec 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Test Plan

Test Result

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Dec 3, 2025

Choose a reason for hiding this comment

Uh oh!

ywang96 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

linyueqian commented Dec 3, 2025

Uh oh!

ywang96 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Gaohan123 left a comment

Choose a reason for hiding this comment

Uh oh!

Gaohan123 Dec 3, 2025

Choose a reason for hiding this comment

Uh oh!

linyueqian Dec 3, 2025

Choose a reason for hiding this comment

Uh oh!

Gaohan123 Dec 4, 2025

Choose a reason for hiding this comment

Uh oh!

linyueqian Dec 4, 2025

Choose a reason for hiding this comment

Uh oh!

Gaohan123 left a comment

Choose a reason for hiding this comment

Uh oh!

linyueqian commented Dec 4, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

linyueqian commented Dec 3, 2025 •

edited

Loading

ywang96 left a comment •

edited

Loading

ywang96 left a comment •

edited

Loading