[BugFix] Fix Qwen3 TTS 0.6B profile run hang (#995)

[marksverdhei]

Purpose

Fix Qwen3-TTS-12Hz-0.6B-Base hanging during server startup (profile/warmup run).

Resolves #995

Root Cause

During vLLM's profile/warmup run, forward() is called with dummy token IDs and empty runtime_additional_information. For the Base task type, this triggers generate_voice_clone() in ICL mode with degenerate inputs (1-second silent audio clip, placeholder ref_text). The 0.6B model cannot converge from this input and generates indefinitely, never producing an EOS token. The EngineCore times out waiting for the worker to finish, emitting:

No available shared memory broadcast block found in 60 seconds.

The 1.7B models are robust enough to produce EOS from the same degenerate input, which is why only 0.6B is affected.

Fix

Cap max_new_tokens to 2 when text is empty (profile run detection), so the full generation pipeline executes — preserving KV-cache profiling behaviour — but exits quickly even when the model cannot converge from degenerate dummy inputs.

Test Plan

# 1. Build Docker image
docker build -t vllm-omni-q3tts:latest -f docker/Dockerfile.ci .

# 2. Start server with 0.6B Base model
docker run -d \
  --name qwen3-tts-server \
  --gpus '"device=0"' \
  -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm-omni-q3tts:latest \
  vllm serve Qwen/Qwen3-TTS-12Hz-0.6B-Base \
    --omni --host 0.0.0.0 --port 8000 \
    --gpu-memory-utilization 0.9 --enforce-eager

# 3. Verify server starts (previously hung here)
curl http://localhost:8000/health

# 4. Send a TTS request
curl -X POST http://localhost:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Hello, this is a test.",
    "model": "Qwen/Qwen3-TTS-12Hz-0.6B-Base",
    "task_type": "Base",
    "language": "English",
    "ref_audio": "data:audio/wav;base64,<base64_encoded_wav>",
    "ref_text": "Reference text for the audio.",
    "x_vector_only_mode": true,
    "max_new_tokens": 256
  }' -o output.wav

Test Result

Tested on RTX 3090 (Ampere, sm_86), Docker, vllm-omni v0.14.0rc1:

• Before fix: Server hangs at No available shared memory broadcast block found in 60 seconds — never starts
• After fix: Profile run completes with capped generation (max_new_tokens=2), full pipeline executes, server starts successfully
• 0.6B Base inference: HTTP 200, produces valid 3.34s non-silent 24kHz audio in 2.9s (x_vector_only voice cloning mode)
• Approach: Generation step cap preserves full pipeline execution during warmup (KV-cache profiling compatible) while bounding degenerate generation

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft.

BEFORE SUBMITTING, PLEASE READ https://github.com/vllm-project/vllm-omni/blob/main/CONTRIBUTING.md

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6eba044eea

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[hsliuustc0106]

@linyueqian PTAL

[tzhouam]

Hello, thank you for your effort and contribution.

I reviewed your code and design, and I’m concerned that the “shortcut forward” path could lead to unexpected behavior in KV-cache management, especially if we adopt vLLM’s paged attention in the future.

Would using a dummy input text help avoid this issue?

[Copilot]

Pull request overview

This PR fixes a critical hang issue where the Qwen3-TTS-12Hz-0.6B-Base model would indefinitely generate during vLLM's profile/warmup run at server startup. The root cause is that during profile runs, the forward() method receives empty text input and degenerate runtime information, causing the 0.6B model to fail to converge and never produce an EOS token, leading to a timeout.

Changes:

• Short-circuits the forward() method in Qwen3TTSModelForGeneration when text input is empty
• Returns a dummy 1-second silent audio tensor immediately during profile runs
• Prevents the 0.6B model from hanging during server startup while maintaining compatibility with larger models

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[marksverdhei]

Thanks for the feedback @tzhouam — that's a valid concern. I tested the dummy text approach: substituting text = "Hello." when empty so the full pipeline executes. Unfortunately it still hangs, because during profile runs it's not only text that's missing — _dummy_run() calls the model without runtime_additional_information in kwargs at all, so ref_audio, ref_text, and all other fields also fall back to degenerate defaults (e.g. a 1-second silent audio clip for ref_audio). The 0.6B model can't converge from that combination.

I'm looking into supplying valid dummy values for all the missing inputs so the full pipeline can execute during warmup, but that may be a larger change. What approach would you prefer here? Some options:

1. Construct realistic dummy runtime_additional_information at the runner level so the model gets valid inputs during profile runs
2. Add a generation step cap (e.g. max_new_tokens) during profile runs to bound execution even if the model doesn't converge
3. Keep the current short-circuit as a stopgap while working on a more complete fix

Happy to go whichever direction works best for the project.

[Gaohan123]

Thanks for the feedback @tzhouam — that's a valid concern. I tested the dummy text approach: substituting text = "Hello." when empty so the full pipeline executes. Unfortunately it still hangs, because during profile runs it's not only text that's missing — _dummy_run() calls the model without runtime_additional_information in kwargs at all, so ref_audio, ref_text, and all other fields also fall back to degenerate defaults (e.g. a 1-second silent audio clip for ref_audio). The 0.6B model can't converge from that combination.

I'm looking into supplying valid dummy values for all the missing inputs so the full pipeline can execute during warmup, but that may be a larger change. What approach would you prefer here? Some options:

1. Construct realistic dummy runtime_additional_information at the runner level so the model gets valid inputs during profile runs
2. Add a generation step cap (e.g. max_new_tokens) during profile runs to bound execution even if the model doesn't converge
3. Keep the current short-circuit as a stopgap while working on a more complete fix

Happy to go whichever direction works best for the project.

Currently, I think choice 2 is better. Thanks.

[marksverdhei]

Applied and tested the approach.

[Gaohan123]

Could you please check the CI error? Thanks

[marksverdhei]

Could you please check the CI error? Thanks

Not obvious to me that the fail was triggered by this patch. I'm not entirely sure...

[marksverdhei]

Could you please check the CI error? Thanks

Not obvious to me that the fail was triggered by this patch. I'm not entirely sure...

[Automated analysis] I reproduced the CI environment and investigated both failures. Neither appears to be caused by this PR's changes:

1. test_qwen3_omni_talker — Fails with torch._dynamo.exc.BackendCompilerFailed during torch.compile. This is a pre-existing issue with torch.compile + Qwen3 Omni talker that is already tracked and was fixed independently in PR [BugFix] Fix Qwen3 Omni talker mtp torch.compile startup error #1104 ([BugFix] Fix Qwen3 Omni talker mtp torch.compile startup error). The failing code path is in torch._inductor and unrelated to the profile-run changes in this PR.

2. test_qwen3_tts — Fails with an assertion on audio duration: assert 2 <= duration <= 20 where duration ≈ 0.02s. This is an inference quality issue — the model produces near-silent output for the test prompt. This test also fails on main (confirmed by checking the base branch CI), so it is a flaky/pre-existing failure unrelated to this PR.

In summary, both CI failures are reproducible on main without this PR's changes.

---

This is an automated message generated by a CI analysis tool.

[hsliuustc0106]

lgtm

[hsliuustc0106]

@Gaohan123 PTAL

[Gaohan123]

LGTM. Thanks!