[bugfix] support text + audio mixed output

[GG-li]

PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS (AT THE BOTTOM) HAVE BEEN CONSIDERED.

Purpose

As mentioned in #540 ,if we set output modality is audio, the output will be text + audio.

Test Plan

Server:

vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --omni --port 8888

Curl:

curl http://localhost:8888/v1/chat/completions -H "Content-Type: application/json" -d '{"model": "Qwen/Qwen3-Omni-30B-A3B-Instruct", "messages": [{"role": "user", "content": "Describe vLLM in brief."}], "modalities": ["audio"]}'

Test Result

Output before fix:

{"id":"chatcmpl-a6a266195a78440a","object":"chat.completion","created":1767754518,"model":"Qwen/Qwen3-Omni-30B-A3B-Instruct","choices":[{"index":0,"message":{"role":"assistant","content":"vLLM is an open-source library designed for efficient inference and serving of large language models (LLMs). It optimizes memory usage and computational efficiency through techniques like PagedAttention, which reduces memory fragmentation and improves throughput by managing attention keys and values more effectively. vLLM supports popular LLM architectures such as LLaMA, LLaMA2, and others, and enables high-throughput, low-latency model serving with support for continuous batching and dynamic scheduling. Its primary goal is to accelerate LLM inference while minimizing resource consumption, making it ideal for deployment in production environments.","refusal":null,"annotations":null,"audio":null,"function_call":null,"tool_calls":[],"reasoning":null,"reasoning_content":null},"logprobs":null,"finish_reason":"stop","stop_reason":null,"token_ids":null},{"index":0,"message":{"role":"assistant","content":null,"refusal":null,"annotations":null,"audio":{"id":"audio-e1b491c019cb4736","data":"UklGRngxIABX......JAAgABQAIAAYABwA=","expires_at":1767840988,"transcript":""},"function_call":null,"tool_calls":[],"reasoning":null,"reasoning_content":null},"logprobs":null,"finish_reason":"stop","stop_reason":null,"token_ids":null}],"service_tier":null,"system_fingerprint":null,"usage":{"prompt_tokens":15,"total_tokens":134,"completion_tokens":119,"prompt_tokens_details":null},"prompt_logprobs":null,"prompt_token_ids":null,"kv_transfer_params":null}

Output After Fix :

{"id":"chatcmpl-9434083195f9255f","object":"chat.completion","created":1767750392,"model":"Qwen/Qwen3-Omni-30B-A3B-Instruct","choices":[{"index":0,"message":{"role":"assistant","content":null,"refusal":null,"annotations":null,"audio":{"id":"audio-79963b2733804118","data":"UklGRnjyIgBXQV....../+//3/+//9//3////+//z//f/+////+//6//3////9//r/+v/8//n/+f/3//r/+v/7//z/+//4//n/+//6//n/9//8//v/+P/5//z//f/4//n/9//0//X/+f/5//f/9v/2//b/9/8=","expires_at":1767836939,"transcript":""},"function_call":null,"tool_calls":[],"reasoning":null,"reasoning_content":null},"logprobs":null,"finish_reason":"stop","stop_reason":null,"token_ids":null}],"service_tier":null,"system_fingerprint":null,"usage":{"prompt_tokens":0,"total_tokens":0,"completion_tokens":0,"prompt_tokens_details":null},"prompt_logprobs":null,"prompt_token_ids":null,"kv_transfer_params":null}

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft.

BEFORE SUBMITTING, PLEASE READ https://github.com/vllm-project/vllm-omni/blob/main/CONTRIBUTING.md (anything written below this line will be removed by GitHub Actions)

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 966ebdf103

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Preserve usage when filtering modalities

For non‑streaming requests that set modalities to exclude text (e.g., [“audio”]), this filter skips the text omni_outputs, which is the only path that populates usage/prompt_token_ids/kv_transfer_params. The response then returns zero usage even though tokens were consumed, while streaming still reports prompt tokens, so clients that depend on usage will see a regression. Consider deriving usage from omni_outputs.request_output even when text output is filtered.

Useful? React with 👍 / 👎.

[hsliuustc0106]

@fake0fan PTAL

[fake0fan]

Okay, I understand. However, if I really want to output both audio and text, how should I configure it? Should I set it to text+audio? I haven't found any related attempts in the current code. Could you try it?

[GG-li]

Okay, I understand. However, if I really want to output both audio and text, how should I configure it? Should I set it to text+audio? I haven't found any related attempts in the current code. Could you try it?

OK, I understand

[GG-li]

Okay, I understand. However, if I really want to output both audio and text, how should I configure it? Should I set it to ? I haven't found any related attempts in the current code. Could you try it?text+audio

I find no modifications are required; simply set "modalities": ["text","audio"]. Just like:

curl http://localhost:8888/v1/chat/completions -H "Content-Type: application/json" -d '{"model": "Qwen/Qwen3-Omni-30B-A3B-Instruct", "messages": [{"role": "user", "content": "Describe vLLM in brief."}], "modalities": ["text","audio"]}' 

[fake0fan]

LGTM

[david6666666]

LGTM