Skip to content

[bugfix] support text + audio mixed output #843

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
david6666666 merged 6 commits into from
Jan 21, 2026
+10 −0
Merged

[bugfix] support text + audio mixed output #843

Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
966ebdf
修复
GG-li Jan 19, 2026
761e5a6
Merge branch 'main' into bugfix-540
GG-li Jan 20, 2026
da79e2c
Merge branch 'main' into bugfix-540
hsliuustc0106 Jan 20, 2026
c4d8c0c
Merge branch 'main' into bugfix-540
GG-li Jan 21, 2026
8fa561c
Merge branch 'main' into bugfix-540
GG-li Jan 21, 2026
5f477e0
Merge branch 'main' into bugfix-540
GG-li Jan 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions vllm_omni/entrypoints/openai/serving_chat.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1324,11 +1324,21 @@ async def chat_completion_full_generator(
prompt_token_ids = None
kv_transfer_params = None

# Build requested modalities set for filtering
requested_modalities = (
set(request.modalities) if hasattr(request, "modalities") and request.modalities else None
)

for omni_outputs in final_outputs:
choices_data = []
if omni_outputs.request_output is not None and not getattr(omni_outputs.request_output, "finished", False):
continue

# Filter outputs based on requested modalites
if requested_modalities is not None and omni_outputs.final_output_type not in requested_modalities:
logger.warning(f"final output type: {omni_outputs.final_output_type} is not needed by the request")
continue
Comment on lines +1337 to +1340
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot Jan 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve usage when filtering modalities

For non‑streaming requests that set modalities to exclude text (e.g., [“audio”]), this filter skips the text omni_outputs, which is the only path that populates usage/prompt_token_ids/kv_transfer_params. The response then returns zero usage even though tokens were consumed, while streaming still reports prompt tokens, so clients that depend on usage will see a regression. Consider deriving usage from omni_outputs.request_output even when text output is filtered.

Useful? React with 👍 / 👎.


if omni_outputs.final_output_type == "text":
(
choices_data,
Expand Down