[Test] Add full test for Qwen3-Omni-30B-A3B-Instruct by yenuo26 · Pull Request #720 · vllm-project/vllm-omni

yenuo26 · 2026-01-09T09:59:33Z

PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS (AT THE BOTTOM) HAVE BEEN CONSIDERED.

Purpose

This PR is intended to add full test for Qwen3-Omni-30B-A3B-Instruct
design and plan, please refer to the #723

Test Plan

python -m pytest -sv tests/e2e/online_serving/test_qwen3_omni_expansion.py::test_text_to_text_001
python -m pytest -sv tests/e2e/online_serving/test_qwen3_omni_expansion.py::test_text_to_text_audio_001

Test Result

2 passed, 2 warnings in 299.74s (0:04:59)

Essential Elements of an Effective PR Description Checklist

The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
The test plan, such as providing test command.
The test results, such as pasting the results comparison before and after, or e2e results
(Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
(Optional) Release notes update. If your change is user facing, please update the release notes draft.

BEFORE SUBMITTING, PLEASE READ https://github.com/vllm-project/vllm-omni/blob/main/CONTRIBUTING.md (anything written below this line will be removed by GitHub Actions)

Signed-off-by: wangyu31577 <[email protected]>

… full_test

Signed-off-by: wangyu31577 <[email protected]>

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3023809d85

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2026-01-09T10:04:00Z

tests/conftest.py

+import psutil
 import pytest
 import torch
+import whisper
+import yaml


Avoid import-time dependency on whisper/psutil

Because tests/conftest.py is imported for every test run, the top-level import whisper/import psutil will raise ImportError in any environment that doesn’t preinstall those packages, causing test collection to fail even when the new e2e tests aren’t selected. The repo’s pyproject.toml doesn’t declare these dependencies, so a default dev/CI setup is likely to hit this; consider moving the imports inside the helper functions/tests that need them or adding the packages to test/dev extras.

Useful? React with 👍 / 👎.

@yenuo26 please pay attention to AI's review, this could help us improve efficiency and avoid obvious mistakes

@yenuo26 please pay attention to AI's review, this could help us improve efficiency and avoid obvious mistakes

get

chatgpt-codex-connector · 2026-01-09T10:04:00Z

tests/conftest.py

+    output_path = f"{yaml_path.split('.')[0]}_{int(time.time())}.yaml"
+    with open(output_path, "w", encoding="utf-8") as f:


Preserve dotted paths when writing modified YAML

Building the output filename with yaml_path.split('.')[0] drops everything after the first dot in the entire path, so a path like /tmp/.config/stage.yaml or config.v1.yaml will be rewritten into a different directory/name (e.g., /tmp/_<ts>.yaml), which can overwrite the wrong file or fail to be found later. Using Path(yaml_path).with_name(Path(yaml_path).stem + f"_{ts}").with_suffix(".yaml") (or similar) preserves the original directory and base name.

Useful? React with 👍 / 👎.

tzhouam · 2026-01-15T16:01:57Z

Should we consider a better way to manage variables such as concurrency size and message type? Hardcoding them inside the function impacts maintainability and increases the risk of missing or duplication in our test coverage.

Signed-off-by: wangyu31577 <[email protected]>

yenuo26 · 2026-01-16T08:39:20Z

Should we consider a better way to manage variables such as concurrency size and message type? Hardcoding them inside the function impacts maintainability and increases the risk of missing or duplication in our test coverage.

I have added the following two functions to solve this problem.

def get_prompt(prompt_type="text_only"):
    prompts = {
        "text_only": "What is the capital of China?",
        "mix": "What is recited in the audio? What is in this image? Describe the video briefly.",
    }
    return prompts.get(prompt_type, prompts["text_only"])


def get_max_batch_size(size_type="few"):
    batch_sizes = {"few": 5, "medium": 100, "large": 256}
    return batch_sizes.get(size_type, 5)

tzhouam

LGTM

Signed-off-by: wangyu31577 <[email protected]>

hsliuustc0106 · 2026-01-16T10:40:31Z

pyproject.toml

    "pytest-cov>=4.0.0",
    "mypy==1.11.1",
    "pre-commit==4.0.1",
+    "openai-whisper"


any version?

Signed-off-by: wangyu31577 <[email protected]>

hsliuustc0106

lgtm, thx

Signed-off-by: wangyu31577 <[email protected]> Co-authored-by: wangyu31577 <[email protected]> Signed-off-by: Chen Yang <[email protected]>

Signed-off-by: wangyu31577 <[email protected]> Co-authored-by: wangyu31577 <[email protected]>

wangyu31577 and others added 9 commits January 8, 2026 21:22

add full test

ac2e967

Signed-off-by: wangyu31577 <[email protected]>

add label

2a513dd

Signed-off-by: wangyu31577 <[email protected]>

add full test

356786a

Signed-off-by: wangyu31577 <[email protected]>

add full test

a89948a

Signed-off-by: wangyu31577 <[email protected]>

add full test

1c106d5

Signed-off-by: wangyu31577 <[email protected]>

Merge branch 'vllm-project:main' into full_test

30ef18d

add full test

290e5c7

Signed-off-by: wangyu31577 <[email protected]>

Merge branch 'full_test' of https://github.com/yenuo26/vllm-omni into…

1d2ba89

… full_test

pre-commit

3023809

Signed-off-by: wangyu31577 <[email protected]>

chatgpt-codex-connector bot reviewed Jan 9, 2026

View reviewed changes

yenuo26 changed the title ~~Add full test for Qwen3-Omni-30B-A3B-Instruct~~ [Test] Add full test for Qwen3-Omni-30B-A3B-Instruct Jan 9, 2026

yenuo26 mentioned this pull request Jan 9, 2026

[RFC]: Add E2E full test for Qwen3-Omni-30B-A3B-Instruct #723

Open

12 tasks

Merge branch 'vllm-project:main' into full_test

38863b1

ZJY0516 requested review from Gaohan123 and hsliuustc0106 January 11, 2026 02:35

hsliuustc0106 requested a review from tzhouam January 11, 2026 03:15

david6666666 mentioned this pull request Jan 14, 2026

[Feature]: E2E Testing: Add coverage for different modal inputs/outputs (Diffusion vs. AR models). JiusiServe/vllm-omni#68

Open

4 tasks

david6666666 added this to the v0.14.0rc1 milestone Jan 14, 2026

yenuo26 and others added 2 commits January 16, 2026 16:22

Merge branch 'vllm-project:main' into full_test

59faf3b

Add prompt and batch size getter functions

b37dfca

Signed-off-by: wangyu31577 <[email protected]>

tzhouam approved these changes Jan 16, 2026

View reviewed changes

tzhouam added the ready label to trigger buildkite CI label Jan 16, 2026

wangyu31577 added 2 commits January 16, 2026 18:30

add dependencies

f6e46e6

Signed-off-by: wangyu31577 <[email protected]>

add dependencies

35b9c1c

Signed-off-by: wangyu31577 <[email protected]>

hsliuustc0106 reviewed Jan 16, 2026

View reviewed changes

add dependencies version

eb41de2

Signed-off-by: wangyu31577 <[email protected]>

hsliuustc0106 approved these changes Jan 16, 2026

View reviewed changes

hsliuustc0106 merged commit 4e23bff into vllm-project:main Jan 16, 2026
7 checks passed

with1015 pushed a commit to with1015/vllm-omni that referenced this pull request Jan 20, 2026

[Test] Add full test for Qwen3-Omni-30B-A3B-Instruct (vllm-project#720)

1b07c2e

Signed-off-by: wangyu31577 <[email protected]> Co-authored-by: wangyu31577 <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

[Test] Add full test for Qwen3-Omni-30B-A3B-Instruct#720

[Test] Add full test for Qwen3-Omni-30B-A3B-Instruct#720
hsliuustc0106 merged 15 commits intovllm-project:mainfrom
yenuo26:full_test

yenuo26 commented Jan 9, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector bot left a comment

Uh oh!

chatgpt-codex-connector bot Jan 9, 2026

Uh oh!

hsliuustc0106 Jan 16, 2026

Uh oh!

yenuo26 Jan 16, 2026

Uh oh!

chatgpt-codex-connector bot Jan 9, 2026

Uh oh!

tzhouam commented Jan 15, 2026

Uh oh!

yenuo26 commented Jan 16, 2026

Uh oh!

tzhouam left a comment

Uh oh!

hsliuustc0106 Jan 16, 2026

Uh oh!

yenuo26 Jan 16, 2026

Uh oh!

hsliuustc0106 left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		output_path = f"{yaml_path.split('.')[0]}_{int(time.time())}.yaml"
		with open(output_path, "w", encoding="utf-8") as f:

Comments

Conversation

yenuo26 commented Jan 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Test Plan

Test Result

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Jan 9, 2026

Choose a reason for hiding this comment

Uh oh!

hsliuustc0106 Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

yenuo26 Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot Jan 9, 2026

Choose a reason for hiding this comment

Uh oh!

tzhouam commented Jan 15, 2026

Uh oh!

yenuo26 commented Jan 16, 2026

Uh oh!

tzhouam left a comment

Choose a reason for hiding this comment

Uh oh!

hsliuustc0106 Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

yenuo26 Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

hsliuustc0106 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

yenuo26 commented Jan 9, 2026 •

edited

Loading