[Test] Add example test cases for omni online#1086
Merged
hsliuustc0106 merged 11 commits intovllm-project:mainfrom Feb 9, 2026
Merged
[Test] Add example test cases for omni online#1086hsliuustc0106 merged 11 commits intovllm-project:mainfrom
hsliuustc0106 merged 11 commits intovllm-project:mainfrom
Conversation
Signed-off-by: wangyu31577 <[email protected]>
Signed-off-by: wangyu31577 <[email protected]>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5fdb4b1438
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
added 2 commits
January 29, 2026 22:27
Signed-off-by: wangyu31577 <[email protected]>
Signed-off-by: wangyu31577 <[email protected]>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds example pytest-based “online serving” test cases for Qwen Omni models (Qwen2.5-Omni and Qwen3-Omni), including a CI-oriented stage config and some test utility refactoring.
Changes:
- Added example online serving tests that execute the shipped client scripts/shell scripts and assert on returned text/audio.
- Added a Qwen2.5-Omni CI stage config YAML for multi-stage server startup.
- Refactored Whisper-based audio transcription helpers in
tests/conftest.pyand added an optionalportoverride toOmniServer.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 7 comments.
| File | Description |
|---|---|
| tests/examples/online_serving/test_qwen3_omni.py | New example online-serving tests for Qwen3-Omni using example clients and simple output assertions. |
| tests/examples/online_serving/test_qwen2_5_omni.py | New example online-serving tests for Qwen2.5-Omni using example clients and simple output assertions. |
| tests/e2e/stage_configs/qwen2_5_omni_ci.yaml | New multi-stage CI stage config for Qwen2.5-Omni. |
| tests/conftest.py | Refactors audio transcription helpers and adds optional port parameter support for OmniServer. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <[email protected]> Signed-off-by: Hongsheng Liu <[email protected]>
- Updated `convert_audio_file_to_text` to conditionally clear GPU cache based on availability. - Changed `port` type annotation in `OmniServer` class to support optional integer. - Modified `omni_server` fixture documentation to clarify its scope. - Enhanced `extract_content_after_keyword` function to raise an error if keywords are not found, improving error handling in tests. Signed-off-by: yenuo26 <[email protected]>
…sts and adjust timeout settings - Reduced timeout for "Omni Model Test with H100" from 180 to 60 minutes. - Added a new test step for "Omni Model Test" with specific commands and environment settings. - Included additional test script `test_qwen3_omni.py` in the H100 test step. Signed-off-by: yenuo26 <[email protected]>
Contributor
Author
|
@hsliuustc0106 @david6666666 Please help add the "ready" label. |
YanickSchraner
pushed a commit
to YanickSchraner/vllm-omni
that referenced
this pull request
Feb 20, 2026
Signed-off-by: wangyu31577 <[email protected]> Signed-off-by: Hongsheng Liu <[email protected]> Signed-off-by: yenuo26 <[email protected]> Co-authored-by: wangyu31577 <[email protected]> Co-authored-by: Hongsheng Liu <[email protected]> Co-authored-by: Copilot <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS (AT THE BOTTOM) HAVE BEEN CONSIDERED.
Purpose
Add example test cases for omni online
Test Plan
pytest -sv test_qwen2_5_omni.py --html=report.html --self-contained-html
pytest -sv test_qwen3_omni.py --html=report.html --self-contained-html
Test Result
tests/examples/online_serving/test_qwen2_5_omni.py::test_stream_001[omni_server0]::setup('/workspace/vllm-omni/tests/examples/online_serving/test_qwen2_5_omni.py', 239, 'Skipped: There is a known issue with stream error.')tests/examples/online_serving/test_qwen2_5_omni.py::test_send_multimodal_request_001[omni_server0]tests/examples/online_serving/test_qwen2_5_omni.py::test_send_multimodal_request_002[omni_server0]tests/examples/online_serving/test_qwen2_5_omni.py::test_send_multimodal_request_003[omni_server0]tests/examples/online_serving/test_qwen2_5_omni.py::test_modality_control_001[omni_server0]tests/examples/online_serving/test_qwen2_5_omni.py::test_modality_control_002[omni_server0]tests/examples/online_serving/test_qwen2_5_omni.py::test_modality_control_003[omni_server0]Summary:
⚠️ 1 Skipped
✅ 0 Failed
✅ 6 Passed
✅ 0 Expected failures
✅ 0 Unexpected passes
✅ 0 Errors
✅ 0 Reruns
tests/examples/online_serving/test_qwen_omni.py::test_send_multimodal_request_001[omni_server0]tests/examples/online_serving/test_qwen_omni.py::test_send_multimodal_request_002[omni_server]tests/examples/online_serving/test_qwen_omni.py::test_send_multimodal_request_003[omni_server]tests/examples/online_serving/test_qwen_omni.py::test_modality_control_001tests/examples/online_serving/test_qwen_omni.py::test_modality_control_002tests/examples/online_serving/test_qwen_omni.py::test_modality_control_003tests/examples/online_serving/test_qwen_omni.py::test_stream_001[omni_server]Summary:
⏱️ Total time: 00:06:50
✅ 7 Passed
✅ 0 Failed
✅ 0 Skipped
✅ 0 Expected failures
✅ 0 Unexpected passes
✅ 0 Errors
✅ 0 Reruns
BEFORE SUBMITTING, PLEASE READ https://github.com/vllm-project/vllm-omni/blob/main/CONTRIBUTING.md (anything written below this line will be removed by GitHub Actions)