Skip to content

[Bugfix][Qwen3-TTS] Add compatibility for both @check_model_inputs and @check_model_inputs().#965

Closed
gcanlin wants to merge 3 commits intovllm-project:mainfrom
gcanlin:check_inputs
Closed

[Bugfix][Qwen3-TTS] Add compatibility for both @check_model_inputs and @check_model_inputs().#965
gcanlin wants to merge 3 commits intovllm-project:mainfrom
gcanlin:check_inputs

Conversation

@gcanlin
Copy link
Contributor

@gcanlin gcanlin commented Jan 26, 2026 •
edited
Loading

PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS (AT THE BOTTOM) HAVE BEEN CONSIDERED.

Purpose

Fix #963.

Test Plan

python end2end.py --query-type Base
python end2end.py --query-type Base --use-batch-sample

Test Result

INFO 01-26 09:13:01 [log_utils.py:550] {'type': 'request_level_metrics',
INFO 01-26 09:13:01 [log_utils.py:550]  'request_id': '0_a52daf69-e539-4590-bc54-372bccd25e40',
INFO 01-26 09:13:01 [log_utils.py:550]  'e2e_time_ms': 8056.328058242798,
INFO 01-26 09:13:01 [log_utils.py:550]  'e2e_tpt': 287.72600208009993,
INFO 01-26 09:13:01 [log_utils.py:550]  'e2e_total_tokens': 28,
INFO 01-26 09:13:01 [log_utils.py:550]  'transfers_total_time_ms': 0.0,
INFO 01-26 09:13:01 [log_utils.py:550]  'transfers_total_bytes': 0,
INFO 01-26 09:13:01 [log_utils.py:550]  'stages': {0: {'stage_gen_time_ms': 7991.328954696655,
INFO 01-26 09:13:01 [log_utils.py:550]                 'num_tokens_out': 0,
INFO 01-26 09:13:01 [log_utils.py:550]                 'num_tokens_in': 28}}}
Processed prompts: 100%|███████████████████████████████████| 1/1 [00:08<00:00,  8.06s/req, est. speed stage-0 tok/s: 3.48, avg e2e_lat: 0.0ms]
INFO 01-26 09:13:01 [omni.py:840] [Summary] {'e2e_requests': 1,1 [00:08<00:00,  8.06s/req, est. speed stage-0 tok/s: 3.48, avg e2e_lat: 0.0ms]
INFO 01-26 09:13:01 [omni.py:840]  'e2e_total_time_ms': 8058.4917068481445,
INFO 01-26 09:13:01 [omni.py:840]  'e2e_sum_time_ms': 8056.328058242798,
INFO 01-26 09:13:01 [omni.py:840]  'e2e_total_tokens': 28,
INFO 01-26 09:13:01 [omni.py:840]  'e2e_avg_time_per_request_ms': 8056.328058242798,
INFO 01-26 09:13:01 [omni.py:840]  'e2e_avg_tokens_per_s': 3.475528776581028,
INFO 01-26 09:13:01 [omni.py:840]  'wall_time_ms': 8058.4917068481445,
INFO 01-26 09:13:01 [omni.py:840]  'final_stage_id': {'0_a52daf69-e539-4590-bc54-372bccd25e40': 0},
INFO 01-26 09:13:01 [omni.py:840]  'stages': [{'stage_id': 0,
INFO 01-26 09:13:01 [omni.py:840]              'requests': 1,
INFO 01-26 09:13:01 [omni.py:840]              'tokens': 28,
INFO 01-26 09:13:01 [omni.py:840]              'total_time_ms': 8056.776285171509,
INFO 01-26 09:13:01 [omni.py:840]              'avg_time_per_request_ms': 8056.776285171509,
INFO 01-26 09:13:01 [omni.py:840]              'avg_tokens_per_s': 3.4753354206364127}],
INFO 01-26 09:13:01 [omni.py:840]  'transfers': []}
Adding requests:   0%|                                                                                                  | 0/1 [00:08<?, ?it/s]
Request ID: 0_a52daf69-e539-4590-bc54-372bccd25e40, Saved audio to output_audio/output_0_a52daf69-e539-4590-bc54-372bccd25e40.wav

INFO 01-26 09:15:53 [log_utils.py:550] {'type': 'request_level_metrics',
INFO 01-26 09:15:53 [log_utils.py:550]  'request_id': '1_d4515d52-59f1-4cc9-ad81-feabd1827eff',
INFO 01-26 09:15:53 [log_utils.py:550]  'e2e_time_ms': 17422.70851135254,
INFO 01-26 09:15:53 [log_utils.py:550]  'e2e_tpt': 791.94129597057,
INFO 01-26 09:15:53 [log_utils.py:550]  'e2e_total_tokens': 22,
INFO 01-26 09:15:53 [log_utils.py:550]  'transfers_total_time_ms': 0.0,
INFO 01-26 09:15:53 [log_utils.py:550]  'transfers_total_bytes': 0,
INFO 01-26 09:15:53 [log_utils.py:550]  'stages': {0: {'stage_gen_time_ms': 8563.502788543701,
INFO 01-26 09:15:53 [log_utils.py:550]                 'num_tokens_out': 0,
INFO 01-26 09:15:53 [log_utils.py:550]                 'num_tokens_in': 22}}}
Processed prompts: 100%|████████████████████████████████| 2/2 [00:17<00:00,  8.71s/req, est. speed stage-0 tok/s: 2.87, avg e2e_lat: 8863.3ms]
INFO 01-26 09:15:53 [omni.py:840] [Summary] {'e2e_requests': 2,00:17<00:00,  8.68s/req, est. speed stage-0 tok/s: 2.87, avg e2e_lat: 8863.3ms]
INFO 01-26 09:15:53 [omni.py:840]  'e2e_total_time_ms': 17424.197912216187,
INFO 01-26 09:15:53 [omni.py:840]  'e2e_sum_time_ms': 26285.984754562378,
INFO 01-26 09:15:53 [omni.py:840]  'e2e_total_tokens': 50,
INFO 01-26 09:15:53 [omni.py:840]  'e2e_avg_time_per_request_ms': 13142.992377281189,
INFO 01-26 09:15:53 [omni.py:840]  'e2e_avg_tokens_per_s': 1.9021543406823156,
INFO 01-26 09:15:53 [omni.py:840]  'wall_time_ms': 17424.197912216187,
INFO 01-26 09:15:53 [omni.py:840]  'final_stage_id': {'0_b9b00e74-355c-4b24-b707-d3dc55b11612': 0,
INFO 01-26 09:15:53 [omni.py:840]                     '1_d4515d52-59f1-4cc9-ad81-feabd1827eff': 0},
INFO 01-26 09:15:53 [omni.py:840]  'stages': [{'stage_id': 0,
INFO 01-26 09:15:53 [omni.py:840]              'requests': 2,
INFO 01-26 09:15:53 [omni.py:840]              'tokens': 50,
INFO 01-26 09:15:53 [omni.py:840]              'total_time_ms': 17423.3341217041,
INFO 01-26 09:15:53 [omni.py:840]              'avg_time_per_request_ms': 8711.66706085205,
INFO 01-26 09:15:53 [omni.py:840]              'avg_tokens_per_s': 2.8697148118003097}],
INFO 01-26 09:15:53 [omni.py:840]  'transfers': []}
Adding requests:   0%|                                                                                                  | 0/2 [00:17<?, ?it/s]
Request ID: 0_b9b00e74-355c-4b24-b707-d3dc55b11612, Saved audio to output_audio/output_0_b9b00e74-355c-4b24-b707-d3dc55b11612.wav
Request ID: 1_d4515d52-59f1-4cc9-ad81-feabd1827eff, Saved audio to output_audio/output_1_d4515d52-59f1-4cc9-ad81-feabd1827eff.wav
Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft.

BEFORE SUBMITTING, PLEASE READ https://github.com/vllm-project/vllm-omni/blob/main/CONTRIBUTING.md (anything written below this line will be removed by GitHub Actions)

@gcanlin gcanlin changed the title [Bugfix] Switched @check_model_inputs to @check_model_inputs() [Bugfix][Qwen3-TTS] Switched @check_model_inputs to @check_model_inputs() Jan 26, 2026
@david6666666
Copy link
Collaborator

david6666666 commented Jan 26, 2026

LGTM, thx for fix

@david6666666 david6666666 added the ready label to trigger buildkite CI label Jan 26, 2026
@david6666666 david6666666 enabled auto-merge (squash) January 26, 2026 12:36
@gcanlin
Copy link
Contributor Author

gcanlin commented Jan 26, 2026

@david6666666 Hi, thanks for reviewing! But after discussing with @qibaoyuan, we think this PR isn't the correct way to solve the problem. It only works for transformers v4.57.3 or early versions but not v4.57.6. We need a more general solution:)

@gcanlin gcanlin marked this pull request as draft January 26, 2026 12:48
auto-merge was automatically disabled January 26, 2026 12:48

Pull request was converted to draft

@gcanlin
compatality
13b2d68 
Signed-off-by: gcanlin <[email protected]>
@gcanlin gcanlin changed the title [Bugfix][Qwen3-TTS] Switched @check_model_inputs to @check_model_inputs() [Bugfix][Qwen3-TTS] Add compatibility for both @check_model_inputs and @check_model_inputs(). Jan 27, 2026
@gcanlin gcanlin marked this pull request as ready for review January 27, 2026 01:14
@gcanlin
Copy link
Contributor Author

gcanlin commented Jan 27, 2026

@qibaoyuan @david6666666 I add compatibility to support @check_model_inputs and @check_model_inputs() as workaround. We can merge it first. And in the future will fix it.

@gcanlin
Copy link
Contributor Author

gcanlin commented Jan 27, 2026

Notice that #968 removes @check_model_inputs, after investigating its role, we really don't need it in vLLM-Omni.

@gcanlin gcanlin closed this Jan 27, 2026
@gcanlin gcanlin mentioned this pull request Jan 27, 2026
Closed
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@david6666666 david6666666 david6666666 approved these changes

@hsliuustc0106 hsliuustc0106 Awaiting requested review from hsliuustc0106 hsliuustc0106 is a code owner

Assignees

No one assigned

Labels

ready label to trigger buildkite CI

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Bug][Qwen-TTS]: check_model_inputs lost the parentheses

2 participants

@gcanlin @david6666666