Skip to content

Update rope_scaling to rope_parameters in preparation for Transformers v5 #28542

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
vllm-bot merged 73 commits into from
Nov 19, 2025
Merged

Update rope_scaling to rope_parameters in preparation for Transformers v5 #28542

vllm-bot merged 73 commits into from
Nov 19, 2025
+544 −912

Conversation

@hmellor
Copy link
Member

@hmellor hmellor commented Nov 12, 2025 •
edited by github-actions bot
Loading

In Transformers v5:

  • rope_scaling is now called rope_parameters
  • rope_theta now lives inside rope_parameters
  • rope_parameters may be nested for models which have different RoPE parameters for each layer type (i.e. Gemma & ModernBERT)

This PR adds forward compatibility for Transformesr v5 RoPE config by:

  • Moving any found config.rope_scaling to config.rope_parameters
  • Moving any found config.rope_theta to config.rope_parameters.rope_theta
  • Performs parch_rope_parameters on all nested configs if present
  • Performs patch_rope_parameters_dict on all nested RoPE parameters if present
  • Globally renaming rope_scaling to rope_parameters
  • get_rope:
    • Remove base as an argument because it no longer needs to be passed separately
    • If rope_parameters is None, default to rope base of 10000 which seems to be a universal default
  • Any models which do not use this 10000 default have it set using the set_default_rope_theta helper

Note, the errors triggered by disable_sliding_window when used with rope scaling models have been removed. It's been left as a follow up task remove disable_sliding_window completely as it is no longer relevant.

@mergify
Copy link

mergify bot commented Nov 12, 2025

Documentation preview: https://vllm--28542.org.readthedocs.build/en/28542/

@mergify mergify bot added documentation Improvements or additions to documentation llama Related to Llama models performance Performance-related issues qwen Related to Qwen models gpt-oss Related to GPT-OSS models speculative-decoding labels Nov 12, 2025
hmellor added 11 commits November 12, 2025 11:12
@hmellor
More renaming in transformers utils
889b900 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Fix patch_rope_parameters for when rope_scaling was explicitly `N…
50b1a87 
…one`

Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Update Gemma3 and Gemma3n
bd182e0 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Merge branch 'main' into update-rope-config
4c61e2e
@hmellor
Get rope_theta from the new location too
65c8658 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Fix condition for non gemma3 models
5d65739 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Make Transformers backend torch compile check work with new rope params
b4e1967 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Re-enable a load of Transformers nightly tests which are now fixed
ee77bd7 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Update the custom configs
df4c007 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Make sure scaling factor always exists
325ff8d 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
A couple more models that now init on v5
11c23a7 
Signed-off-by: Harry Mellor <[email protected]>
@mergify mergify bot added the ci/build label Nov 13, 2025
@hmellor
Update Commandr
4ea113c 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Update Qwen3Next
59b0f27 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Update Olmo2
064441b 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Update models which can default to 10000
002fb90 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Fix nemotron config
99c5d47 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Fix ernie 4.5 vl
c38e8bb 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Fix benchmarks/tests where get_rope is called with positional argum…
eebe73c 
…ents

Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Merge branch 'main' into update-rope-config
540a46b
@hmellor
Fix get_rope kwargs in vision transformers
a60b5ec 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Update new model
00f2853 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Missed positional args
717a704 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Fix nemotron config validation
a9fa3b0 
Signed-off-by: Harry Mellor <[email protected]>
@vllm-bot vllm-bot merged commit a8b7030 into vllm-project:main Nov 19, 2025
55 of 57 checks passed
@github-project-automation github-project-automation bot moved this from To Triage to Done in gpt-oss Issues & Enhancements Nov 19, 2025
@hmellor hmellor deleted the update-rope-config branch November 19, 2025 18:32
Victor49152 pushed a commit to Victor49152/vllm that referenced this pull request Nov 20, 2025
@hmellor @Victor49152
Update rope_scaling to rope_parameters in preparation for Transfo…
f38f901 
…rmers v5 (vllm-project#28542)

Signed-off-by: Harry Mellor <[email protected]>
@DarkLight1337 DarkLight1337 mentioned this pull request Nov 20, 2025
Merged
5 tasks
@hl475 hl475 mentioned this pull request Nov 20, 2025
Merged
5 tasks
LuminolT pushed a commit to LuminolT/vllm that referenced this pull request Nov 21, 2025
@hmellor @LuminolT
Update rope_scaling to rope_parameters in preparation for Transfo…
1cb26d6 
…rmers v5 (vllm-project#28542)

Signed-off-by: Harry Mellor <[email protected]>
Signed-off-by: LuminolT <[email protected]>
@juliendenize juliendenize mentioned this pull request Nov 21, 2025
Merged
5 tasks
bigPYJ1151 pushed a commit that referenced this pull request Nov 25, 2025
@hmellor @bigPYJ1151
Update rope_scaling to rope_parameters in preparation for Transfo…
4eb2166 
…rmers v5 (#28542)

Signed-off-by: Harry Mellor <[email protected]>
Signed-off-by: jiang1.li <[email protected]>
bringlein pushed a commit to bringlein/vllm that referenced this pull request Nov 26, 2025
@hmellor @bringlein
Update rope_scaling to rope_parameters in preparation for Transfo…
a73f405 
…rmers v5 (vllm-project#28542)

Signed-off-by: Harry Mellor <[email protected]>
devpatelio pushed a commit to SumanthRH/vllm that referenced this pull request Nov 29, 2025
@hmellor @devpatelio
Update rope_scaling to rope_parameters in preparation for Transfo…
4bca638 
…rmers v5 (vllm-project#28542)

Signed-off-by: Harry Mellor <[email protected]>
@Potabk Potabk mentioned this pull request Nov 29, 2025
Closed
@wangxiyuan wangxiyuan mentioned this pull request Dec 1, 2025
Merged
kitaekatt pushed a commit to kitaekatt/vllm that referenced this pull request Dec 1, 2025
@hmellor @kitaekatt
Update rope_scaling to rope_parameters in preparation for Transfo…
a458031 
…rmers v5 (vllm-project#28542)

Signed-off-by: Harry Mellor <[email protected]>
wangxiyuan added a commit to vllm-project/vllm-ascend that referenced this pull request Dec 2, 2025
@wangxiyuan @Potabk @zhangxinyuehfad
upgrade vLLM to main (#4608)
7f2673e 
1. fix vllm-project/vllm#28542
The model structure modifications we involved in are:
     - Qwen2.5-VL(still exist some patch)
     - Qwen2-VL
     - Qwen2
     - DeepSeek series
     - Qwen-moe series
2. fix vllm-project/vllm#29121
   the output token now  type changed from np to `list[list[int]]`

3. fix vllm-project/vllm#29262
    `xformers` backend for multimodal now has been deprecated
4. fix vllm-project/vllm#29342

5. fix vllm-project/vllm#28579
6. fix vllm-project/vllm#28718
7. fix vllm-project/vllm#28665
8. fix vllm-project/vllm#26847
vllm introduced the `optimization-level`, some default config has been
changed, and the param `--enforce-eager` has been deprecated
9. fix http://github.com/vllm-project/vllm/pull/29223 it retuns tuple
for sampler.
10. fix vllm-project/vllm#29471 we'll remove the
related patch to avoid this kind of error.

Co-authored-by: hfadzxy <[email protected]>
Co-authored-by: wangli <[email protected]>


- vLLM version: v0.11.2

---------

Signed-off-by: wangxiyuan <[email protected]>
Signed-off-by: wangli <[email protected]>
Signed-off-by: hfadzxy <[email protected]>
Co-authored-by: wangli <[email protected]>
Co-authored-by: hfadzxy <[email protected]>
ChenCangtao pushed a commit to ChenCangtao/vllm-ascend that referenced this pull request Dec 3, 2025
@wangxiyuan @Potabk @zhangxinyuehfad
upgrade vLLM to main (vllm-project#4608)
28a9496 
1. fix vllm-project/vllm#28542
The model structure modifications we involved in are:
     - Qwen2.5-VL(still exist some patch)
     - Qwen2-VL
     - Qwen2
     - DeepSeek series
     - Qwen-moe series
2. fix vllm-project/vllm#29121
   the output token now  type changed from np to `list[list[int]]`

3. fix vllm-project/vllm#29262
    `xformers` backend for multimodal now has been deprecated
4. fix vllm-project/vllm#29342

5. fix vllm-project/vllm#28579
6. fix vllm-project/vllm#28718
7. fix vllm-project/vllm#28665
8. fix vllm-project/vllm#26847
vllm introduced the `optimization-level`, some default config has been
changed, and the param `--enforce-eager` has been deprecated
9. fix http://github.com/vllm-project/vllm/pull/29223 it retuns tuple
for sampler.
10. fix vllm-project/vllm#29471 we'll remove the
related patch to avoid this kind of error.

Co-authored-by: hfadzxy <[email protected]>
Co-authored-by: wangli <[email protected]>


- vLLM version: v0.11.2

---------

Signed-off-by: wangxiyuan <[email protected]>
Signed-off-by: wangli <[email protected]>
Signed-off-by: hfadzxy <[email protected]>
Co-authored-by: wangli <[email protected]>
Co-authored-by: hfadzxy <[email protected]>
Mercykid-bash pushed a commit to Mercykid-bash/vllm-ascend that referenced this pull request Dec 4, 2025
@wangxiyuan @Potabk
@zhangxinyuehfad @mercykid
upgrade vLLM to main (vllm-project#4608)
963fafa 
1. fix vllm-project/vllm#28542
The model structure modifications we involved in are:
     - Qwen2.5-VL(still exist some patch)
     - Qwen2-VL
     - Qwen2
     - DeepSeek series
     - Qwen-moe series
2. fix vllm-project/vllm#29121
   the output token now  type changed from np to `list[list[int]]`

3. fix vllm-project/vllm#29262
    `xformers` backend for multimodal now has been deprecated
4. fix vllm-project/vllm#29342

5. fix vllm-project/vllm#28579
6. fix vllm-project/vllm#28718
7. fix vllm-project/vllm#28665
8. fix vllm-project/vllm#26847
vllm introduced the `optimization-level`, some default config has been
changed, and the param `--enforce-eager` has been deprecated
9. fix http://github.com/vllm-project/vllm/pull/29223 it retuns tuple
for sampler.
10. fix vllm-project/vllm#29471 we'll remove the
related patch to avoid this kind of error.

Co-authored-by: hfadzxy <[email protected]>
Co-authored-by: wangli <[email protected]>

- vLLM version: v0.11.2

---------

Signed-off-by: wangxiyuan <[email protected]>
Signed-off-by: wangli <[email protected]>
Signed-off-by: hfadzxy <[email protected]>
Co-authored-by: wangli <[email protected]>
Co-authored-by: hfadzxy <[email protected]>
Signed-off-by: Che Ruan <[email protected]>
Mercykid-bash pushed a commit to Mercykid-bash/vllm-ascend that referenced this pull request Dec 4, 2025
@wangxiyuan @Potabk
@zhangxinyuehfad @mercykid
upgrade vLLM to main (vllm-project#4608)
4faeb15 
1. fix vllm-project/vllm#28542
The model structure modifications we involved in are:
     - Qwen2.5-VL(still exist some patch)
     - Qwen2-VL
     - Qwen2
     - DeepSeek series
     - Qwen-moe series
2. fix vllm-project/vllm#29121
   the output token now  type changed from np to `list[list[int]]`

3. fix vllm-project/vllm#29262
    `xformers` backend for multimodal now has been deprecated
4. fix vllm-project/vllm#29342

5. fix vllm-project/vllm#28579
6. fix vllm-project/vllm#28718
7. fix vllm-project/vllm#28665
8. fix vllm-project/vllm#26847
vllm introduced the `optimization-level`, some default config has been
changed, and the param `--enforce-eager` has been deprecated
9. fix http://github.com/vllm-project/vllm/pull/29223 it retuns tuple
for sampler.
10. fix vllm-project/vllm#29471 we'll remove the
related patch to avoid this kind of error.

Co-authored-by: hfadzxy <[email protected]>
Co-authored-by: wangli <[email protected]>

- vLLM version: v0.11.2

---------

Signed-off-by: wangxiyuan <[email protected]>
Signed-off-by: wangli <[email protected]>
Signed-off-by: hfadzxy <[email protected]>
Co-authored-by: wangli <[email protected]>
Co-authored-by: hfadzxy <[email protected]>
Signed-off-by: Che Ruan <[email protected]>
charlotte12l pushed a commit to charlotte12l/vllm that referenced this pull request Dec 5, 2025
@hmellor @charlotte12l
Update rope_scaling to rope_parameters in preparation for Transfo…
63ca75e 
…rmers v5 (vllm-project#28542)

Signed-off-by: Harry Mellor <[email protected]>
Signed-off-by: Xingyu Liu <[email protected]>
Meihan-chen pushed a commit to Meihan-chen/vllm-ascend that referenced this pull request Dec 5, 2025
@wangxiyuan @Potabk
@zhangxinyuehfad @Meihan-chen
upgrade vLLM to main (vllm-project#4608)
a7f6a8c 
1. fix vllm-project/vllm#28542
The model structure modifications we involved in are:
     - Qwen2.5-VL(still exist some patch)
     - Qwen2-VL
     - Qwen2
     - DeepSeek series
     - Qwen-moe series
2. fix vllm-project/vllm#29121
   the output token now  type changed from np to `list[list[int]]`

3. fix vllm-project/vllm#29262
    `xformers` backend for multimodal now has been deprecated
4. fix vllm-project/vllm#29342

5. fix vllm-project/vllm#28579
6. fix vllm-project/vllm#28718
7. fix vllm-project/vllm#28665
8. fix vllm-project/vllm#26847
vllm introduced the `optimization-level`, some default config has been
changed, and the param `--enforce-eager` has been deprecated
9. fix http://github.com/vllm-project/vllm/pull/29223 it retuns tuple
for sampler.
10. fix vllm-project/vllm#29471 we'll remove the
related patch to avoid this kind of error.

Co-authored-by: hfadzxy <[email protected]>
Co-authored-by: wangli <[email protected]>


- vLLM version: v0.11.2

---------

Signed-off-by: wangxiyuan <[email protected]>
Signed-off-by: wangli <[email protected]>
Signed-off-by: hfadzxy <[email protected]>
Co-authored-by: wangli <[email protected]>
Co-authored-by: hfadzxy <[email protected]>
Zhathw pushed a commit to Zhathw/vllm that referenced this pull request Dec 6, 2025
@hmellor @Zhathw
Update rope_scaling to rope_parameters in preparation for Transfo…
eb77702 
…rmers v5 (vllm-project#28542)

Signed-off-by: Harry Mellor <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@chatgpt-codex-connector chatgpt-codex-connector[bot] chatgpt-codex-connector[bot] left review comments

@noooop noooop Awaiting requested review from noooop noooop is a code owner

@patrickvonplaten patrickvonplaten Awaiting requested review from patrickvonplaten patrickvonplaten is a code owner

@sighingnow sighingnow Awaiting requested review from sighingnow sighingnow is a code owner

@mgoin mgoin Awaiting requested review from mgoin mgoin is a code owner

@tlrmchlsmth tlrmchlsmth Awaiting requested review from tlrmchlsmth tlrmchlsmth is a code owner

@WoosukKwon WoosukKwon Awaiting requested review from WoosukKwon WoosukKwon is a code owner

@yewentao256 yewentao256 Awaiting requested review from yewentao256 yewentao256 is a code owner

@simon-mo simon-mo Awaiting requested review from simon-mo

@youkaichao youkaichao Awaiting requested review from youkaichao youkaichao is a code owner

@robertgshaw2-redhat robertgshaw2-redhat Awaiting requested review from robertgshaw2-redhat robertgshaw2-redhat is a code owner

@houseroad houseroad Awaiting requested review from houseroad houseroad is a code owner

@ProExpertProg ProExpertProg Awaiting requested review from ProExpertProg ProExpertProg is a code owner

Assignees

No one assigned

Labels

ci/build deepseek Related to DeepSeek models documentation Improvements or additions to documentation gpt-oss Related to GPT-OSS models llama Related to Llama models performance Performance-related issues qwen Related to Qwen models ready ONLY add when PR is ready to merge/full CI is needed speculative-decoding

Projects

gpt-oss Issues & Enhancements
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@hmellor @vllm-bot