Rename clashing method names for vLLM model protocol #27583
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Harry Mellor <[email protected]>
hmellor
requested review from
DarkLight1337,
NickLucche,
benchislett,
luccafong,
patrickvonplaten,
sighingnow and
ywang96
as code owners
|
Documentation preview: https://vllm--27583.org.readthedocs.build/en/27583/ |
mergify
bot
added
documentation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Comment on lines
+64
to
79
| def _check_vllm_model_embed_input_ids(model: type[object] | object) -> bool: | ||
| model_embed_input_ids = getattr(model, "embed_input_ids", None) | ||
| if not callable(model_embed_input_ids): | ||
| model_get_input_embeddings = getattr(model, "get_input_embeddings", None) | ||
| if callable(model_get_input_embeddings): | ||
| logger.warning( | ||
| "`get_input_embeddings` for vLLM models is deprecated and will be " | ||
| "removed in v0.13.0 or v1.0.0, whichever is earlier. Please rename " | ||
| "this method to `embed_input_ids`." | ||
| ) | ||
| model.embed_input_ids = model_get_input_embeddings | ||
| logger.warning( | ||
| "The model (%s) is missing the `get_input_embeddings` method.", | ||
| "The model (%s) is missing the `embed_input_ids` method.", | ||
| model, | ||
| ) | ||
| return False |
There was a problem hiding this comment.
The new _check_vllm_model_embed_input_ids always returns False when a model still exposes the deprecated get_input_embeddings method, even after binding it to embed_input_ids. As a result, is_vllm_model will reject any third‑party models that follow the old interface, contradicting the intended deprecation fallback and potentially breaking out‑of‑tree integrations until they rename their method. The check should return True when the legacy method exists and is wired up, otherwise the compatibility path never actually works.
Useful? React with 👍 / 👎.
Member
Author
|
As per discussion with @ywang96, I'll be pausing work on this until an ongoing new model is added as to not disrupt it. |
Signed-off-by: Harry Mellor <[email protected]>
Signed-off-by: Harry Mellor <[email protected]>
github-project-automation
bot
moved this from To Triage
to Ready
in gpt-oss Issues & Enhancements
DarkLight1337
added
the
ready
Member
Author
|
Failing extended pooling test also failed last night's nightly https://buildkite.com/vllm/ci/builds/38575/steps/canvas?sid=019a7670-4e2e-49b6-8137-1777aa2125f5 |
Member
Author
|
Failing mm extended 1 test also failed last night's nightly https://buildkite.com/vllm/ci/builds/38575/steps/canvas?sid=019a7670-4e31-45f4-867d-a91104a9c070 Failing mm extended 3 test also failed last night's nightly https://buildkite.com/vllm/ci/builds/38575/steps/canvas?sid=019a7670-4e32-4856-89cf-83eaf9f00b2c |
github-project-automation
bot
moved this from Ready
to Done
in gpt-oss Issues & Enhancements
5 tasks
geodavic
pushed a commit
to geodavic/vllm
that referenced
this pull request
Nov 16, 2025
) Signed-off-by: Harry Mellor <[email protected]> Signed-off-by: George D. Torres <[email protected]>
bwasti
pushed a commit
to bwasti/vllm
that referenced
this pull request
Nov 17, 2025
) Signed-off-by: Harry Mellor <[email protected]> Signed-off-by: Bram Wasti <[email protected]>
wangxiyuan
added a commit
to vllm-project/vllm-ascend
that referenced
this pull request
Nov 26, 2025
Bump vLLM version to v0.11.2 What's broken and changed by vLLM: 1. structured_output is broken by vllm-project/vllm#26866 2. get_mrope_input_positions is broken by vllm-project/vllm#28399 3. graph mode is broken by vllm-project/vllm#25110 we'll upgrade torch to 2.8 to fix the problem later 4. embedding is broken by vllm-project/vllm#27583 5. `get_attn_backend_cls` and attention backend is broken are broken by vllm-project/vllm#28534 6. spec decode is broken by vllm-project/vllm#28771 7. sp feature is broken by vllm-project/vllm#27126 8. mtp is broken by vllm-project/vllm#27922 9. lora is broken by vllm-project/vllm#21068 10. execute_model is broken by vllm-project/vllm#26866 11. `VLLM_DISABLE_SHARED_EXPERTS_STREAM` env is broken by vllm-project/vllm#28159 12. kv cahe is broken by vllm-project/vllm#27753 13. dp is broken by vllm-project/vllm#25110 What's broken and changed by ourself: 1. qwen vl is broken by vllm-project/vllm#28455 We'll remove model files in the future to avoid this kind of error 2. Engine core is broken by vllm-project/vllm#23691 We'll remove the patch file in the future. 3. Ascend scheduler is broken by vllm-project/vllm#28733 We'll remove ascend scheudler later. 4. qwen3-next is broken by vllm-project/vllm#28083 We'll remove model files in the future to avoid this kind of error 5. qwen vl is broken by vllm-project/vllm#27764. We'll remove model files in the future Known issue: 1. ray doesn't work 2. the accuracy of qwen3-next is not correct 3. qwen3-vl is broken 4. prefix cache+ ascend scheduler + deepseek v2 lite is broken. Co-authored-by: MengqingCao <[email protected]> Co-authored-by: hfadzxy <[email protected]> Co-authored-by: leo-pony <[email protected]> Co-authored-by: 22dimensions <[email protected]> Co-authored-by: shen-shanshan <[email protected]> - vLLM version: v0.11.2 --------- Signed-off-by: wangxiyuan <[email protected]> Signed-off-by: MengqingCao <[email protected]> Signed-off-by: hfadzxy <[email protected]> Signed-off-by: leo-pony <[email protected]> Co-authored-by: MengqingCao <[email protected]> Co-authored-by: hfadzxy <[email protected]> Co-authored-by: leo-pony <[email protected]>
Kurumi5210
pushed a commit
to lidenghui1110/vllm-ascend
that referenced
this pull request
Nov 26, 2025
Bump vLLM version to v0.11.2 What's broken and changed by vLLM: 1. structured_output is broken by vllm-project/vllm#26866 2. get_mrope_input_positions is broken by vllm-project/vllm#28399 3. graph mode is broken by vllm-project/vllm#25110 we'll upgrade torch to 2.8 to fix the problem later 4. embedding is broken by vllm-project/vllm#27583 5. `get_attn_backend_cls` and attention backend is broken are broken by vllm-project/vllm#28534 6. spec decode is broken by vllm-project/vllm#28771 7. sp feature is broken by vllm-project/vllm#27126 8. mtp is broken by vllm-project/vllm#27922 9. lora is broken by vllm-project/vllm#21068 10. execute_model is broken by vllm-project/vllm#26866 11. `VLLM_DISABLE_SHARED_EXPERTS_STREAM` env is broken by vllm-project/vllm#28159 12. kv cahe is broken by vllm-project/vllm#27753 13. dp is broken by vllm-project/vllm#25110 What's broken and changed by ourself: 1. qwen vl is broken by vllm-project/vllm#28455 We'll remove model files in the future to avoid this kind of error 2. Engine core is broken by vllm-project/vllm#23691 We'll remove the patch file in the future. 3. Ascend scheduler is broken by vllm-project/vllm#28733 We'll remove ascend scheudler later. 4. qwen3-next is broken by vllm-project/vllm#28083 We'll remove model files in the future to avoid this kind of error 5. qwen vl is broken by vllm-project/vllm#27764. We'll remove model files in the future Known issue: 1. ray doesn't work 2. the accuracy of qwen3-next is not correct 3. qwen3-vl is broken 4. prefix cache+ ascend scheduler + deepseek v2 lite is broken. Co-authored-by: MengqingCao <[email protected]> Co-authored-by: hfadzxy <[email protected]> Co-authored-by: leo-pony <[email protected]> Co-authored-by: 22dimensions <[email protected]> Co-authored-by: shen-shanshan <[email protected]> - vLLM version: v0.11.2 --------- Signed-off-by: wangxiyuan <[email protected]> Signed-off-by: MengqingCao <[email protected]> Signed-off-by: hfadzxy <[email protected]> Signed-off-by: leo-pony <[email protected]> Co-authored-by: MengqingCao <[email protected]> Co-authored-by: hfadzxy <[email protected]> Co-authored-by: leo-pony <[email protected]> Signed-off-by: Kurumi5210 <[email protected]>
devpatelio
pushed a commit
to SumanthRH/vllm
that referenced
this pull request
Nov 29, 2025
) Signed-off-by: Harry Mellor <[email protected]>
845473182
pushed a commit
to 845473182/vllm-ascend
that referenced
this pull request
Nov 29, 2025
Bump vLLM version to v0.11.2 What's broken and changed by vLLM: 1. structured_output is broken by vllm-project/vllm#26866 2. get_mrope_input_positions is broken by vllm-project/vllm#28399 3. graph mode is broken by vllm-project/vllm#25110 we'll upgrade torch to 2.8 to fix the problem later 4. embedding is broken by vllm-project/vllm#27583 5. `get_attn_backend_cls` and attention backend is broken are broken by vllm-project/vllm#28534 6. spec decode is broken by vllm-project/vllm#28771 7. sp feature is broken by vllm-project/vllm#27126 8. mtp is broken by vllm-project/vllm#27922 9. lora is broken by vllm-project/vllm#21068 10. execute_model is broken by vllm-project/vllm#26866 11. `VLLM_DISABLE_SHARED_EXPERTS_STREAM` env is broken by vllm-project/vllm#28159 12. kv cahe is broken by vllm-project/vllm#27753 13. dp is broken by vllm-project/vllm#25110 What's broken and changed by ourself: 1. qwen vl is broken by vllm-project/vllm#28455 We'll remove model files in the future to avoid this kind of error 2. Engine core is broken by vllm-project/vllm#23691 We'll remove the patch file in the future. 3. Ascend scheduler is broken by vllm-project/vllm#28733 We'll remove ascend scheudler later. 4. qwen3-next is broken by vllm-project/vllm#28083 We'll remove model files in the future to avoid this kind of error 5. qwen vl is broken by vllm-project/vllm#27764. We'll remove model files in the future Known issue: 1. ray doesn't work 2. the accuracy of qwen3-next is not correct 3. qwen3-vl is broken 4. prefix cache+ ascend scheduler + deepseek v2 lite is broken. Co-authored-by: MengqingCao <[email protected]> Co-authored-by: hfadzxy <[email protected]> Co-authored-by: leo-pony <[email protected]> Co-authored-by: 22dimensions <[email protected]> Co-authored-by: shen-shanshan <[email protected]> - vLLM version: v0.11.2 --------- Signed-off-by: wangxiyuan <[email protected]> Signed-off-by: MengqingCao <[email protected]> Signed-off-by: hfadzxy <[email protected]> Signed-off-by: leo-pony <[email protected]> Co-authored-by: MengqingCao <[email protected]> Co-authored-by: hfadzxy <[email protected]> Co-authored-by: leo-pony <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
get_input_embeddingsis a getter method of alltransformers.PreTrainedModels.This name had been reused in vLLM to call
forwardon the model's embedding layer.Since the method in vLLM is not actually a getter and causes a confusing clash with the getter in Transformers, this PR:
get_input_embeddingstoembed_input_idsget_multimodal_embeddingstoembed_multimodalfor consistencyVllmModelandSupportsMultiModalso that third party vLLM models should still work