-
-
Notifications
You must be signed in to change notification settings - Fork 11.6k
[V1]SupportsV0Only protocol for model definitions
#13959
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[V1]SupportsV0Only protocol for model definitions
#13959
Conversation
Signed-off-by: Roger Wang <[email protected]>
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add 🚀 |
|
This pull request has merge conflicts that must be resolved before it can be |
Signed-off-by: Roger Wang <[email protected]>
Signed-off-by: Roger Wang <[email protected]>
|
@robertgshaw2-redhat I turned on CI - feel free to review this whenever you have time. |
Signed-off-by: Roger Wang <[email protected]>
Signed-off-by: Roger Wang <[email protected]> Signed-off-by: Louis Ulmer <[email protected]>
Signed-off-by: Roger Wang <[email protected]>
Repurposed from #13943 to add a
SupportsV0Onlyprotocol to models that are not compatible with vLLM V1 so that we can programmatically check the compatibility such as in #13726.Doing this protocol instead of
SupportsV1also helps more easily track these models to migrate them to V1 as well as touching fewer files, since most models are now supported by V1 vLLM.One caveat is that
PixtralHFis currently not compatible with V1 but there's no way for us to differentiate it from Llava sincetransformersusesLlavaForConditionalGenerationdefinition for both models.