-
-
Notifications
You must be signed in to change notification settings - Fork 11.7k
[XPU] work around for sp, avoid custom op import error #28822
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: Kunshang Ji <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request addresses an import error for SequenceParallelismPass on non-CUDA platforms by making the import conditional. While this fixes the immediate issue, it introduces a potential runtime NameError if sequence parallelism is enabled on an unsupported platform. I've provided a suggestion to create a dummy pass for unsupported platforms, which makes the code more robust by preventing crashes and gracefully disabling the feature with a warning. This approach is more resilient to configuration errors.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for fixing, we also validated this with vllm-ascend. This should also be backported to release/v0.11.1 branch
Sounds good - will cherry pick there! |
Signed-off-by: Kunshang Ji <[email protected]>
Signed-off-by: Kunshang Ji <[email protected]> Signed-off-by: jiang1.li <[email protected]>
…28822) Signed-off-by: Kunshang Ji <[email protected]>
…28822) Signed-off-by: Kunshang Ji <[email protected]>
…28822) Signed-off-by: Kunshang Ji <[email protected]>
Purpose
#27126 add import
SequenceParallelismPass, which break xpu path. we should follow L16-L20, import only for cuda_alike platform.Test Plan
CI.
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.