Fix docker python version #2845
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
Author
|
#2716 caused the issue |
Contributor
Author
|
@simon-mo @WoosukKwon can someone take a look? |
NikolaBorisov
force-pushed
the
fix-docker-python
branch
from
a79af40 to
437b2ea
Compare
Contributor
Author
|
#2811 Also broke the docker images. This PR should fix it as well |
Contributor
|
LGTM! |
Collaborator
|
Ah great catch! can you help refactor the Dockerfile so the test target uses 3.8 and everything else uses latest stable? |
Collaborator
|
I guess this will close #2773 as well? |
simon-mo
approved these changes
Feb 13, 2024
Contributor
Author
|
Yes this will fix #2773. I don't think we can have the python 3.8 just in the test target, because the issue was that c code was build with 3.8 but then it could not be imported in 3.10. I think to make sure 3.8 works we could build a different docker file or target that starts with 3.8 python and install vllm from pip? You will not catch it fast. |
Docker images were broken because we build the extensions with
python3.8 but then we try to run them with other version of python.
This results in exception error
File "/workspace/vllm/model_executor/layers/quantization/awq.py", line 6, in <module>
from vllm._C import ops
ModuleNotFoundError: No module named 'vllm._C'
I think if we want to make sure vllm works in older versions of python
we should do something else.
Also fix for the cupy dependency
The docker build was not working with the requirement file
NikolaBorisov
force-pushed
the
fix-docker-python
branch
from
437b2ea to
34a0ad7
Compare
xjpang
pushed a commit
to xjpang/vllm
that referenced
this pull request
Feb 20, 2024
xjpang
pushed a commit
to xjpang/vllm
that referenced
this pull request
Feb 22, 2024
xjpang
pushed a commit
to xjpang/vllm
that referenced
this pull request
Mar 4, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Docker images were broken because we build the extensions with python3.8 but then we try to run them with other version of python.
This results in exception error
I think if we want to make sure vllm works in older versions of python we should do something else.
To reproduce
docker build .docker run ...It crashes with the above exception. I think as part of the CI there should be something that checks if the docker build containers work.