[Installation]: GitHub access required during install for vllm >=0.4.1 (for cu12-libnccl.so.2.18.1) #4360
Description
Your current environment
Current environment has no internet access. Using a local pypi mirror to install packages. Using the CUDA version of vllm on an Nvidia A10.
How you are installing vllm
Using a local pypi mirror to install packages in an environment without internet access.
The following works:
pip install -vvv vllm==0.4.0.post1The following does not work:
pip install -vvv vllm==0.4.1 Running setup.py (path:/tmp/pip-install-wot5g_wr/vllm-nccl-cu12_7614e24f467d4cef9205802e1b0b89cb/setup.py) egg_info for package vllm-nccl-cu12
Created temporary directory: /tmp/pip-pip-egg-info-lwr60hk0
Running command python setup.py egg_info
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "/tmp/pip-install-wot5g_wr/vllm-nccl-cu12_7614e24f467d4cef9205802e1b0b89cb/setup.py", line 83, in <module>
if get_md5_hash(destination) != file_hash:
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-install-wot5g_wr/vllm-nccl-cu12_7614e24f467d4cef9205802e1b0b89cb/setup.py", line 43, in get_md5_hash
with open(file_path, "rb") as f: # Open file in binary read mode
^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '<my home dir>/.config/vllm/nccl/cu12/libnccl.so.2.18.1'
Downloading nccl package from https://github.com/vllm-project/vllm-nccl/releases/download/v0.1.0/cu12-libnccl.so.2.18.1
Failed to download nccl package from https://github.com/vllm-project/vllm-nccl/releases/download/v0.1.0/cu12-libnccl.so.2.18.1
<urlopen error [Errno 110] Connection timed out>The install process errors out when running the setup.py, which is expected in this scenario as the environment does not have internet access to be able to pull a file from github.
From looking at the diff vs version 0.4.0.post1 I can see that 0.4.1 has a new requirements-cuda.txt which adds vllm-nccl-cu12>=2.18,<2.19
I think the code erroring out is vllm-nccl setup.py which is attempting to grab stuff from github.
From my point of view it would be amazing if there was a way to install vllm using only files from pypi (or a pypi mirror).
I think I might be able to work around this by fetching the file outside of the installation process and setting the VLLM_NCCL_SO_PATH env var to its location, but I don't think the vllm-nccl setup.py knows to check this, so would still fail on install unless I am able to put the .so file in the user's home directory ahead of time, or find a way to skip attempting to install vllm-nccl.