File tree Expand file tree Collapse file tree 1 file changed +30
-0
lines changed
Expand file tree Collapse file tree 1 file changed +30
-0
lines changed Original file line number Diff line number Diff line change @@ -27,6 +27,36 @@ container to access the host's shared memory. vLLM uses PyTorch, which uses shar
2727memory to share data between processes under the hood, particularly for tensor parallel inference.
2828:::
2929
30+ :::{note}
31+ Optional dependencies are not included in order to avoid licensing issues (e.g. < gh-issue:8030 > ).
32+
33+ If you need to use those dependencies (having accepted the license terms),
34+ create a custom Dockerfile on top of the base image with an extra layer that installs them:
35+
36+ ``` Dockerfile
37+ FROM vllm/vllm-openai:v0.7.3
38+
39+ # e.g. install the `audio` and `video` optional dependencies
40+ # NOTE: Make sure the version of vLLM matches the base image!
41+ RUN uv pip install --system vllm[audio,video]==0.7.3
42+ ```
43+
44+ :::
45+
46+ :::{tip}
47+ Some new models may only be available on the main branch of [ HF Transformers] ( https://github.com/huggingface/transformers ) .
48+
49+ To use the development version of ` transformers ` , create a custom Dockerfile on top of the base image
50+ with an extra layer that installs their code from source:
51+
52+ ``` Dockerfile
53+ FROM vllm/vllm-openai:latest
54+
55+ RUN uv pip install --system git+https://github.com/huggingface/transformers.git
56+ ```
57+
58+ :::
59+
3060(deployment-docker-build-image-from-source)=
3161
3262## Building vLLM's Docker Image from Source
You can’t perform that action at this time.
0 commit comments