Skip to content

Commit 04b01ac

Browse files
DarkLight1337lulmer
authored andcommitted
[Doc] Dockerfile instructions for optional dependencies and dev transformers (vllm-project#13699)
Signed-off-by: Louis Ulmer <[email protected]>
1 parent 9ad4433 commit 04b01ac

File tree

1 file changed

+30
-0
lines changed

1 file changed

+30
-0
lines changed

docs/source/deployment/docker.md

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,36 @@ container to access the host's shared memory. vLLM uses PyTorch, which uses shar
2727
memory to share data between processes under the hood, particularly for tensor parallel inference.
2828
:::
2929

30+
:::{note}
31+
Optional dependencies are not included in order to avoid licensing issues (e.g. <gh-issue:8030>).
32+
33+
If you need to use those dependencies (having accepted the license terms),
34+
create a custom Dockerfile on top of the base image with an extra layer that installs them:
35+
36+
```Dockerfile
37+
FROM vllm/vllm-openai:v0.7.3
38+
39+
# e.g. install the `audio` and `video` optional dependencies
40+
# NOTE: Make sure the version of vLLM matches the base image!
41+
RUN uv pip install --system vllm[audio,video]==0.7.3
42+
```
43+
44+
:::
45+
46+
:::{tip}
47+
Some new models may only be available on the main branch of [HF Transformers](https://github.com/huggingface/transformers).
48+
49+
To use the development version of `transformers`, create a custom Dockerfile on top of the base image
50+
with an extra layer that installs their code from source:
51+
52+
```Dockerfile
53+
FROM vllm/vllm-openai:latest
54+
55+
RUN uv pip install --system git+https://github.com/huggingface/transformers.git
56+
```
57+
58+
:::
59+
3060
(deployment-docker-build-image-from-source)=
3161

3262
## Building vLLM's Docker Image from Source

0 commit comments

Comments
 (0)