Skip to content

[CI/Build] Dockerfile.cpu improvements #7298

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 8, 2024
Merged

[CI/Build] Dockerfile.cpu improvements #7298

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .dockerignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,4 @@
vllm/*.so
/.venv
/build
dist
30 changes: 21 additions & 9 deletions Dockerfile.cpu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,37 +2,49 @@

FROM ubuntu:22.04 AS cpu-test-1

RUN apt-get update -y \
&& apt-get install -y curl git wget vim numactl gcc-12 g++-12 python3 python3-pip libtcmalloc-minimal4 libnuma-dev \
RUN --mount=type=cache,target=/var/cache/apt \
apt-get update -y \
&& apt-get install -y curl ccache git wget vim numactl gcc-12 g++-12 python3 python3-pip libtcmalloc-minimal4 libnuma-dev \
&& update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-12 10 --slave /usr/bin/g++ g++ /usr/bin/g++-12

# https://intel.github.io/intel-extension-for-pytorch/cpu/latest/tutorials/performance_tuning/tuning_guide.html
# intel-openmp provides additional performance improvement vs. openmp
# tcmalloc provides better memory allocation efficiency, e.g, holding memory in caches to speed up access of commonly-used objects.
RUN pip install intel-openmp
RUN --mount=type=cache,target=/root/.cache/pip \
pip install intel-openmp

ENV LD_PRELOAD="/usr/lib/x86_64-linux-gnu/libtcmalloc_minimal.so.4:/usr/local/lib/libiomp5.so:$LD_PRELOAD"

RUN echo 'ulimit -c 0' >> ~/.bashrc

RUN pip install https://intel-extension-for-pytorch.s3.amazonaws.com/ipex_dev/cpu/intel_extension_for_pytorch-2.4.0%2Bgitfbaa4bc-cp310-cp310-linux_x86_64.whl

RUN pip install --upgrade pip \
&& pip install wheel packaging ninja "setuptools>=49.4.0" numpy
ENV PIP_EXTRA_INDEX_URL=https://download.pytorch.org/whl/cpu
RUN --mount=type=cache,target=/root/.cache/pip \
--mount=type=bind,src=requirements-build.txt,target=requirements-build.txt \
pip install --upgrade pip && \
pip install -r requirements-build.txt
Copy link
Member

@bigPYJ1151 bigPYJ1151 Aug 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM, thanks for the improvement!

The only thing I concerned is using requirements-build.txt will install torch with many CUDA dependencies, which looks a bit redundant and increases download time.

Copy link
Contributor Author

@dtrifiro dtrifiro Aug 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will not: PIP_EXTRA_INDEX_URL is set before the RUN command, resulting in the following dependencies being installed:

Package           Version
----------------- ---------
cmake             3.30.2
filelock          3.15.4
fsspec            2024.6.1
Jinja2            3.1.4
MarkupSafe        2.1.5
mpmath            1.3.0
networkx          3.3
ninja             1.11.1.1
packaging         24.1
pip               24.1.2
setuptools        70.3.0
sympy             1.13.1
torch             2.4.0+cpu
typing_extensions 4.12.2
wheel             0.43.0

One possible issue might come up with some package managers such as uv:

export UV_EXTRA_INDEX_URL=https://download.pytorch.org/whl/cpu
uv pip install -r requirements-build.txt

will fail due to the way uv resolves local version identifiers: in order for this to succeed, the +cpu specifier should be explicitly added (torch==2.40+cpu).

Copy link
Contributor Author

@dtrifiro dtrifiro Aug 8, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that this PIP_EXTRA_INDEX url also works for PEP517 style builds, so running

export PIP_EXTRA_INDEX_URL=https://download.pytorch.org/whl/cpu 
export VLLM_TARGET_DEVICE=cpu 
pip install git+https://github.com/vllm-project/vllm

will also install the cpu version when creating the isolated build venv.


FROM cpu-test-1 AS build

COPY ./ /workspace/vllm

WORKDIR /workspace/vllm

RUN pip install -v -r requirements-cpu.txt --extra-index-url https://download.pytorch.org/whl/cpu
RUN --mount=type=cache,target=/root/.cache/pip \
--mount=type=bind,src=requirements-common.txt,target=requirements-common.txt \
--mount=type=bind,src=requirements-cpu.txt,target=requirements-cpu.txt \
pip install -v -r requirements-cpu.txt

COPY ./ ./

# Support for building with non-AVX512 vLLM: docker build --build-arg VLLM_CPU_DISABLE_AVX512="true" ...
ARG VLLM_CPU_DISABLE_AVX512
ENV VLLM_CPU_DISABLE_AVX512=${VLLM_CPU_DISABLE_AVX512}

RUN VLLM_TARGET_DEVICE=cpu python3 setup.py install
ENV CCACHE_DIR=/root/.cache/ccache
RUN --mount=type=cache,target=/root/.cache/pip \
--mount=type=cache,target=/root/.cache/ccache \
VLLM_TARGET_DEVICE=cpu python3 setup.py bdist_wheel && \
Copy link
Contributor Author

@dtrifiro dtrifiro Aug 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A significant issue here is that the torch version installed by requirements-cpu.txt will override whatever is defined in requirements-cpu.txt.

pip install dist/*.whl

Copy link
Contributor

@maxdebayser maxdebayser Aug 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do the files in dist stick around in the container image? Can they be removed with && rm -r dist or is there some dependency on these files after pip install?

Copy link
Contributor Author

@dtrifiro dtrifiro Aug 8, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that's correct, good catch. I did not bother too much about cleaning the image as I think this is clear this is mostly a CI image.

If there's an interest in making this production ready, we can set up multi-stage builds and have a final deployment stage without dev/build dependencies.

I can remove this if others think this is worth removing in this PR

Copy link
Member

@mgoin mgoin Aug 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't a big overhead, we can remove later

WORKDIR /workspace/

Expand Down