vLLM recently moved the path of docker files, will break unit tests

[edlee123]

Priority

P3-Medium

OS type

Ubuntu

Hardware type

Xeon-GNR

Installation method

• Pull docker images from hub.docker.com
• Build docker images from source
• Other

Deploy method

• Docker
• Docker Compose
• Kubernetes Helm Charts
• Other

Running nodes

Single Node

What's the version?

Building vLLM Dockerfiles from source.

Description

vLLM moved recently docker files to a dedicated folder in a recent PR:

vllm-project/vllm@e6e3c55

What it means is that if we have CI tests that are pulling down the latest vLLM and building Docker files, it may not be able to find the dockerfiles in the original path.

It also means we should update the ChatQnA/docker_image_build/build.yaml since building ChatQna from this yaml may not work as originally intended.

Reproduce steps

Please take a look at the new location of vLLM docker files: vllm-project/vllm@e6e3c55

And then review unit tests that pull down vLLM github, and the path specified to build the vLLM Dockerfile.

Raw log

Attachments

No response

[xiguiw]

@edlee123

Thank you very much for notifying us this change!

That's very important and can save efforts to debug the potential issue in the near future.

Currently, OPEA get the latest stable version to build docker images.

So this change will break the docker build when the next VLLM tag is created.

Please be notified this change @chensuyue, @lvliang-intel, @lkk12014402 @joshuayao @ftian1

[xiguiw]

Code and script that are affected by this change:

1. GenAComps
   

   GenAIComps/comps/third_parties/vllm/src/build_docker_vllm.sh

   Line 36 in 2b0cd08

   # Build the docker image for vLLM based on the hardware mode

./comps/llms/deployment/docker_compose/test_llms_faq-generation_vllm.sh:20: git clone https://github.com/vllm-project/vllm.git
./comps/dataprep/src/README_finance.md:50:git clone https://github.com/HabanaAI/vllm-fork.git
./comps/agent/src/README.md:132:git clone https://github.com/HabanaAI/vllm-fork.git
./comps/lvms/src/README.md:74:git clone https://github.com/HabanaAI/vllm-fork.git
./comps/third_parties/vllm/

./.github/workflows/_comps-workflow.yml:74: git clone https://github.com/vllm-project/vllm.git vllm-openvino
./.github/workflows/_comps-workflow.yml:78: git clone https://github.com/HabanaAI/vllm-fork.git && cd vllm-fork
./.github/workflows/push-image-build.yml:94: git clone https://github.com/vllm-project/vllm.git vllm-openvino
./.github/workflows/push-image-build.yml:98: git clone https://github.com/HabanaAI/vllm-fork.git && cd vllm-fork
./tests/llms/test_llms_doc-summarization_vllm_on_intel_hpu.sh:20: git clone https://github.com/HabanaAI/vllm-fork.git
./tests/llms/test_llms_doc-summarization_vllm.sh:21: git clone https://github.com/vllm-project/vllm.git
./tests/llms/test_llms_faq-generation_vllm_on_intel_hpu.sh:20: git clone https://github.com/HabanaAI/vllm-fork.git
./tests/llms/test_llms_faq-generation_vllm.sh:21: git clone https://github.com/vllm-project/vllm.git
./tests/llms/test_llms_text-generation_service_vllm_on_intel_hpu.sh:20: git clone https://github.com/HabanaAI/vllm-fork.git
./tests/dataprep/test_dataprep_redis_finance_on_intel_hpu.sh:34: git clone https://github.com/HabanaAI/vllm-fork.git
./tests/agent/test_agent_langchain_on_intel_hpu.sh:59: git clone https://github.com/HabanaAI/vllm-fork.git
./tests/agent/build_vllm_gaudi.sh:9: git clone https://github.com/HabanaAI/vllm-fork.git
./tests/agent/sql_agent_test/test_sql_agent.sh:108: git clone https://github.com/HabanaAI/vllm-fork.git
./tests/guardrails/test_guardrails_hallucination_detection_on_intel_hpu.sh:14: git clone https://github.com/HabanaAI/vllm-fork.git
./tests/lvms/test_lvms_vllm_on_intel_hpu.sh:21: git clone https://github.com/HabanaAI/vllm-fork.git
./tests/third_parties/test_third_parties_vllm_openvino.sh:20: git clone https://github.com/vllm-project/vllm.git vllm-openvino

2. GenAIExamples.

a. test_script of each examples

https://github.com/opea-project/GenAIExamples/blob/8fe2d5d0be06bcaa42770cb1bed97831e920c345/ChatQnA/tests/test_compose_on_xeon.sh#L34

b). docker_image_build.yaml of each example

c) ./.github/workflows/_build_image.yml:
build docker image

[edlee123]

Glad to help! It has been good to learn too, and thank you for the test_compose_on_xeon.sh example (cloning to last stable vLLM tag when writing unit tests).

@edlee123

Thank you very much for notifying us this change!

That's very important and can save efforts to debug the potential issue in the near future.

Currently, OPEA get the latest stable version to build docker images.

So this change will break the docker build when the next VLLM tag is created.

Please be notified this change @chensuyue, @lvliang-intel, @lkk12014402 @joshuayao @ftian1

[chensuyue]

The new tag of vllm has been released, https://github.com/vllm-project/vllm/tree/v0.8.3, we should update all our vllm image building path. @ZePan110 please follow.