add test script by srinarayan-srikanthan · Pull Request #2199 · opea-project/GenAIExamples

srinarayan-srikanthan · 2025-08-14T03:48:36Z

Description

Test script for remote endpoint

Issues

List the issue or RFC link this PR is working on. If there is no such link, please mark it as n/a.

Type of change

List the type of change like below. Please delete options that are not relevant.

Bug fix (non-breaking change which fixes an issue)
[ x ] New feature (non-breaking change which adds new functionality)
Breaking change (fix or feature that would break existing design and interface)
Others (enhancement, documentation, validation, etc.)

Dependencies

NA

Tests

Describe the tests that you ran to verify your changes.

Signed-off-by: Ubuntu <azureuser@denvr-inf.kifxisxbiwme5gt4kkwqsfdjuh.dx.internal.cloudapp.net>

github-actions · 2025-08-14T03:48:48Z

Dependency Review

✅ No vulnerabilities or license issues found.

Scanned Files

None

for more information, see https://pre-commit.ci

ChatQnA/tests/test_compose_remote_on_xeon.sh

CodeGen/tests/test_compose_remote_on_xeon.sh

DocSum/tests/test_compose_remote_on_xeon.sh

alexsin368 · 2025-08-15T01:23:47Z

ChatQnA/tests/test_compose_remote_on_xeon.sh

+    fi
+}
+
+function stop_docker() {


add a line to stop the vllm-cpu docker container

alexsin368 · 2025-08-15T01:24:31Z

CodeGen/tests/test_compose_remote_on_xeon.sh

+    fi
+}
+
+function stop_docker() {


add a line to stop the vllm-cpu docker container

alexsin368 · 2025-08-15T01:25:27Z

DocSum/tests/test_compose_remote_on_xeon.sh

+        "stream=False"
+}
+
+function stop_docker() {


add a line to stop the vllm-cpu docker container

ProductivitySuite/tests/test_compose_remote_on_xeon.sh

alexsin368 · 2025-08-15T01:26:21Z

ProductivitySuite/tests/test_compose_remote_on_xeon.sh

+    fi
+}
+
+function stop_docker() {


add a line to stop the vllm-cpu docker container

alexsin368 · 2025-08-15T01:31:21Z

ChatQnA/tests/test_compose_remote_on_xeon.sh

+
+    # Start Docker Containers
+    docker compose -f compose_remote.yaml -f compose.telemetry.yaml up -d --quiet-pull > ${LOG_PATH}/start_services_with_compose_remote.log
+}


add logic to wait for the longest docker container to spin up, usually the backend service. example: https://github.com/opea-project/GenAIExamples/blob/main/ChatQnA/tests/test_compose_on_xeon.sh#L47

alexsin368 · 2025-08-15T01:31:32Z

CodeGen/tests/test_compose_remote_on_xeon.sh

+    export API_KEY=$TEST_KEY
+    export LLM_MODEL_ID=TinyLlama/TinyLlama-1.1B-Chat-v1.0
+    # Start Docker Containers
+    docker compose -f compose_remote.yaml up -d > ${LOG_PATH}/start_services_with_compose.log


add logic to wait for the longest docker container to spin up, usually the backend service

alexsin368 · 2025-08-15T01:32:38Z

DocSum/tests/test_compose_remote_on_xeon.sh

+    export API_KEY=$TEST_KEY
+    export LLM_MODEL_ID=TinyLlama/TinyLlama-1.1B-Chat-v1.0
+    docker compose -f compose_remote.yaml up -d > ${LOG_PATH}/start_services_with_compose_remote.log
+    sleep 1m


Sleeping for 1 minute is ok from experience waiting for the docker containers to spin up. But if you want to make it a closed-loop check, add logic to wait for the longest docker container to spin up, usually the backend service

alexsin368 · 2025-08-15T01:33:09Z

ProductivitySuite/tests/test_compose_remote_on_xeon.sh

+
+    # Start Docker Containers
+    docker compose -f compose_remote.yaml up -d > ${LOG_PATH}/start_services_with_compose_remote.log
+    sleep 30s


Sleeping for 30 seconds is ok, but if you want to make it a closed-loop check, add logic to wait for the longest docker container to spin up, usually the backend service

alexsin368

TEST_KEY needs to be set somewhere, vllm-cpu docker image as a dummy remote endpoint uses a fixed version image, and need to add logic to wait for docker containers to spin up before proceeding to validate services.

ChatQnA/tests/test_compose_remote_on_xeon.sh

louie-tsai · 2025-08-15T23:27:18Z

ChatQnA/tests/test_compose_remote_on_xeon.sh

+    echo "Build all the images with --no-cache, check docker_image_build.log for details..."
+    service_list="chatqna chatqna-ui dataprep retriever nginx"
+    docker compose -f build.yaml build ${service_list} --no-cache > ${LOG_PATH}/docker_image_build.log
+    docker run --env "VLLM_SKIP_WARMUP=true" -p 8000:8000 --ipc=host public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.9.2 --model TinyLlama/TinyLlama-1.1B-Chat-v1.0 --api-key $TEST_KEY


why this docker run command inside docker build function? you could move it to start services and do sleep in between.

CodeGen/tests/test_compose_remote_on_xeon.sh

DocSum/tests/test_compose_remote_on_xeon.sh

CodeGen/tests/test_compose_remote_on_xeon.sh

DocSum/tests/test_compose_remote_on_xeon.sh

Copilot

Pull Request Overview

This PR adds test scripts for remote endpoint testing across different OPEA services. The scripts are designed to validate remote LLM endpoint configurations for ChatQnA, CodeGen, DocSum, and ProductivitySuite services running on Intel Xeon processors.

Key changes include:

Addition of comprehensive test scripts for four different OPEA services with remote endpoint support
Implementation of microservice and megaservice validation functions
Integration of frontend testing capabilities for UI components

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 9 comments.

File	Description
ProductivitySuite/tests/test_compose_remote_on_xeon.sh	Test script for ProductivitySuite with remote endpoint validation, including ChatQnA and CodeGen services
DocSum/tests/test_compose_remote_on_xeon.sh	Document summarization service test script supporting text, audio, and video input formats
CodeGen/tests/test_compose_remote_on_xeon.sh	Code generation service test script with microservice validation and frontend testing
ChatQnA/tests/test_compose_remote_on_xeon.sh	Chat Q&A service test script with embedding, retrieval, and reranking microservice validation

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-08-19T20:59:29Z

ProductivitySuite/tests/test_compose_remote_on_xeon.sh

+IMAGE_REPO=${IMAGE_REPO:-"opea"}
+IMAGE_TAG=${IMAGE_TAG:-"latest"}
+echo "REGISTRY=IMAGE_REPO=${IMAGE_REPO}"
+echo "TAG=IMAGE_TAG=${IMAGE_TAG}"


The echo statements are displaying variable names instead of values. Should be echo "REGISTRY=${IMAGE_REPO}" and echo "TAG=${IMAGE_TAG}"

Suggested change

echo "TAG=IMAGE_TAG=${IMAGE_TAG}"

echo "REGISTRY=${IMAGE_REPO}"

echo "TAG=${IMAGE_TAG}"

Copilot · 2025-08-19T20:59:30Z

DocSum/tests/test_compose_remote_on_xeon.sh

+WORKPATH=$(dirname "$PWD")
+LOG_PATH="$WORKPATH/tests"
+echo "REGISTRY=IMAGE_REPO=${IMAGE_REPO}"
+echo "TAG=IMAGE_TAG=${IMAGE_TAG}"


The echo statements are displaying variable names instead of values. Should be echo "REGISTRY=${IMAGE_REPO}" and echo "TAG=${IMAGE_TAG}"

Suggested change

echo "TAG=IMAGE_TAG=${IMAGE_TAG}"

echo "REGISTRY=${IMAGE_REPO}"

echo "TAG=${IMAGE_TAG}"

Copilot · 2025-08-19T20:59:30Z

CodeGen/tests/test_compose_remote_on_xeon.sh

+IMAGE_REPO=${IMAGE_REPO:-"opea"}
+IMAGE_TAG=${IMAGE_TAG:-"latest"}
+echo "REGISTRY=IMAGE_REPO=${IMAGE_REPO}"
+echo "TAG=IMAGE_TAG=${IMAGE_TAG}"


The echo statements are displaying variable names instead of values. Should be echo "REGISTRY=${IMAGE_REPO}" and echo "TAG=${IMAGE_TAG}"

Suggested change

echo "TAG=IMAGE_TAG=${IMAGE_TAG}"

echo "REGISTRY=${IMAGE_REPO}"

echo "TAG=${IMAGE_TAG}"

Copilot · 2025-08-19T20:59:31Z

ChatQnA/tests/test_compose_remote_on_xeon.sh

+IMAGE_REPO=${IMAGE_REPO:-"opea"}
+IMAGE_TAG=${IMAGE_TAG:-"latest"}
+echo "REGISTRY=IMAGE_REPO=${IMAGE_REPO}"
+echo "TAG=IMAGE_TAG=${IMAGE_TAG}"


The echo statements are displaying variable names instead of values. Should be echo "REGISTRY=${IMAGE_REPO}" and echo "TAG=${IMAGE_TAG}"

Suggested change

echo "TAG=IMAGE_TAG=${IMAGE_TAG}"

echo "REGISTRY=${IMAGE_REPO}"

echo "TAG=${IMAGE_TAG}"

Copilot · 2025-08-19T20:59:31Z

ProductivitySuite/tests/test_compose_remote_on_xeon.sh

+
+    echo "::group::validate_frontend"
+    validate_frontend
+    echo "::endgroup::"


The validate_frontend function is called but never defined in this script, which will cause a runtime error.

Copilot · 2025-08-19T20:59:31Z

ProductivitySuite/tests/test_compose_remote_on_xeon.sh

+    echo "Build all the images with --no-cache, check docker_image_build.log for details..."
+    docker compose -f build.yaml build --no-cache > ${LOG_PATH}/docker_image_build.log
+    docker run --env "VLLM_SKIP_WARMUP=true" -p 8000:8000 --ipc=host public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.9.2 --model TinyLlama/TinyLlama-1.1B-Chat-v1.0 --api-key $TEST_KEY
+    docker images && sleep 1s


This Docker run command is executed without proper backgrounding or container cleanup. Consider running in detached mode with -d flag and implementing proper container lifecycle management.

Suggested change

docker images && sleep 1s

container_id=$(docker run -d --env "VLLM_SKIP_WARMUP=true" -p 8000:8000 --ipc=host public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.9.2 --model TinyLlama/TinyLlama-1.1B-Chat-v1.0 --api-key $TEST_KEY)

docker images && sleep 1s

# Cleanup the container after use

docker stop $container_id

docker rm $container_id

Copilot · 2025-08-19T20:59:31Z

DocSum/tests/test_compose_remote_on_xeon.sh

+    service_list="docsum docsum-gradio-ui whisper llm-docsum"
+    docker compose -f build.yaml build ${service_list} --no-cache > ${LOG_PATH}/docker_image_build.log
+    docker run --env "VLLM_SKIP_WARMUP=true" -p 8000:8000 --ipc=host public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.9.2 --model TinyLlama/TinyLlama-1.1B-Chat-v1.0 --api-key $TEST_KEY
+    docker images && sleep 1s


This Docker run command is executed without proper backgrounding or container cleanup. Consider running in detached mode with -d flag and implementing proper container lifecycle management.

Suggested change

docker images && sleep 1s

VLLM_CONTAINER_ID=$(docker run -d --env "VLLM_SKIP_WARMUP=true" -p 8000:8000 --ipc=host public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.9.2 --model TinyLlama/TinyLlama-1.1B-Chat-v1.0 --api-key $TEST_KEY)

docker images && sleep 1s

# Cleanup: stop and remove the container

docker stop $VLLM_CONTAINER_ID

docker rm $VLLM_CONTAINER_ID

Copilot · 2025-08-19T20:59:32Z

CodeGen/tests/test_compose_remote_on_xeon.sh

+
+    docker compose -f build.yaml build ${service_list} --no-cache > ${LOG_PATH}/docker_image_build.log
+    docker run --env "VLLM_SKIP_WARMUP=true" -p 8000:8000 --ipc=host public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.9.2 --model TinyLlama/TinyLlama-1.1B-Chat-v1.0 --api-key $TEST_KEY
+    docker images && sleep 1s


This Docker run command is executed without proper backgrounding or container cleanup. Consider running in detached mode with -d flag and implementing proper container lifecycle management.

Suggested change

docker images && sleep 1s

docker run -d --name vllm_test_container --env "VLLM_SKIP_WARMUP=true" -p 8000:8000 --ipc=host public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.9.2 --model TinyLlama/TinyLlama-1.1B-Chat-v1.0 --api-key $TEST_KEY

# Wait for the container to be ready (optional: implement healthcheck/wait logic here)

docker images && sleep 1s

# Cleanup: stop and remove the test container

docker stop vllm_test_container && docker rm vllm_test_container

Copilot · 2025-08-19T20:59:32Z

ChatQnA/tests/test_compose_remote_on_xeon.sh

+    service_list="chatqna chatqna-ui dataprep retriever nginx"
+    docker compose -f build.yaml build ${service_list} --no-cache > ${LOG_PATH}/docker_image_build.log
+    docker run --env "VLLM_SKIP_WARMUP=true" -p 8000:8000 --ipc=host public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.9.2 --model TinyLlama/TinyLlama-1.1B-Chat-v1.0 --api-key $TEST_KEY
+    docker images && sleep 1s


This Docker run command is executed without proper backgrounding or container cleanup. Consider running in detached mode with -d flag and implementing proper container lifecycle management.

Suggested change

docker images && sleep 1s

VLLM_CONTAINER_ID=$(docker run -d --env "VLLM_SKIP_WARMUP=true" -p 8000:8000 --ipc=host public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.9.2 --model TinyLlama/TinyLlama-1.1B-Chat-v1.0 --api-key $TEST_KEY)

docker images && sleep 1s

# Stop and remove the container after use

docker stop $VLLM_CONTAINER_ID

docker rm $VLLM_CONTAINER_ID

CICD-at-OPEA · 2025-09-19T22:38:06Z

This PR is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

CICD-at-OPEA · 2025-09-27T22:37:29Z

This PR was closed because it has been stalled for 7 days with no activity.

add test script

9c7a242

Signed-off-by: Ubuntu <azureuser@denvr-inf.kifxisxbiwme5gt4kkwqsfdjuh.dx.internal.cloudapp.net>

Copilot AI review requested due to automatic review settings August 14, 2025 03:48

srinarayan-srikanthan requested review from hteeyeoh, jaswanth8888, letonghan, lvliang-intel and yao531441 as code owners August 14, 2025 03:48

srinarayan-srikanthan self-assigned this Aug 14, 2025

srinarayan-srikanthan assigned louie-tsai and alexsin368 Aug 14, 2025

[pre-commit.ci] auto fixes from pre-commit.com hooks

3da05e7

for more information, see https://pre-commit.ci

louie-tsai requested review from louie-tsai and removed request for Copilot August 14, 2025 03:49

louie-tsai mentioned this pull request Aug 14, 2025

[Feature] Add remote inference endpoint support in CI/CD #2005

Open

13 tasks

alexsin368 reviewed Aug 15, 2025

View reviewed changes

ChatQnA/tests/test_compose_remote_on_xeon.sh Show resolved Hide resolved

alexsin368 reviewed Aug 15, 2025

View reviewed changes

ChatQnA/tests/test_compose_remote_on_xeon.sh Show resolved Hide resolved

alexsin368 reviewed Aug 15, 2025

View reviewed changes

CodeGen/tests/test_compose_remote_on_xeon.sh Show resolved Hide resolved

alexsin368 reviewed Aug 15, 2025

View reviewed changes

DocSum/tests/test_compose_remote_on_xeon.sh Show resolved Hide resolved

alexsin368 reviewed Aug 15, 2025

View reviewed changes

DocSum/tests/test_compose_remote_on_xeon.sh Show resolved Hide resolved

alexsin368 reviewed Aug 15, 2025

View reviewed changes

ProductivitySuite/tests/test_compose_remote_on_xeon.sh Show resolved Hide resolved

alexsin368 reviewed Aug 15, 2025

View reviewed changes

alexsin368 requested changes Aug 15, 2025

View reviewed changes

alexsin368 mentioned this pull request Aug 15, 2025

Codetrans: enable remote endpoints #2195

Merged

4 tasks

louie-tsai reviewed Aug 15, 2025

View reviewed changes

Merge branch 'main' into cicd

be31cec

Copilot AI review requested due to automatic review settings August 19, 2025 20:57

Copilot AI reviewed Aug 19, 2025

View reviewed changes

Merge branch 'main' into cicd

b9c106f

CICD-at-OPEA added the Stale label Sep 19, 2025

CICD-at-OPEA closed this Sep 27, 2025

	echo "TAG=IMAGE_TAG=${IMAGE_TAG}"
	echo "REGISTRY=${IMAGE_REPO}"
	echo "TAG=${IMAGE_TAG}"

-    docker images && sleep 1s
+    container_id=$(docker run -d --env "VLLM_SKIP_WARMUP=true" -p 8000:8000 --ipc=host public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.9.2 --model TinyLlama/TinyLlama-1.1B-Chat-v1.0 --api-key $TEST_KEY)
+    docker images && sleep 1s
+    # Cleanup the container after use
+    docker stop $container_id
+    docker rm $container_id

-    docker images && sleep 1s
+    VLLM_CONTAINER_ID=$(docker run -d --env "VLLM_SKIP_WARMUP=true" -p 8000:8000 --ipc=host public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.9.2 --model TinyLlama/TinyLlama-1.1B-Chat-v1.0 --api-key $TEST_KEY)
+    docker images && sleep 1s
+    # Cleanup: stop and remove the container
+    docker stop $VLLM_CONTAINER_ID
+    docker rm $VLLM_CONTAINER_ID

-    docker images && sleep 1s
+    docker run -d --name vllm_test_container --env "VLLM_SKIP_WARMUP=true" -p 8000:8000 --ipc=host public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.9.2 --model TinyLlama/TinyLlama-1.1B-Chat-v1.0 --api-key $TEST_KEY
+    # Wait for the container to be ready (optional: implement healthcheck/wait logic here)
+    docker images && sleep 1s
+    # Cleanup: stop and remove the test container
+    docker stop vllm_test_container && docker rm vllm_test_container

Conversation

srinarayan-srikanthan commented Aug 14, 2025

Description

Issues

Type of change

Dependencies

Tests

Uh oh!

github-actions bot commented Aug 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Dependency Review

Scanned Files

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alexsin368 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

CICD-at-OPEA commented Sep 19, 2025

Uh oh!

CICD-at-OPEA commented Sep 27, 2025

github-actions bot commented Aug 14, 2025 •

edited

Loading