Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions comps/embeddings/deployment/docker_compose/compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ services:
image: ${REGISTRY:-opea}/embedding:${TAG:-latest}
container_name: tei-embedding-server
ports:
- "${EMBEDDER_PORT:-10201}:6000"
- "${EMBEDDER_PORT:-6000}:6000"
ipc: host
environment:
no_proxy: ${no_proxy}
Expand All @@ -45,7 +45,7 @@ services:
image: ${REGISTRY:-opea}/embedding:${TAG:-latest}
container_name: ovms-embedding-server
ports:
- "${EMBEDDER_PORT:-10201}:6000"
- "${EMBEDDER_PORT:-6000}:6000"
ipc: host
environment:
no_proxy: ${no_proxy}
Expand All @@ -60,7 +60,7 @@ services:
image: ${REGISTRY:-opea}/embedding:${TAG:-latest}
container_name: pg-embedding-server
ports:
- ${EMBEDDER_PORT:-10202}:6000
- ${EMBEDDER_PORT:-6000}:6000
ipc: host
environment:
no_proxy: ${no_proxy}
Expand All @@ -75,7 +75,7 @@ services:
image: ${REGISTRY:-opea}/embedding:${TAG:-latest}
container_name: clip-embedding-server
ports:
- ${EMBEDDER_PORT:-10203}:6000
- ${EMBEDDER_PORT:-6000}:6000
ipc: host
environment:
no_proxy: ${no_proxy}
Expand Down
4 changes: 2 additions & 2 deletions comps/rerankings/deployment/docker_compose/compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ services:
image: ${REGISTRY:-opea}/reranking:${TAG:-latest}
container_name: reranking-server
ports:
- ${RERANK_PORT:-10700}:8000
- ${RERANK_PORT:-8000}:8000
ipc: host
environment:
no_proxy: ${no_proxy}
Expand Down Expand Up @@ -51,7 +51,7 @@ services:
image: ${REGISTRY:-opea}/reranking:${TAG:-latest}
container_name: reranking-ovms-server
ports:
- "${RERANK_PORT:-10201}:8000"
- "${RERANK_PORT:-8000}:8000"
ipc: host
environment:
no_proxy: ${no_proxy}
Expand Down
14 changes: 7 additions & 7 deletions comps/rerankings/src/README_ovms.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ It speedup starting the service and avoids copying the model from Internet each
- For Xeon CPU:

```bash
your_port=12005
your_port=8001
docker run -p $your_port:8001 -v ./models:/models --name ovms-rerank-serving \
openvino/model_server:2025.0 --port 8001 --config_path /models/config_reranking.json
```
Expand All @@ -53,11 +53,11 @@ It speedup starting the service and avoids copying the model from Internet each
2. Run the reranking microservice and connect it to the OVMS service:

```bash
export OVMS_RERANKING_PORT=12005
export OVMS_RERANKING_PORT=8001
export OVMS_RERANKING_ENDPOINT="http://localhost:${OVMS_RERANKING_PORT}"
export MODEL_ID=BAAI/bge-reranker-large

docker run -d --rm --name="reranking-ovms-server" -e LOGFLAG=True -p 10700:8000 --ipc=host -e OVMS_RERANKING_ENDPOINT=$OVMS_RERANKING_ENDPOINT -e RERANK_COMPONENT_NAME="OPEA_OVMS_RERANKING" -e MODEL_ID=$MODEL_ID opea/reranking:comps
docker run -d --rm --name="reranking-ovms-server" -e LOGFLAG=True -p 8000:8000 --ipc=host -e OVMS_RERANKING_ENDPOINT=$OVMS_RERANKING_ENDPOINT -e RERANK_COMPONENT_NAME="OPEA_OVMS_RERANKING" -e MODEL_ID=$MODEL_ID opea/reranking:comps
```

## 📦 3. Start Microservice with docker compose
Expand All @@ -71,7 +71,7 @@ Deploy both the OVMS Reranking Service and the Reranking Microservice using Dock
```bash
export MODEL_ID="BAAI/bge-reranker-large"
export OVMS_RERANKING_PORT=12005
export RERANK_PORT=10700
export RERANK_PORT=8000
export host_ip=$(hostname -I | awk '{print $1}')
export OVMS_RERANKING_ENDPOINT="http://${host_ip}:${OVMS_RERANKING_PORT}"
export TAG=comps
Expand Down Expand Up @@ -99,7 +99,7 @@ Deploy both the OVMS Reranking Service and the Reranking Microservice using Dock
- Verify the reranking service is running:

```bash
curl http://localhost:10700/v1/health_check \
curl http://localhost:8000/v1/health_check \
-X GET \
-H 'Content-Type: application/json'
```
Expand All @@ -109,14 +109,14 @@ Deploy both the OVMS Reranking Service and the Reranking Microservice using Dock
- Execute reranking process by providing query and documents

```bash
curl http://localhost:10700/v1/reranking -X POST -H 'Content-Type: application/json' \
curl http://localhost:8000/v1/reranking -X POST -H 'Content-Type: application/json' \
-d '{"initial_query":"What is Deep Learning?", "retrieved_docs": [{"text":"Deep Learning is not..."}, {"text":"Deep learning is..."}]}'
```

- You can add the parameter `top_n` to specify the return number of the reranker model, default value is 1.

```bash
curl http://localhost:10700/v1/reranking \
curl http://localhost:8000/v1/reranking \
-X POST \
-d '{"initial_query":"What is Deep Learning?", "retrieved_docs": [{"text":"Deep Learning is not..."}, {"text":"Deep learning is..."}], "top_n":2}' \
-H 'Content-Type: application/json'
Expand Down
10 changes: 5 additions & 5 deletions comps/rerankings/src/README_tei.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ This README provides set-up instructions and comprehensive details regarding the
export host_ip=$(hostname -I | awk '{print $1}')
export TEI_RERANKING_ENDPOINT="http://${host_ip}:${TEI_RERANKING_PORT}"

docker run -d --name="reranking-tei-server" -e LOGFLAG=True -p 10700:8000 --ipc=host -e http_proxy=$http_proxy -e https_proxy=$https_proxy -e TEI_RERANKING_ENDPOINT=$TEI_RERANKING_ENDPOINT -e HF_TOKEN=$HF_TOKEN -e RERANK_COMPONENT_NAME="OPEA_TEI_RERANKING" opea/reranking:comps
docker run -d --name="reranking-tei-server" -e LOGFLAG=True -p 8000:8000 --ipc=host -e http_proxy=$http_proxy -e https_proxy=$https_proxy -e TEI_RERANKING_ENDPOINT=$TEI_RERANKING_ENDPOINT -e HF_TOKEN=$HF_TOKEN -e RERANK_COMPONENT_NAME="OPEA_TEI_RERANKING" opea/reranking:comps
```

## 📦 2. Start Microservice with docker compose
Expand All @@ -81,7 +81,7 @@ Deploy both the TEI Reranking Service and the Reranking Microservice using Docke
```bash
export RERANK_MODEL_ID="BAAI/bge-reranker-base"
export TEI_RERANKING_PORT=12005
export RERANK_PORT=10700
export RERANK_PORT=8000
export host_ip=$(hostname -I | awk '{print $1}')
export TEI_RERANKING_ENDPOINT="http://${host_ip}:${TEI_RERANKING_PORT}"
export TAG=comps
Expand Down Expand Up @@ -115,7 +115,7 @@ Deploy both the TEI Reranking Service and the Reranking Microservice using Docke
- Verify the reranking service is running:

```bash
curl http://localhost:10700/v1/health_check \
curl http://localhost:8000/v1/health_check \
-X GET \
-H 'Content-Type: application/json'
```
Expand All @@ -125,7 +125,7 @@ Deploy both the TEI Reranking Service and the Reranking Microservice using Docke
- Execute reranking process by providing query and documents

```bash
curl http://localhost:10700/v1/reranking \
curl http://localhost:8000/v1/reranking \
-X POST \
-d '{"initial_query":"What is Deep Learning?", "retrieved_docs": [{"text":"Deep Learning is not..."}, {"text":"Deep learning is..."}]}' \
-H 'Content-Type: application/json'
Expand All @@ -134,7 +134,7 @@ Deploy both the TEI Reranking Service and the Reranking Microservice using Docke
- You can add the parameter `top_n` to specify the return number of the reranker model, default value is 1.

```bash
curl http://localhost:10700/v1/reranking \
curl http://localhost:8000/v1/reranking \
-X POST \
-d '{"initial_query":"What is Deep Learning?", "retrieved_docs": [{"text":"Deep Learning is not..."}, {"text":"Deep learning is..."}], "top_n":2}' \
-H 'Content-Type: application/json'
Expand Down
10 changes: 5 additions & 5 deletions comps/rerankings/src/README_videoqna.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ Then we could send the `top_n` videos to the downstream LVM.

```bash
docker run -d --name "reranking-videoqna-server" \
-p 10703:8000 \
-p 8000:8000 \
--ipc=host \
-e no_proxy=${no_proxy} \
-e http_proxy=${http_proxy} \
Expand All @@ -50,7 +50,7 @@ Deploy both the Videoqna Reranking Service and the Reranking Microservice using

```bash
export TEI_RERANKING_PORT=12006
export RERANK_PORT=10703
export RERANK_PORT=8000
export host_ip=$(hostname -I | awk '{print $1}')
export TEI_RERANKING_ENDPOINT="http://${host_ip}:${TEI_RERANKING_PORT}"
export TAG=comps
Expand All @@ -75,7 +75,7 @@ Deploy both the Videoqna Reranking Service and the Reranking Microservice using
- Verify the reranking service is running:

```bash
curl http://localhost:10703/v1/health_check \
curl http://localhost:8000/v1/health_check \
-X GET \
-H 'Content-Type: application/json'
```
Expand All @@ -85,7 +85,7 @@ Deploy both the Videoqna Reranking Service and the Reranking Microservice using
- Execute reranking process by providing query and documents

```bash
curl http://localhost:10703/v1/reranking \
curl http://localhost:8000/v1/reranking \
-X POST \
-d '{"initial_query":"What is Deep Learning?", "retrieved_docs": [{"text":"Deep Learning is not..."}, {"text":"Deep learning is..."}]}' \
-H 'Content-Type: application/json'
Expand All @@ -94,7 +94,7 @@ Deploy both the Videoqna Reranking Service and the Reranking Microservice using
- You can add the parameter `top_n` to specify the return number of the reranker model, default value is 1.

```bash
curl http://localhost:10703/v1/reranking \
curl http://localhost:8000/v1/reranking \
-X POST \
-d '{"initial_query":"What is Deep Learning?", "retrieved_docs": [{"text":"Deep Learning is not..."}, {"text":"Deep learning is..."}], "top_n":2}' \
-H 'Content-Type: application/json'
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ services:
host_ip: ${host_ip}
HF_TOKEN: ${HF_TOKEN}
healthcheck:
test: ["CMD", "curl", "-f", "http://${host_ip}:${TEI_EMBEDDER_PORT}/health"]
test: ["CMD", "curl", "-f", "http://localhost:80/health"]
interval: 10s
timeout: 6s
retries: 48
Expand All @@ -38,7 +38,7 @@ services:
https_proxy: ${https_proxy}
host_ip: ${host_ip}
healthcheck:
test: ["CMD", "curl", "-f", "http://${host_ip}:${TEI_RERANKING_PORT}/health"]
test: ["CMD", "curl", "-f", "http://localhost:80/health"]
interval: 10s
timeout: 6s
retries: 48
Expand Down Expand Up @@ -66,7 +66,7 @@ services:
- SYS_NICE
ipc: host
healthcheck:
test: ["CMD", "curl", "-f", "http://${host_ip}:${TEI_RERANKING_PORT}/health"]
test: ["CMD", "curl", "-f", "http://localhost:80/health"]
interval: 10s
timeout: 6s
retries: 48
Loading