Skip to content

bugfix GraphRAG updated docker compose and env settings to fix issues post refactor#1567

Merged
rbrugaro merged 14 commits intoopea-project:mainfrom
rbrugaro:GRAG_example
Mar 4, 2025
Merged

bugfix GraphRAG updated docker compose and env settings to fix issues post refactor#1567
rbrugaro merged 14 commits intoopea-project:mainfrom
rbrugaro:GRAG_example

Conversation

@rbrugaro
Copy link
Copy Markdown
Collaborator

Validated readme commands work as expected:

# download pdf file
wget https://raw.githubusercontent.com/opea-project/GenAIComps/v1.1/comps/retrievers/redis/data/nke-10k-2023.pdf
# upload pdf file with dataprep
curl -X POST "http://${host_ip}:11103/v1/dataprep/ingest" \
    -H "Content-Type: multipart/form-data" \
    -F "files=@./nke-10k-2023.pdf"
curl http://${host_ip}:8888/v1/graphrag \
    -H "Content-Type: application/json"  \
    -d '{"messages": [{"role": "user","content": "where do Nike subsidiaries operate?
    "}]}'

@rbrugaro rbrugaro requested a review from ashahba as a code owner February 19, 2025 01:00
@github-actions
Copy link
Copy Markdown

github-actions bot commented Feb 19, 2025

Dependency Review

✅ No vulnerabilities or license issues found.

Scanned Files

@xiguiw
Copy link
Copy Markdown
Collaborator

xiguiw commented Feb 25, 2025

@rbrugaro
The CI error message is:
Error response from daemon: No such container: neo4j-apoc

Is the container name typo? Would you please check it?

  • [[ neo4j-apoc == \e\x\t\r\a\c\t_\g\r\a\p\h_\n\e\o\4\j ]]
  • [[ neo4j-apoc == \n\e\o\4\j-\a\p\o\c ]]
    Mega service start duration is 503 s
    ++ curl --silent --write-out 'HTTPSTATUS:%{http_code}' 100.83.111.229:7474
  • HTTP_RESPONSE=HTTPSTATUS:000
    ++ echo HTTPSTATUS:000
    ++ tr -d '\n'
    ++ sed -e 's/.*HTTPSTATUS://'
  • HTTP_STATUS=000
    ++ echo HTTPSTATUS:000
    ++ sed -e 's/HTTPSTATUS:.*//g'
  • RESPONSE_BODY=
  • docker logs neo4j-apoc
    Error response from daemon: No such container: neo4j-apoc
  • '[' 000 -ne 200 ']'
  • echo '[ neo4j-apoc ] HTTP status is not 200. Received status was 000'
  • exit 1
    [ neo4j-apoc ] HTTP status is not 200. Received status was 000

@rbrugaro
Copy link
Copy Markdown
Collaborator Author

#1592

@Spycsh
Copy link
Copy Markdown
Collaborator

Spycsh commented Mar 4, 2025

From the log it seems there are some remaining issues on frontend tests. @rbrugaro have you checked the UI locally? Is it working successfully? Maybe there are some proxies not set correctly that lead to a 124 timeout on frontend tests?
image

@rbrugaro rbrugaro merged commit 8dac9d1 into opea-project:main Mar 4, 2025
14 checks passed
chyundunovDatamonsters pushed a commit to chyundunovDatamonsters/OPEA-GenAIExamples that referenced this pull request Mar 4, 2025
… post refactor (opea-project#1567)

Signed-off-by: rbrugaro <[email protected]>
Signed-off-by: Rita Brugarolas Brufau <[email protected]>
Co-authored-by: chen, suyue <[email protected]>
Co-authored-by: WenjiaoYue <[email protected]>
Signed-off-by: Chingis Yundunov <[email protected]>
@rbrugaro
Copy link
Copy Markdown
Collaborator Author

rbrugaro commented Mar 5, 2025

Fixes #1509

jedwards-habana pushed a commit to jedwards-habana/GenAIExamples that referenced this pull request Mar 11, 2025
… post refactor (opea-project#1567)

Signed-off-by: rbrugaro <[email protected]>
Signed-off-by: Rita Brugarolas Brufau <[email protected]>
Co-authored-by: chen, suyue <[email protected]>
Co-authored-by: WenjiaoYue <[email protected]>
Signed-off-by: Edwards, James A <[email protected]>
chyundunovDatamonsters pushed a commit to chyundunovDatamonsters/OPEA-GenAIExamples that referenced this pull request Mar 21, 2025
… post refactor (opea-project#1567)

Signed-off-by: rbrugaro <[email protected]>
Signed-off-by: Rita Brugarolas Brufau <[email protected]>
Co-authored-by: chen, suyue <[email protected]>
Co-authored-by: WenjiaoYue <[email protected]>
Signed-off-by: Chingis Yundunov <[email protected]>
chyundunovDatamonsters pushed a commit to chyundunovDatamonsters/OPEA-GenAIExamples that referenced this pull request May 16, 2025
… post refactor (opea-project#1567)

Signed-off-by: rbrugaro <[email protected]>
Signed-off-by: Rita Brugarolas Brufau <[email protected]>
Co-authored-by: chen, suyue <[email protected]>
Co-authored-by: WenjiaoYue <[email protected]>
Signed-off-by: Chingis Yundunov <[email protected]>
letonghan pushed a commit that referenced this pull request Sep 17, 2025
* Enlarge DocSum prompt buffer
Follow PR opea-project/GenAIComps#1471

Signed-off-by: XinyaoWa <[email protected]>
letonghan added a commit that referenced this pull request Sep 17, 2025
* Fix image build issue (#1553)

Signed-off-by: chensuyue <[email protected]>
Signed-off-by: Anthony Mahanna <[email protected]>

* Unified default port number for the same service in text2graph and text2sql (#1554)

Signed-off-by: Yao, Qing <[email protected]>
Signed-off-by: Anthony Mahanna <[email protected]>

* new: `OpeaArangoDataprep` (#2)

* new: `third_parties/arangodb`

* new: `OpeaArangoDataprep`

* cleanup

* fix: `vllm` instead of `tgi`

* fix: dataprep compsoe

* cleanup

Signed-off-by: Anthony Mahanna <[email protected]>

* new: `OpeaArangoRetriever` (#3)

* new: `OpeaArangoRetriever`

* cleanup

Signed-off-by: Anthony Mahanna <[email protected]>

* new: deps

Signed-off-by: Anthony Mahanna <[email protected]>

* fix typo: `test_retrievers_arango.sh`

Signed-off-by: Anthony Mahanna <[email protected]>

* updated retriever-arango compose file

Signed-off-by: Anthony Mahanna <[email protected]>

* correction

Signed-off-by: Anthony Mahanna <[email protected]>

* add json-repair to dataprep-arango requirements

Signed-off-by: Anthony Mahanna <[email protected]>

* Fix network error, change WORKPATH

Signed-off-by: Anthony Mahanna <[email protected]>

* extra time for health check retriever

Signed-off-by: Anthony Mahanna <[email protected]>

* extended retriever healthcheck 90secs

Signed-off-by: Anthony Mahanna <[email protected]>

* correction

Signed-off-by: Anthony Mahanna <[email protected]>

* Update arangodb.py

Signed-off-by: Anthony Mahanna <[email protected]>

* Removing hugging face token requirement from test file

Signed-off-by: Anthony Mahanna <[email protected]>

* Update test_dataprep_arango with network tests and additional logs

Signed-off-by: Anthony Mahanna <[email protected]>

* Running CI after docker rate limit

Signed-off-by: Anthony Mahanna <[email protected]>

* Base case remove HF_token, no additional tests

Signed-off-by: Anthony Mahanna <[email protected]>

* Adding VLLM check and logs, currently VLLM not working in CI/CD

Signed-off-by: Anthony Mahanna <[email protected]>

* cleanup: compose.yaml

Signed-off-by: Anthony Mahanna <[email protected]>

* update: arangodb healthcheck

Signed-off-by: Anthony Mahanna <[email protected]>

* cleanup

Signed-off-by: Anthony Mahanna <[email protected]>

* cleanup: retriever test

Signed-off-by: Anthony Mahanna <[email protected]>

* fix: typo

Signed-off-by: Anthony Mahanna <[email protected]>

* rem: unused vars

Signed-off-by: Anthony Mahanna <[email protected]>

* fix: indent

Signed-off-by: Anthony Mahanna <[email protected]>

* temp: swap vllm healthcheck with sleep

Signed-off-by: Anthony Mahanna <[email protected]>

* fix: typo

Signed-off-by: Anthony Mahanna <[email protected]>

* fix: component name typo

Signed-off-by: Anthony Mahanna <[email protected]>

* fix: support `EmbedDoc` for retriever

Signed-off-by: Anthony Mahanna <[email protected]>

* fix: `getattr`

Signed-off-by: Anthony Mahanna <[email protected]>

* fix: CURL command

Signed-off-by: Anthony Mahanna <[email protected]>

* revert 60614841

Signed-off-by: Anthony Mahanna <[email protected]>

* Update xtune file and change DDP paramter (#1552)

Signed-off-by: jilongwa <[email protected]>
Signed-off-by: Anthony Mahanna <[email protected]>

* add N/A option (#1561)

Signed-off-by: ZhangJianyu <[email protected]>
Co-authored-by: ZhangJianyu <[email protected]>
Signed-off-by: Anthony Mahanna <[email protected]>

* Test latest gaudi docker container (#1477)

Update base gaudi container into the latest version, docker pull vault.habana.ai/gaudi-docker/1.20.1/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest, https://docs.habana.ai/en/latest/Installation_Guide/Additional_Installation/Docker_Installation.html#use-intel-gaudi-containers

Signed-off-by: chensuyue <[email protected]>
Signed-off-by: Anthony Mahanna <[email protected]>

* fix audioqna male voice setting (#1559)

Co-authored-by: Letong Han <[email protected]>
Signed-off-by: Anthony Mahanna <[email protected]>

* added error handling for lvm (#1556)

Signed-off-by: okhleif-IL <[email protected]>
Signed-off-by: Anthony Mahanna <[email protected]>

* enable mysql db for sql agent (#1431)

Signed-off-by: cheehook <[email protected]>
Signed-off-by: Anthony Mahanna <[email protected]>

* Enlarge DocSum prompt buffer (#1567)

* Enlarge DocSum prompt buffer
Follow PR opea-project/GenAIComps#1471

Signed-off-by: XinyaoWa <[email protected]>
Signed-off-by: Anthony Mahanna <[email protected]>

* Update vLLM parameter max-seq-len-to-capture (#1565)

Signed-off-by: lvliang-intel <[email protected]>
Signed-off-by: Anthony Mahanna <[email protected]>

* fix: lint

Signed-off-by: Anthony Mahanna <[email protected]>

* fix: missing import

Signed-off-by: Anthony Mahanna <[email protected]>

* new: healtcheck for dataprep-arangodb

Signed-off-by: Anthony Mahanna <[email protected]>

* update: arangodb readmes

Signed-off-by: Anthony Mahanna <[email protected]>

* cleanup: test_dataprep_arango.sh

Signed-off-by: Anthony Mahanna <[email protected]>

* cleanup: test_dataprep_arango.sh (PT2)

Signed-off-by: Anthony Mahanna <[email protected]>

* cleanup: test_dataprep_arango.sh (PT3)

Signed-off-by: Anthony Mahanna <[email protected]>

* update: test_dataprep_arango.sh

Signed-off-by: Anthony Mahanna <[email protected]>

* fix: whitespace

Signed-off-by: Anthony Mahanna <[email protected]>

* Remove Transformers versions from requirements.txt file (#1547)

* Remove Transformers versions from requirements.txt file

Signed-off-by: Abolfazl Shahbazi <[email protected]>
Signed-off-by: Anthony Mahanna <[email protected]>

* Remove index_names from files for dataprep-get request  (#1569)

* remove index_names from files fot get request

Signed-off-by: Mustafa <[email protected]>

* update the tests

Signed-off-by: Mustafa <[email protected]>

* update the tests

Signed-off-by: Mustafa <[email protected]>

* update the tests

Signed-off-by: Mustafa <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add validation check for 'all' as an index_name

Signed-off-by: Mustafa <[email protected]>

* fix for readme file

Signed-off-by: Mustafa <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: Mustafa <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Abolfazl Shahbazi <[email protected]>
Signed-off-by: Anthony Mahanna <[email protected]>

* Upgrade Optimum Habana version to fix security check issue (#1571)

Signed-off-by: lvliang-intel <[email protected]>
Signed-off-by: Anthony Mahanna <[email protected]>

* Make llamaguard compatible with both TGI and vLLM (#1581)

Signed-off-by: lvliang-intel <[email protected]>
Signed-off-by: Anthony Mahanna <[email protected]>

* Fix Dockerfile error and add CI test for IPEX (#1585)

* Fix Dockerfile error and add CI teat

Signed-off-by: lvliang-intel <[email protected]>
Signed-off-by: Anthony Mahanna <[email protected]>

* Reduce multilang tts docker image size (#1587)

* fix audioqna male voice setting

* reduce multilang tts docker image size

Signed-off-by: Anthony Mahanna <[email protected]>

* unset OPENAI_KEY in CI test (#1586)

Signed-off-by: Rita Brugarolas <[email protected]>
Signed-off-by: Anthony Mahanna <[email protected]>

* Add AWS Credentials for CD test (#1588)

* Fix CD test issue

Signed-off-by: ZePan110 <[email protected]>
Signed-off-by: Anthony Mahanna <[email protected]>

* update: shorten ingest_dataprep.txt

Signed-off-by: Anthony Mahanna <[email protected]>

* revert: a4d943e7

Signed-off-by: Anthony Mahanna <[email protected]>

* new: `DataprepRequest` model (#1525)

* new: `DataprepRequest`

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: docstrings

* rem: `ingest_from_graphDB`

* new: dep injection

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: verbose `input` processing

* attempt: replace `kwargs` with params

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* rem: `db_type`

ref: opea-project/GenAIComps#1525 (comment)

* attempt: require `base`

* Revert "attempt: require `base`"

This reverts commit 620ca6bc1d0b1a23f81f8d86168d65088712cfdf.

* new: `DataprepRequest`

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: docstrings

* rem: `ingest_from_graphDB`

* new: dep injection

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: verbose `input` processing

* attempt: replace `kwargs` with params

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* rem: `db_type`

ref: opea-project/GenAIComps#1525 (comment)

* attempt: require `base`

* Revert "attempt: require `base`"

This reverts commit 620ca6bc1d0b1a23f81f8d86168d65088712cfdf.

* Fix dataprep request class issue of Redis (#1)

* new: `DataprepRequest`

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: docstrings

* rem: `ingest_from_graphDB`

* new: dep injection

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: verbose `input` processing

* attempt: replace `kwargs` with params

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* rem: `db_type`

ref: opea-project/GenAIComps#1525 (comment)

* attempt: require `base`

* Revert "attempt: require `base`"

This reverts commit 620ca6bc1d0b1a23f81f8d86168d65088712cfdf.

* fix dataprep request class of redis

Signed-off-by: letonghan <[email protected]>

* revert change in redis.py

Signed-off-by: letonghan <[email protected]>

---------

Signed-off-by: letonghan <[email protected]>
Co-authored-by: Anthony Mahanna <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Anthony Mahanna <[email protected]>
Co-authored-by: Liang Lv <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* revert: `DataprepRequest` for multimodal

* revert: `DataprepRequest` for multimodal (PT2)

* fix: conditionally fetch unique `DataprepRequest` attributes

* fix bugs in dataprep util script

Signed-off-by: letonghan <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* revert change of pgvector

Signed-off-by: letonghan <[email protected]>

* fix indices bug for redis

Signed-off-by: letonghan <[email protected]>

* minor fix for redis

Signed-off-by: letonghan <[email protected]>

* ingest file into rag_redis_test

Signed-off-by: letonghan <[email protected]>

* update indice name

Signed-off-by: letonghan <[email protected]>

---------

Signed-off-by: letonghan <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Liang Lv <[email protected]>
Co-authored-by: Letong Han <[email protected]>
Co-authored-by: letonghan <[email protected]>
Signed-off-by: Anthony Mahanna <[email protected]>

* revert: bc4445cb

Signed-off-by: Anthony Mahanna <[email protected]>

* revert: d17f6aac

Signed-off-by: Anthony Mahanna <[email protected]>

* Revert "new: `DataprepRequest` model (#1525)" (#1592)

This reverts commit 88947ab.

Signed-off-by: Anthony Mahanna <[email protected]>

* add hyperlinks

Signed-off-by: Anthony Mahanna <[email protected]>

* revert: 4eb9ec4f

Signed-off-by: Anthony Mahanna <[email protected]>

* new: ArangoDBDataprepRequest

Signed-off-by: Anthony Mahanna <[email protected]>

* fix: lint

Signed-off-by: Anthony Mahanna <[email protected]>

* cleanup: delete_files

Signed-off-by: Anthony Mahanna <[email protected]>

* remove: env mutation

Signed-off-by: Anthony Mahanna <[email protected]>

* fix: move openai key env var to top of file

Signed-off-by: Anthony Mahanna <[email protected]>

---------

Signed-off-by: chensuyue <[email protected]>
Signed-off-by: Anthony Mahanna <[email protected]>
Signed-off-by: Yao, Qing <[email protected]>
Signed-off-by: jilongwa <[email protected]>
Signed-off-by: ZhangJianyu <[email protected]>
Signed-off-by: okhleif-IL <[email protected]>
Signed-off-by: cheehook <[email protected]>
Signed-off-by: XinyaoWa <[email protected]>
Signed-off-by: lvliang-intel <[email protected]>
Signed-off-by: Abolfazl Shahbazi <[email protected]>
Signed-off-by: Mustafa <[email protected]>
Signed-off-by: Rita Brugarolas <[email protected]>
Signed-off-by: ZePan110 <[email protected]>
Signed-off-by: letonghan <[email protected]>
Co-authored-by: chen, suyue <[email protected]>
Co-authored-by: Yao Qing <[email protected]>
Co-authored-by: lasyasn <[email protected]>
Co-authored-by: Ajay Kallepalli <[email protected]>
Co-authored-by: jilongW <[email protected]>
Co-authored-by: Neo Zhang Jianyu <[email protected]>
Co-authored-by: ZhangJianyu <[email protected]>
Co-authored-by: Spycsh <[email protected]>
Co-authored-by: Letong Han <[email protected]>
Co-authored-by: Omar Khleif <[email protected]>
Co-authored-by: cheehook <[email protected]>
Co-authored-by: XinyaoWa <[email protected]>
Co-authored-by: Liang Lv <[email protected]>
Co-authored-by: Abolfazl Shahbazi <[email protected]>
Co-authored-by: Mustafa <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: rbrugaro <[email protected]>
Co-authored-by: ZePan110 <[email protected]>
Co-authored-by: letonghan <[email protected]>
cogniware-devops pushed a commit to Cogniware-Inc/GenAIExamples that referenced this pull request Dec 19, 2025
… post refactor (opea-project#1567)

Signed-off-by: rbrugaro <[email protected]>
Signed-off-by: Rita Brugarolas Brufau <[email protected]>
Co-authored-by: chen, suyue <[email protected]>
Co-authored-by: WenjiaoYue <[email protected]>
Signed-off-by: cogniware-devops <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants