Refactor dataprep microservice#1153

Merged

chensuyue merged 90 commits intomainfrom

lvl/dataprep_refactor

Jan 20, 2025

Collaborator

lvliang-intel commented Jan 15, 2025

Description

Refactor dataprep microservice, including opensearch, elasticsearch, pinecone, pgvector, neo4j, qdrant, vdms.

Issues

n/a.

Type of change

List the type of change like below. Please delete options that are not relevant.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds new functionality)
Breaking change (fix or feature that would break existing design and interface)
Others (enhancement, documentation, validation, etc.)

Dependencies

None, just code refactor

Tests

Local test and CI test.

lvliang-intel added 24 commits

January 13, 2025 17:49


          refactor dataprep code

eafa1b2

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'main' of https://github.com/opea-project/GenAIComps int…

ec817d7

…o lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Dataprep code refactor

7c545a5

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'main' of https://github.com/opea-project/GenAIComps int…

ff6cdfc

…o lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'main' of https://github.com/opea-project/GenAIComps int…

b0558dd

…o lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

ce3504c

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

29cc5c6

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

218da1b

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

1c8d870

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

3c0c6c3

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

b1635c0

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

2768c00

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

cab0a5d

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

7f66f78

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

f439f1c

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

6d931bc

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

8a4559c

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

4d92e05

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

4d6f1f9

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

632cbaa

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

c95c7b2

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

18dc889

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

0c82e24

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

31672f9

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

lvliang-intel requested review from XinyuYe-Intel, ZePan110, chensuyue, ftian1 and letonghan as code owners

January 15, 2025 15:11


          Merge branch 'main' into lvl/dataprep_refactor

c7fc298

lvliang-intel and others added 13 commits

January 19, 2025 02:49


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

6d848f2

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

744541d

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

40b3aac

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

07e49f2

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

35fc7be

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'main' of https://github.com/opea-project/GenAIComps int…

b256387

…o lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          [pre-commit.ci] auto fixes from pre-commit.com hooks

9f9b2b0

for more information, see https://pre-commit.ci


          Merge branch 'main' of https://github.com/opea-project/GenAIComps int…

0ad3e95

…o lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

9ec463f

…ct/GenAIComps into lvl/dataprep_refactor


          [pre-commit.ci] auto fixes from pre-commit.com hooks

52cc446

for more information, see https://pre-commit.ci


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

9d91d73

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

31a0dfb

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          [pre-commit.ci] auto fixes from pre-commit.com hooks

efb0f22

for more information, see https://pre-commit.ci

XinyuYe-Intel approved these changes

View reviewed changes

lvliang-intel added 6 commits

January 19, 2025 21:44


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

6f874eb

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'main' of https://github.com/opea-project/GenAIComps int…

dc709da

…o lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

a4813ff

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

6ca467c

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'main' of https://github.com/opea-project/GenAIComps int…

4d91a12

…o lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

9d12f72

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

chensuyue mentioned this pull request

For test only #1178

Closed

chensuyue merged commit c223a9d into main

chensuyue deleted the lvl/dataprep_refactor branch

January 20, 2025 12:30

chensuyue mentioned this pull request

Update GraphRAG to be compatible with latest component changes opea-project/GenAIExamples#1427

Merged

4 tasks

chensuyue pushed a commit to opea-project/GenAIExamples that referenced this pull request


          Adapt code for dataprep microservice refactor (#1408)

0f7e5a3

opea-project/GenAIComps#1153

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

smguggen pushed a commit to opea-aws-proserve/GenAIComps that referenced this pull request


          Refactor dataprep microservice (opea-project#1153)

8f4d887

Refactor dataprep microservice, including opensearch, elasticsearch, pinecone, pgvector, neo4j, qdrant, vdms.

Signed-off-by: lvliang-intel <liang1.lv@intel.com>
Co-authored-by: chen, suyue <suyue.chen@intel.com>

chyundunovDatamonsters pushed a commit to chyundunovDatamonsters/OPEA-GenAIExamples that referenced this pull request


          Adapt code for dataprep microservice refactor (opea-project#1408)

fd5c9fe

opea-project/GenAIComps#1153

Signed-off-by: lvliang-intel <liang1.lv@intel.com>
Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>

chyundunovDatamonsters pushed a commit to chyundunovDatamonsters/OPEA-GenAIExamples that referenced this pull request


          Adapt code for dataprep microservice refactor (opea-project#1408)

fcfc586

opea-project/GenAIComps#1153

Signed-off-by: lvliang-intel <liang1.lv@intel.com>
Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>

madison-evans pushed a commit to SAPD-Intel/GenAIComps that referenced this pull request


          Refactor dataprep microservice (opea-project#1153)

c708cc3

Refactor dataprep microservice, including opensearch, elasticsearch, pinecone, pgvector, neo4j, qdrant, vdms.

Signed-off-by: lvliang-intel <liang1.lv@intel.com>
Co-authored-by: chen, suyue <suyue.chen@intel.com>

cogniware-devops pushed a commit to Cogniware-Inc/GenAIExamples that referenced this pull request


          Adapt code for dataprep microservice refactor (opea-project#1408)

d25283d

opea-project/GenAIComps#1153

Signed-off-by: lvliang-intel <liang1.lv@intel.com>
Signed-off-by: cogniware-devops <ambarish.desai@cogniware.ai>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

chensuyue chensuyue left review comments

yao531441 yao531441 approved these changes

lkk12014402 lkk12014402 approved these changes

XinyuYe-Intel XinyuYe-Intel approved these changes

lianhao lianhao left review comments

letonghan Awaiting requested review from letonghan

ZePan110 Awaiting requested review from ZePan110 ZePan110 is a code owner

ftian1 Awaiting requested review from ftian1 ftian1 is a code owner

XinyaoWa Awaiting requested review from XinyaoWa

Labels

None yet