PR: OpenSearch integration components for OPEA

[cameronmorin]

Description

These proposed changes create the OPEA components necessary to integrate OpenSearch with GenAIExamples, specifically tested & validated with ChatQnA

Issues

n/a

Type of change

List the type of change like below. Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds new functionality)
• Breaking change (fix or feature that would break existing design and interface)
• Others (enhancement, documentation, validation, etc.)

Dependencies

opensearch-py

Tests

I tested the components independently, as well as end-to-end by creating my own ChatQnA cluster replacing the redis microservices (dataprep, retriever, and vectorstore) with the OpenSearch components I've created here.

[lvliang-intel]

@cameronmorin,
Thanks for your contribution.
Please check the pre-commit CI issues, looks like the yaml file has format issue.

[letonghan]

Hi @cameronmorin , thanks for your contribution. Please check out the comments above, and remember to add CI test scripts like this for each of the new components. Please name the test scripts following the schema of test/${component}/test_${component}_${storage}_${framework}.sh.
Thank you.

[cameronmorin]

Hi @letonghan ! Thank you so much for reviewing my PR and leaving your helpful comments. I've addressed the simple ones of removing unnecessary comments & files, and I'm in the process of adding the test scripts you mentioned that are required.

[ashahba]

Hi @cameronmorin and thanks for this PR!
I just did a quick look and it needs those 2 files fixed for formatting.

I need to do deeper review once those changes are made.

Thanks.

[joshuayao]

Hi @cameronmorin

Thanks for your PR. Could you please check the failures in the CI test? Thanks.

[ashahba]

Why not combine some of these commands together like:

apt-get update
apt-get install default-jre libtesseract-dev poppler-utils tesseract-ocr

[smguggen]

Would you mind linking me to the OPEA style guidelines because I'm confused as to what the standards are. This snippet is identical to existing code:

[smguggen]

This pull request can be cancelled, changes have been merged in newer PR

[cameronmorin]

Lastly, I've tried to address the DCO comment by rebasing my branch in order to sign all of my commits, however now I'm seeing issues with commits that I didn't author after addressing & signing all of my commits. I'll try to see if I can fix these as well, but if it comes to it after I address all of the comments and get the tests working, can I move my changes to a new PR with a single signed commit from me to be merged instead?

If not, some help or advice with the commit signing would also be appreciated :) Thanks!

[codecov]

Codecov Report

Attention: Patch coverage is 33.33333% with 2 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
comps/cores/mega/gateway.py	0.00%	1 Missing ⚠️
comps/version.py	0.00%	1 Missing ⚠️

Files with missing lines	Coverage Δ
comps/cores/proto/api_protocol.py	96.12% <100.00%> (+0.06%)	⬆️
comps/cores/mega/gateway.py	28.14% <0.00%> (ø)
comps/version.py	0.00% <0.00%> (ø)

[chensuyue]

Lastly, I've tried to address the DCO comment by rebasing my branch in order to sign all of my commits, however now I'm seeing issues with commits that I didn't author after addressing & signing all of my commits. I'll try to see if I can fix these as well, but if it comes to it after I address all of the comments and get the tests working, can I move my changes to a new PR with a single signed commit from me to be merged instead?

If not, some help or advice with the commit signing would also be appreciated :) Thanks!

The rebase seems not correct, the diff file including some history change.

[chensuyue]

Follow the CI reminder to fix the issue, https://github.com/opea-project/GenAIComps/actions/runs/12091595827/job/33765700276?pr=908

[cameronmorin]

Lastly, I've tried to address the DCO comment by rebasing my branch in order to sign all of my commits, however now I'm seeing issues with commits that I didn't author after addressing & signing all of my commits. I'll try to see if I can fix these as well, but if it comes to it after I address all of the comments and get the tests working, can I move my changes to a new PR with a single signed commit from me to be merged instead?
If not, some help or advice with the commit signing would also be appreciated :) Thanks!

The rebase seems not correct, the diff file including some history change.

I agree, I thought the rebase seemed off when I initially performed it because I saw a lot of historical changes when I rebased to sign all of the commits in the log. I tried to rebase manually and only sign the commits that I authored, but I still saw the historical changes being added to my PR in addition to the DCO still failing. I just pushed a change that fixed the DCO, however I still see the historical changes added to the PR. Would it be a bad idea to resolve all of the reviewer comments, and split my changes into a fresh branch / PR that doesn't have to rebase historical changes in order to merge that instead of this PR? I'd want to link this PR so that all conversations and reviews persist and are tracked historically, but I'm not sure how to clear up the other changes flowing into my branch.

[cameronmorin]

I've moved the changes over to a new PR in order to resolve the history and DCO issues. Please review & approve that PR instead since all tests are passing with the same file changes and new git log.

[joshuayao]

Hi @cameronmorin, thanks for your contribution. I will close this PR.