[RFC] OPEA Inference Microservices (OIM) by poussa · Pull Request #327 · opea-project/docs

poussa · 2025-03-10T10:17:00Z

RFC for a new OPEA feature: OPEA Inference Microservices (OIM). OIM will address LLM inference deployment challenges via Kubernetes operator, custom resources, profiles and standardised container images.

mkbhanda

Minor typos and a question. Thank you @poussa !

community/rfcs/25-03-07-OPEA-001-OIM-Operator.md

eero-t

Looks OK, but I have few minor suggestions.

community/rfcs/25-03-07-OPEA-001-OIM-Operator.md

Yu-amd · 2025-03-11T21:59:14Z

Hi team,

I raised a feature request for the OIM operator here: opea-project/GenAIExamples#1525 Please review and consider adding this as MVP.

Thank you,
Yu

ashahba

Thanks @poussa
The PR looks good overall but please address @eero-t 's comment where you see fit.
Also we need a few lines like a high level summary for the description.

AlexHe99 · 2025-03-12T03:09:29Z

Really the feature required of OPEA!

poussa · 2025-03-12T07:45:37Z

Thanks @poussa The PR looks good overall but please address @eero-t 's comment where you see fit. Also we need a few lines like a high level summary for the description.

@eero-t comments addressed and description updated.

community/rfcs/25-03-07-OPEA-001-OIM-Operator.md

eero-t · 2025-03-12T19:20:13Z

Thanks @poussa The PR looks good overall but please address @eero-t 's comment where you see fit.

That's done now.

Also we need a few lines like a high level summary for the description.

Could you suggest something for that, or is it good now?

community/rfcs/25-03-07-OPEA-001-OIM-Operator.md

poussa · 2025-03-14T06:45:46Z

@mkbhanda @ftian1 @ashahba I have now addressed all the feedback. Time for merge? PTAL.

Signed-off-by: Sakari Poussa <[email protected]>

ashahba

LGTM!

* [RFC] OPEA Inference Microservices (OIM) Signed-off-by: Sakari Poussa <[email protected]> * review fixes Signed-off-by: Sakari Poussa <[email protected]> * review fixes Signed-off-by: Sakari Poussa <[email protected]> * review fixes Signed-off-by: Sakari Poussa <[email protected]> * RFC: add OIM operator diagram Signed-off-by: Sakari Poussa <[email protected]> * review fixes, add picture Signed-off-by: Sakari Poussa <[email protected]> --------- Signed-off-by: Sakari Poussa <[email protected]> Signed-off-by: Haim Barad <[email protected]>

* Add rfc for Routing Agent Signed-off-by: Haim Barad <[email protected]> * Add blogs page to OPEA (#318) * Create index.rst * Update index.rst Signed-off-by: Haim Barad <[email protected]> * Update index.rst (#329) * Update index.rst Added "Moving from OpenAI to Opensource using OPEA" blog post * Updated index.rst with commit message * Updated index.rst with the right dates. Signed off by: chrisahsiong23 <[email protected]> * Update index.rst to pass DCO Signed-off-by: chrisahsiong23 <[email protected]> --------- Signed-off-by: chrisahsiong23 <[email protected]> Signed-off-by: Haim Barad <[email protected]> * Add GitHub Action to check and close stale issues and PRs (#332) Signed-off-by: Sun, Xuehao <[email protected]> Signed-off-by: Haim Barad <[email protected]> * [Bug: 181] Getting started quide mentions Intel Tiber Cloud (#321) Fixes #181 Co-authored-by: Ghosh, Soumyadip <[email protected]> Signed-off-by: Piroozan, Nariman <[email protected]> Signed-off-by: Jaini, Pallavi <[email protected]> Signed-off-by: Kavulya, Soila <[email protected]> Signed-off-by: Rajabose, Shifani <[email protected]> Signed-off-by: Shifani Rajabose <[email protected]> Co-authored-by: Malini Bhandaru <[email protected]> Signed-off-by: Haim Barad <[email protected]> * Add doc for contributing vectorDB. Signed-off-by: Katherine Druckman <[email protected]> Signed-off-by: Haim Barad <[email protected]> * Apply suggestions from code review Co-authored-by: Malini Bhandaru <[email protected]> Signed-off-by: Katherine Druckman <[email protected]> Signed-off-by: Haim Barad <[email protected]> * Apply suggestions from code review Co-authored-by: Malini Bhandaru <[email protected]> Signed-off-by: Katherine Druckman <[email protected]> Signed-off-by: Haim Barad <[email protected]> * Fix the URL for add_vectorDB.md (#334) * Fix the URL for add_vectorDB.md Signed-off-by: Abolfazl Shahbazi <[email protected]> * minor formattingupdate to CONTRIBUTING.md Signed-off-by: Abolfazl Shahbazi <[email protected]> --------- Signed-off-by: Abolfazl Shahbazi <[email protected]> Signed-off-by: Haim Barad <[email protected]> * Add AMD OPEA blog (#339) Signed-off-by: Yu Wang <[email protected]> Signed-off-by: Haim Barad <[email protected]> * [RFC] OPEA Inference Microservices (OIM) (#327) * [RFC] OPEA Inference Microservices (OIM) Signed-off-by: Sakari Poussa <[email protected]> * review fixes Signed-off-by: Sakari Poussa <[email protected]> * review fixes Signed-off-by: Sakari Poussa <[email protected]> * review fixes Signed-off-by: Sakari Poussa <[email protected]> * RFC: add OIM operator diagram Signed-off-by: Sakari Poussa <[email protected]> * review fixes, add picture Signed-off-by: Sakari Poussa <[email protected]> --------- Signed-off-by: Sakari Poussa <[email protected]> Signed-off-by: Haim Barad <[email protected]> * Add 'Document Summarization' to the list of OPEA blogs (#337) Signed-off-by: Abolfazl Shahbazi <[email protected]> Signed-off-by: Haim Barad <[email protected]> * Add workflow executor agent strategy rfc (#77) * Add workflow executor agent strategy rfc Signed-off-by: JoshuaL3000 <[email protected]> * Remove space in name. Add title Signed-off-by: JoshuaL3000 <[email protected]> --------- Signed-off-by: JoshuaL3000 <[email protected]> Co-authored-by: Ying Hu <[email protected]> Signed-off-by: Haim Barad <[email protected]> * add chapter Paublications Signed-off-by: Haim Barad <[email protected]> * add date item in blogs Signed-off-by: Haim Barad <[email protected]> * update the content by table Signed-off-by: Haim Barad <[email protected]> * Hybridrag Framework (#341) * Hybrid RAG PR request Signed-off-by: Raghava, Sharath <[email protected]> * Signing off PR Signed-off-by: Raghava, Sharath <[email protected]> * filing RFC HybridRAG Signed-off-by: Raghava, Sharath <[email protected]> --------- Signed-off-by: Raghava, Sharath <[email protected]> Co-authored-by: Ying Hu <[email protected]> Signed-off-by: Haim Barad <[email protected]> * [Bug: 234] Replaced Low res image with High res image on top-left of the docs website (#325) * [Bug: 234] Replaced Low res image with High res image on top-left of the docs website Fixes #234 Co-authored-by: Piroozan, Nariman <[email protected]> Signed-off-by: Ghosh, Soumyadip <[email protected]> Signed-off-by: Jaini, Pallavi <[email protected]> Signed-off-by: Kavulya, Soila <[email protected]> Signed-off-by: Rajabose, Shifani <[email protected]> Signed-off-by: Shifani Rajabose <[email protected]> * Add blogs page to OPEA (#318) * Create index.rst * Update index.rst Signed-off-by: Shifani Rajabose <[email protected]> --------- Signed-off-by: Shifani Rajabose <[email protected]> Co-authored-by: Pranav Gupta <[email protected]> Co-authored-by: Malini Bhandaru <[email protected]> Signed-off-by: Haim Barad <[email protected]> * Update conf.py Changed datatype to fix the error/warning Signed-off-by: Haim Barad <[email protected]> * Add build_chatbot_blog Signed-off-by: Haim Barad <[email protected]> * Fefine the format Signed-off-by: Haim Barad <[email protected]> * A brief introduction of OPEA in first part Signed-off-by: Haim Barad <[email protected]> * reformat AudioQnA sample guide (#323) Signed-off-by: intel-whye <[email protected]> Co-authored-by: intel-whye <[email protected]> Co-authored-by: Rachel R <[email protected]> Co-authored-by: Ying Hu <[email protected]> Signed-off-by: Haim Barad <[email protected]> * Revert "Add build_chatbot_blog" (#342) * Revert "A brief introduction of OPEA in first part" This reverts commit 9342278. * Revert "Fefine the format" This reverts commit fd1e2f2. * Revert "Add build_chatbot_blog" This reverts commit 0126778. Signed-off-by: Haim Barad <[email protected]> * doc:Add emeritus code owners page (#312) * doc:Add emeritus code owners page Signed-off-by: Wang,Le3 <[email protected]> * remove lines Signed-off-by: Wang,Le3 <[email protected]> --------- Signed-off-by: Wang,Le3 <[email protected]> Signed-off-by: Haim Barad <[email protected]> * Added link about routellm. Signed-off-by: Haim Barad <[email protected]> * Added note for text-based inputs Signed-off-by: Haim Barad <[email protected]> * DCO checkin this commit... Signed-off-by: Haim Barad <[email protected]> --------- Signed-off-by: Haim Barad <[email protected]> Signed-off-by: chrisahsiong23 <[email protected]> Signed-off-by: Sun, Xuehao <[email protected]> Signed-off-by: Katherine Druckman <[email protected]> Signed-off-by: Abolfazl Shahbazi <[email protected]> Signed-off-by: Yu Wang <[email protected]> Signed-off-by: Sakari Poussa <[email protected]> Signed-off-by: JoshuaL3000 <[email protected]> Signed-off-by: Raghava, Sharath <[email protected]> Signed-off-by: Shifani Rajabose <[email protected]> Signed-off-by: intel-whye <[email protected]> Signed-off-by: Wang,Le3 <[email protected]> Co-authored-by: Pranav Gupta <[email protected]> Co-authored-by: chrisahsiong23 <[email protected]> Co-authored-by: Sun, Xuehao <[email protected]> Co-authored-by: Shifani Rajabose <[email protected]> Co-authored-by: Malini Bhandaru <[email protected]> Co-authored-by: Katherine Druckman <[email protected]> Co-authored-by: kdruckman <[email protected]> Co-authored-by: Abolfazl Shahbazi <[email protected]> Co-authored-by: Yu-amd <[email protected]> Co-authored-by: Sakari Poussa <[email protected]> Co-authored-by: JoshuaL3000 <[email protected]> Co-authored-by: Ying Hu <[email protected]> Co-authored-by: ZhangJianyu <[email protected]> Co-authored-by: intelsharath <[email protected]> Co-authored-by: Wang, Xigui <[email protected]> Co-authored-by: Hao Ruan <[email protected]> Co-authored-by: intel-whye <[email protected]> Co-authored-by: Rachel R <[email protected]> Co-authored-by: xiguiw <[email protected]> Co-authored-by: wangleflex <[email protected]>

poussa requested review from chensuyue, chickenrae, ftian1, mkbhanda, preethivenkatesh and tomlenth as code owners March 10, 2025 10:17

mkbhanda reviewed Mar 10, 2025

View reviewed changes

eero-t reviewed Mar 11, 2025

View reviewed changes

ashahba requested changes Mar 12, 2025

View reviewed changes

poussa force-pushed the oim-rfc branch 2 times, most recently from 9865d85 to 5b8dd15 Compare March 12, 2025 07:37

eero-t reviewed Mar 12, 2025

View reviewed changes

community/rfcs/25-03-07-OPEA-001-OIM-Operator.md Outdated Show resolved Hide resolved

poussa force-pushed the oim-rfc branch from 057d77d to 6b3b38b Compare March 12, 2025 17:48

eero-t approved these changes Mar 12, 2025

View reviewed changes

joshuayao requested review from lkk12014402 and lvliang-intel March 13, 2025 00:27

ftian1 reviewed Mar 13, 2025

View reviewed changes

community/rfcs/25-03-07-OPEA-001-OIM-Operator.md Show resolved Hide resolved

lkk12014402 reviewed Mar 13, 2025

View reviewed changes

community/rfcs/25-03-07-OPEA-001-OIM-Operator.md Show resolved Hide resolved

poussa force-pushed the oim-rfc branch from 7ff741c to 59c96c0 Compare March 14, 2025 07:24

ftian1 approved these changes Mar 20, 2025

View reviewed changes

poussa added 5 commits March 20, 2025 10:29

[RFC] OPEA Inference Microservices (OIM)

2263e8d

Signed-off-by: Sakari Poussa <[email protected]>

review fixes

fa3ce75

Signed-off-by: Sakari Poussa <[email protected]>

review fixes

66e8fb5

Signed-off-by: Sakari Poussa <[email protected]>

review fixes

ed10953

Signed-off-by: Sakari Poussa <[email protected]>

RFC: add OIM operator diagram

67ae8cf

Signed-off-by: Sakari Poussa <[email protected]>

review fixes, add picture

78b930a

Signed-off-by: Sakari Poussa <[email protected]>

poussa force-pushed the oim-rfc branch from 59c96c0 to 78b930a Compare March 20, 2025 08:30

yinghu5 self-requested a review March 20, 2025 08:51

yinghu5 approved these changes Mar 20, 2025

View reviewed changes

lvliang-intel approved these changes Mar 20, 2025

View reviewed changes

ashahba approved these changes Mar 21, 2025

View reviewed changes

ashahba merged commit 5d4eda9 into opea-project:main Mar 21, 2025
4 checks passed

eero-t mentioned this pull request Mar 25, 2025

Need documentation for air-gapped (offline) on-prem deployment opea-project/GenAIExamples#1405

Closed

yinghu5 mentioned this pull request Mar 26, 2025

[Feature] [OIM] Make inference backend configurable opea-project/GenAIExamples#1525

Open

eero-t mentioned this pull request Mar 31, 2025

Provide a clear working example of how to orchestrate multiple services opea-project/GenAIComps#1272

Closed

Conversation

poussa commented Mar 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mkbhanda left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

eero-t left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Yu-amd commented Mar 11, 2025

Uh oh!

ashahba left a comment

Choose a reason for hiding this comment

Uh oh!

AlexHe99 commented Mar 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

poussa commented Mar 12, 2025

Uh oh!

Uh oh!

eero-t commented Mar 12, 2025

Uh oh!

Uh oh!

Uh oh!

poussa commented Mar 14, 2025

Uh oh!

ashahba left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

poussa commented Mar 10, 2025 •

edited

Loading

AlexHe99 commented Mar 12, 2025 •

edited

Loading