Agent: add support for remote server by alexsin368 · Pull Request #1644 · opea-project/GenAIComps

alexsin368 · 2025-05-02T00:42:38Z

Description

Add support for usage of a remote server inference endpoint. Supports Enterprise Inference.

Related PR: opea-project/GenAIExamples#1900

Issues

N/A

Type of change

List the type of change like below. Please delete options that are not relevant.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds new functionality)
Breaking change (fix or feature that would break existing design and interface)
Others (enhancement, documentation, validation, etc.)

Dependencies

None

Tests

Tested with AgentQnA consuming this agent microservice where chat completions were showing on the UI.

Signed-off-by: alexsin368 <alex.sin@intel.com>

for more information, see https://pre-commit.ci

comps/agent/src/integrations/utils.py

comps/agent/src/integrations/config.py

Signed-off-by: alexsin368 <alex.sin@intel.com>

…GenAIComps into agent-remote-service Signed-off-by: alexsin368 <alex.sin@intel.com>

for more information, see https://pre-commit.ci

alexsin368

removed use_remote_service

Copilot

Pull Request Overview

This PR adds support for using a remote server inference endpoint by enabling the use of an API key when initializing the ChatOpenAI client. Key changes include:

Updating the ChatOpenAI initialization in utils.py to append "/v1" to the remote endpoint when an API key is provided.
Adding a new argparse argument "--api_key" and corresponding environment variable handling in config.py.
Updating README.md to document how to use remote servers with the new API key support.

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File	Description
comps/agent/src/integrations/utils.py	Adds conditional initialization of ChatOpenAI using an API key.
comps/agent/src/integrations/config.py	Supports reading the api_key from environment variables.
comps/agent/src/README.md	Updates documentation to instruct on running LLMs with remote servers.

Comments suppressed due to low confidence (1)

comps/agent/src/README.md:119

[nitpick] The documentation uses 'LLM_ENDPOINT_URL' while the configuration code expects 'llm_endpoint_url'. Consider aligning the environment variable naming for consistency.

- `LLM_ENDPOINT_URL`=<inference-endpoint>

yinghu5

LGTM, thank you!

comps/agent/src/integrations/utils.py

Signed-off-by: alexsin368 <alex.sin@intel.com>

comps/agent/src/integrations/utils.py

Signed-off-by: alexsin368 <alex.sin@intel.com>

for more information, see https://pre-commit.ci

This reverts commit b7c4acf.

Signed-off-by: alexsin368 <alex.sin@intel.com>

…GenAIComps into agent-remote-service Signed-off-by: alexsin368 <alex.sin@intel.com>

for more information, see https://pre-commit.ci

Signed-off-by: ZePan110 <ze.pan@intel.com> Signed-off-by: alexsin368 <alex.sin@intel.com>

Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com> Co-authored-by: Lv,Liang1 <liang1.lv@intel.com> Signed-off-by: alexsin368 <alex.sin@intel.com>

Signed-off-by: ZePan110 <ze.pan@intel.com> Signed-off-by: alexsin368 <alex.sin@intel.com>

Signed-off-by: alexsin368 <alex.sin@intel.com>

for more information, see https://pre-commit.ci Signed-off-by: alexsin368 <alex.sin@intel.com>

…pea-project#1643) * bug fix for dataingest url Signed-off-by: Mustafa <mustafa.cetin@intel.com> * add validation function Signed-off-by: Mustafa <mustafa.cetin@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * validation update Signed-off-by: Mustafa <mustafa.cetin@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update validation function Signed-off-by: Mustafa <mustafa.cetin@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Mustafa <mustafa.cetin@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: alexsin368 <alex.sin@intel.com>

…roject#1645) * Add MariaDB Vector third-party service MariaDB Vector was introduced since MariaDB Server 11.7 Signed-off-by: Razvan-Liviu Varzaru <razvan@mariadb.org> * Add retriever MariaDB Vector integration Signed-off-by: Razvan-Liviu Varzaru <razvan@mariadb.org> * Add dataprep MariaDB Vector integration Signed-off-by: Razvan-Liviu Varzaru <razvan@mariadb.org> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix CI failures - md5 is used for the primary key not as a security hash - fixed mariadb readme headers Signed-off-by: Razvan-Liviu Varzaru <razvan@mariadb.org> --------- Signed-off-by: Razvan-Liviu Varzaru <razvan@mariadb.org> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: alexsin368 <alex.sin@intel.com>

Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: alexsin368 <alex.sin@intel.com>

…oject#1676) * Expand test matrix, find all tests use 3rd party Dockerfiles Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: alexsin368 <alex.sin@intel.com>

Update README.md for first entry of OPEA Signed-off-by: alexsin368 <alex.sin@intel.com>

Signed-off-by: lvliang-intel <liang1.lv@intel.com> Signed-off-by: alexsin368 <alex.sin@intel.com>

…#1635) Signed-off-by: alexsin368 <alex.sin@intel.com>

* Fix huggingfacehub API upgrade issue Signed-off-by: lvliang-intel <liang1.lv@intel.com> Signed-off-by: alexsin368 <alex.sin@intel.com>

Signed-off-by: alexsin368 <alex.sin@intel.com>

Signed-off-by: lvliang-intel <liang1.lv@intel.com> Signed-off-by: alexsin368 <alex.sin@intel.com>

…pea-project#1694) Signed-off-by: lvliang-intel <liang1.lv@intel.com> Signed-off-by: alexsin368 <alex.sin@intel.com>

Signed-off-by: alexsin368 <alex.sin@intel.com>

This reverts commit b7c4acf. Signed-off-by: alexsin368 <alex.sin@intel.com>

Signed-off-by: alexsin368 <alex.sin@intel.com>

…GenAIComps into agent-remote-service Signed-off-by: alexsin368 <alex.sin@intel.com>

for more information, see https://pre-commit.ci

alexsin368 · 2025-05-15T01:42:22Z

need help resolving DCO. steps i followed are not working

louie-tsai

looks good to me

alexsin368 · 2025-05-15T22:00:44Z

ready to merge!

alexsin368 added 2 commits April 30, 2025 17:56

add support for remote server

369d5dc

Signed-off-by: alexsin368 <alex.sin@intel.com>

add steps to enable remote server

0f6191d

Signed-off-by: alexsin368 <alex.sin@intel.com>

alexsin368 requested review from lkk12014402 and minmin-intel as code owners May 2, 2025 00:42

[pre-commit.ci] auto fixes from pre-commit.com hooks

71f1608

for more information, see https://pre-commit.ci

alexsin368 mentioned this pull request May 2, 2025

AgentQnA - add support for remote server opea-project/GenAIExamples#1900

Merged

4 tasks

louie-tsai reviewed May 2, 2025

View reviewed changes

comps/agent/src/integrations/utils.py Outdated Show resolved Hide resolved

comps/agent/src/integrations/utils.py Outdated Show resolved Hide resolved

comps/agent/src/integrations/config.py Outdated Show resolved Hide resolved

alexsin368 and others added 3 commits May 2, 2025 16:58

remove use_remote_service

bbcda06

Signed-off-by: alexsin368 <alex.sin@intel.com>

Merge branch 'agent-remote-service' of https://github.com/alexsin368/…

03f81c6

…GenAIComps into agent-remote-service Signed-off-by: alexsin368 <alex.sin@intel.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

45cf931

for more information, see https://pre-commit.ci

alexsin368 commented May 3, 2025

View reviewed changes

yinghu5 requested review from Copilot, xuechendi and yinghu5 May 7, 2025 08:35

Copilot AI reviewed May 7, 2025

View reviewed changes

Merge branch 'main' into agent-remote-service

35c67df

yinghu5 requested a review from ftian1 as a code owner May 12, 2025 01:49

yinghu5 approved these changes May 12, 2025

View reviewed changes

Merge branch 'main' into agent-remote-service

836c0dd

minmin-intel reviewed May 13, 2025

View reviewed changes

comps/agent/src/integrations/utils.py Outdated Show resolved Hide resolved

add OpenAI models instructions, fix format of commands

4899f79

Signed-off-by: alexsin368 <alex.sin@intel.com>

louie-tsai reviewed May 14, 2025

View reviewed changes

comps/agent/src/integrations/utils.py Outdated Show resolved Hide resolved

minmin-intel approved these changes May 15, 2025

View reviewed changes

alexsin368 and others added 7 commits May 14, 2025 17:34

Merge branch 'main' into agent-remote-service

101d133

simplify ChatOpenAI instantiation

b7c4acf

Signed-off-by: alexsin368 <alex.sin@intel.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

6586657

for more information, see https://pre-commit.ci

Revert "simplify ChatOpenAI instantiation"

d288734

This reverts commit b7c4acf.

add back check and logic for llm_engine, set openai_key argument

848368f

Signed-off-by: alexsin368 <alex.sin@intel.com>

Merge branch 'agent-remote-service' of https://github.com/alexsin368/…

7d01d77

…GenAIComps into agent-remote-service Signed-off-by: alexsin368 <alex.sin@intel.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

53aaaa5

for more information, see https://pre-commit.ci

ZePan110 and others added 24 commits May 14, 2025 18:34

Provide ARCH option for lvm-video-llama image build (opea-project#1630)

a70201f

Signed-off-by: ZePan110 <ze.pan@intel.com> Signed-off-by: alexsin368 <alex.sin@intel.com>

Add sglang microservice for supporting llama4 model (opea-project#1640)

212e612

Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com> Co-authored-by: Lv,Liang1 <liang1.lv@intel.com> Signed-off-by: alexsin368 <alex.sin@intel.com>

Remove invalid codeowner. (opea-project#1642)

5fc478e

Signed-off-by: ZePan110 <ze.pan@intel.com> Signed-off-by: alexsin368 <alex.sin@intel.com>

add support for remote server

1fe684c

Signed-off-by: alexsin368 <alex.sin@intel.com>

add steps to enable remote server

bd68f54

Signed-off-by: alexsin368 <alex.sin@intel.com>

remove use_remote_service

23f1f56

Signed-off-by: alexsin368 <alex.sin@intel.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

d1d2ac1

for more information, see https://pre-commit.ci Signed-off-by: alexsin368 <alex.sin@intel.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

a9d9ad7

for more information, see https://pre-commit.ci Signed-off-by: alexsin368 <alex.sin@intel.com>

update PR reviewers (opea-project#1651)

5e1656f

Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: alexsin368 <alex.sin@intel.com>

Expand test matrix, find all tests use 3rd party Dockerfiles (opea-pr…

69fea0d

…oject#1676) * Expand test matrix, find all tests use 3rd party Dockerfiles Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: alexsin368 <alex.sin@intel.com>

fix the typo of README.md Comp (opea-project#1679)

388c264

Update README.md for first entry of OPEA Signed-off-by: alexsin368 <alex.sin@intel.com>

Fix request handle timeout issue (opea-project#1687)

3b42858

Signed-off-by: lvliang-intel <liang1.lv@intel.com> Signed-off-by: alexsin368 <alex.sin@intel.com>

FEAT: Enable OPEA microservices to start as MCP servers (opea-project…

928e0f7

…#1635) Signed-off-by: alexsin368 <alex.sin@intel.com>

Fix huggingface_hub API upgrade issue (opea-project#1691)

9be8f9f

* Fix huggingfacehub API upgrade issue Signed-off-by: lvliang-intel <liang1.lv@intel.com> Signed-off-by: alexsin368 <alex.sin@intel.com>

add OpenAI models instructions, fix format of commands

0ffa6a6

Signed-off-by: alexsin368 <alex.sin@intel.com>

Fix dataprep opensearch ingest issue (opea-project#1697)

f83070c

Signed-off-by: lvliang-intel <liang1.lv@intel.com> Signed-off-by: alexsin368 <alex.sin@intel.com>

Fix embedding issue with ArangoDB due to deprecated HuggingFace API (o…

72bc23b

…pea-project#1694) Signed-off-by: lvliang-intel <liang1.lv@intel.com> Signed-off-by: alexsin368 <alex.sin@intel.com>

simplify ChatOpenAI instantiation

b2d93ff

Signed-off-by: alexsin368 <alex.sin@intel.com>

Revert "simplify ChatOpenAI instantiation"

78001b0

This reverts commit b7c4acf. Signed-off-by: alexsin368 <alex.sin@intel.com>

add back check and logic for llm_engine, set openai_key argument

1f4b746

Signed-off-by: alexsin368 <alex.sin@intel.com>

Merge branch 'agent-remote-service' of https://github.com/alexsin368/…

79e0407

…GenAIComps into agent-remote-service Signed-off-by: alexsin368 <alex.sin@intel.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

45376b9

for more information, see https://pre-commit.ci

louie-tsai approved these changes May 15, 2025

View reviewed changes

yinghu5 merged commit fcf7a5a into opea-project:main May 16, 2025
13 checks passed

alexsin368 deleted the agent-remote-service branch May 16, 2025 23:37

joshuayao mentioned this pull request May 19, 2025

[Feature] Enable remote inference endpoints for examples opea-project/GenAIExamples#1973

Closed

27 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Agent: add support for remote server#1644

Agent: add support for remote server#1644
yinghu5 merged 40 commits intoopea-project:mainfrom
alexsin368:agent-remote-service

alexsin368 commented May 2, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alexsin368 left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

yinghu5 left a comment

Uh oh!

Uh oh!

Uh oh!

alexsin368 commented May 15, 2025

Uh oh!

louie-tsai left a comment

Uh oh!

alexsin368 commented May 15, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

11 participants

Conversation

alexsin368 commented May 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Issues

Type of change

Dependencies

Tests

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alexsin368 left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

yinghu5 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

alexsin368 commented May 15, 2025

Uh oh!

louie-tsai left a comment

Choose a reason for hiding this comment

Uh oh!

alexsin368 commented May 15, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

11 participants

alexsin368 commented May 2, 2025 •

edited

Loading