Skip to content

Fix request handle timeout issue#1687

Merged
chensuyue merged 1 commit intoopea-project:mainfrom
lvliang-intel:fix_request_timeout_issue
May 12, 2025
Merged

Fix request handle timeout issue#1687
chensuyue merged 1 commit intoopea-project:mainfrom
lvliang-intel:fix_request_timeout_issue

Conversation

@lvliang-intel
Copy link
Copy Markdown
Collaborator

@lvliang-intel lvliang-intel commented May 12, 2025

Description

Fix request handle timeout issue

Issues

1936

Type of change

List the type of change like below. Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds new functionality)
  • Breaking change (fix or feature that would break existing design and interface)
  • Others (enhancement, documentation, validation, etc.)

Dependencies

None

Tests

Local test and CI test

@lvliang-intel lvliang-intel mentioned this pull request May 12, 2025
Closed
10 tasks
@codecov
Copy link
Copy Markdown

codecov bot commented May 12, 2025 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Files with missing lines Coverage Δ
comps/cores/mega/orchestrator.py 90.43% <100.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@chensuyue chensuyue merged commit 8b957fe into opea-project:main May 12, 2025
16 checks passed
@eero-t
Copy link
Copy Markdown
Contributor

eero-t commented May 12, 2025 •
edited
Loading

Fix request handle timeout issue

This is not a fix, but a CI band-aid. How long the responses take depends on the LLM model, underlying HW and query options, so it should not be hard-coded, but configurable. E.g. with startup options for application timeout default & accepted range, and query parameter for changing the default withing that range.

And orchestrator error handling would also need to be improved; it should not just throw an exception and break user connection when error is returned, but return some meaningful error.

@eero-t eero-t mentioned this pull request May 12, 2025
Merged
4 tasks
alexsin368 pushed a commit to alexsin368/GenAIComps that referenced this pull request May 15, 2025
@lvliang-intel @alexsin368
Fix request handle timeout issue (opea-project#1687)
3b42858 
Signed-off-by: lvliang-intel <[email protected]>
Signed-off-by: alexsin368 <[email protected]>
yinghu5 added a commit that referenced this pull request May 16, 2025
@alexsin368 @pre-commit-ci
@yinghu5 @ZePan110 @lvliang-intel @MSCetin37 @RazvanLiviuVarzaru @chensuyue @Spycsh
Agent: add support for remote server (#1644)
fcf7a5a 
* add support for remote server

Signed-off-by: alexsin368 <[email protected]>

* add steps to enable remote server

Signed-off-by: alexsin368 <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* remove use_remote_service

Signed-off-by: alexsin368 <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add OpenAI models instructions, fix format of commands

Signed-off-by: alexsin368 <[email protected]>

* simplify ChatOpenAI instantiation

Signed-off-by: alexsin368 <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Revert "simplify ChatOpenAI instantiation"

This reverts commit b7c4acf.

* add back check and logic for llm_engine, set openai_key argument

Signed-off-by: alexsin368 <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Provide ARCH option for lvm-video-llama image build (#1630)

Signed-off-by: ZePan110 <[email protected]>
Signed-off-by: alexsin368 <[email protected]>

* Add sglang microservice for supporting llama4 model (#1640)

Signed-off-by: Ye, Xinyu <[email protected]>
Co-authored-by: Lv,Liang1 <[email protected]>
Signed-off-by: alexsin368 <[email protected]>

* Remove invalid codeowner. (#1642)

Signed-off-by: ZePan110 <[email protected]>
Signed-off-by: alexsin368 <[email protected]>

* add support for remote server

Signed-off-by: alexsin368 <[email protected]>

* add steps to enable remote server

Signed-off-by: alexsin368 <[email protected]>

* remove use_remote_service

Signed-off-by: alexsin368 <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: alexsin368 <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: alexsin368 <[email protected]>

* bug fix for chunk_size and overlap cause error in dataprep ingestion (#1643)

* bug fix for dataingest url

Signed-off-by: Mustafa <[email protected]>

* add validation function

Signed-off-by: Mustafa <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* validation update

Signed-off-by: Mustafa <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update validation function

Signed-off-by: Mustafa <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: Mustafa <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: alexsin368 <[email protected]>

* MariaDB Vector integrations for retriever & dataprep services (#1645)

* Add MariaDB Vector third-party service

MariaDB Vector was introduced since MariaDB Server 11.7

Signed-off-by: Razvan-Liviu Varzaru <[email protected]>

* Add retriever MariaDB Vector integration

Signed-off-by: Razvan-Liviu Varzaru <[email protected]>

* Add dataprep MariaDB Vector integration

Signed-off-by: Razvan-Liviu Varzaru <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix CI failures

- md5 is used for the primary key not as a security hash
- fixed mariadb readme headers

Signed-off-by: Razvan-Liviu Varzaru <[email protected]>

---------

Signed-off-by: Razvan-Liviu Varzaru <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: alexsin368 <[email protected]>

* update PR reviewers (#1651)

Signed-off-by: chensuyue <[email protected]>
Signed-off-by: alexsin368 <[email protected]>

* Expand test matrix, find all tests use 3rd party Dockerfiles (#1676)

* Expand test matrix, find all tests use 3rd party Dockerfiles

Signed-off-by: chensuyue <[email protected]>
Signed-off-by: alexsin368 <[email protected]>

* fix the typo of README.md Comp (#1679)

Update README.md for first entry of OPEA

Signed-off-by: alexsin368 <[email protected]>

* Fix request handle timeout issue (#1687)

Signed-off-by: lvliang-intel <[email protected]>
Signed-off-by: alexsin368 <[email protected]>

* FEAT: Enable OPEA microservices to start as MCP servers (#1635)

Signed-off-by: alexsin368 <[email protected]>

* Fix huggingface_hub API upgrade issue (#1691)

* Fix huggingfacehub API upgrade issue

Signed-off-by: lvliang-intel <[email protected]>
Signed-off-by: alexsin368 <[email protected]>

* add OpenAI models instructions, fix format of commands

Signed-off-by: alexsin368 <[email protected]>

* Fix dataprep opensearch ingest issue (#1697)

Signed-off-by: lvliang-intel <[email protected]>
Signed-off-by: alexsin368 <[email protected]>

* Fix embedding issue with ArangoDB due to deprecated HuggingFace API (#1694)

Signed-off-by: lvliang-intel <[email protected]>
Signed-off-by: alexsin368 <[email protected]>

* simplify ChatOpenAI instantiation

Signed-off-by: alexsin368 <[email protected]>

* Revert "simplify ChatOpenAI instantiation"

This reverts commit b7c4acf.

Signed-off-by: alexsin368 <[email protected]>

* add back check and logic for llm_engine, set openai_key argument

Signed-off-by: alexsin368 <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: alexsin368 <[email protected]>
Signed-off-by: ZePan110 <[email protected]>
Signed-off-by: Ye, Xinyu <[email protected]>
Signed-off-by: Mustafa <[email protected]>
Signed-off-by: Razvan-Liviu Varzaru <[email protected]>
Signed-off-by: chensuyue <[email protected]>
Signed-off-by: lvliang-intel <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ying Hu <[email protected]>
Co-authored-by: ZePan110 <[email protected]>
Co-authored-by: Liang Lv <[email protected]>
Co-authored-by: Mustafa <[email protected]>
Co-authored-by: Razvan Liviu Varzaru <[email protected]>
Co-authored-by: chen, suyue <[email protected]>
Co-authored-by: Spycsh <[email protected]>
This was referenced Jun 23, 2025
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@chensuyue chensuyue chensuyue approved these changes

@letonghan letonghan letonghan approved these changes

@ftian1 ftian1 Awaiting requested review from ftian1 ftian1 is a code owner

@Spycsh Spycsh Awaiting requested review from Spycsh Spycsh is a code owner

@lkk12014402 lkk12014402 Awaiting requested review from lkk12014402 lkk12014402 is a code owner

@minmin-intel minmin-intel Awaiting requested review from minmin-intel minmin-intel is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@lvliang-intel @eero-t @chensuyue @letonghan