Skip to content

[BugFix] Fix bugs in /v1/abort_requests interface from PR(#6992)#7176

Merged
Jiang-Jia-Jun merged 6 commits into
PaddlePaddle:developfrom
qwes5s5:abort_requests_fix
Apr 21, 2026
Merged

[BugFix] Fix bugs in /v1/abort_requests interface from PR(#6992)#7176
Jiang-Jia-Jun merged 6 commits into
PaddlePaddle:developfrom
qwes5s5:abort_requests_fix

Conversation

@qwes5s5
Copy link
Copy Markdown
Collaborator

@qwes5s5 qwes5s5 commented Apr 3, 2026

Motivation

💡 If this PR is a Cherry Pick, the PR title needs to follow the format by adding the [Cherry-Pick] label at the very beginning and appending the original PR ID at the end. For example, [Cherry-Pick][CI] Add check trigger and logic(#5191)

💡 如若此PR是Cherry Pick,PR标题需遵循格式,在最开始加上[Cherry-Pick]标签,以及最后面加上原PR ID,例如[Cherry-Pick][CI] Add check trigger and logic(#5191)

It was discovered that PR(#6992) introduced bugs that may trigger 500 errors or finish_reason validation failures when using multimodal or completions interfaces.

Modifications

Modified finish_reason to include the "abort" value.

Usage or Command

Accuracy Tests

Checklist

  • Add at least a tag in the PR title.
    • Tag list: [[FDConfig],[APIServer],[Engine], [Scheduler], [PD Disaggregation], [Executor], [Graph Optimization], [Speculative Decoding], [RL], [Models], [Quantization], [Loader], [OP], [KVCache], [DataProcessor], [BugFix], [Docs], [CI], [Optimization], [Feature], [Benchmark], [Others], [XPU], [HPU], [GCU], [DCU], [Iluvatar], [Metax]]
    • You can add new tags based on the PR content, but the semantics must be clear.
  • Format your code, run pre-commit before commit.
  • Add unit tests. Please write the reason in this PR if no unit tests.
  • Provide accuracy results.
  • If the current PR is submitting to the release branch, make sure the PR has been submitted to the develop branch, then cherry-pick it to the release branch with the [Cherry-Pick] PR tag.

@paddle-bot
Copy link
Copy Markdown

paddle-bot Bot commented Apr 3, 2026

Thanks for your contribution!

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Apr 3, 2026

Codecov Report

❌ Patch coverage is 66.66667% with 6 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (develop@3c8c82d). Learn more about missing BASE report.

Files with missing lines Patch % Lines
fastdeploy/engine/common_engine.py 40.00% 5 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             develop    #7176   +/-   ##
==========================================
  Coverage           ?   73.90%           
==========================================
  Files              ?      398           
  Lines              ?    55000           
  Branches           ?     8618           
==========================================
  Hits               ?    40649           
  Misses             ?    11632           
  Partials           ?     2719           
Flag Coverage Δ
GPU 73.90% <66.66%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@qwes5s5 qwes5s5 requested a review from Jiang-Jia-Jun April 9, 2026 07:25
PaddlePaddle-bot

This comment was marked as outdated.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

该 PR 用于修复 PR(#6992) 引入的 /v1/abort_requests 相关问题,避免在多模态/Completion 场景下触发 500 或 OpenAI 协议 finish_reason 校验失败,并补强 abort 清理路径的健壮性。

Changes:

  • 扩展 OpenAI 协议响应的 finish_reason 枚举,加入 "abort" 以通过校验。
  • 优化 abort 等待/清理逻辑:对已结束请求做过滤与集合清理,并在资源回收时清理 abort 相关集合,避免残留导致异常。
  • 更新/修复相关单测,使其与新的过滤逻辑一致。

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
tests/engine/test_common_engine.py 调整 _wait_abort_complete 相关用例,补齐 requests 字典以匹配新过滤逻辑
fastdeploy/entrypoints/openai/protocol.py finish_reason 增加 "abort",修复协议校验失败
fastdeploy/engine/sched/resource_manager_v1.py abort/finish 回收路径使用 discard 并补充清理 abort 集合
fastdeploy/engine/common_engine.py _wait_abort_complete 增强:过滤已结束目标并清理 abort 集合,降低卡住/500 风险

Comment thread fastdeploy/engine/common_engine.py
Comment thread fastdeploy/entrypoints/openai/protocol.py
PaddlePaddle-bot

This comment was marked as outdated.

Jiang-Jia-Jun
Jiang-Jia-Jun previously approved these changes Apr 14, 2026
PaddlePaddle-bot

This comment was marked as outdated.

@qwes5s5 qwes5s5 requested a review from Jiang-Jia-Jun April 17, 2026 03:11
PaddlePaddle-bot

This comment was marked as outdated.

PaddlePaddle-bot

This comment was marked as outdated.

@xyxinyang
Copy link
Copy Markdown
Collaborator

日志归属合理

Copy link
Copy Markdown

@PaddlePaddle-bot PaddlePaddle-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Code Review | 2026-04-21 11:57:28

📋 Review 摘要

PR 概述:修复 PR(#6992) 引入的 abort 请求接口 bug,包括 finish_reason 校验失败和潜在 500 错误
变更范围:Engine abort 流程、资源管理器清理逻辑、OpenAI 协议定义
影响面 TagEngine Scheduler APIServer

问题

级别 文件 概述
🔴 Bug common_engine.py:1596 getattr 在属性为 None 时不会返回 fallback,仍会触发 TypeError

详见行内评论。

总体评价

本 PR 的整体修复方向正确:removediscard 消除 KeyErrorfinish_reason 新增 "abort" 值、_wait_abort_complete 增加已完成请求的提前退出逻辑,这些都是合理的 bugfix。但 eos_token_idsgetattr fallback 存在一个遗留 bug——当属性存在但值为 None 时仍会崩溃,需要修复后才能安全合入。

engine_recv_first_token_time=request.metrics.engine_recv_first_token_time if request.metrics else now,
request_start_time=request.metrics.arrival_time if request.metrics else now,
)
eos_token_ids = getattr(request, "eos_token_ids", [0])
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Bug getattr(request, "eos_token_ids", [0])eos_token_idsNone 时不会返回 fallback 值 [0]

Request 类中 eos_token_ids 定义为 Optional[list[int]] = None(request.py:92/149),该属性始终存在但可能为 Nonegetattr 只在属性不存在时才返回默认值,当属性存在但值为 None 时,getattr 仍然返回 None,后续 eos_token_ids[0] 会触发 TypeError: 'NoneType' object is not subscriptable

建议修复:

eos_token_ids = getattr(request, "eos_token_ids", None) or [0]

或更明确地:

eos_token_ids = request.eos_token_ids if request.eos_token_ids else [0]

@Jiang-Jia-Jun Jiang-Jia-Jun merged commit 8883757 into PaddlePaddle:develop Apr 21, 2026
34 of 38 checks passed
Jiang-Jia-Jun pushed a commit that referenced this pull request Apr 22, 2026
@qwes5s5 qwes5s5 deleted the abort_requests_fix branch April 28, 2026 07:04
xiaoguoguo626807 pushed a commit to xiaoguoguo626807/FastDeploy that referenced this pull request May 7, 2026
…e#6992) (PaddlePaddle#7176)

* abort api bug fix

* bug fix

---------

Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants