Skip to content

[Feature] Config eviction_duration#7125

Merged
Smilencelsy merged 4 commits intoPaddlePaddle:developfrom
mouxinqq:develop
Apr 1, 2026
Merged

[Feature] Config eviction_duration#7125
Smilencelsy merged 4 commits intoPaddlePaddle:developfrom
mouxinqq:develop

Conversation

@mouxinqq
Copy link
Copy Markdown
Contributor

@mouxinqq mouxinqq commented Apr 1, 2026

更新radix tree默认驱逐时间,新增用户可配置驱逐时间,并新增mixed也可用token counter

@paddle-bot
Copy link
Copy Markdown

paddle-bot Bot commented Apr 1, 2026

Thanks for your contribution!

@paddle-bot paddle-bot Bot added the contributor External developers label Apr 1, 2026
Copy link
Copy Markdown

@PaddlePaddle-bot PaddlePaddle-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Code Review | 2026-04-01

📋 Review 摘要

PR 概述:新增 radix tree 缓存节点的可配置驱逐时间(eviction_duration),将默认值从 5 分钟调整为 30 分钟
变更范围:golang_router/config、scheduler/handler、文档
影响面 Tag[Scheduler] [Docs]

📝 PR 规范检查

PR 标题和描述基本符合规范。建议在描述中补充修改原因:

描述模板(可直接复制):

## Motivation
原默认驱逐时间(5分钟)过短,导致缓存命中率较低。调整为30分钟并支持用户自定义配置。

## Modifications
- 新增 eviction-duration-mins 配置项
- 更新默认驱逐时间为 30 分钟
- 更新文档和示例配置

问题

级别 文件 概述
❓ 疑问 docs/router.md:198 文档标注默认值为 5 分钟,但代码实际默认值为 30 分钟
❓ 疑问 config.example.yaml:12 示例配置值 30 与文档标注默认值 5 不一致

总体评价

功能实现正确,配置传递链路完整。但文档、示例配置、代码默认值之间存在不一致,需要作者确认预期的默认值应该是 5 分钟还是 30 分钟,并统一所有位置的说明。

Comment thread docs/online_serving/router.md Outdated
prefill-policy: "cache_aware" # Prefill scheduling policy in PD mode
decode-policy: "request_num" # Decode scheduling policy in PD mode
eviction-interval-secs: 60 # Cache eviction interval for CacheAware scheduling
eviction-duration-mins: 5 # Eviction duration for cache-aware radix tree nodes (minutes); default: 5
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❓ 疑问 默认值不一致

文档注释标注 default: 5,但代码中 config.go 的默认值已改为 30 分钟。

请确认预期的默认值:

  • 如果默认值应为 30 分钟,请更新此处注释为 default: 30
  • 如果默认值应为 5 分钟,请将 config.go 中的默认值改回 5

prefill-policy: "cache_aware"
decode-policy: "request_num"
eviction-interval-secs: 60
eviction-duration-mins: 30 # eviction duration for cache-aware radix tree nodes (minutes); default: 5
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❓ 疑问 示例值与注释矛盾

示例配置设置为 30,但注释写 default: 5。这会让用户困惑实际的默认行为。

建议:

  1. 如果 30 分钟是推荐值,注释改为 default: 30
  2. 或者删除注释中的 default 说明,仅保留功能描述

@CLAassistant
Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


mouxin seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@Smilencelsy Smilencelsy merged commit 6cae9b1 into PaddlePaddle:develop Apr 1, 2026
36 of 38 checks passed
xiaoguoguo626807 pushed a commit to xiaoguoguo626807/FastDeploy that referenced this pull request May 7, 2026
* [Feature] Config eviction_duration

* [Feature] Config eviction_duration

* [Feature] Config eviction_duration

* [Feature] Config eviction_duration

---------

Co-authored-by: mouxin <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor External developers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants