[Feature] Config eviction_duration#7125
Conversation
|
Thanks for your contribution! |
PaddlePaddle-bot
left a comment
There was a problem hiding this comment.
🤖 AI Code Review | 2026-04-01
📋 Review 摘要
PR 概述:新增 radix tree 缓存节点的可配置驱逐时间(eviction_duration),将默认值从 5 分钟调整为 30 分钟
变更范围:golang_router/config、scheduler/handler、文档
影响面 Tag:[Scheduler] [Docs]
📝 PR 规范检查
PR 标题和描述基本符合规范。建议在描述中补充修改原因:
描述模板(可直接复制):
## Motivation
原默认驱逐时间(5分钟)过短,导致缓存命中率较低。调整为30分钟并支持用户自定义配置。
## Modifications
- 新增 eviction-duration-mins 配置项
- 更新默认驱逐时间为 30 分钟
- 更新文档和示例配置
问题
| 级别 | 文件 | 概述 |
|---|---|---|
| ❓ 疑问 | docs/router.md:198 |
文档标注默认值为 5 分钟,但代码实际默认值为 30 分钟 |
| ❓ 疑问 | config.example.yaml:12 |
示例配置值 30 与文档标注默认值 5 不一致 |
总体评价
功能实现正确,配置传递链路完整。但文档、示例配置、代码默认值之间存在不一致,需要作者确认预期的默认值应该是 5 分钟还是 30 分钟,并统一所有位置的说明。
| prefill-policy: "cache_aware" # Prefill scheduling policy in PD mode | ||
| decode-policy: "request_num" # Decode scheduling policy in PD mode | ||
| eviction-interval-secs: 60 # Cache eviction interval for CacheAware scheduling | ||
| eviction-duration-mins: 5 # Eviction duration for cache-aware radix tree nodes (minutes); default: 5 |
There was a problem hiding this comment.
❓ 疑问 默认值不一致
文档注释标注 default: 5,但代码中 config.go 的默认值已改为 30 分钟。
请确认预期的默认值:
- 如果默认值应为 30 分钟,请更新此处注释为
default: 30 - 如果默认值应为 5 分钟,请将
config.go中的默认值改回5
| prefill-policy: "cache_aware" | ||
| decode-policy: "request_num" | ||
| eviction-interval-secs: 60 | ||
| eviction-duration-mins: 30 # eviction duration for cache-aware radix tree nodes (minutes); default: 5 |
There was a problem hiding this comment.
❓ 疑问 示例值与注释矛盾
示例配置设置为 30,但注释写 default: 5。这会让用户困惑实际的默认行为。
建议:
- 如果 30 分钟是推荐值,注释改为
default: 30 - 或者删除注释中的 default 说明,仅保留功能描述
|
mouxin seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
* [Feature] Config eviction_duration * [Feature] Config eviction_duration * [Feature] Config eviction_duration * [Feature] Config eviction_duration --------- Co-authored-by: mouxin <[email protected]>
更新radix tree默认驱逐时间,新增用户可配置驱逐时间,并新增mixed也可用token counter