[RL] Support cpu tensor broadcast#7833
Conversation
|
Thanks for your contribution! |
CI报告基于以下代码生成（30分钟更新一次）: 1 任务总览存在 1 个 required 失败任务，需处理后方可合并。
2 任务状态汇总2.1 Required任务 : 9/10 通过
2.2 可选任务 — 28/32 通过
3 失败详情（仅 required）Run FastDeploy Unit Tests and Coverage / run_tests_with_coverage — 覆盖率不达标（置信度: 高）Run FastDeploy Unit Tests and Coverage / run_tests_with_coverage
覆盖率详情:
根因详情: 关键日志: 修复建议:
修复建议摘要: 为 dynamic_weight_manager.py L353-357 新增代码添加单元测试 关联变更: 链接: 查看日志 |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #7833 +/- ##
==========================================
Coverage ? 63.31%
==========================================
Files ? 462
Lines ? 64284
Branches ? 9854
==========================================
Hits ? 40700
Misses ? 20815
Partials ? 2769
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
CI报告基于以下代码生成（30分钟更新一次）: 1 任务总览
2 任务状态汇总2.1 Required任务 : 7/10 通过
2.2 可选任务 — 28/31 通过
3 失败详情（仅 required）Run Stable Tests / stable_tests — 测试失败（置信度: 低）Run Stable Tests / stable_tests
根因详情: 关键日志: 修复建议:
修复建议摘要: 确认 gloo backend 可用，或添加环境兼容性检查 关联变更: Extracted partial CE model tasks to run in CI. / run_ce_cases — 测试失败（置信度: 低）Extracted partial CE model tasks to run in CI. / run_ce_cases
根因详情: 关键日志: 修复建议:
修复建议摘要: 将 while 循环内的 new_group() 移至循环外，避免重复创建 关联变更: |
CI报告基于以下代码生成（30分钟更新一次）: 1 任务总览所有 Required 任务全部通过 ✅，建议合并（3 个 Optional 任务失败，不阻塞合并）。
2 任务状态汇总2.1 Required任务 : 4/4 通过
2.2 可选任务 — 17/20 通过
3 失败详情（仅 required）无 required 失败任务。 |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
PaddlePaddle-bot
left a comment
PaddlePaddle-bot
left a comment
There was a problem hiding this comment.
🤖 Paddle-CI-Agent | pr_review |
2026-05-18 12:15:33
📋 Review 摘要
PR 概述：将 RL 场景下 CPU tensor 信号广播从 NCCL/broadcast_object_list 路径切换到独立 gloo backend，消除无谓的 GPU kernel 调用和 D2H 同步拷贝
变更范围：fastdeploy/rl/dynamic_weight_manager.py、fastdeploy/worker/worker_process.py
影响面 Tag：[RL]
问题
| 级别 | 文件 | 概述 |
|---|---|---|
| 🟡 建议 | fastdeploy/rl/dynamic_weight_manager.py:357 |
循环内重复调用私有 API _get_group_map_by_name()，建议缓存引用 |
| 📝 PR 规范 | — | ## Modifications、## Usage or Command、## Accuracy Tests 段落内容为空，Checklist 全部未勾选 |
📝 PR 规范检查
标题格式合规（含官方 [RL] Tag）。描述中 ## Modifications、## Usage or Command、## Accuracy Tests 三个段落仅有占位注释、无实际内容，Checklist 全部未勾选，需补全。
PR 描述建议（可直接复制，必须复刻 checklist §D2 模板的完整结构）：
## Motivation
paddle 建立通信组时默认的 backend 是 NCCL，此时 `paddle.distributed.broadcast` 不支持广播 CPU tensor，`paddle.distributed.broadcast_object_list` 仍会调用 GPU kernel 并引入 DtoH 同步拷贝。通过改用 gloo backend 建组，实现纯 CPU socket 广播，完全绕开 NCCL 和 GPU，nsys 上不会有任何 CUDA kernel 和 DtoH/HtoD 操作。
## Modifications
- `fastdeploy/worker/worker_process.py`：
- `__init__` 中（当 `ranks > 1` 时）使用 `gloo` backend 创建独立通信组 `self.gloo_group`
- `_broadcast_model_weights_signal` 改用 `paddle.full(..., device="cpu")` 创建 CPU tensor + `paddle.distributed.broadcast` 广播，替换原来的 `broadcast_object_list`
- `event_loop_normal` 中两处调用从 `group=None` 改为 `group=self.gloo_group`
- `fastdeploy/rl/dynamic_weight_manager.py`：
- `clear_parameters` 中在调用全局 `shutdown_process_group()` 前，先将不具备 `shutdown()` 方法的 ProcessGroupGloo 从 paddle 注册表中移除，避免 AttributeError
## Usage or Command
N/A
## Accuracy Tests
N/A — 此 PR 不涉及模型前向计算变更，仅修改进程间通信机制。
## Checklist
- [x] Add at least a tag in the PR title.
- Tag list: [`[FDConfig]`,`[APIServer]`,`[Engine]`, `[Scheduler]`, `[PD Disaggregation]`, `[Executor]`, `[Graph Optimization]`, `[Speculative Decoding]`, `[RL]`, `[Models]`, `[Quantization]`, `[Loader]`, `[OP]`, `[KVCache]`, `[DataProcessor]`, `[BugFix]`, `[Docs]`, `[CI]`, `[Optimization]`, `[Feature]`, `[Benchmark]`, `[Others]`, `[XPU]`, `[HPU]`, `[GCU]`, `[DCU]`, `[Iluvatar]`, `[Metax]`]
- You can add new tags based on the PR content, but the semantics must be clear.
- [ ] Format your code, run `pre-commit` before commit.
- [ ] Add unit tests. Please write the reason in this PR if no unit tests.
- [x] Provide accuracy results.
- [ ] If the current PR is submitting to the `release` branch, make sure the PR has been submitted to the `develop` branch, then cherry-pick it to the `release` branch with the `[Cherry-Pick]` PR tag.总体评价
整体方案思路清晰正确，gloo backend 确实能完全绕开 NCCL 避免 GPU 同步；cleanup 代码中使用私有 API 且循环内重复调用建议小幅改进，无阻塞性问题。
|
|
||
| for name, pg in list(_get_group_map_by_name().items()): | ||
| if pg.process_group is not None and not hasattr(pg.process_group, "shutdown"): | ||
| _get_group_map_by_name().pop(name, None) |
There was a problem hiding this comment.
🟡 建议 在循环内重复调用 _get_group_map_by_name() 且使用私有 API
当前代码在循环内调用了两次 _get_group_map_by_name()——第一次 .items() 用于遍历，第二次 .pop() 用于删除。若 Paddle 内部实现返回同一个可变 dict 引用，则当前代码可正常工作；但若内部实现将来改为返回副本，则 pop 会静默失效，gloo group 无法被清除，shutdown_process_group() 仍会抛出 AttributeError。
同时，_get_group_map_by_name 是 Paddle 内部私有 API（以 _ 开头），可能在 Paddle 版本升级时无预警地变更或删除。
建议缓存引用，同时做防御性注释：
group_map = _get_group_map_by_name() # internal API, cache once
for name, pg in list(group_map.items()):
if pg.process_group is not None and not hasattr(pg.process_group, "shutdown"):
group_map.pop(name, None)
6 participants
Motivation
paddle 建立通信组时默认的 backend 是NCCL，此时 paddle.distributed.broadcast 不支持广播CPU tensor，paddle.distributed.broadcast_object_list 仍会调用GPU kernel并引入DtoH同步拷贝。
Modifications
Usage or Command
Accuracy Tests
Checklist
[FDConfig],[APIServer],[Engine],[Scheduler],[PD Disaggregation],[Executor],[Graph Optimization],[Speculative Decoding],[RL],[Models],[Quantization],[Loader],[OP],[KVCache],[DataProcessor],[BugFix],[Docs],[CI],[Optimization],[Feature],[Benchmark],[Others],[XPU],[HPU],[GCU],[DCU],[Iluvatar],[Metax]]pre-commitbefore commit.releasebranch, make sure the PR has been submitted to thedevelopbranch, then cherry-pick it to thereleasebranch with the[Cherry-Pick]PR tag.