-
Notifications
You must be signed in to change notification settings - Fork 876
[Docathon][Add CN Doc No.50-51] #6375
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
28 changes: 28 additions & 0 deletions
28
docs/api/paddle/incubate/distributed/fleet/recompute_hybrid_cn.rst
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| .. _cn_api_paddle_incubate_distributed_fleet_recompute_hybrid: | ||
|
|
||
| recompute_hybrid | ||
| ------------------------------- | ||
|
|
||
| .. py:function:: paddle.incubate.distributed.fleet.recompute_hybrid(ctx, function, *args, **kwargs) | ||
| 在混合并行场景中,重新计算中间激活以节省内存。当前的混合并行重新计算具有局限性。 | ||
|
|
||
| #它不能处理以下几种情况: | ||
|
|
||
| |序号|情况描述| | ||
| | ---- | ---- | | ||
| |1 |重新计算的计算输出,有不需要梯度的张量| | ||
| |2 |前向输出张量没有梯度。通过 detach()可以暂时解决这个问题| | ||
| |3 |这里,我们只使用 float dtype 来区分输出张量中是否需要梯度| | ||
|
|
||
| 参数 | ||
| :::::::::::: | ||
|
|
||
| - **ctx** (dict) – 包括“mp_group”、“offload”和“partition”键。键“mp_group”(Group),表示活动被拆分到哪个组。键“offload”(bool,可选,默认值为 False)表示是否卸载到 cpu。键'partition'(bool,可选,默认值为 False)表示是否拆分 mp_group 中的活动。 | ||
| - **function** (paddle.nn.Layer) - 层序列的层,描述模型的前向通道的一部分,其中间激活将在前向阶段被释放以节省内存,并将在后向阶段被重新计算以进行梯度计算。 | ||
| - ***args** (Tensor) - 函数的输入(元组)。 | ||
| - ****kwargs** (Dict) - 函数的输入(字典)。 | ||
|
|
||
| 返回 | ||
| ::::::::: | ||
|
|
||
| args 和 kwargs 上的函数输出。 | ||
25 changes: 25 additions & 0 deletions
25
docs/api/paddle/incubate/distributed/fleet/recompute_sequential_cn.rst
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| .. _cn_api_paddle_incubate_distributed_fleet_recompute_sequential: | ||
|
|
||
| recompute_sequential | ||
| ------------------------------- | ||
|
|
||
| .. py:function:: paddle.incubate.distributed.fleet.recompute_sequential(ctx, functions, *args, **kwargs) | ||
| 重新计算中间激活以节省“顺序”模型的内存。使用“ctx”来传输一些上下文参数,它类似于“recompute_hybrid”API。 | ||
|
|
||
| 参数 | ||
| :::::::::::: | ||
|
|
||
| - **ctx** (dict) – 包含“segments”和“preserve_rng_state”键,键“segments”(int,默认 1)表示要在模型中创建的块的数量,键“preserve_rng_state”(bool,可选,默认=True)表示是否保存向前 rng。如果为 True,则在执行反向传播的正向重新计算时,将恢复最后一个正向 rng 值。 | ||
| - **function** (paddle.nn.Sequential) - 层序列的层,描述模型的前向通道的一部分,其中间激活将在前向阶段被释放以节省内存,并将在后向阶段被重新计算以进行梯度计算。 | ||
| - ***args** (Tensor) - 函数的输入(元组)。 | ||
| - ****kwargs** (Dict) - 函数的输入(字典)。 | ||
|
|
||
| 返回 | ||
| ::::::::: | ||
|
|
||
| args 和 kwargs 上的函数输出。 | ||
|
|
||
| 代码示例 | ||
| :::::::::::: | ||
|
|
||
| COPY-FROM: paddle.incubate.distributed.fleet.recompute_sequential |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
渲染有问题,可以看下。