Enhance backward process#18700
Conversation
chengduoZH
force-pushed
the
fix_prun_backward
branch
from
2c1c92a to
b619b11
Compare
There was a problem hiding this comment.
Note that the label is not persistable in fluid.layers.cross_entropy.
There was a problem hiding this comment.
这个解释https://github.com/PaddlePaddle/Paddle/pull/18700/files#r305269149 可以加入单测代码中么？
这个单测的意思是，如果有并不需要的label，也能保持正确么？
test=develop
chengduoZH
force-pushed
the
fix_prun_backward
branch
from
b619b11 to
a59069f
Compare
test=develop
There was a problem hiding this comment.
这个解释https://github.com/PaddlePaddle/Paddle/pull/18700/files#r305269149 可以加入单测代码中么？
这个单测的意思是，如果有并不需要的label，也能保持正确么？
python/paddle/fluid/backward.py
Outdated
| return op_descs | ||
|
|
||
|
|
||
| def __find_not_need_ops(grad_op_descs, ops, input_grad_names_set): |
There was a problem hiding this comment.
- 可以多加一些注释，说一下整体剪枝的逻辑么，比如是先转成图再剪？
- 会有性能上的影响么？
There was a problem hiding this comment.
同@luotao02
虽然issue里面有详情，也建议在PR里面描述一下。
There was a problem hiding this comment.
可以多加一些注释，说一下整体剪枝的逻辑么，比如是先转成图再剪？
Done
会有性能上的影响么？
时间复杂度是O(N)，与之前构建backward过程的时间复杂度是一样的。
test=develop
chengduoZH
force-pushed
the
fix_prun_backward
branch
from
ab1e51c to
23a8872
Compare
test=develop
test=develop
| Pruning Program with Structural Analysis Method of Computational Graph. | ||
| The nodes of the computational graph composed of backward OPS should be | ||
| interconnected. If there are unconnected sub-graphs in the computational graph, | ||
| these sub-graphs should be cut off. |
There was a problem hiding this comment.
Maybe better to add comments for grad_op_descs, ops, input_grad_names_set
test=develop
luotao1
left a comment
luotao1
left a comment
There was a problem hiding this comment.
LGTM for update backward.py
4 participants
Prune the unnecessary op in backward.
Related PR #15955
Fix #15283