ctc grad scale same with espnet#34729
Merged
ZeyuChen merged 26 commits intoPaddlePaddle:developfrom Aug 17, 2021
Merged
Conversation
|
Hi, It's a test PR, it will not trigger CI. If you want to trigger CI, please remove |
Shixiaowei02
previously approved these changes
Aug 11, 2021
zhangting2020
approved these changes
Aug 16, 2021
Contributor
zhangting2020
left a comment
zhangting2020
left a comment
There was a problem hiding this comment.
LGTM for skip_check_grad_ci
zh794390558
added a commit
to zh794390558/Paddle
that referenced
this pull request
Oct 26, 2021
This reverts commit 10f9644.
This was referenced Oct 26, 2021
zh794390558
added a commit
to zh794390558/Paddle
that referenced
this pull request
Oct 27, 2021
This reverts commit 10f9644.
ZeyuChen
pushed a commit
that referenced
this pull request
Oct 28, 2021
ZeyuChen
pushed a commit
that referenced
this pull request
Oct 28, 2021
ghost
pushed a commit
to piotrekobi/Paddle
that referenced
this pull request
Nov 3, 2021
* Revert "Align CTC grad scale same with ESPNet (PaddlePaddle#34729)" This reverts commit 10f9644. * ctc grad compute on gpu
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR types
New features
PR changes
OPs
Describe
CTC 算 loss有两种norm方式，一种是 batch_size 的 reduce，一种是 element_wise 的reduce。
这个PR是修正这两种 norm 方式下 ctc grad 的计算。
目前 paddle 的 norm 方式和 espnet 是有差异的，而且现在实现的方式并不常用。
所以新增两种 norm 方式，具体实现同 https://github.com/espnet/warp-ctc/blob/pytorch_bindings/pytorch_binding/warpctc_pytorch/__init__.py#L39
具体使用时需要用户自己根据使用的norm方式选在对应的选项对 grad 做处理。
修改文件过多是因为 ctc 实现了 LodTensor。
doc fix ctc api for #34729 docs#3709
grad支持GPU操作
espnet:
paddle:
paddle-gpu:
paddle-cpu vs paddle-gpu
测试case和espnet不同