-
Notifications
You must be signed in to change notification settings - Fork 5.9k
Added cast op oneDNN kernel for bf16/fp32 datatypes casting(FWD/BWD) #33056
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Thanks for your contribution! |
jczaja
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
|
@jakpiase Hi Please rename the PR title or make it more clear because I will use these titles for release note. |
python/paddle/fluid/tests/unittests/mkldnn/test_cast_mkldnn_op.py
Outdated
Show resolved
Hide resolved
python/paddle/fluid/tests/unittests/mkldnn/test_cast_mkldnn_op.py
Outdated
Show resolved
Hide resolved
python/paddle/fluid/tests/unittests/mkldnn/test_cast_mkldnn_op.py
Outdated
Show resolved
Hide resolved
arlesniak
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
python/paddle/fluid/tests/unittests/mkldnn/test_cast_mkldnn_op.py
Outdated
Show resolved
Hide resolved
arlesniak
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
|
@luotao1 Could you please start your review? |
| self.check_grad_with_place( | ||
| core.CPUPlace(), ["X"], | ||
| "Out", | ||
| check_dygraph=False, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Paddle/python/paddle/fluid/tests/unittests/op_test.py
Lines 1076 to 1090 in 394d8d4
| def check_output_with_place(self, | |
| place, | |
| atol=0, | |
| no_check_set=None, | |
| equal_nan=False, | |
| check_dygraph=True, | |
| inplace_atol=None): | |
| self.infer_dtype_from_inputs_outputs(self.inputs, self.outputs) | |
| if self.dtype == np.float64 and \ | |
| self.op_type not in op_threshold_white_list.NEED_FIX_FP64_CHECK_OUTPUT_THRESHOLD_OP_LIST: | |
| atol = 0 | |
| if self.is_bfloat16_op(): | |
| check_dygraph = False | |
| if hasattr(self, 'force_fp32_output') and getattr( |
since you add
check_dygraph = False when bf16_op in check_output_with_place, could you add same code in check_grad_with_place? Thus, you can decrease one approve
@jakpiase @lidanqing-intel @jczaja
@wzzju Please note this.
PR types
New features
PR changes
OPs
Describe
Added cast op for 4 types of casts (fp32->fp32, fp32->bf16, bf16->fp32, bf16->bf16) for enabling Word2Vec model. OneDNN kernel can be used both in FWD and BWD.