[AMP] add state_dict and load_state_dict and unittest for class GradScaler #34300
Conversation
|
Thanks for your contribution! |
python/paddle/amp/grad_scaler.py
Outdated
| # required: gpu,xpu | ||
| import paddle | ||
| paddle.set_device('gpu') |
There was a problem hiding this comment.
新增的两行注释和代码在中英文文档预览里都没有看到，是生成预览之后新添加的？另外，其他代码示例是否也需要添加类似的代码？
There was a problem hiding this comment.
thanks，新增的注释和代码是在预览之后新增加的，我去更新一下预览代码，paddle.set_device('gpu')可以不添加，后面的GradScaler初始化的时候会针对device进行提醒，这里添加的这行代码已删除。
| Args: | ||
| new_init_loss_scaling(int): The new_init_loss_scaling used to update initial loss scaling factor. | ||
| new_init_loss_scaling(float): The new_init_loss_scaling used to update initial loss scaling factor. |
There was a problem hiding this comment.
确认一下类型变更之后对原来的情况是否完全兼容？比如参数类型检查是否有相关设置
There was a problem hiding this comment.
thanks，原本数据类型是float，初次添加注释的时候错误提交了int，这里修改为了正确的数据类型float。
| print('save_load:', out_use_state_dict[0], out_no_state_dict[0]) | ||
| self.assertTrue( | ||
| np.allclose( | ||
| out_use_state_dict[0], out_no_state_dict[0], atol=1.e-2)) |
There was a problem hiding this comment.
thanks，it is equal after set flag FLAGS_cudnn_deterministic=True.
| paddle.save(scaler.state_dict(), 'ResNet_model.pdparams') | ||
| dict_load = paddle.load('ResNet_model.pdparams') | ||
| scaler.load_state_dict(dict_load) |
There was a problem hiding this comment.
check if the state value are equal
There was a problem hiding this comment.
thanks，the state values are euqal.
4 participants
PR types
New features
PR changes
APIs
Describe
add state_dict and load_state_dict and unittest for class GradScaler
中文文档链接：http://10.136.157.23:8090/documentation/docs/zh/api/paddle/amp/GradScaler_cn.html?reviewVersion=jenkins-doc-review-2-191
英文文档链接：http://10.136.157.23:8090/documentation/docs/zh/api/paddle/amp/GradScaler_cn.html?reviewVersion=jenkins-doc-review-2-191