[AMP] add get() and set() for Grad_scaler by zhangbo9674 · Pull Request #33835 · PaddlePaddle/Paddle

zhangbo9674 · 2021-06-29T07:57:18Z

PR types

New features

PR changes

APIs

Describe

Add get() and set() function for amp properties：

scaler = paddle.amp.GradScaler(init_loss_scaling=1024)
print("get_enable=",scaler.get_enable())
print("get_decr_every_n_nan_or_inf=",scaler.get_decr_every_n_nan_or_inf())
print("get_decr_ratio=",scaler.get_decr_ratio())
print("get_incr_every_n_steps=",scaler.get_incr_every_n_steps())
print("get_incr_ratio=",scaler.get_incr_ratio())
print("get_init_loss_scaling=",scaler.get_init_loss_scaling())
print("get_use_dynamic_loss_scaling=",scaler.get_use_dynamic_loss_scaling())

scaler.set_decr_every_n_nan_or_inf(2)
print("get_decr_every_n_nan_or_inf=",scaler.get_decr_every_n_nan_or_inf())
scaler.set_decr_ratio(0.1)
print("get_decr_ratio=",scaler.get_decr_ratio())
scaler.set_incr_every_n_steps(200)
print("get_incr_every_n_steps=",scaler.get_incr_every_n_steps())
scaler.set_incr_ratio(3)
print("get_incr_ratio=",scaler.get_incr_ratio())
scaler.set_init_loss_scaling(100)
print("get_init_loss_scaling=",scaler.get_init_loss_scaling())

paddle-bot-old · 2021-06-29T07:57:27Z

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

TCChenlong · 2021-06-30T05:18:31Z

+    def is_use_loss_scaling(self):
+        """
+        Enable loss scaling or not.
+


这里需要加一个 Returns：然后说明返回类型，以及什么情况下会返回什么样的值

TCChenlong · 2021-06-30T05:18:56Z

+    def is_use_dynamic_loss_scaling(self):
+        """
+        Whether to use dynamic loss scaling.
+


同上，需要补充 Returns

TCChenlong · 2021-06-30T05:19:29Z

+
+        Examples:
+            .. code-block:: python
+                import paddle


.. code & import 这两行中间需要加一个空行否则预览会有问题

TCChenlong · 2021-06-30T05:19:37Z

+
+        Examples:
+            .. code-block:: python
+                import paddle


TCChenlong · 2021-06-30T05:19:44Z

+
+        Examples:
+            .. code-block:: python
+                import paddle


TCChenlong · 2021-06-30T05:19:53Z

+    def get_init_loss_scaling(self):
+        """
+        Return the initial loss scaling factor.
+


需要加Returns 同上

zhiqiu · 2021-06-30T08:00:19Z

        """
        return super(GradScaler, self).minimize(optimizer, *args, **kwargs)
+
+    def is_enable(self):


is_enable -> is_enabled

zhiqiu · 2021-06-30T08:04:34Z

+        """
+        return super(GradScaler, self).is_enable()
+
+    def is_use_dynamic_loss_scaling(self):


is_use_dynamic_loss_scaling -> is_dynamic_loss_scaling_used?

zhiqiu

LGTM

TCChenlong · 2021-06-30T10:43:21Z

+                                               incr_every_n_steps=1000,
+                                               decr_every_n_nan_or_inf=2,
+                                               use_dynamic_loss_scaling=True)
+                enable = scaler.get_enable()


scaler.get_enable -> scaler.is_enable

TCChenlong · 2021-06-30T10:44:03Z

+                                               incr_every_n_steps=1000,
+                                               decr_every_n_nan_or_inf=2,
+                                               use_dynamic_loss_scaling=True)
+                use_dynamic_loss_scaling = scaler.get_use_dynamic_loss_scaling()


scaler.get_use_dynamic_loss_scaling -> scaler.is_use_dynamic_loss_scaling

TCChenlong · 2021-06-30T10:44:14Z

+
+        Examples:
+            .. code-block:: python
+                import paddle


TCChenlong

LGTM

zhiqiu

LGTM

add get and set for Grad_scaler

4f462cc

refine some API name and comments

38b9226

TCChenlong reviewed Jun 30, 2021

View reviewed changes

refine API name and comments

5b59179

zhiqiu reviewed Jun 30, 2021

View reviewed changes

zhiqiu previously approved these changes Jun 30, 2021

View reviewed changes

TCChenlong reviewed Jun 30, 2021

View reviewed changes

refine some comments

d80e842

zhangbo9674 dismissed zhiqiu’s stale review via d80e842 June 30, 2021 11:15

TCChenlong approved these changes Jul 1, 2021

View reviewed changes

zhiqiu approved these changes Jul 1, 2021

View reviewed changes

zhiqiu merged commit 8568734 into PaddlePaddle:develop Jul 1, 2021

zhangbo9674 deleted the dev/amp_get_set branch September 14, 2022 02:22

Conversation

zhangbo9674 commented Jun 29, 2021

PR types

PR changes

Describe

Uh oh!

paddle-bot-old Bot commented Jun 29, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zhiqiu left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

TCChenlong left a comment

Choose a reason for hiding this comment

Uh oh!

zhiqiu left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants