Add loss, save_inference_model and logger by LielinJiang · Pull Request #53 · PaddlePaddle/hapi

LielinJiang · 2020-04-24T12:54:37Z

move Loss from model.py to loss.py
add save_inference_model
add logger
add unittest for multiple gpus train,eval and predict; transform; datasets; models;loss;logger.
add some comments
fix bug

heavengate · 2020-04-27T03:11:49Z

hapi/logger.py

+    local_rank = ParallelEnv().local_rank
+    if local_rank == 0:
+        ch = logging.StreamHandler(stream=sys.stdout)
+        ch.setLevel(logging.DEBUG)


should be set as log_level ? same as below

done, thanks

heavengate · 2020-04-27T03:12:00Z

hapi/loss.py

@@ -0,0 +1,103 @@
+# Copyright (c) 2019 PaddlePaddle Authors. All Rights Reserved.


2019 -> 2020

done, thanks

heavengate · 2020-04-27T03:12:37Z

hapi/loss.py

+__all__ = ['Loss', 'CrossEntropy', 'SoftmaxWithCrossEntropy']
+
+
+def to_list(value):


move these common functions to utils.py ?

done, thanks

heavengate · 2020-04-27T03:16:02Z

hapi/test/dist_mnist.py

@@ -0,0 +1,179 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.


move tests directory to under hapi/ will make whl contains test scripts, do we mean this? OTOH, move tests directory here, tests directory outside should be deleted?

看了下incubate目录，fleet下也有个test文件夹。把根目录下的tests删除了。到时候挪到框架中，还可以再变一下tests的目录

qingqing01 · 2020-04-27T04:56:30Z

examples/bmn/modeling.py

-from hapi.download import get_weights_path
+from hapi.model import Model
+from hapi.loss import Loss
+from hapi.download import get_weights_path_from_url


这个名字有点长，是否从url，可以内部判断吧，可以直接用download吗？返回weight的path

qingqing01 · 2020-04-27T05:02:13Z

hapi/model.py

+            params_filename(str|None): The name of file to save all related parameters.
+                                            If it is set None, parameters will be saved
+                                            in separate files .
+            export_for_deployment(bool): If True, programs are modified to only support


下面调用fluid.io.save_inference_model时始终传入的 infer_prog，而export_for_deployment=False时描述是为了训练。这里觉得可以先把export_for_deployment参数去掉

done，thanks

qingqing01 · 2020-04-27T05:07:39Z

hapi/test/dist_mnist.py

+        return x
+
+
+class MNIST(Model):


内部已经封装了LeNet了，觉得可以用，这里就不用写这些了

done, thanks

guoshengCS · 2020-04-27T06:43:26Z

hapi/logger.py

+    logger.propagate = False
+
+    format_str = '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
+    logging.basicConfig(format=format_str, level=log_level)


Maybe we'd better not to use logging.basicConfig, it seems make user's logging setting ignored afterwards. See PaddlePaddle/Paddle#17786

done, thanks.

guoshengCS · 2020-04-27T06:45:02Z

hapi/logger.py

+from paddle.fluid.dygraph.parallel import ParallelEnv
+
+
+@functools.lru_cache()


Would there be python version incompatibility for functools.lru_cache

removed, thanks.

LielinJiang added 12 commits April 24, 2020 12:22

move Loss to loss.py

e26905f

update download name

e08a624

fix windows label bug

37c7612

fix some bugs

e7d5356

add logger

b2cbdda

add logger

254fbcc

add unit test

5ac55a4

move loss to loss.py

d2c796b

refine save inference model

b9463b7

add license

32d5c59

remove no used comments

a7742a7

add unittest for evaluate and predict

6c77091

LielinJiang requested review from guoshengCS, heavengate, qingqing01 and xyzhou-puck April 27, 2020 03:09

heavengate reviewed Apr 27, 2020

View reviewed changes

resolve confilct

0033bc5

qingqing01 reviewed Apr 27, 2020

View reviewed changes

LielinJiang added 4 commits April 27, 2020 05:35

update log level set

c3930dc

move commen functions to utils.py

ee496dc

clean test code

b3ec7f3

remove no used import

4f78ebf

LielinJiang requested review from heavengate and qingqing01 April 27, 2020 05:46

move tests folder

f4f461b

heavengate previously approved these changes Apr 27, 2020

View reviewed changes

guoshengCS reviewed Apr 27, 2020

View reviewed changes

fix incompatibility and remove basicConfig

3e9f805

LielinJiang dismissed heavengate’s stale review via 3e9f805 April 27, 2020 08:56

LielinJiang requested review from guoshengCS and heavengate April 27, 2020 08:57

heavengate approved these changes Apr 27, 2020

View reviewed changes

qingqing01 approved these changes Apr 27, 2020

View reviewed changes

LielinJiang merged commit 2c6d409 into PaddlePaddle:master Apr 27, 2020

		@@ -0,0 +1,103 @@
		# Copyright (c) 2019 PaddlePaddle Authors. All Rights Reserved.

		__all__ = ['Loss', 'CrossEntropy', 'SoftmaxWithCrossEntropy']


		def to_list(value):

		@@ -0,0 +1,179 @@
		# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.

		from paddle.fluid.dygraph.parallel import ParallelEnv


		@functools.lru_cache()

Conversation

LielinJiang commented Apr 24, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

heavengate Apr 27, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

heavengate Apr 27, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LielinJiang Apr 27, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

LielinJiang commented Apr 24, 2020 •

edited

Loading

heavengate Apr 27, 2020 •

edited

Loading

heavengate Apr 27, 2020 •

edited

Loading

LielinJiang Apr 27, 2020 •

edited

Loading