Skip to content

create a test for quantized resnet50 - extended#16532

Closed
wojtuss wants to merge 3 commits intoPaddlePaddle:developfrom
sfraczek:sfraczek/analyzer_int8_resnet50_test2
Closed

create a test for quantized resnet50 - extended#16532
wojtuss wants to merge 3 commits intoPaddlePaddle:developfrom
sfraczek:sfraczek/analyzer_int8_resnet50_test2

Conversation

@wojtuss
Copy link

@wojtuss wojtuss commented Mar 28, 2019

This is an extended version of #16399.
It supports inference on a full ImageNet dataset preprocessed using the tool #16529

With the use of this test we get the following accuracy results on the preprocessed full imagenet val dataset:
ResNet50:
Avg top1 INT8 accuracy: 0.7640
Avg top1 FP32 accuracy: 0.7663
MobileNet-v1:
Avg top1 INT8 accuracy: 0.7039
Avg top1 FP32 accuracy: 0.7078

@sfraczek
Copy link

sfraczek commented Mar 28, 2019

It also contains bug fixes and other review fixes.

cfg->SwitchSpecifyInputNames(false);
cfg->SetCpuMathLibraryNumThreads(FLAGS_paddle_num_threads);
cfg->EnableMKLDNN();
cfg->pass_builder()->SetPasses(

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@luotao1 without this function it didn't work right due to bug with fuse passes being copied many times.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean the same bug in #16174?

@wojtuss
Copy link
Author

wojtuss commented Mar 28, 2019

@luotao1, this version requires a dataset with a different name on the server. @lidanqing-intel will send you the data.

@luotao1
Copy link
Contributor

luotao1 commented Mar 29, 2019

Is #16399 closed due to this PR?

@luotao1
Copy link
Contributor

luotao1 commented Mar 29, 2019

I upload the data and rerun the CI.

@luotao1
Copy link
Contributor

luotao1 commented Mar 29, 2019

[02:21:05]        Start 164: test_analyzer_int8_resnet50
[02:21:06]147/610 Test #164: test_analyzer_int8_resnet50 .....................***Exception: SegFault  0.40 sec
[02:21:06]WARNING: Logging before InitGoogleLogging() is written to STDERR
[02:21:06]W0329 02:21:05.684131 57099 init.cc:83] Cannot enable P2P access from 0 to 1
[02:21:06]W0329 02:21:05.684239 57099 init.cc:83] Cannot enable P2P access from 1 to 0
[02:21:06][==========] Running 2 tests from 1 test case.
[02:21:06][----------] Global test environment set-up.
[02:21:06][----------] 2 tests from Analyzer_int8_resnet50
[02:21:06][ RUN      ] Analyzer_int8_resnet50.quantization
[02:21:06]/paddle/paddle/fluid/inference/tests/api/analyzer_int8_image_classification_tester.cc:122: Failure
[02:21:06]Failed
[02:21:06]Couldn't open file: /root/.cache/inference_demo/int8/data.bin
[02:21:06]
[02:21:06]        Start 165: test_analyzer_int8_mobilenet
[02:21:06]148/610 Test #165: test_analyzer_int8_mobilenet ....................***Exception: SegFault  0.40 sec
[02:21:06]WARNING: Logging before InitGoogleLogging() is written to STDERR
[02:21:06]W0329 02:21:06.362413 57101 init.cc:83] Cannot enable P2P access from 0 to 1
[02:21:06]W0329 02:21:06.362510 57101 init.cc:83] Cannot enable P2P access from 1 to 0
[02:21:06][==========] Running 2 tests from 1 test case.
[02:21:06][----------] Global test environment set-up.
[02:21:06][----------] 2 tests from Analyzer_int8_resnet50
[02:21:06][ RUN      ] Analyzer_int8_resnet50.quantization
[02:21:06]/paddle/paddle/fluid/inference/tests/api/analyzer_int8_image_classification_tester.cc:122: Failure
[02:21:06]Failed
[02:21:06]Couldn't open file: /root/.cache/inference_demo/int8/data.bin
[02:21:06]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants