Skip to content

Add fusion_lstm INT8 PTQ#35334

Merged
jczaja merged 3 commits intoPaddlePaddle:developfrom
wozna:ptq_fusion_lstm
Sep 6, 2021
Merged

Add fusion_lstm INT8 PTQ#35334
jczaja merged 3 commits intoPaddlePaddle:developfrom
wozna:ptq_fusion_lstm

Conversation

@wozna
Copy link

@wozna wozna commented Aug 31, 2021 •
edited
Loading

PR types

New features

PR changes

OPs

Describe

This PR:

  • adds fusion_lstm op to post-training quantization with calculation scales
  • adds UT for this
  • adds test with PTQ inference for lstm model
  • changes MKLDNNMemoryFormat for any for inputs in fusion_lstm op. Thanks to this, OneDNN can choose the most optimal format.

@paddle-bot-old
Copy link

paddle-bot-old bot commented Aug 31, 2021

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@wozna wozna changed the title Add fusion_lstm INT8 PTQ [DO NOT MERGE] Add fusion_lstm INT8 PTQ Aug 31, 2021
@wozna wozna changed the title [DO NOT MERGE] Add fusion_lstm INT8 PTQ Add fusion_lstm INT8 PTQ Sep 2, 2021
@lidanqing-vv
Copy link
Contributor

lidanqing-vv commented Sep 3, 2021 •
edited
Loading

@XieYunshen Hi, could you please approve this PR which added a UT and set its timeout property as 120). Thanks !

 0. You must have one QA (XieYunshen(Recommend) or chalsliu) approval for setting parameter RUN_TYPE as EXCLUSIVE, DIST, NIGHTLY, EXCLUSIVE:NIGHTLY or DISTNIGHTLY, or setting TIMEOUT properties.

@wzzju

@wozna wozna requested review from jakpiase and jczaja September 3, 2021 09:55
@lidanqing-vv lidanqing-vv self-requested a review September 3, 2021 12:04
lidanqing-vv
lidanqing-vv approved these changes Sep 3, 2021
View reviewed changes
Copy link
Contributor

@lidanqing-vv lidanqing-vv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lidanqing-vv
Copy link
Contributor

lidanqing-vv commented Sep 6, 2021 •
edited
Loading

@baoachun @juncaipeng Hi, This is for lstm post-training calibration. INT8 accuracy is the same as FP32 and peformance are improved.
Keeping the post-training calibration, we could use this support to generalize it to continuous integration CI 40 models INT8. This is what we are working now, but we are still solving version compatibility issues.
@wozna Please put acc and perf here. Thanks!

jczaja
jczaja approved these changes Sep 6, 2021
View reviewed changes
Copy link
Contributor

@jczaja jczaja left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jczaja jczaja merged commit 7ef04da into PaddlePaddle:develop Sep 6, 2021
AnnaTrainingG pushed a commit to AnnaTrainingG/Paddle that referenced this pull request Sep 29, 2021
@AnnaTrainingG
Add fusion_lstm INT8 PTQ (PaddlePaddle#35334)
d9a0341 
* Add fusion_lstm INT8 PTQ

* Correct mkldnn_cache_capacity and enable fc_lstm_fuse_pass only for this test

* Change mkldnn_cache_capacity
@wozna wozna deleted the ptq_fusion_lstm branch February 24, 2023 16:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@XieYunshen XieYunshen XieYunshen approved these changes

@jakpiase jakpiase Awaiting requested review from jakpiase

+2 more reviewers

@jczaja jczaja jczaja approved these changes

@lidanqing-vv lidanqing-vv lidanqing-vv approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

int8 Intel

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@wozna @lidanqing-vv @jczaja @XieYunshen