[paddle inference cpu]update cpu inference

[bukejiyu]

PR types

New features

PR changes

Others

Description

pcard-71500
相关pr #8634
1.增加fp16_int8混合精度推理 8463B bs=1 输入/输出=128/15 首token时延 463.775ms next token时延 33.409ms
2.支持 cachekv int8
3.更新readme

[paddle-bot]

Thanks for your contribution!

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 157 lines in your changes missing coverage. Please review.

Project coverage is 53.82%. Comparing base (a2f9558) to head (1d80c2d).
Report is 229 commits behind head on develop.

Files with missing lines	Patch %	Lines
...erimental/transformers/fused_transformer_layers.py	0.00%	127 Missing ⚠️
...dlenlp/experimental/transformers/llama/modeling.py	0.00%	27 Missing ⚠️
...enlp/experimental/transformers/generation_utils.py	0.00%	3 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #8984      +/-   ##
===========================================
- Coverage    54.24%   53.82%   -0.43%     
===========================================
  Files          650      652       +2     
  Lines       104307   104468     +161     
===========================================
- Hits         56583    56231     -352     
- Misses       47724    48237     +513     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.