Add Sharding V1 broadcast and V2 allgather overlap optimize

[iosmers]

PR types

Performance optimization

PR changes

Others

Description

1、sharding 支持 broadcast overlap forward, allgather overlap forward
2、同时为sharding stage1 overlap优化添加一些约束项
allgather优化只针对stage1 v2
broadcast overlap优化只针对stage1 v1
logging_step必须大于1
使用Use_Casual_mask环境变量避免DToH的sync

[paddle-bot]

Thanks for your contribution!

[codecov]

Codecov Report

Attention: Patch coverage is 17.69231% with 428 lines in your changes missing coverage. Please review.

Project coverage is 53.97%. Comparing base (7aaa788) to head (4205525).
Report is 258 commits behind head on develop.

Files with missing lines	Patch %	Lines
paddlenlp/transformers/qwen/modeling_3D_auto.py	16.38%	393 Missing ⚠️
paddlenlp/trainer/training_args.py	0.00%	12 Missing ⚠️
...erimental/transformers/fused_transformer_layers.py	0.00%	7 Missing ⚠️
paddlenlp/transformers/gpt/modeling_auto.py	0.00%	4 Missing ⚠️
paddlenlp/peft/lora/lora_layers.py	0.00%	2 Missing ⚠️
paddlenlp/transformers/gemma/modeling.py	50.00%	2 Missing ⚠️
paddlenlp/transformers/gpt/modeling.py	50.00%	2 Missing ⚠️
paddlenlp/transformers/llama/modeling.py	60.00%	2 Missing ⚠️
paddlenlp/transformers/mixtral/modeling.py	50.00%	2 Missing ⚠️
paddlenlp/transformers/qwen/modeling.py	60.00%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #8499      +/-   ##
===========================================
- Coverage    54.25%   53.97%   -0.29%     
===========================================
  Files          617      618       +1     
  Lines        96368    96825     +457     
===========================================
- Hits         52288    52259      -29     
- Misses       44080    44566     +486     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[wawltor]

LGTM