[AutoParallel] fix the grad_clip logic of auto_hybrid_pp

[zty-king]

PR Category

Auto Parallel

PR Types

Performance

Description

• 当前新的动半pp的GradientClipByGlobalNorm逻辑存在问题，计算结果错误，需要修复。
• 第一版修复的逻辑导致以下训练性能下降比较严重，同时发现一些重要的可优化点，因此做一些优化
  
• 优化如下：
• 1. 将all_gather+sum操作 修改为 all_reduce
• 2. 将每次调用时创建新的group的逻辑，修改成get_sub_mesh，并根据sub_mesh获取sub_group，从而防止大量group的创建导致性能下降。

[paddle-bot]

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 55.55556% with 4 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (develop@68835a8). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
python/paddle/nn/clip.py	55.55%	4 Missing ⚠️

❌ Your patch status has failed because the patch coverage (55.55%) is below the target coverage (90.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##             develop   #74409   +/-   ##
==========================================
  Coverage           ?   55.55%           
==========================================
  Files              ?        1           
  Lines              ?        9           
  Branches           ?        0           
==========================================
  Hits               ?        5           
  Misses             ?        4           
  Partials           ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[zty-king]

/re-run all-failed

[zty-king]

本地测试如下：

[xuxinyi389]

感觉 is_pp_enable 逻辑不够简洁，可以改成下方代码：

# Check for auto hybrid pipeline parallelism and source mesh existence
if flag_auto_hybrid_pp and src_mesh is not None:
    g_mesh = dist.get_mesh()
    
    # Check if mesh exists and pipeline parallelism is enabled ("pp" dim size > 1)
    if g_mesh and "pp" in g_mesh.dim_names and g_mesh.get_dim_size("pp") > 1:
        
        # Get the pipeline parallelism subgroup for communication
        pp_group = g_mesh.get_submesh_with_dim("pp").get_group("pp")
        
        # Perform all-reduce on the local tensor value across the PP group
        global_norm_var_local = global_norm_var._local_value()
        dist.all_reduce(
            global_norm_var_local,
            op=dist.ReduceOp.SUM,
            group=pp_group,
        )
        
        # Re-shard the tensor with the reduced value
        global_norm_var = dist.shard_tensor(
            global_norm_var_local,
            global_norm_var.process_mesh,
            global_norm_var.placements,
        )

[zty-king]

/re-run all-failed

[xuxinyi389]

LGTM

[From00]

LGTM