Skip to content

[AutoParallel] Support deepseekv2 with DP/MP#9862

Merged
wawltor merged 9 commits into
PaddlePaddle:developfrom
xuxinyi389:suppoort_deepseekv2_autoparallel
Feb 17, 2025
Merged

[AutoParallel] Support deepseekv2 with DP/MP#9862
wawltor merged 9 commits into
PaddlePaddle:developfrom
xuxinyi389:suppoort_deepseekv2_autoparallel

Conversation

@xuxinyi389
Copy link
Copy Markdown
Collaborator

@xuxinyi389 xuxinyi389 commented Feb 13, 2025

PR types

New features

PR changes

Models

Description

  1. deepseekv2/3 单卡模型,
  2. 中层API配置DP(Sharding)/MP跑通
image

@paddle-bot
Copy link
Copy Markdown

paddle-bot Bot commented Feb 13, 2025

Thanks for your contribution!

@codecov
Copy link
Copy Markdown

codecov Bot commented Feb 13, 2025

Codecov Report

Attention: Patch coverage is 22.38494% with 371 lines in your changes missing coverage. Please review.

Project coverage is 51.61%. Comparing base (58fc49f) to head (495d123).
Report is 347 commits behind head on develop.

Files with missing lines Patch % Lines
...addlenlp/transformers/deepseek_v2/modeling_auto.py 17.40% 337 Missing ⚠️
...addlenlp/transformers/deepseek_v3/modeling_auto.py 50.00% 34 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #9862      +/-   ##
===========================================
- Coverage    52.14%   51.61%   -0.53%     
===========================================
  Files          734      740       +6     
  Lines       116261   117708    +1447     
===========================================
+ Hits         60622    60753     +131     
- Misses       55639    56955    +1316     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@DrownFish19 DrownFish19 changed the title suppoort_deepseekv2_autoparallel_with_DP/MP [AutoParallel] Support deepseekv2 with DP/MP Feb 13, 2025
Comment thread paddlenlp/transformers/deepseek_v2/modeling_auto.py Outdated
Comment thread paddlenlp/transformers/deepseek_v2/modeling_auto.py Outdated
Comment thread paddlenlp/transformers/deepseek_v2/modeling_auto.py Outdated
Comment thread paddlenlp/transformers/deepseek_v2/modeling_auto.py
Comment thread paddlenlp/transformers/deepseek_v2/modeling_auto.py
DrownFish19
DrownFish19 previously approved these changes Feb 14, 2025
Copy link
Copy Markdown
Collaborator

@DrownFish19 DrownFish19 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Copy Markdown
Collaborator

@DrownFish19 DrownFish19 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wawltor wawltor merged commit 5eeb7aa into PaddlePaddle:develop Feb 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants