Skip to content

add common models and common params unit test for llm predictor.#9711

Merged
yuanlehome merged 7 commits into
PaddlePaddle:developfrom
aooxin:dev_20241227
Jan 3, 2025
Merged

add common models and common params unit test for llm predictor.#9711
yuanlehome merged 7 commits into
PaddlePaddle:developfrom
aooxin:dev_20241227

Conversation

@aooxin
Copy link
Copy Markdown
Contributor

@aooxin aooxin commented Dec 27, 2024

PR types

Others

PR changes

Others

Description

添加llm推理中常见模型和常见参数的单测
在A30单卡暂时耗时如下:
15 passed in 1507.02s (0:18:27)

覆盖下面的模型

  模型名称
1 Qwen2.5-7B-Instruct-tiny-nhl2
2 Qwen1.5-MoE-A2.7B-Chat-tiny-nhl2
3 Llama-2-7b-chat-tiny-nhl2
4 Meta-Llama-3.1-8B-Instruct-tiny-nhl2

Qwen2.5-1.5B-Instruct在参数测试中覆盖
70B和72B的在多卡中测试

覆盖下面的参数

两个 model:Qwen2.5-1.5B-Instruct和meta-llama/Llama-3.2-3B-Instruct

参数名称
use_fake_parameter True
inference_model False
append_attn True

覆盖分布式多卡推理场景

  模型名称
2 Qwen2.5-72B-Instruct-tiny-nhl2
3 Llama-2-70b-chat-tiny-nhl2
4 Meta-Llama-3.1-70B-Instruct-tiny-nhl2

@paddle-bot
Copy link
Copy Markdown

paddle-bot Bot commented Dec 27, 2024

Thanks for your contribution!

@codecov
Copy link
Copy Markdown

codecov Bot commented Dec 27, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 53.07%. Comparing base (3aa9f4c) to head (863ff0b).
Report is 271 commits behind head on develop.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #9711      +/-   ##
===========================================
- Coverage    53.30%   53.07%   -0.23%     
===========================================
  Files          724      718       -6     
  Lines       118271   114473    -3798     
===========================================
- Hits         63039    60759    -2280     
+ Misses       55232    53714    -1518     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@yuanlehome yuanlehome self-requested a review January 2, 2025 02:56
@yuanlehome yuanlehome merged commit bb85eb4 into PaddlePaddle:develop Jan 3, 2025
@aooxin aooxin deleted the dev_20241227 branch January 16, 2025 09:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants