Skip to content

[PD] Add different TP sizes support for no-MLA models #6793

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
zhyncs merged 31 commits into from
Jun 25, 2025
Merged

[PD] Add different TP sizes support for no-MLA models #6793

Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
6ed77eb
support decode_tp > prefill_tp
Hongbosherlock May 28, 2025
9294eba
Merge branch 'sgl-project:main' into support_tp
Hongbosherlock May 29, 2025
9889917
support prefill_tp > decode_tp
Hongbosherlock May 29, 2025
3b38371
support different tp for no-MLA
Hongbosherlock May 30, 2025
72c5e08
Merge branch 'sgl-project:main' into support_tp
Hongbosherlock May 30, 2025
2ab0a14
clean code
Hongbosherlock May 31, 2025
3c8581f
optimize:remove group_concurrent_contiguous
Hongbosherlock Jun 2, 2025
c80399f
Merge branch 'sgl-project:main' into support_tp
Hongbosherlock Jun 2, 2025
9ba6379
fix Prefill TP > Decode TP
Hongbosherlock Jun 4, 2025
01c3387
Improve code robustness
Hongbosherlock Jun 4, 2025
6f547f2
minor fix
Hongbosherlock Jun 4, 2025
17b7dbc
format code
Hongbosherlock Jun 4, 2025
4b7c575
Merge branch 'sgl-project:main' into support_tp
Hongbosherlock Jun 7, 2025
93e5d21
support page_size>1
Hongbosherlock Jun 10, 2025
243a527
fix conflicts
Hongbosherlock Jun 10, 2025
91d5287
clean code
Hongbosherlock Jun 10, 2025
ba9c2d8
fix kv_indices_len
Hongbosherlock Jun 10, 2025
be2bb04
Merge branch 'main' into support_tp
Hongbosherlock Jun 11, 2025
ff6f488
some fixs
Hongbosherlock Jun 11, 2025
2293eab
Update conn.py
ShangmingCai Jun 11, 2025
886bd68
Merge branch 'main' into support_tp
ShangmingCai Jun 11, 2025
efbb101
update kvargs
Hongbosherlock Jun 12, 2025
4567b28
Merge branch 'main' into support_tp
Hongbosherlock Jun 12, 2025
a55245a
updates
Hongbosherlock Jun 12, 2025
000ccdb
resolve conflict
Hongbosherlock Jun 16, 2025
bc2690b
Merge branch 'sgl-project:main' into support_tp
Hongbosherlock Jun 23, 2025
ba65ef8
Merge branch 'main' into support_tp
ShangmingCai Jun 24, 2025
6eb9634
Fix sync status
ShangmingCai Jun 24, 2025
484a717
Merge branch 'main' into support_tp
ShangmingCai Jun 24, 2025
6eda16d
add batch transfer
Hongbosherlock Jun 24, 2025
f5bc385
Merge branch 'main' into support_tp
ShangmingCai Jun 25, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Loading