POC: CUDA tensor parallel#1018

Closed

ikawrakow wants to merge 15 commits intomainfrom

Commits on Nov 27, 2025

Remove most of split mode row
Iwan Kawrakow
committed
WIP
Iwan Kawrakow
committed
WIP: also allocate the KV cache using tensor split
Iwan Kawrakow
committed
WIP: it runs with wrong result
Iwan Kawrakow
committed
WIP
Iwan Kawrakow
committed
This works, but it is slow
Iwan Kawrakow
committed
This is slightly better
Iwan Kawrakow
committed
Playing games with the scheduler
Iwan Kawrakow
committed
Fix attn split
Iwan Kawrakow
committed
Show memory used per device
Iwan Kawrakow
committed
Make it work with partial offload
Iwan Kawrakow
committed
Allow for f16 source in fused_rms_norm
Iwan Kawrakow
committed
This results in faster PP.
Iwan Kawrakow
committed
Rename split mode "row" to split mode "graph"
Iwan Kawrakow
committed

Commits on Nov 28, 2025

Leave FFN partial results as f16
Iwan Kawrakow
committed