[Speculative decoding][Re-take] Enable TP>1 speculative decoding#4840

Merged

cadedaniel merged 20 commits intovllm-project:mainfrom

comaniac:cody/tp_gt_1

May 16, 2024

Commits on May 9, 2024

hacks for tp gt 1
cadedaniel
committed

Commits on May 14, 2024

wip clean up spec worker
cadedaniel
committed
fix
cadedaniel
committed
fix
cadedaniel
committed
fix
cadedaniel
committed
Merge remote-tracking branch 'upstream/main' into tp_gt_1
cadedaniel
committed
fix
cadedaniel
committed
clean
cadedaniel
committed
clean
cadedaniel
committed
format
cadedaniel
committed
fix nccl failures
cadedaniel
committed
lint
cadedaniel
committed
ngram test
cadedaniel
committed
comment
cadedaniel
committed
format
cadedaniel
committed

Commits on May 15, 2024

Commits on May 16, 2024