Skip to content

Fix n_batch_size not set to context size for draft model

e448b5c
Select commit
Loading
Failed to load commit list.
Merged

server: improve speed of speculative decoding #1119

Fix n_batch_size not set to context size for draft model
e448b5c
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs