Skip to content

[Bugfix] Only add Attention.kv_scale if kv cache quantization is enabled#5936

Merged
mgoin merged 2 commits intomainfrom
bugfix-explicit-kv-scale
Jun 28, 2024
Merged

[Bugfix] Only add `Attention.kv_scale` if kv cache quantization is enabled#5936
mgoin merged 2 commits intomainfrom
bugfix-explicit-kv-scale

Commits

Commits on Jun 27, 2024

Commits on Jun 28, 2024