feat: QuantizedRotatingKVCache + KVSplit (K/V different bits)#1074

Open

deceptech-packet-ninja wants to merge 3 commits intoml-explore:mainfrom

deceptech-packet-ninja:feat/kv-cache-improvements

Commits on Mar 30, 2026

feat: add --kv-bits, --kv-group-size, --quantized-kv-start to server

deceptech-packet-ninja
and
claude
committed
feat: QuantizedRotatingKVCache + KVSplit (K/V different bits)

deceptech-packet-ninja
and
claude
committed

Commits on Apr 3, 2026

merge: resolve conflict with upstream batch generation refactor

deceptech-packet-ninja
and
claude
committed