Shape Mismatch Value_Cache for phi4 mini reasoning

[sempervictus]

Describe the bug

2025-06-18T05:43:57.006891Z  INFO mistralrs_core::prefix_cacher: PrefixCacherV2 is enabled. Expect higher multi-turn throughput for both text and multimodal.
2025-06-18T05:44:13.295510Z ERROR mistralrs_core::engine: step - Model failed with error: Msg("shape mismatch value_cache [1024, 2, 128, 32], expected (1024, 8, 16, 32, 8)")
2025-06-18T05:44:13.295566Z  INFO mistralrs_core: Dummy run completed in 16.29086093s.
2025-06-18T05:44:13.295949Z ERROR mistralrs_core::engine: step - Model failed with error: Msg("shape mismatch value_cache [1024, 2, 128, 32], expected (1024, 8, 16, 32, 8)")

thread '<unnamed>' panicked at /mistralrs/mistralrs-core/src/engine/mod.rs:417:25:
called `Result::unwrap()` on an `Err` value: SendError { .. }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Latest commit or version

Which commit or version you ran with.

[sempervictus]

Appears to be NCCL-related

[EricLBuehler]

Hi @sempervictus - this has been fixed in #1481 and is now working nicely with:

cargo run --features cuda,nccl -- -i --isq 8 run -m microsoft/Phi-4-mini-reasoning