[MPK] Development Roadmap Q3 2025

[jiazhihao]

This issue summarizes the main ongoing tasks for the Mirage Persistent Kernel project. We welcome feedback, contributions, and collaborations from the community!

High-Priority Tasks

• Speculative decoding: a key advantage of MPK is achieving extremely low-latency for LLM serving. Speculative decoding offers a lossless approach to further reducing LLM latency. [MPK] Supporting Lookahead Decoding in MPK #329 [MPK] Supporting Prompt Lookup Decoding in MPK #328
• Quantization (FP8/INT8/INT4/FP4): LLM decoding is largely bottleneck by memory access. This task adds support for 8-bit and 4-bit quantization methods in MPK. [MPK] Various Quantization Support #330
• Enhanced support for Hopper and Blackwell architectures. This task extends MPK to support next-generation architectures such as NVIDIA Blackwell. Support Mirage on Blackwell GPU Architecture #262 Optimize the padding for Hopper GPU transpiler #265 Flexible cluster mapping #184
• Paged attention and continuous batching: we should update our attention tasks to support paging and continuous batching [Feature Request] - qwen3-8B demo; larger batch size #358
• Performance optimizations: reuse in-shared-memory tensors between tasks to reduce device memory access [MPK] Manage the shared memory usage per SM #331

Mid-Priority Tasks

• Support for MoE models and expert-parallelism [MPK] Supporting MoE models in MPK #332
• Paged and radix attention: the current attention task implements flashattention without paging or prefix reusing, both of which are critical for batched serving.
• Integration into existing LLM serving platform (vllm, sglang) vLLM integration #342
• Support for non-Transformer architectures such as [MPK] Potential megakernel support for pLSTM #324