[Performance] Introducing Prefix-Cached Chunked Prefill with flash-attn backend and 10% throughput gained under prompt <1K#6819
Closed
Juelianqvq wants to merge 1 commit intovllm-project:mainfrom
Juelianqvq:main
Closed
[Performance] Introducing Prefix-Cached Chunked Prefill with flash-attn backend and 10% throughput gained under prompt <1K#6819Juelianqvq wants to merge 1 commit intovllm-project:mainfrom Juelianqvq:main
Juelianqvq wants to merge 1 commit intovllm-project:mainfrom
Juelianqvq:main
