[CI Failure] Fix fp8 kv cache on <SM90

[mgoin]

Purpose

The "Quantization Test" has been broken for a few days due to FA2 being chosen for fp8 kv cache on SM80 and SM89 after this PR removed V0 fallback #25033

https://buildkite.com/vllm/ci/builds/31806/steps/canvas?jid=0199716c-38f4-4409-affd-a1c35d55bc0e

[2025-09-22T14:14:38Z] (EngineCore_DP0 pid=12418) NotImplementedError: FlashAttention does not support fp8 kv-cache on this device.

Test Plan

Test Result

pytest -s -v "tests/quantization/test_fp8.py::test_kv_cache_model_load_and_run[False-neuralmagic/Meta-Llama-3-8B-Instruct-FP8-KV]" works on L40s

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[gemini-code-assist]

Code Review

This pull request aims to fix an issue where FlashAttention was incorrectly chosen for fp8 kv-cache on devices with compute capability less than 9.0, leading to a NotImplementedError. The change correctly modifies the attention backend selection logic to fall back to the Triton backend in this scenario. My main concern is that a pre-check in is_kv_cache_dtype_supported might still fail because it uses a simplified logic that doesn't account for this new fallback, potentially preventing the fix from being effective. I've left a specific comment detailing this critical issue. This should be addressed to ensure the fix is complete.

[chatgpt-codex-connector]

Codex Review: Here are some suggestions.

Reply with @codex fix comments to fix any unresolved comments.

About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you open a pull request for review, mark a draft as ready, or comment "@codex review". If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex fix this CI failure" or "@codex address that feedback".