[Performance] Support FP8 flashinfer TRTLLM MOE on Qwen3 and Qwen-3next

[jiahanc]

Purpose

• Integrate multiple routing methods for FP8 flashinfer trtllm MOE, currently only DS and Llama4
• Add FP8 flashinfer trtllm MOE support on Qwen3 and Qwen3-next

Test Plan

Qwen3-Next-80B-A3B-Instruct-FP8 on 2xB200 TP2

VLLM_USE_FLASHINFER_MOE_FP8=1 VLLM_FLASHINFER_MOE_BACKEND=latency VLLM_USE_DEEP_GEMM=0 VLLM_USE_TRTLLM_ATTENTION=0 VLLM_ATTENTION_BACKEND=FLASH_ATTN vllm serve Qwen/Qwen3-Next-80B-A3B-Instruct-FP8 \
    --max-num-batched-tokens 8192 \
    --max-model-len 16384 \
    --no-enable-prefix-caching \
    --async-scheduling \
    --compilation_config.pass_config.enable_fi_allreduce_fusion true \
    --compilation_config.pass_config.enable_noop true \
    --compilation_config.cudagraph_mode FULL_DECODE_ONLY \
    --compilation_config.splitting_ops [] \
    -tp 2 

lm_eval --model local-completions --tasks gsm8k --model_args model=Qwen/Qwen3-Next-80B-A3B-Instruct-FP8,base_url=http://0.0.0.0:8000/v1/completions,max_retries=3,tokenized_requests=False,timeout=1200,max_gen_toks=2048,max_length=8192 --batch_size 2048 --trust_remote_code --limit 0.5

Qwen3-30B-A3B-Instruct-2507-FP8 on 2xB200 TP2

VLLM_USE_FLASHINFER_MOE_FP8=1 VLLM_FLASHINFER_MOE_BACKEND=latency VLLM_USE_DEEP_GEMM=0 vllm serve Qwen/Qwen3-30B-A3B-Instruct-2507-FP8 \
    --max-num-batched-tokens 8192 \
    --max-model-len 16384 \
    --no-enable-prefix-caching \
    --async-scheduling \
    --compilation_config.pass_config.enable_fi_allreduce_fusion true \
    --compilation_config.pass_config.enable_noop true \
    --compilation_config.cudagraph_mode FULL_DECODE_ONLY \
    --compilation_config.splitting_ops [] \
    -tp 2

lm_eval --model local-completions --tasks gsm8k --model_args model=Qwen/Qwen3-30B-A3B-Instruct-2507-FP8,base_url=http://0.0.0.0:8000/v1/completions,max_retries=3,tokenized_requests=False,timeout=1200,max_gen_toks=2048,max_length=8192 --batch_size 2048 --trust_remote_code --limit 0.5

Test Result

Qwen3-Next-80B-A3B-Instruct-FP8

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.9515|±  |0.0084|
|     |       |strict-match    |     5|exact_match|↑  |0.9197|±  |0.0106|

Qwen/Qwen3-30B-A3B-Instruct-2507-FP8

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.9379|±  |0.0094|
|     |       |strict-match    |     5|exact_match|↑  |0.9364|±  |0.0095|

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @jiahanc.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[jiahanc]

Qwen3-Next-80B-A3B-Instruct-FP8 on 1xB200 1k/1k benchmark

[jiahanc]

@mgoin @pavanimajety may you help review the PR?
pre-commit failure is unrelated to any changed file. Might be introduced in other PRs

[mxz297]

If this PR is merged, can vllm still run with older flashinfer? We are internally just upgrading to flashinfer nightly-v0.4.1-20251027. This seems to bump flashinfer version again. Is it possible to consider some backward compatibility with older flashinfer version?

cc @houseroad @yeqcharlotte

[jiahanc]

If this PR is merged, can vllm still run with older flashinfer? We are internally just upgrading to flashinfer nightly-v0.4.1-20251027. This seems to bump flashinfer version again. Is it possible to consider some backward compatibility with older flashinfer version?

cc @houseroad @yeqcharlotte

Hi @mxz297 ,
There is no api change compared to v0.4.1. :)

[pavanimajety]

@alexm-redhat / @mgoin Could you please review? Thanks!

[pavanimajety]

LGTM, thanks for the PR

[jiahanc]

There is already 1 PR update FI version:#27952
Merge 27952 first and will rebase my PR

[jiahanc]

There is already 1 PR update FI version:#27952 Merge 27952 first and will rebase my PR

Rebased. Ready to merge 😄

[mgoin]

LGTM to get in now. We should make an issue to use RoutingMethod more broadly