[ModelOpt] Load w13/w2_input_scale for all experts, nvfp4

[wenscarl]

This matches trt-llm usage. Use all experts' input scales to compute alpha and quantize. This change doesn't affect perf.

Purpose

Test Plan

VLLM_WORKER_MULTIPROC_METHOD="spawn" \
VLLM_USE_STANDALONE_COMPILE=0 \
VLLM_USE_FLASHINFER_MOE_FP4=1 \
VLLM_FLASHINFER_MOE_BACKEND="throughput" \
lm_eval --model vllm --model_args pretrained=nvidia/DeepSeek-R1-FP4,quantization=modelopt_fp4,data_parallel_size=8,enable_expert_parallel=True,tensor_parallel_size=1,max_model_len=2048,enforce_eager=True --trust_remote_code --tasks gsm8k --num_fewshot 5 --batch_size auto

Test Result

After change:

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.9431|±  |0.0064|
|     |       |strict-match    |     5|exact_match|↑  |0.9401|±  |0.0065|```

Before change:

Tasks	Version	Filter	n-shot	Metric		Value		Stderr
gsm8k	3	flexible-extract	5	exact_match	↑	0.934	±	0.0068
		strict-match	5	exact_match	↑	0.931	±	0.0070

---
<details>
<summary> Essential Elements of an Effective PR Description Checklist </summary>

- [ ] The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
- [ ] The test plan, such as providing test command.
- [ ] The test results, such as pasting the results comparison before and after, or e2e results
- [ ] (Optional) The necessary documentation update, such as updating `supported_models.md` and `examples` for a new model.
- [ ] (Optional) Release notes update. If your change is user facing, please update the release notes draft in the [Google Doc](https://docs.google.com/document/d/1YyVqrgX4gHTtrstbq8oWUImOyPCKSGnJ7xtTpmXzlRs/edit?tab=t.0).
</details>

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @wenscarl.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

[leejnau]

For the nvidia/DeepSeek-R1-0528-FP4-v2 model, in both TP4 and DP4 modes, with the FlashInfer backend, this PR raises the accuracy from ~2% to ~95%.

server (TP4):

VLLM_USE_FLASHINFER_MOE_FP4=1 VLLM_FLASHINFER_MOE_BACKEND="throughput" vllm serve nvidia/DeepSeek-R1-0528-FP4-v2 --quantization="modelopt_fp4" --trust-remote-code --gpu_memory_utilization=0.8 --tensor-parallel-size 4 --data-parallel-size 1

client:

python3 tests/evals/gsm8k/gsm8k_eval.py
commit: 29350922c64a808a6de3b0e31fbadc2aebd6ba3f
Accuracy: 0.025
commit: 281be34de010fd4b106341e8aa3996f01f121c61
Accuracy: 0.955

server (DP4):

VLLM_USE_FLASHINFER_MOE_FP4=1 VLLM_FLASHINFER_MOE_BACKEND="throughput" vllm serve nvidia/DeepSeek-R1-0528-FP4-v2 --quantization="modelopt_fp4" --trust-remote-code --gpu_memory_utilization=0.8 --tensor-parallel-size 1 --data-parallel-size 4

client:

python3 tests/evals/gsm8k/gsm8k_eval.py
commit: 29350922c64a808a6de3b0e31fbadc2aebd6ba3f
Accuracy: 0.022
commit: 281be34de010fd4b106341e8aa3996f01f121c61
Accuracy: 0.955

[mgoin]

Nit: looks like these comments are out of date

[mgoin]

Would be good to put these three lines together and leave a comment on what use_global_sf means in this case since we are in fused_moe/layer.py

[mgoin]

LGTM, although checking for local attrs in fused_moe/layer.py doesn't feel good to keep doing I don't have a better option atm