[Bugfix][Quantization] Fix FP8 + EP

[tlrmchlsmth]

Fp8MoEMethod is reaching into the layer to get the number of experts during process_weights_after_loading, which isn't right when using expert parallelism.

VLLM_TEST_ENABLE_EP=1 vllm serve neuralmagic/Mixtral-8x7B-Instruct-v0.1-FP8 --tensor-parallel-size 2

Would hit the following error:

ERROR 02-24 19:29:54 [engine.py:400]   File "/home/tms/vllm/vllm/model_executor/model_loader/loader.py", line 167, in _process_weights_after_loading
ERROR 02-24 19:29:54 [engine.py:400]     quant_method.process_weights_after_loading(module)
ERROR 02-24 19:29:54 [engine.py:400]   File "/home/tms/vllm/vllm/model_executor/layers/quantization/fp8.py", line 651, in process_weights_after_loading
ERROR 02-24 19:29:54 [engine.py:400]     layer.w13_weight[expert_id][start:start +
ERROR 02-24 19:29:54 [engine.py:400]     ~~~~~~~~~~~~~~~~^^^^^^^^^^^
ERROR 02-24 19:29:54 [engine.py:400] IndexError: index 4 is out of bounds for dimension 0 with size 4

I changed num_experts to global_num_experts and added a local_num_experts as well, and fixed similar spots in quark_moe.py and compressed_tensors_moe.py.

I also fixed up a couple of other spots where we are looking at the layer's num_experts for heuristics.

vllm/vllm/model_executor/layers/quantization/gptq_marlin.py

Line 156 in 1f0ae3e

if layer.num_experts > 32:

vllm/vllm/model_executor/layers/quantization/awq_marlin.py

Line 139 in 1f0ae3e

if layer.num_experts > 32:

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[mgoin]

Nice catch

[mgoin]

Lovely, thanks!