DeepSeek-V3/R1 MoE load balance deployment and inference using EPLB

[feliang-git]

Motivation

To achieve load balance during inference, a crucial feature is to load expert by a given combination, instead of loading them sequentially. Given that the load traffic monitor tool and DeepSeek EPLB algorithm are ready, this functionality would be the next step in achieving MoE expert load balance.

Modifications

• Modified EPMoE class weight_loading and forward methods to support expert mapping.
• Modified DeepseekV2MoE class. Passing MoE expert mapping tensor.
• Added a server start option --enable-eplb-moe to turn on MoE expert load balance.

Checklist

• Format your code according to the Code Formatting with Pre-Commit.
• Add unit tests as outlined in the Running Unit Tests.
• Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
• Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
• For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
• Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR.

[liz-badada]

Better pass the num_layers info as well as num_experts.

[jokerwyt]

Hi, any progress on this PR? Does it work on --enable-ep-moe or --enable-deepep-moe? @feliang-git

[feliang-git]

Hi, any progress on this PR? Does it work on --enable-ep-moe or --enable-deepep-moe? @feliang-git

[feliang-git]

Hi, any progress on this PR? Does it work on --enable-ep-moe or --enable-deepep-moe? @feliang-git

Hi @jokerwyt, there is another pr working on implementing EPLB. Please refer to pr#5295 for more detail. Thanks!