Skip to content

[bugfix] Apply routed scaling factor to cutlass_fused_experts_fp8 #8688

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kushanam merged 1 commit into from
Aug 2, 2025
Merged

[bugfix] Apply routed scaling factor to cutlass_fused_experts_fp8 #8688

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion python/sglang/srt/layers/quantization/fp8.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1039,7 +1039,7 @@ def apply(
from sglang.srt.layers.moe.cutlass_moe import cutlass_fused_experts_fp8

topk_weights, topk_ids, _ = topk_output
return cutlass_fused_experts_fp8(
output = cutlass_fused_experts_fp8(
x,
layer.w13_weight.transpose(1, 2),
layer.w2_weight.transpose(1, 2),
Expand All @@ -1062,6 +1062,10 @@ def apply(
self.problem_sizes2,
use_fp8_blockscale=True,
)
# TODO: Fuse into select_experts
if routed_scaling_factor is not None:
output *= routed_scaling_factor
return output
# Expert fusion with FP8 quantization
return fused_experts(
x,
Expand Down
Loading