[NVIDIA] Qwen3.5 h200 sglang MTP

[hshrivastava-droid]

Summary

Add benchmark configuration and script for Qwen3.5-397B-A17B-FP8 on H200 with SGLang MTP (Multi-Token Prediction) using EAGLE speculative decoding.

Changes

New benchmark script: benchmarks/single_node/qwen3.5_fp8_h200_mtp.sh

• SGLang launch server with EAGLE speculative decoding:
  • --speculative-algorithm EAGLE
  • --speculative-num-steps 3
  • --speculative-num-draft-tokens 4
  • --speculative-eagle-topk 1
• FP8 quantization with FP8 E4M3 KV cache
• FlashInfer attention backend with allreduce fusion enabled
• Chunked prefill (16384), radix cache disabled
• Supports TP8/EP8 with --expert-parallel-size

New config: qwen3.5-fp8-h200-sglang-mtp in nvidia-master.yaml

• Image: lmsysorg/sglang:v0.5.9-cu129-amd64
• Model: Qwen/Qwen3.5-397B-A17B-FP8
• Two sequence length configs: 1024/1024 and 8192/1024
• Search space: TP8, EP8, concurrency 4–128 with MTP spec-decoding

perf-changelog.yaml

• Added changelog entry for the new qwen3.5-fp8-h200-sglang-mtp config

[kedarpotdar-nv]

@claude add a PR description based on changes

[Klaud-Cold]

Claude finished @kedarpotdar-nv's task in 46s —— View job

---

Adding PR Description

• Gather context from PR changes
• Write and update PR description

PR description has been added with a summary of all changes:

• New benchmark script for Qwen3.5-397B-A17B-FP8 on H200 with SGLang MTP (EAGLE speculative decoding)
• New qwen3.5-fp8-h200-sglang-mtp config in nvidia-master.yaml (TP8/EP8, conc 4–128, two seq-len configs)
• Perf-changelog entry for the new config

[Oseltamivir]

LGTM, Recipe in https://cookbook.sglang.io/autoregressive/Qwen/Qwen3.5