PDD reproduction: central algorithmic claims align on a compact teacher

This repository reconstructs the central algorithmic claims of Parallel Decoding Distillation for Fast Image and Video Generation (arXiv:2607.26004): a stop-gradient on-policy parallel decoder should beat naive large steps, midpoint targets should beat Euler targets, and one trained decoder should expose a useful 1/2/4/8-NFE trade-off.

Assessment: partially reproduced in this bounded CIFAR-10 setup. Across two 160,000-update grid-8 seeds, midpoint PDD reduced 1-NFE KID × 1,000 to 280.34, versus 409.06 for naive Euler and 472.04 for matched Euler-target PDD; paired trajectory MSE was 0.0875, versus 0.6265 for the naive control. The paper's non-comparable ImageNet number likewise favors midpoint—FID 2.69 versus 2.73. The same checkpoints improved to mean KID 46.89 at 8 NFE without retraining.

The deliberate substitution is a public google/ddpm-cifar10-32 teacher and public UofT CIFAR-10 images instead of the paper's SiT-XL ImageNet and 14B–22B media models. Formal evidence used 1,024 fixed seeds, a 128-NFE midpoint reference, trajectory MSE, paired Inception features, bounded KID, diversity, and measured latency. Runs used OpenResearch Kubernetes on NVIDIA RTX PRO 6000 Blackwell GPUs, peaking at 16 concurrent GPUs.

• Illustrated report
• Self-contained marimo tutorial
• Compact measured results
• Figure-generation code

Experiment log

Every experiment used the immutable exact command bash run.sh; hyperparameters live in each linked branch.

Branch / experiment	Purpose or change	Exact run command	Assessment / outcome	Compute
main	Public report, notebook, and implementation	Not run as an experiment (publication surface)	Presentation only	—
Naive coarse baseline	Public teacher, real CIFAR-10, 128-NFE reference, naive 1/2/4/8-NFE control	bash run.sh	Reference KID × 1,000 6.91; naive KID 409.06 → 176.71	Kubernetes, 1× RTX PRO 6000 Blackwell
Euler PDD, 8k seed 0	Reconstructed on-policy PD loss with matched Euler targets	bash run.sh	Worse than naive; replicated over 8 seeds	Kubernetes, 1× RTX PRO 6000 Blackwell
Midpoint PDD, 8k seed 0	Same decoder with midpoint teacher targets	bash run.sh	Worse than Euler and naive; replicated over 8 seeds	Kubernetes, 1× RTX PRO 6000 Blackwell
Euler PDD, 80k	Tenfold convergence extension	bash run.sh	Beats naive at 4/8 NFE; worse at 1/2	Kubernetes, 1× RTX PRO 6000 Blackwell
Midpoint PDD, 80k	Matched tenfold convergence extension	bash run.sh	Beats naive KID at every NFE; replicated	Kubernetes, 1× RTX PRO 6000 Blackwell
Selected grid-8 Euler	160k updates at 1e-4; matched target control	bash run.sh	Two-seed mean KID 472.04 → 132.90	Kubernetes, 1× RTX PRO 6000 Blackwell
Selected grid-8 midpoint	160k updates at 1e-4; selected reconstruction	bash run.sh	Two-seed mean KID 280.34 → 46.89	Kubernetes, 1× RTX PRO 6000 Blackwell
Grid-8 midpoint, 300k	Paper-length training robustness	bash run.sh	KID 279.17 → 47.41; no material gain over selected 160k model	Kubernetes, 1× RTX PRO 6000 Blackwell
Frozen-trunk midpoint	Isolate output-head learning from trunk drift	bash run.sh	Numerically identical to full training	Kubernetes, 1× RTX PRO 6000 Blackwell

Implementation map

• reproduce.py contains the VP probability-flow conversion, parallel output heads, detached intra-block rollout, Euler/midpoint targets, fused block sampler, metrics, and latency benchmark.
• config.json is the baseline experiment configuration; each experiment branch changes code/config rather than the run command.
• .orx/k8s.yaml requests one GPU per independent seed or diagnostic. This avoided an NCCL failure specific to the cluster image while still filling all 16 GPUs with independent work.
• run.sh installs pinned public dependencies and prints machine-readable terminal evidence.

To inspect the evidence interactively without rerunning GPU training, open the Molab notebook above. For a local notebook:

marimo edit notebooks/pdd_cifar10_reproduction.py