Skip to content

Conversation

@bghira
Copy link
Owner

@bghira bghira commented Aug 5, 2025

Currently very VRAM-heavy due to lack of VSA and USP in SimpleTuner.

@bghira bghira requested a review from Copilot August 5, 2025 20:41
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds FastWan DMD (Distribution Matching Distillation) as a new distillation method to SimpleTuner. DMD is a memory-intensive distillation technique that uses a generator (student) and fake score transformer (discriminator) to achieve high-quality distillation in fewer steps (typically 3-step).

Key changes:

  • Implements DMD distillation infrastructure with generator and fake score transformer components
  • Adds comprehensive DMD configuration options and training logic
  • Updates documentation to include DMD usage examples and comparison with DCM

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File Description
helpers/distillation/factory.py Adds DMD enum value and factory method for creating DMD distillers
helpers/distillation/dmd/distiller.py Complete DMD implementation with generator/discriminator training logic
documentation/distillation/WAN_DCM.md Minor update to validation steps from 50 to 4
documentation/distillation/FASTWAN_DMD.md Comprehensive DMD documentation with configuration examples
Comments suppressed due to low confidence (1)

helpers/distillation/dmd/distiller.py:322

  • [nitpick] The function name '_pred_noise_to_pred_video' is misleading as it suggests video-specific processing, but the function performs generic noise-to-clean conversion that works for any latent type. Consider renaming to '_pred_noise_to_clean_latents' or '_convert_noise_prediction_to_clean'.
    def _pred_noise_to_pred_video(self, pred_noise, noise_input, timestep):

@bghira bghira merged commit 78205fc into main Aug 5, 2025
1 check passed
@bghira bghira deleted the feature/dmd-fastwan branch August 5, 2025 22:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants