[Model] Support stable diffusion3 by iwzbi · Pull Request #439 · vllm-project/vllm-omni

iwzbi · 2025-12-23T13:08:20Z

PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS (AT THE BOTTOM) HAVE BEEN CONSIDERED.

Purpose

Support stable diffusion3 #311

Test Plan

Test Result

Essential Elements of an Effective PR Description Checklist

The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
The test plan, such as providing test command.
The test results, such as pasting the results comparison before and after, or e2e results
(Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
(Optional) Release notes update. If your change is user facing, please update the release notes draft.

BEFORE SUBMITTING, PLEASE READ https://github.com/vllm-project/vllm-omni/blob/main/CONTRIBUTING.md (anything written below this line will be removed by GitHub Actions)

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2025-12-23T13:11:09Z

vllm_omni/diffusion/registry.py

        "longcat_image",
        "pipeline_longcat_image_edit",
        "LongcatImageEditPipeline",
+    "StableDiffusion3Pipeline": (


Fix malformed diffusion model registry entry

The _DIFFUSION_MODELS dictionary is syntactically invalid because the LongCatImageEditPipeline entry is missing its closing parenthesis and trailing comma before the StableDiffusion3Pipeline entry, leaving the file unparsable. Importing vllm_omni.diffusion.registry will raise a SyntaxError, preventing any diffusion model (including the new SD3 pipeline) from being registered or loaded.

Useful? React with 👍 / 👎.

hsliuustc0106 · 2025-12-25T00:43:10Z

could you please try to use this model generate "vllm-omni supports stable diffusion" or something similar :)

iwzbi · 2025-12-25T03:26:30Z

could you please try to use this model generate "vllm-omni supports stable diffusion" or something similar :)

It really depends on prompt engineering, this is the best I can get after a few runs:

iwzbi · 2025-12-25T03:34:49Z

@hsliuustc0106 By the way, this PR does not include sequence parallelism and diffusion caching. I plan to support these features on SD3 and provide benchmark scripts in separate PRs.

Signed-off-by: iwzbi <[email protected]>

ZJY0516 · 2025-12-26T03:02:01Z

@hsliuustc0106 By the way, this PR does not include sequence parallelism and diffusion caching. I plan to support these features on SD3 and provide benchmark scripts in separate PRs.

Does this support cache-dit?

ZJY0516 · 2025-12-26T03:07:00Z

vllm_omni/diffusion/models/sd3/pipeline_sd3.py

+            timestep = t.expand(latents.shape[0]).to(device=latents.device, dtype=latents.dtype)
+
+            # Forward pass for positive prompt (or unconditional if no CFG)
+            # cache_branch is passed to hook for CFG-aware state management


There is no cache branch here. Remove this line

vllm_omni/diffusion/models/sd3/pipeline_sd3.py

Signed-off-by: iwzbi <[email protected]>

ZJY0516

LGTM

SamitHuang · 2025-12-27T09:02:54Z

can you update the supported model list as well? For the test plan, an example script will be easier to verify than a Jupyter notebook

Signed-off-by: iwzbi <[email protected]>

iwzbi · 2025-12-27T10:56:10Z

can you update the supported model list as well? For the test plan, an example script will be easier to verify than a Jupyter notebook

sure. added this model to supported_models.md. For the test, Would it be better to have SD3 covered by the functionality of examples/offline_inference/text_to_image/text_to_image.py than to write a separate example specifically for SD3? I originally wanted to add SD3's CI test to tests/e2e/offline_inference/test_t2i_model.py, but I couldn't find a suitable small parameter SD3 model...

hsliuustc0106 · 2025-12-27T13:53:04Z

@hsliuustc0106 By the way, this PR does not include sequence parallelism and diffusion caching. I plan to support these features on SD3 and provide benchmark scripts in separate PRs.

please submit a follow-up PR, and then we are going to announce it to the public SD3 is ready in vllm-omni :)

hsliuustc0106 · 2025-12-27T20:19:04Z

please also remember to submit a PR to vllm/recipes

iwzbi · 2025-12-28T00:18:16Z

@hsliuustc0106 By the way, this PR does not include sequence parallelism and diffusion caching. I plan to support these features on SD3 and provide benchmark scripts in separate PRs.

please submit a follow-up PR, and then we are going to announce it to the public SD3 is ready in vllm-omni :)

No problem. I will finish it in two days.

iwzbi · 2025-12-28T00:20:54Z

vllm/recipes

No problem

hsliuustc0106 · 2025-12-28T02:04:59Z

check this for reference vllm-project/recipes#184

Signed-off-by: iwzbi <[email protected]> Signed-off-by: iwzbi <[email protected]> Co-authored-by: iwzbi <[email protected]> Co-authored-by: Hongsheng Liu <[email protected]> Signed-off-by: wangyu31577 <[email protected]>

Signed-off-by: iwzbi <[email protected]> Signed-off-by: iwzbi <[email protected]> Co-authored-by: iwzbi <[email protected]> Co-authored-by: Hongsheng Liu <[email protected]> Signed-off-by: HonestDeng <[email protected]>

Signed-off-by: iwzbi <[email protected]> Signed-off-by: iwzbi <[email protected]> Co-authored-by: iwzbi <[email protected]> Co-authored-by: Hongsheng Liu <[email protected]>

iwzbi requested a review from hsliuustc0106 as a code owner December 23, 2025 13:08

chatgpt-codex-connector bot reviewed Dec 23, 2025

View reviewed changes

iwzbi force-pushed the dev/support_sd3 branch 2 times, most recently from 28ed378 to 72779e1 Compare December 23, 2025 14:03

iwzbi and others added 6 commits December 25, 2025 12:15

go

5034506

Signed-off-by: iwzbi <[email protected]>

go

9a8cce1

Signed-off-by: iwzbi <[email protected]>

go

135f49b

Signed-off-by: iwzbi <[email protected]>

go

3621b20

Signed-off-by: iwzbi <[email protected]>

go

e9661a8

Signed-off-by: iwzbi <[email protected]>

go

e50f0d8

Signed-off-by: iwzbi <[email protected]>

iwzbi force-pushed the dev/support_sd3 branch from ab49162 to e50f0d8 Compare December 25, 2025 04:16

hsliuustc0106 added 2 commits December 25, 2025 23:56

Merge branch 'main' into dev/support_sd3

f1a75d2

Merge branch 'main' into dev/support_sd3

30103f2

hsliuustc0106 requested a review from ZJY0516 December 26, 2025 01:18

hsliuustc0106 added the ready label to trigger buildkite CI label Dec 26, 2025

ZJY0516 reviewed Dec 26, 2025

View reviewed changes

go

cfe9a46

Signed-off-by: iwzbi <[email protected]>

hsliuustc0106 removed the ready label to trigger buildkite CI label Dec 26, 2025

iwzbi requested a review from ZJY0516 December 27, 2025 06:34

ZJY0516 approved these changes Dec 27, 2025

View reviewed changes

Merge branch 'main' into dev/support_sd3

71399f1

go

7d6ecac

Signed-off-by: iwzbi <[email protected]>

hsliuustc0106 approved these changes Dec 27, 2025

View reviewed changes

hsliuustc0106 added the ready label to trigger buildkite CI label Dec 27, 2025

hsliuustc0106 merged commit c9eace8 into vllm-project:main Dec 27, 2025
7 checks passed

hsliuustc0106 mentioned this pull request Dec 27, 2025

[RFC]: DiT model and feature support enhancement #85

Closed

58 tasks

david6666666 mentioned this pull request Jan 9, 2026

[Feature]: vLLM-Omni model owner JiusiServe/vllm-omni#25

Open

17 tasks

SamitHuang mentioned this pull request Jan 13, 2026

[Bug]: AttributeError: dual_attention_layers in Stable Diffusion 3 #636

Closed

1 task

david6666666 mentioned this pull request Jan 16, 2026

vLLM-Omni Model Support #808

Open

55 tasks

Comments

Conversation

iwzbi commented Dec 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Test Plan

Test Result

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Dec 23, 2025

Choose a reason for hiding this comment

Uh oh!

hsliuustc0106 commented Dec 25, 2025

Uh oh!

iwzbi commented Dec 25, 2025

Uh oh!

iwzbi commented Dec 25, 2025

Uh oh!

ZJY0516 commented Dec 26, 2025

Uh oh!

ZJY0516 Dec 26, 2025

Choose a reason for hiding this comment

Uh oh!

iwzbi Dec 26, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ZJY0516 left a comment

Choose a reason for hiding this comment

Uh oh!

SamitHuang commented Dec 27, 2025

Uh oh!

iwzbi commented Dec 27, 2025

Uh oh!

Uh oh!

hsliuustc0106 commented Dec 27, 2025

Uh oh!

hsliuustc0106 commented Dec 27, 2025

Uh oh!

iwzbi commented Dec 28, 2025

Uh oh!

iwzbi commented Dec 28, 2025

Uh oh!

hsliuustc0106 commented Dec 28, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

iwzbi commented Dec 23, 2025 •

edited

Loading