[CI]: Bagel E2E Smoked Test

[princepride]

Purpose

Add E2E Bagel unit test I have discussed in: #400

Add end-to-end unit tests for Bagel text-to-image generation to validate the model's image generation pipeline. This PR includes:

• Shared Memory Connector Test: Tests Bagel text2img generation using the default shared memory connector for inter-stage communication
• Mooncake Connector Test: Tests Bagel text2img generation using the Mooncake connector for distributed inter-stage communication
• Buildkite CI Integration: Added a new CI step to run the Bagel text2img tests on H100 GPU
• The tests validate that generated images match expected reference pixel values within a ±5 tolerance, ensuring consistent and reproducible image generation output.

Test Plan

pytest -s -v tests/e2e/offline_inference/test_bagel_text2img.py

Result

root@deepseek-v3-2-vllm-77f648c586-w95x2:/proj-tango-pvc/users/zhipeng.wang/workspace/vllm-omni# pytest tests/model_executor/models/bagel/test_bagel_text2img.py -v
========================================================== test session starts ==========================================================
platform linux -- Python 3.12.12, pytest-9.0.2, pluggy-1.6.0 -- /usr/bin/python3.12
cachedir: .pytest_cache
rootdir: /proj-tango-pvc/users/zhipeng.wang/workspace/vllm-omni
configfile: pyproject.toml
plugins: shard-0.1.2, timeout-2.4.0, cov-7.0.0, forked-1.6.0, asyncio-1.3.0, rerunfailures-16.1, anyio-4.12.0
asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collected 2 items                                                                                                                       
Running 2 items in this shard: tests/model_executor/models/bagel/test_bagel_text2img.py::test_bagel_text2img_shared_memory_connector, tests/model_executor/models/bagel/test_bagel_text2img.py::test_bagel_text2img_mooncake_connector

tests/model_executor/models/bagel/test_bagel_text2img.py::test_bagel_text2img_shared_memory_connector PASSED                      [ 50%]
tests/model_executor/models/bagel/test_bagel_text2img.py::test_bagel_text2img_mooncake_connector PASSED                           [100%]E0129 01:14:04.789839 2952173 master_client.cpp:208] RPC call failed: End of file

[princepride]

@codex review

[princepride]

@congw729 PTAL

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7915a89633

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[Gaohan123]

I think it is not a Unit Test, which is just for api check and protection. Your PR is more like an E2E Smoked Test (ST), which is more suitable to put in e2e folder.

[princepride]

I think it is not a Unit Test, which is just for api check and protection. Your PR is more like an E2E Smoked Test (ST), which is more suitable to put in e2e folder.

Thank you for your feedback, I will change

[princepride]

@Gaohan123 @congw729 PTAL, I hope this test can be merged as soon as possible to prevent new PR unintentionally modified the relevant code.

[hsliuustc0106]

@yenuo26 PTAL

[Copilot]

Pull request overview

This PR adds end-to-end tests for Bagel text-to-image generation to strengthen CI coverage for diffusion-based models. The tests validate image generation output using pixel-level comparisons and support both SharedMemory and Mooncake connectors for inter-stage communication.

Changes:

• Added comprehensive E2E tests for Bagel text2img generation with two connector types (SharedMemory and Mooncake)
• Updated GPU memory utilization in Bagel stage configurations to support single-GPU testing (redistributed from 0.4/0.4 to 0.35/0.55)
• Added Buildkite CI step for automated testing on H100 GPUs
• Added mooncake-transfer-engine dependency to development requirements

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 17 comments.

Show a summary per file

File	Description
tests/e2e/offline_inference/test_bagel_text2img.py	New test file with two test functions validating Bagel text2img generation using reference pixel comparisons
vllm_omni/model_executor/stage_configs/bagel.yaml	Adjusted GPU memory utilization to accommodate both stages on single GPU (0.35/0.55 split)
vllm_omni/model_executor/stage_configs/bagel_multiconnector.yaml	Updated device allocation to use single GPU and adjusted memory utilization; now both stages run on device 0
.buildkite/pipeline.yml	Added new CI step for Bagel text2img tests on H100 GPU with 30-minute timeout
pyproject.toml	Added mooncake-transfer-engine==0.3.8.post1 to dev dependencies

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[hsliuustc0106]

@princepride could you please add the benchmark result for diffusion models under benchmark folder, maybe for the dit-only model

[princepride]

DiT-only Model: BAGEL-7B-MoT (Text-to-Image)

Hardware: NVIDIA H200 141GB

Server command:

vllm serve ByteDance-Seed/BAGEL-7B-MoT --omni --port 8099

Benchmark command:

python3 benchmarks/diffusion/diffusion_benchmark_serving.py \
    --base-url http://localhost:8099 \
    --model ByteDance-Seed/BAGEL-7B-MoT \
    --task t2i \
    --dataset vbench \
    --num-prompts 5

Results:

================= Serving Benchmark Result =================
Model:                                   ByteDance-Seed/BAGEL-7B-MoT
Dataset:                                 vbench         
Task:                                    t2i            
--------------------------------------------------
Benchmark duration (s):                  53.55          
Request rate:                            inf            
Max request concurrency:                 1              
Successful requests:                     5/5              
--------------------------------------------------
Request throughput (req/s):              0.09           
Latency Mean (s):                        10.7092        
Latency Median (s):                      10.0302        
Latency P99 (s):                         12.6974        

============================================================

[princepride]

@hsliuustc0106 PTAL

[princepride]

@hsliuustc0106

DiT-only Model: BAGEL-7B-MoT (Text-to-Image)

Hardware: NVIDIA H200 141GB

Server command:

vllm serve ByteDance-Seed/BAGEL-7B-MoT --omni --port 8099

Benchmark command:

python3 benchmarks/diffusion/diffusion_benchmark_serving.py \
    --base-url http://localhost:8099 \
    --model ByteDance-Seed/BAGEL-7B-MoT \
    --task t2i \
    --dataset vbench \
    --num-prompts 5

Results:

================= Serving Benchmark Result =================
Model:                                   ByteDance-Seed/BAGEL-7B-MoT
Dataset:                                 vbench         
Task:                                    t2i            
--------------------------------------------------
Benchmark duration (s):                  53.55          
Request rate:                            inf            
Max request concurrency:                 1              
Successful requests:                     5/5              
--------------------------------------------------
Request throughput (req/s):              0.09           
Latency Mean (s):                        10.7092        
Latency Median (s):                      10.0302        
Latency P99 (s):                         12.6974        

============================================================

[Gaohan123]

Please resolve CI failures

[hsliuustc0106]

Please resolve CI failures

it looks like HF downloading problem

[princepride]

@hsliuustc0106 Ready to merge. 😊