integrate cosmos-transfer1 + guided generation to I4H workflows #129

guopengf · 2025-05-08T18:01:53Z

Description

Integrate cosmos-transfer1 with our proposed guided generation pipeline to the I4H workflow.

workflows/robotic_ultrasound/tests/helpers.py

workflows/robotic_ultrasound/scripts/simulation/launch_transfer.sh

...ts/simulation/environments/cosmos_transfer1/config/generated_prompts_two_seperate_views.json

tools/env_setup_cosmos_transfer1.sh

workflows/robotic_ultrasound/tests/helpers.py

...ic_ultrasound/scripts/simulation/environments/cosmos_transfer1/guided_generation_pipeline.py

...imulation/environments/cosmos_transfer1/config/inference/cosmos-1-diffusion-control2world.py

...ic_ultrasound/scripts/simulation/environments/cosmos_transfer1/guided_generation_pipeline.py

...lows/robotic_ultrasound/scripts/simulation/environments/cosmos_transfer1/model/model_ctrl.py

Signed-off-by: Pengfei Guo <[email protected]>

workflows/robotic_ultrasound/tests/test_simulation/test_integration_cosmos_transfer1.py

workflows/robotic_ultrasound/scripts/simulation/environments/cosmos_transfer1/transfer.py

workflows/robotic_ultrasound/scripts/simulation/README.md

tools/env_setup_cosmos_transfer1.sh

workflows/robotic_ultrasound/scripts/simulation/environments/cosmos_transfer1/transfer.py

mingxin-zheng · 2025-05-23T08:55:25Z

/build

mingxin-zheng · 2025-05-23T10:02:51Z

======================================================================
ERROR: test_integration_cosmos_transfer1 (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_integration_cosmos_transfer1
Traceback (most recent call last):
  File "/root/miniconda3/envs/i4h-env/lib/python3.10/unittest/loader.py", line 154, in loadTestsFromName
    module = __import__(module_name)
  File "/home/jenkins/agent/workspace/pre-merge/workflows/robotic_ultrasound/tests/test_simulation/test_integration_cosmos_transfer1.py", line 25, in <module>
    from cosmos_transfer1.checkpoints import BASE_7B_CHECKPOINT_PATH
ModuleNotFoundError: No module named 'cosmos_transfer1'

tools/run_all_tests.py

Signed-off-by: YunLiu <[email protected]>

KumoLiu · 2025-05-23T10:34:33Z

/build

Signed-off-by: YunLiu <[email protected]>

…fer1-integrate Signed-off-by: YunLiu <[email protected]>

KumoLiu · 2025-05-25T12:41:45Z

also tested locally:

  0%|          | 0/1 [00:00<?, ?it/s]
100%|██████████| 1/1 [00:04<00:00,  4.82s/it]
100%|██████████| 1/1 [00:04<00:00,  4.82s/it]
..
  0%|          | 0/100 [00:00<?, ?it/s]
100%|██████████| 100/100 [00:00<00:00, 5653.62it/s]
.
----------------------------------------------------------------------
Ran 3 tests in 314.762s

Signed-off-by: YunLiu <[email protected]>

workflows/robotic_ultrasound/tests/test_simulation/test_integration_cosmos_transfer1.py

KumoLiu · 2025-05-26T06:02:33Z

/build

KumoLiu · 2025-05-26T07:07:42Z

/build

KumoLiu · 2025-05-26T07:17:24Z

/build

KumoLiu · 2025-05-26T08:13:38Z

/build

mingxin-zheng · 2025-05-26T08:15:24Z

/build

KumoLiu · 2025-05-26T10:49:28Z

/build

KumoLiu · 2025-05-26T13:37:18Z

/build

Signed-off-by: YunLiu <[email protected]>

KumoLiu · 2025-05-27T02:52:07Z

/build

KumoLiu · 2025-05-27T06:45:07Z

/build

KumoLiu · 2025-05-27T09:08:17Z

/build

### Description Integrate[ cosmos-transfer1](https://github.com/nvidia-cosmos/cosmos-transfer1/blob/861254a98a01d4e23f3e41d8a213d002a737e1d3/examples/inference_cosmos_transfer1_7b.md) with our proposed guided generation pipeline to the I4H workflow. --------- Signed-off-by: Pengfei Guo <[email protected]> Signed-off-by: YunLiu <[email protected]> Co-authored-by: YunLiu <[email protected]> Co-authored-by: Mingxin Zheng <[email protected]>

Fixes duplicated install of cudnn introduced by #129 Signed-off-by: Mingxin Zheng <[email protected]> Co-authored-by: YunLiu <[email protected]>

add guided generation pipline

690b717

guopengf self-assigned this May 8, 2025

guopengf added 6 commits May 8, 2025 14:49

h5 batch inference support

b06bf94

improve format

59ddcb9

prepare test case

723be4b

lint

b53778e

update test case

dc66ae8

format

0941b69

guopengf marked this pull request as ready for review May 12, 2025 02:43

guopengf requested review from KumoLiu and Nic-Ma May 12, 2025 02:44

Nic-Ma reviewed May 12, 2025

View reviewed changes

workflows/robotic_ultrasound/tests/helpers.py Outdated Show resolved Hide resolved

KumoLiu reviewed May 13, 2025

View reviewed changes

...ic_ultrasound/scripts/simulation/environments/cosmos_transfer1/guided_generation_pipeline.py Show resolved Hide resolved

KumoLiu reviewed May 13, 2025

View reviewed changes

...imulation/environments/cosmos_transfer1/config/inference/cosmos-1-diffusion-control2world.py Show resolved Hide resolved

KumoLiu reviewed May 13, 2025

View reviewed changes

...ic_ultrasound/scripts/simulation/environments/cosmos_transfer1/guided_generation_pipeline.py Outdated Show resolved Hide resolved

Nic-Ma reviewed May 14, 2025

View reviewed changes

...ic_ultrasound/scripts/simulation/environments/cosmos_transfer1/guided_generation_pipeline.py Outdated Show resolved Hide resolved

Nic-Ma reviewed May 14, 2025

View reviewed changes

...lows/robotic_ultrasound/scripts/simulation/environments/cosmos_transfer1/model/model_ctrl.py Outdated Show resolved Hide resolved

guopengf and others added 5 commits May 14, 2025 09:14

update test cases

1494470

add readme

f5800f4

Merge branch 'main' into pengfeig/cosmos-transfer1-integrate

6d50d78

Signed-off-by: Pengfei Guo <[email protected]>

add readme figure

1d85d0c

fix link

c980304

KumoLiu reviewed May 15, 2025

View reviewed changes

tools/env_setup_cosmos_transfer1.sh Outdated Show resolved Hide resolved

KumoLiu reviewed May 15, 2025

View reviewed changes

workflows/robotic_ultrasound/scripts/simulation/environments/cosmos_transfer1/transfer.py Show resolved Hide resolved

guopengf and others added 4 commits May 15, 2025 09:27

update dependency install and fix issues

c757b45

remve change for install_deps.py

220535b

Merge branch 'main' into pengfeig/cosmos-transfer1-integrate

0ebe16f

Merge branch 'main' into pengfeig/cosmos-transfer1-integrate

979e0ab

Merge branch 'main' into pengfeig/cosmos-transfer1-integrate

164aa5d

mingxin-zheng reviewed May 23, 2025

View reviewed changes

tools/run_all_tests.py Show resolved Hide resolved

fix

67e1e0b

Signed-off-by: YunLiu <[email protected]>

KumoLiu added 2 commits May 25, 2025 16:30

increase timeout for cosmos-transfer test

01128e3

Signed-off-by: YunLiu <[email protected]>

Merge remote-tracking branch 'origin/main' into pengfeig/cosmos-trans…

6efb7c3

…fer1-integrate Signed-off-by: YunLiu <[email protected]>

skip if not enough gpu

3bfc8e8

Signed-off-by: YunLiu <[email protected]>

KumoLiu reviewed May 26, 2025

View reviewed changes

workflows/robotic_ultrasound/tests/test_simulation/test_integration_cosmos_transfer1.py Show resolved Hide resolved

increase pi0 eval timeout

ad6cb9f

Signed-off-by: YunLiu <[email protected]>

mingxin-zheng merged commit 3351057 into main May 27, 2025
4 checks passed

KumoLiu deleted the pengfeig/cosmos-transfer1-integrate branch May 27, 2025 15:09

mingxin-zheng mentioned this pull request May 28, 2025

Fix robot us docker build #147

Merged

Nic-Ma pushed a commit that referenced this pull request May 30, 2025

Fix robot us docker build (#147)

55c5b5f

Fixes duplicated install of cudnn introduced by #129 Signed-off-by: Mingxin Zheng <[email protected]> Co-authored-by: YunLiu <[email protected]>

integrate cosmos-transfer1 + guided generation to I4H workflows #129

integrate cosmos-transfer1 + guided generation to I4H workflows #129

Uh oh!

Conversation

guopengf commented May 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mingxin-zheng commented May 23, 2025

Uh oh!

mingxin-zheng commented May 23, 2025

Uh oh!

Uh oh!

KumoLiu commented May 23, 2025

Uh oh!

KumoLiu commented May 25, 2025

Uh oh!

Uh oh!

KumoLiu commented May 26, 2025

Uh oh!

KumoLiu commented May 26, 2025

Uh oh!

KumoLiu commented May 26, 2025

Uh oh!

KumoLiu commented May 26, 2025

Uh oh!

mingxin-zheng commented May 26, 2025

Uh oh!

KumoLiu commented May 26, 2025

Uh oh!

KumoLiu commented May 26, 2025

Uh oh!

KumoLiu commented May 27, 2025

Uh oh!

KumoLiu commented May 27, 2025

Uh oh!

KumoLiu commented May 27, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

guopengf commented May 8, 2025 •

edited

Loading