Skip to content

Commit 9b6220c

Browse files
authored
Revamp recipe tutorials (#1308)
Signed-off-by: Ananth Subramaniam <[email protected]>
1 parent 6756e35 commit 9b6220c

14 files changed

+1268
-493
lines changed

README.md

Lines changed: 5 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -163,34 +163,13 @@ For more details on supported models, see our documentation:
163163

164164
#### Launching Recipes
165165

166-
All recipes are ready to train out of the box, using mock data by default. For an example of how to override the default configuration through YAML or Hydra-style CLI overrides, please have a look at this [script](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/recipes/llama/pretrain_llama3_8b.py). The script can then be launched with `torchrun`. For example, with the aforementioned script:
166+
For a conceptual overview of how recipes are structured, overridden, and launched with either `torchrun` or NeMo-Run, read the [Using Recipes guide](https://docs.nvidia.com/nemo/megatron-bridge/latest/recipe-usage.html).
167167

168-
```sh
169-
torchrun --nproc-per-node=2 pretrain_llama3_8b.py model.tensor_model_parallel_size=1 <additional overrides ...>
170-
```
171-
172-
Optionally, Megatron Bridge also supports launching with [NeMo-Run](https://github.com/NVIDIA-NeMo/Run). See the following examples for reference on launching with NeMo-Run:
173-
174-
- [pretrain_llama3_8b_nemo_run_script.py](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/recipes/llama/pretrain_llama3_8b_nemo_run_script.py)
175-
- [pretrain_llama3_8b_nemo_run_partial.py](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/recipes/llama/pretrain_llama3_8b_nemo_run_partial.py)
176-
177-
These examples can also be run as-is with the Llama 3 8B recipe (with NeMo-Run installed).
178-
179-
Launch Llama 3 8B pretraining with NeMo-Run's `run.Script`:
168+
Runnable tutorials live in `tutorials/recipes/llama` that covers:
180169

181-
```sh
182-
uv run python pretrain_llama3_8b_nemo_run_script.py \
183-
--nproc-per-node=2 \
184-
model.pipeline_model_parallel_size=1 \
185-
train.train_iters=10 # this script passes Hydra-style overrides to the target script
186-
```
187-
188-
Launch Llama 3 8B pretraining with NeMo-Run's `run.Partial`:
189-
190-
```sh
191-
uv run python pretrain_llama3_8b_nemo_run_partial.py \
192-
--nproc-per-node=2
193-
```
170+
- `00_quickstart_pretrain.py` for mock-data pretraining
171+
- `01_quickstart_finetune.py` + LoRA configs
172+
- YAML-driven flows and launch helpers
194173

195174
<!-- ### Vision-Language Models -->
196175

examples/recipes/llama/conf/llama3_8b_pretrain_override_example.yaml

Lines changed: 0 additions & 65 deletions
This file was deleted.

examples/recipes/llama/conf/llama3_8b_pretrain_override_example_megatron_fsdp.yaml

Lines changed: 0 additions & 68 deletions
This file was deleted.

examples/recipes/llama/pretrain_llama3_8b.py

Lines changed: 0 additions & 184 deletions
This file was deleted.

0 commit comments

Comments
 (0)