Skip to content

[2.4.0] pretrain_llama3_8b: AttributeError: 'Namespace' object has no attribute 'use_fsdp_double_buffer' #14392

@pbelevich

Description

@pbelevich

There is use_fsdp_double_buffer in https://github.com/NVIDIA/NeMo/blob/v2.4.0/scripts/performance/llm/pretrain_llama3_8b.py#L73
but there is no use_fsdp_double_buffer in
https://github.com/NVIDIA/NeMo/blob/v2.4.0/scripts/performance/argument_parser.py

launching python -m scripts.performance.llm.pretrain_llama3_8b fails with:

  File "/fsxl/belevich/miniconda3/envs/nemo2/lib/python3.12/site-packages/scripts/performance/llm/pretrain_llama3_8b.py", line 119, in <module>
    recipe = override_recipe_configs(
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/fsxl/belevich/miniconda3/envs/nemo2/lib/python3.12/site-packages/scripts/performance/llm/pretrain_llama3_8b.py", line 73, in override_recipe_configs
    use_fsdp_double_buffer=args.use_fsdp_double_buffer,
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'Namespace' object has no attribute 'use_fsdp_double_buffer'

Metadata

Metadata

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions