ModuleNotFoundError: No module named 'megatron.core' due to missing dependency in requirements

[zhao-huan]

Hi,

I only install the asr part of nemo-toolkit by following code:

pip install git+https://github.com/NVIDIA/NeMo.git@main#egg=nemo_toolkit[asr]

When running the script, I encountered the following error:

Traceback (most recent call last):
  File "./speaker_recognize/sortformer_backend.py", line 5, in <module>
    from nemo.collections.asr.models import SortformerEncLabelModel
  File "/usr/local/lib/python3.12/dist-packages/nemo/collections/asr/__init__.py", line 15, in <module>
    from nemo.collections.asr import data, losses, models, modules
  File "/usr/local/lib/python3.12/dist-packages/nemo/collections/asr/losses/__init__.py", line 15, in <module>
    from nemo.collections.asr.losses.angularloss import AngularSoftmaxLoss
  File "/usr/local/lib/python3.12/dist-packages/nemo/collections/asr/losses/angularloss.py", line 18, in <module>
    from nemo.core.classes import Loss, Typing, typecheck
  File "/usr/local/lib/python3.12/dist-packages/nemo/core/__init__.py", line 16, in <module>
    from nemo.core.classes import *
  File "/usr/local/lib/python3.12/dist-packages/nemo/core/classes/__init__.py", line 33, in <module>
    from nemo.core.classes.modelPT import ModelPT
  File "/usr/local/lib/python3.12/dist-packages/nemo/core/classes/modelPT.py", line 51, in <module>
    from nemo.lightning.callback_group import CallbackGroup
  File "/usr/local/lib/python3.12/dist-packages/nemo/lightning/__init__.py", line 27, in <module>
    from nemo.lightning.fabric.plugins import FabricMegatronMixedPrecision
  File "/usr/local/lib/python3.12/dist-packages/nemo/lightning/fabric/plugins.py", line 27, in <module>
    from nemo.lightning.pytorch.plugins.mixed_precision import (
  File "/usr/local/lib/python3.12/dist-packages/nemo/lightning/pytorch/plugins/__init__.py", line 1, in <module>
    from nemo.lightning.pytorch.plugins.data_sampler import MegatronDataSampler
  File "/usr/local/lib/python3.12/dist-packages/nemo/lightning/pytorch/plugins/data_sampler.py", line 23, in <module>
    from nemo.lightning.megatron_parallel import MegatronStep
  File "/usr/local/lib/python3.12/dist-packages/nemo/lightning/megatron_parallel.py", line 53, in <module>
    from megatron.core import parallel_state
ModuleNotFoundError: No module named 'megatron.core'

The issue appears to be related to the recent update #13437, which introduced the import from nemo.lightning.callback_group import CallbackGroup in nemo/core/classes/modelPT.py. This import chain eventually leads to megatron.core being required, but megatron.core module is not listed as a dependency in the file requirements_asr.txt.

Please add megatron.core module to the project's dependency list.

Thank you for your attention to this issue.

[suiyoubi]

@titu1994 Could you confirm if ASR module is expected to run without mcore ?

@zhao-huan you can install https://github.com/NVIDIA/Megatron-LM/tree/main to unblock your work

[tango4j]

@nithinraok
Importing angularloss eventually leads to megatron core input.
Can we make a quick fix on this?
This is affecting all the ASR model inferences that are using ASR environments.

[nithinraok]

Fixed with #14970

Cc: @chtruong814