Merged
Conversation
Signed-off-by: stevehuang52 <[email protected]>
Signed-off-by: stevehuang52 <[email protected]>
Signed-off-by: stevehuang52 <[email protected]>
Signed-off-by: stevehuang52 <[email protected]>
Signed-off-by: stevehuang52 <[email protected]>
Signed-off-by: stevehuang52 <[email protected]>
Signed-off-by: stevehuang52 <[email protected]>
Signed-off-by: stevehuang52 <[email protected]>
Signed-off-by: stevehuang52 <[email protected]>
stevehuang52
requested review from
KunalDhawan,
nithinraok,
tango4j and
weiqingw4ng
Signed-off-by: stevehuang52 <[email protected]>
Signed-off-by: stevehuang52 <[email protected]>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
CodeQL found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.
Signed-off-by: stevehuang52 <[email protected]>
Signed-off-by: stevehuang52 <[email protected]>
Contributor
|
[🤖]: Hi @stevehuang52 👋, We wanted to let you know that a CICD pipeline for this PR just finished successfully. So it might be time to merge this PR or get some approvals. |
Signed-off-by: stevehuang52 <[email protected]>
Contributor
|
[🤖]: Hi @stevehuang52 👋, We wanted to let you know that a CICD pipeline for this PR just finished successfully. So it might be time to merge this PR or get some approvals. |
Signed-off-by: stevehuang52 <[email protected]>
Signed-off-by: stevehuang52 <[email protected]>
Signed-off-by: stevehuang52 <[email protected]>
Signed-off-by: stevehuang52 <[email protected]>
weiqingw4ng
previously approved these changes
Sep 5, 2025
Collaborator
weiqingw4ng
left a comment
weiqingw4ng
left a comment
There was a problem hiding this comment.
LGTM now. There is no issue with llama-nemotron
Signed-off-by: stevehuang52 <[email protected]>
Signed-off-by: stevehuang52 <[email protected]>
Contributor
|
[🤖]: Hi @stevehuang52 👋, We wanted to let you know that a CICD pipeline for this PR just finished successfully. So it might be time to merge this PR or get some approvals. |
Contributor
|
[🤖]: Hi @stevehuang52 👋, We wanted to let you know that a CICD pipeline for this PR just finished successfully. So it might be time to merge this PR or get some approvals. |
Collaborator
|
Tested the framework run. Approving it. |
tango4j
approved these changes
Sep 6, 2025
ealbasiri
pushed a commit
to ealbasiri/NeMo
that referenced
this pull request
Sep 8, 2025
* update streaming ASR Signed-off-by: stevehuang52 <[email protected]> * add voice agent Signed-off-by: stevehuang52 <[email protected]> * update readme Signed-off-by: stevehuang52 <[email protected]> * update websocket Signed-off-by: stevehuang52 <[email protected]> * update Signed-off-by: stevehuang52 <[email protected]> * update Signed-off-by: stevehuang52 <[email protected]> * update readme Signed-off-by: stevehuang52 <[email protected]> * update Signed-off-by: stevehuang52 <[email protected]> * clean up Signed-off-by: stevehuang52 <[email protected]> * clean up Signed-off-by: stevehuang52 <[email protected]> * fix typo Signed-off-by: stevehuang52 <[email protected]> * fix codeQL Signed-off-by: stevehuang52 <[email protected]> * update cfg Signed-off-by: stevehuang52 <[email protected]> * remove unused Signed-off-by: stevehuang52 <[email protected]> * update readme Signed-off-by: stevehuang52 <[email protected]> * change default models Signed-off-by: stevehuang52 <[email protected]> * fix diar diable Signed-off-by: stevehuang52 <[email protected]> * fix diar diable Signed-off-by: stevehuang52 <[email protected]> * update ux Signed-off-by: stevehuang52 <[email protected]> * update tts Signed-off-by: stevehuang52 <[email protected]> * update readme Signed-off-by: stevehuang52 <[email protected]> * fix and update Signed-off-by: stevehuang52 <[email protected]> * fix asr Signed-off-by: stevehuang52 <[email protected]> * update readmme Signed-off-by: stevehuang52 <[email protected]> * update doc and llm dtype Signed-off-by: stevehuang52 <[email protected]> * refactor and add example prompts Signed-off-by: stevehuang52 <[email protected]> * update readme Signed-off-by: stevehuang52 <[email protected]> * update readme Signed-off-by: stevehuang52 <[email protected]> * clean up Signed-off-by: stevehuang52 <[email protected]> * clean up Signed-off-by: stevehuang52 <[email protected]> * update info on streaming sortformer Signed-off-by: stevehuang52 <[email protected]> * move code to 'nemo/agents/voice_agent' Signed-off-by: stevehuang52 <[email protected]> * update doc Signed-off-by: stevehuang52 <[email protected]> * clean up Signed-off-by: stevehuang52 <[email protected]> * refactor Signed-off-by: stevehuang52 <[email protected]> * update doc Signed-off-by: stevehuang52 <[email protected]> * remove the unnecessary streaming state conversion and import it from sortformer_modules, remove PostProcessingParams Signed-off-by: Weiqing Wang <[email protected]> * Apply isort and black reformatting Signed-off-by: weiqingw4ng <[email protected]> * update doc Signed-off-by: stevehuang52 <[email protected]> * clean up Signed-off-by: stevehuang52 <[email protected]> * fix for llama-nemotron template, and refactor Signed-off-by: stevehuang52 <[email protected]> * fix tts separator Signed-off-by: stevehuang52 <[email protected]> * fix for llama-nemotron Signed-off-by: stevehuang52 <[email protected]> * update cfg Signed-off-by: stevehuang52 <[email protected]> * refactor and update doc Signed-off-by: stevehuang52 <[email protected]> * change default llm to qwen Signed-off-by: stevehuang52 <[email protected]> * update doc Signed-off-by: stevehuang52 <[email protected]> --------- Signed-off-by: stevehuang52 <[email protected]> Signed-off-by: Weiqing Wang <[email protected]> Signed-off-by: weiqingw4ng <[email protected]> Co-authored-by: Taejin Park <[email protected]> Co-authored-by: Kunal Dhawan <[email protected]> Co-authored-by: Weiqing Wang <[email protected]> Co-authored-by: weiqingw4ng <[email protected]> Signed-off-by: Enas Albasiri <[email protected]>
blisc
added a commit
to blisc/NeMo
that referenced
this pull request
Oct 10, 2025
* Support QwenVL for inference API (#14534) * Support QwenVL for inference engine * Apply isort and black reformatting Signed-off-by: meatybobby <[email protected]> * Remove comment out * Reformat * Skip pylint check * Add unit tests * Apply isort and black reformatting Signed-off-by: meatybobby <[email protected]> --------- Signed-off-by: meatybobby <[email protected]> Co-authored-by: meatybobby <[email protected]> * Hyena: Allow to use unfused RMSNorm + TELinear to restore accuracy and some speed (#14542) * Fix sequence packing loss calculation (#14437) * Fix sequence packing loss calculation Signed-off-by: Rayan Dasoriya <[email protected]> * Fix nemo2 path Signed-off-by: Rayan Dasoriya <[email protected]> * Skip pylint Signed-off-by: Rayan Dasoriya <[email protected]> --------- Signed-off-by: Rayan Dasoriya <[email protected]> Co-authored-by: Dmytro Pykhtar <[email protected]> * [Audio]: added streaming mode to SpectrogramToAudio (#14524) * [Audio]: added streaming mode to SpectrogramToAudio Signed-off-by: Rauf <[email protected]> * added time buffer Signed-off-by: Rauf <[email protected]> * renamed Nf -> num_frames Signed-off-by: Rauf <[email protected]> * added AudioToSpectrogram and scale and magnitude power Signed-off-by: Rauf <[email protected]> * added multiple chunking support Signed-off-by: Rauf <[email protected]> * added properties _stream_initialized, _eps, got rid of _prev_spec_frame Signed-off-by: Rauf <[email protected]> * added hanning window Signed-off-by: Rauf <[email protected]> * Apply isort and black reformatting Signed-off-by: nasretdinovr <[email protected]> * added a docstring regarding streaming istft mode Signed-off-by: Rauf <[email protected]> --------- Signed-off-by: Rauf <[email protected]> Signed-off-by: nasretdinovr <[email protected]> Co-authored-by: nasretdinovr <[email protected]> * fix: fix missing rope scaling in exporting llama embedding model (#14523) Signed-off-by: Zhiyu Li <[email protected]> * Update evo2 defaults so converted checkpoints have the right parameters (#14514) * Update evo2 defaults so converted checkpoints have the right parameters Signed-off-by: John St John <[email protected]> * Fix line too long issue Signed-off-by: John St John <[email protected]> * Fix expected changes to configs that are locked into our tests Signed-off-by: John St John <[email protected]> --------- Signed-off-by: John St John <[email protected]> * deprecate t0 scripts (#14585) Signed-off-by: dimapihtar <[email protected]> * cfg typo correction (#14588) Signed-off-by: Malay Nagda <[email protected]> * [Perf script] Add use_te_activation_func and activation_func_fp8_input_store flags (#14522) * Add use te activation func and save act input in fp8 flags Signed-off-by: Guyue Huang <[email protected]> * Fix field name Signed-off-by: Guyue Huang <[email protected]> * Update scripts/performance/vlm/finetune_qwen25vl_32b.py Co-authored-by: malay-nagda <[email protected]> Signed-off-by: Guyue Huang <[email protected]> --------- Signed-off-by: Guyue Huang <[email protected]> Signed-off-by: Guyue Huang <[email protected]> Co-authored-by: malay-nagda <[email protected]> * Modify logging message to signal that RestoreConfig will be used (#14469) * Bump TE and Mcore (#14568) * Bump TE and Mcore Signed-off-by: Charlie Truong <[email protected]> * Use Mcore 69b65 Signed-off-by: Charlie Truong <[email protected]> --------- Signed-off-by: Charlie Truong <[email protected]> * Avoid host-device sync in PTL logging (#14489) * remove sync in logging Signed-off-by: qiyuw <[email protected]> * Apply isort and black reformatting Signed-off-by: WanZzzzzz <[email protected]> * add class and func docstrings in data_sampler.py for pylint Signed-off-by: qiyuw <[email protected]> * Apply isort and black reformatting Signed-off-by: WanZzzzzz <[email protected]> --------- Signed-off-by: qiyuw <[email protected]> Signed-off-by: WanZzzzzz <[email protected]> Co-authored-by: qiyuw <[email protected]> Co-authored-by: WanZzzzzz <[email protected]> * Integrate implicit filter kernel with Hyena layer (#14621) * add 1b arclongcontextconfig Signed-off-by: Farhad Ramezanghorbani <[email protected]> * fix device mess Signed-off-by: Farhad Ramezanghorbani <[email protected]> * add implicit_filter support Signed-off-by: Farhad Ramezanghorbani <[email protected]> * use padded input Signed-off-by: Farhad Ramezanghorbani <[email protected]> * Apply isort and black reformatting Signed-off-by: farhadrgh <[email protected]> * Revert "add 1b arclongcontextconfig" This reverts commit 029969bae07e5c1651abd519640424d4aaece216. --------- Signed-off-by: Farhad Ramezanghorbani <[email protected]> Signed-off-by: farhadrgh <[email protected]> * Fix kv_channels configuration for Gemma2 27b (#14590) * fix gemma2 27b kv dimension Signed-off-by: Ananth Subramaniam <[email protected]> * fix gemma2 27b kv dimension Signed-off-by: Ananth Subramaniam <[email protected]> --------- Signed-off-by: Ananth Subramaniam <[email protected]> * [Flux] small fixes (#14333) * feat: print expert groups on megatron init (#13874) Signed-off-by: Alexander Zhipa <[email protected]> Co-authored-by: Alexander Zhipa <[email protected]> Signed-off-by: CarlosGomes98 <[email protected]> * set a different seed for each dp rank Signed-off-by: CarlosGomes98 <[email protected]> * calculate loss inside autocast Signed-off-by: CarlosGomes98 <[email protected]> * disable per token loss, grad acc fusion Signed-off-by: CarlosGomes98 <[email protected]> * add missing self.seed Signed-off-by: CarlosGomes98 <[email protected]> * black formatting Signed-off-by: CarlosGomes98 <[email protected]> * Apply isort and black reformatting Signed-off-by: gautham-kollu <[email protected]> --------- Signed-off-by: Alexander Zhipa <[email protected]> Signed-off-by: CarlosGomes98 <[email protected]> Signed-off-by: gautham-kollu <[email protected]> Co-authored-by: Alexander Zhipa <[email protected]> Co-authored-by: Alexander Zhipa <[email protected]> Co-authored-by: gautham-kollu <[email protected]> Co-authored-by: gautham-kollu <[email protected]> * [Flux] Add MXFP8 Support (#14473) * [Flux] Add MXFP8 support. Signed-off-by: Wil Kong <[email protected]> * [Flux] Add current and block scaling. Signed-off-by: Wil Kong <[email protected]> --------- Signed-off-by: Wil Kong <[email protected]> * use hf hub to download ckpt (#14638) Signed-off-by: Ao Tang <[email protected]> * Fine-tune embedding models (E5-Large-V2 and LLaMA-3.2-1B) on the allnli triplet dataset with NeMo Framework (#14584) * Create E2E-Embedding-Finetuning Signed-off-by: Hemant Giri <[email protected]> * Update E2E-Embedding-Finetuning Signed-off-by: Hemant Giri <[email protected]> * Delete tutorials/llm/embedding/E2E-Embedding-Finetuning Signed-off-by: Hemant Giri <[email protected]> * Create README.md Signed-off-by: Hemant Giri <[email protected]> * Add files via upload Signed-off-by: Hemant Giri <[email protected]> * Add files via upload This is a notebook for E2E finetuning a embedding model Signed-off-by: Hemant Giri <[email protected]> * Update README.md Signed-off-by: Hemant Giri <[email protected]> * Update README.md Signed-off-by: Hemant Giri <[email protected]> * Delete tutorials/llm/embedding/E2E-Embedding-Finetuning/download_dataset.py Signed-off-by: Hemant Giri <[email protected]> * Delete tutorials/llm/embedding/E2E-Embedding-Finetuning/finetune_e5.py Signed-off-by: Hemant Giri <[email protected]> * Delete tutorials/llm/embedding/E2E-Embedding-Finetuning/finetune_llama1b.py Signed-off-by: Hemant Giri <[email protected]> * Delete tutorials/llm/embedding/E2E-Embedding-Finetuning/import_e5_large.py Signed-off-by: Hemant Giri <[email protected]> * Delete tutorials/llm/embedding/E2E-Embedding-Finetuning/import_llama1b.py Signed-off-by: Hemant Giri <[email protected]> --------- Signed-off-by: Hemant Giri <[email protected]> Co-authored-by: Ao Tang <[email protected]> * [Perf script] Llama and GPT3 perf script use mlp cast fusion Signed-off-by: Guyue Huang <[email protected]> * remove service launch scripts (#14647) Signed-off-by: dimapihtar <[email protected]> * warning instead of error with chat template (#14641) Signed-off-by: jenchen13 <[email protected]> * fix notebook (#14643) Signed-off-by: Chen Cui <[email protected]> * [Audio]: fixed bug in conformet unet (#14626) Signed-off-by: Rauf <[email protected]> * Delete tutorials/llm/llama/biomedical-qa directory (#14653) Signed-off-by: Chen Cui <[email protected]> * Fix code checkout during test (#14658) Signed-off-by: Charlie Truong <[email protected]> * Fix Flux seed as optional Arg (#14652) * fix flux seed as optional Signed-off-by: Ao Tang <[email protected]> * fix fluxcontrolnet Signed-off-by: Ao Tang <[email protected]> * Fix code checkout during test Signed-off-by: Charlie Truong <[email protected]> --------- Signed-off-by: Ao Tang <[email protected]> Signed-off-by: Charlie Truong <[email protected]> Co-authored-by: Charlie Truong <[email protected]> * remove older TTS tutorials (#14660) Signed-off-by: Jason <[email protected]> * Remove PEFT scheme condition from recipe (#14661) * Remove PEFT scheme condition from recipe Signed-off-by: Ali Taghibakhshi <[email protected]> * remove unnecessary peft conditioning 12b --------- Signed-off-by: Ali Taghibakhshi <[email protected]> * Add gpt-oss lora exporter (#14589) * add gpt-oss lora exporter Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * update lora exporter for experts Signed-off-by: Chen Cui <[email protected]> * disallow exporting expert lora since nemo implementation is not equivalent to hf Signed-off-by: Chen Cui <[email protected]> * linting Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * address comment Signed-off-by: Chen Cui <[email protected]> --------- Signed-off-by: Chen Cui <[email protected]> Signed-off-by: cuichenx <[email protected]> Co-authored-by: cuichenx <[email protected]> Co-authored-by: Charlie Truong <[email protected]> * Add NeMo Voice Agent (#14325) * update streaming ASR Signed-off-by: stevehuang52 <[email protected]> * add voice agent Signed-off-by: stevehuang52 <[email protected]> * update readme Signed-off-by: stevehuang52 <[email protected]> * update websocket Signed-off-by: stevehuang52 <[email protected]> * update Signed-off-by: stevehuang52 <[email protected]> * update Signed-off-by: stevehuang52 <[email protected]> * update readme Signed-off-by: stevehuang52 <[email protected]> * update Signed-off-by: stevehuang52 <[email protected]> * clean up Signed-off-by: stevehuang52 <[email protected]> * clean up Signed-off-by: stevehuang52 <[email protected]> * fix typo Signed-off-by: stevehuang52 <[email protected]> * fix codeQL Signed-off-by: stevehuang52 <[email protected]> * update cfg Signed-off-by: stevehuang52 <[email protected]> * remove unused Signed-off-by: stevehuang52 <[email protected]> * update readme Signed-off-by: stevehuang52 <[email protected]> * change default models Signed-off-by: stevehuang52 <[email protected]> * fix diar diable Signed-off-by: stevehuang52 <[email protected]> * fix diar diable Signed-off-by: stevehuang52 <[email protected]> * update ux Signed-off-by: stevehuang52 <[email protected]> * update tts Signed-off-by: stevehuang52 <[email protected]> * update readme Signed-off-by: stevehuang52 <[email protected]> * fix and update Signed-off-by: stevehuang52 <[email protected]> * fix asr Signed-off-by: stevehuang52 <[email protected]> * update readmme Signed-off-by: stevehuang52 <[email protected]> * update doc and llm dtype Signed-off-by: stevehuang52 <[email protected]> * refactor and add example prompts Signed-off-by: stevehuang52 <[email protected]> * update readme Signed-off-by: stevehuang52 <[email protected]> * update readme Signed-off-by: stevehuang52 <[email protected]> * clean up Signed-off-by: stevehuang52 <[email protected]> * clean up Signed-off-by: stevehuang52 <[email protected]> * update info on streaming sortformer Signed-off-by: stevehuang52 <[email protected]> * move code to 'nemo/agents/voice_agent' Signed-off-by: stevehuang52 <[email protected]> * update doc Signed-off-by: stevehuang52 <[email protected]> * clean up Signed-off-by: stevehuang52 <[email protected]> * refactor Signed-off-by: stevehuang52 <[email protected]> * update doc Signed-off-by: stevehuang52 <[email protected]> * remove the unnecessary streaming state conversion and import it from sortformer_modules, remove PostProcessingParams Signed-off-by: Weiqing Wang <[email protected]> * Apply isort and black reformatting Signed-off-by: weiqingw4ng <[email protected]> * update doc Signed-off-by: stevehuang52 <[email protected]> * clean up Signed-off-by: stevehuang52 <[email protected]> * fix for llama-nemotron template, and refactor Signed-off-by: stevehuang52 <[email protected]> * fix tts separator Signed-off-by: stevehuang52 <[email protected]> * fix for llama-nemotron Signed-off-by: stevehuang52 <[email protected]> * update cfg Signed-off-by: stevehuang52 <[email protected]> * refactor and update doc Signed-off-by: stevehuang52 <[email protected]> * change default llm to qwen Signed-off-by: stevehuang52 <[email protected]> * update doc Signed-off-by: stevehuang52 <[email protected]> --------- Signed-off-by: stevehuang52 <[email protected]> Signed-off-by: Weiqing Wang <[email protected]> Signed-off-by: weiqingw4ng <[email protected]> Co-authored-by: Taejin Park <[email protected]> Co-authored-by: Kunal Dhawan <[email protected]> Co-authored-by: Weiqing Wang <[email protected]> Co-authored-by: weiqingw4ng <[email protected]> * Update get_tensor_shapes function whose signature was refactored (#14594) * Update get_tensor_shapes function whose signature changed and wasn't refactored Signed-off-by: Asha Anoosheh <[email protected]> * Bump Mcore commit to latest on 0.14.0 branch Signed-off-by: Charlie Truong <[email protected]> * Bump Mcore Signed-off-by: Charlie Truong <[email protected]> * Set flux fsdp test to optional Signed-off-by: Charlie Truong <[email protected]> * Fix flux test to skip Signed-off-by: Charlie Truong <[email protected]> --------- Signed-off-by: Asha Anoosheh <[email protected]> Signed-off-by: Charlie Truong <[email protected]> Co-authored-by: Charlie Truong <[email protected]> * fixing kernel restarting when transcribing (#14665) * fixing kernel restarting when transcribing Signed-off-by: Weiqing Wang <[email protected]> * fixing the same issue for tutorials/asr/ASR_with_NeMo.ipynb Signed-off-by: Weiqing Wang <[email protected]> * remove the change caused by IDE Signed-off-by: Weiqing Wang <[email protected]> --------- Signed-off-by: Weiqing Wang <[email protected]> * Skip trt-llm and vllm install in install test (#14663) Signed-off-by: Charlie Truong <[email protected]> * Canary tutorial fix (#14673) Signed-off-by: Nune <[email protected]> * Downgrade "datasets" library version in ASR tutorial to ensure compatibility with HF Datasets used (#14679) * downgrade dataset in notebooks to ensure comparibility with HF datsets used Signed-off-by: Kunal Dhawan <[email protected]> * remove env information from notebook Signed-off-by: Kunal Dhawan <[email protected]> --------- Signed-off-by: Kunal Dhawan <[email protected]> * End_to_End_Diarization_Training.ipynb (#14680) Signed-off-by: taejinp <[email protected]> * Fix deepseek export dtype (#14307) * add cast dtype option Signed-off-by: Chen Cui <[email protected]> * linting Signed-off-by: Chen Cui <[email protected]> * fix Signed-off-by: Chen Cui <[email protected]> * add atol option Signed-off-by: Chen Cui <[email protected]> * Update L2_NeMo_2_Conversion_Test_DeepSeek.sh Signed-off-by: Chen Cui <[email protected]> * Update state.py Signed-off-by: Chen Cui <[email protected]> * fix test Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * fix test Signed-off-by: Chen Cui <[email protected]> --------- Signed-off-by: Chen Cui <[email protected]> Signed-off-by: cuichenx <[email protected]> Co-authored-by: cuichenx <[email protected]> Co-authored-by: Charlie Truong <[email protected]> * Delete nemo1 notebooks (#14677) * Delete tutorials/llm/llama/sdg-law-title-generation directory Signed-off-by: Chen Cui <[email protected]> * Delete tutorials/llm/llama/domain-adaptive-pretraining/code/domain_adaptive_pretraining_nemo1.0.ipynb Signed-off-by: Chen Cui <[email protected]> --------- Signed-off-by: Chen Cui <[email protected]> * Bump latest Mcore 020abf01 (#14676) * Bump latest Mcore Signed-off-by: Charlie Truong <[email protected]> * Pin Mcore to 020abf01 Signed-off-by: Charlie Truong <[email protected]> --------- Signed-off-by: Charlie Truong <[email protected]> * correct shapes (#14425) Signed-off-by: CarlosGomes98 <[email protected]> Co-authored-by: gautham-kollu <[email protected]> * Fix for "EncDecRNNTBPEModel transcribe() failed with TypeError" (#14698) * fix decode_ids_to_str for AggregateTokenizer Signed-off-by: andrusenkoau <[email protected]> * minor fix Signed-off-by: andrusenkoau <[email protected]> --------- Signed-off-by: andrusenkoau <[email protected]> * Bump modelopt to 0.35.0 and remove `safe_import("modelopt")` in llm collection (#14656) * Bump modelopt to 0.35.0 and remove safe_import in llm collection Signed-off-by: Keval Morabia <[email protected]> * Update eagle architecture spec setting Signed-off-by: Asha Anoosheh <[email protected]> * Reduce specdec memory usage Signed-off-by: Asha Anoosheh <[email protected]> --------- Signed-off-by: Keval Morabia <[email protected]> Signed-off-by: Asha Anoosheh <[email protected]> Co-authored-by: Charlie Truong <[email protected]> Co-authored-by: Asha Anoosheh <[email protected]> * Tutorial fix (#14699) Signed-off-by: Nune <[email protected]> * Add option for LoRA with Transformer Engine op fuser (#14411) * Initial implementation of fused LoRA Signed-off-by: Tim Moon <[email protected]> * Get fused LoRA to run Signed-off-by: Tim Moon <[email protected]> * Initial work toward tensor-parallel support Missing all-gather op Signed-off-by: Tim Moon <[email protected]> * Enable fused LoRA based on model config Signed-off-by: Tim Moon <[email protected]> * Tweak comments Signed-off-by: Tim Moon <[email protected]> * Add TE version checks Signed-off-by: Tim Moon <[email protected]> * Fix linter warning Signed-off-by: Tim Moon <[email protected]> * Apply isort and black reformatting Signed-off-by: timmoon10 <[email protected]> * Use in-place fork/add ops to enable GEMMs with beta=1 Signed-off-by: Tim Moon <[email protected]> * Add ops directly to te.op.Sequential Signed-off-by: Tim Moon <[email protected]> * Move fused LoRA impl into LoRALinear subclass Signed-off-by: Tim Moon <[email protected]> * Fix bug where fused impl was always disabled Signed-off-by: Tim Moon <[email protected]> * Apply isort and black reformatting Signed-off-by: timmoon10 <[email protected]> * Support wgrad accumulation fusion Signed-off-by: Tim Moon <[email protected]> * Add integration test for TE op fuser Signed-off-by: Tim Moon <[email protected]> * Apply isort and black reformatting Signed-off-by: timmoon10 <[email protected]> * Explicitly list module containers that are compatible with list or dict APIs Mcore subclasses of te.ops.Sequential are iterable, but are not compatible with list API. Signed-off-by: Tim Moon <[email protected]> * Apply isort and black reformatting Signed-off-by: timmoon10 <[email protected]> * Add missing docstring Signed-off-by: Tim Moon <[email protected]> * Apply isort and black reformatting Signed-off-by: timmoon10 <[email protected]> * Update Mcore version Signed-off-by: Tim Moon <[email protected]> * Update Megatron-LM commit Signed-off-by: Tim Moon <[email protected]> * Attempt to support forward hooks in fused LoRA Signed-off-by: Tim Moon <[email protected]> * Apply isort and black reformatting Signed-off-by: timmoon10 <[email protected]> --------- Signed-off-by: Tim Moon <[email protected]> Signed-off-by: timmoon10 <[email protected]> Signed-off-by: Tim Moon <[email protected]> Signed-off-by: gautham-kollu <[email protected]> Co-authored-by: timmoon10 <[email protected]> Co-authored-by: gautham-kollu <[email protected]> Co-authored-by: Charlie Truong <[email protected]> * add load-in-4bit param (#14636) Signed-off-by: dimapihtar <[email protected]> * fp4 support (#14625) Signed-off-by: qiyuw <[email protected]> Co-authored-by: qiyuw <[email protected]> Co-authored-by: gautham-kollu <[email protected]> * Update Reasoning-SFT.ipynb (#14716) Signed-off-by: Chen Cui <[email protected]> * Remove artificial block to vortex fp8 TP (#14684) * Remove artificial block to vortex fp8 TP Signed-off-by: John St John <[email protected]> * Handle sequence_parallel=True TP>1 case properly where theres an all gather Signed-off-by: John St John <[email protected]> --------- Signed-off-by: John St John <[email protected]> * Replace MegatronTokenizer with MegatronLegacyTokenizer (#14721) Signed-off-by: Charlie Truong <[email protected]> * Update ModelCommPGs API from megatron-core (#14578) * update Signed-off-by: yaoyu-33 <[email protected]> * Bump Mcore to b615e73 Signed-off-by: Charlie Truong <[email protected]> * Replace ProcessGroupsCollection with ProcessGroupCollection Signed-off-by: Charlie Truong <[email protected]> * Replace pgs_collection with pg_collection Signed-off-by: Charlie Truong <[email protected]> --------- Signed-off-by: yaoyu-33 <[email protected]> Signed-off-by: Charlie Truong <[email protected]> Co-authored-by: Charlie Truong <[email protected]> * drop speech_llm example suite (#14683) Signed-off-by: yaoyu-33 <[email protected]> * feat: Compatibility modification of megatron-fsdp (#14593) * nvfsdp_update Signed-off-by: Selvaraj Anandaraj <[email protected]> Signed-off-by: jianbinc <[email protected]> * add megatron-fsdp checkpoint support Signed-off-by: jianbinc <[email protected]> * update use_custom_fsdp to use_megatron_fsdp Signed-off-by: jianbinc <[email protected]> * revert back pretrain_llama3_8b.py formt code Signed-off-by: jianbinc <[email protected]> * Apply isort and black reformatting Signed-off-by: shjwudp <[email protected]> * keep use_custom_fsdp as backup and notify this will deprecated on m-core 0.14 Signed-off-by: jianbinc <[email protected]> * Apply isort and black reformatting Signed-off-by: shjwudp <[email protected]> * fix CodeQL check Signed-off-by: jianbinc <[email protected]> --------- Signed-off-by: Selvaraj Anandaraj <[email protected]> Signed-off-by: jianbinc <[email protected]> Signed-off-by: shjwudp <[email protected]> Co-authored-by: Selvaraj Anandaraj <[email protected]> Co-authored-by: shjwudp <[email protected]> * imported get_moe_layer_wise_logging_tracker from megatron core moe_utils (#14694) * imported get_moe_layer_wise_logging_tracker from megatron core moe_utils Signed-off-by: Prathamesh Kalamkar <[email protected]> * Apply isort and black reformatting Signed-off-by: prathamk-tw <[email protected]> * moved import to the top * Apply isort and black reformatting Signed-off-by: prathamk-tw <[email protected]> --------- Signed-off-by: Prathamesh Kalamkar <[email protected]> Signed-off-by: prathamk-tw <[email protected]> Co-authored-by: prathamk-tw <[email protected]> * cast SE weights and activations to fp32 (#14743) Signed-off-by: Elena Rastorgueva <[email protected]> * remove env var (#14739) Signed-off-by: Malay Nagda <[email protected]> * detach arg option for run scripts (#14722) * detach arg option for run scripts Signed-off-by: Malay Nagda <[email protected]> * int dit opt instances Signed-off-by: Malay Nagda <[email protected]> --------- Signed-off-by: Malay Nagda <[email protected]> * Use lhotse dataloader for ASR models to support in-manifest channel selection for multichannel recordings (#14586) * make EncDecCTCModelBPE use lhotse dataloader when transcribing Signed-off-by: Roman Korostik <[email protected]> * make EncDecHybridRNNTCTCBPEModel use lhotse dataloader when transcribing Signed-off-by: Roman Korostik <[email protected]> * make EncDecRNNTBPEModel use lhotse dataloader when transcribing Signed-off-by: Roman Korostik <[email protected]> * clarify some error messages Signed-off-by: Roman Korostik <[email protected]> --------- Signed-off-by: Roman Korostik <[email protected]> * Randomized shard slicing for tarred data (#14558) * Randomized shard slicing for tarred data Signed-off-by: Piotr Żelasko <[email protected]> * Add shuffling shards in untarred sharegpt and multimodal conversation sources Signed-off-by: Piotr Żelasko <[email protected]> * Extend slice_length support to multimodal and sharegpt conversations Signed-off-by: Piotr Żelasko <[email protected]> * Update lhotse requirement version Signed-off-by: Piotr Żelasko <[email protected]> --------- Signed-off-by: Piotr Żelasko <[email protected]> * Data prediction objective for flow matching speech enhancement models (#14749) * flow matching: support x-prediction (data as target for the estimator) Signed-off-by: Roman Korostik <[email protected]> * flow matching: fix model init in x-prediction case Signed-off-by: Roman Korostik <[email protected]> * flow matching: add estimator_target to sampler in example configs Signed-off-by: Roman Korostik <[email protected]> * flow matching: expand tests to include data prediction models Signed-off-by: Roman Korostik <[email protected]> * Apply isort and black reformatting Signed-off-by: racoiaws <[email protected]> --------- Signed-off-by: Roman Korostik <[email protected]> Signed-off-by: racoiaws <[email protected]> Co-authored-by: racoiaws <[email protected]> * Fix Some Failures (#14763) * Use megatron_fsdp instead of custom_fsdp for Flux tests. Signed-off-by: Wil Kong <[email protected]> * Update megatron.core quick_gelu import path. Signed-off-by: Wil Kong <[email protected]> --------- Signed-off-by: Wil Kong <[email protected]> * Support additional Slurm parameters (#14742) * support additional slurm params and test with nemotron4 * fixed parsing of slurm params * fix incorrect parsing due to fallback * add support for all performance scripts * Apply isort and black reformatting * remove unused import --------- Signed-off-by: bdubauski <[email protected]> Signed-off-by: Barys Dubauski <[email protected]> Co-authored-by: Barys Dubauski <[email protected]> Co-authored-by: bdubauski <[email protected]> * [Flux] Remove redundant host & device sync. (#14711) Signed-off-by: Wil Kong <[email protected]> Co-authored-by: gautham-kollu <[email protected]> * [Flux] Add cuda_graph_scope and cache images ids for full iteration cuda graph. (#14744) Signed-off-by: Wil Kong <[email protected]> Co-authored-by: gautham-kollu <[email protected]> * Add transducer timestamps without alignments, timestamps to streaming (#14766) * refactored timestamps, fully identical to previuos Signed-off-by: lilithgrigoryan <[email protected]> * removed alignments from rnnt timestamps Signed-off-by: lilithgrigoryan <[email protected]> * clean up Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * clean up Signed-off-by: lilithgrigoryan <[email protected]> * fix tdt confidence without alignments Signed-off-by: lilithgrigoryan <[email protected]> * minor fix Signed-off-by: lilithgrigoryan <[email protected]> * minor fix Signed-off-by: lilithgrigoryan <[email protected]> * minor fix Signed-off-by: lilithgrigoryan <[email protected]> * minor fix Signed-off-by: lilithgrigoryan <[email protected]> * Add timestamps option to streaming inference script Signed-off-by: Vladimir Bataev <[email protected]> * Fix config params Signed-off-by: Vladimir Bataev <[email protected]> * Fix tdt Signed-off-by: Vladimir Bataev <[email protected]> * fix tdt durations, clean up Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * tests fix, clean up Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * remove starting SOS symbols from beam decodings to match timestamps length Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> --------- Signed-off-by: lilithgrigoryan <[email protected]> Signed-off-by: lilithgrigoryan <[email protected]> Signed-off-by: Vladimir Bataev <[email protected]> Co-authored-by: lilithgrigoryan <[email protected]> Co-authored-by: Vladimir Bataev <[email protected]> * Adding bf16 Sortformer train and inference (#14627) * Adding disabled autocast on bce_loss Signed-off-by: taejinp <[email protected]> * Adding Sortformer BF16 inference Signed-off-by: taejinp <[email protected]> * Adding BF16 inference and adding a config Signed-off-by: taejinp <[email protected]> * Apply isort and black reformatting Signed-off-by: tango4j <[email protected]> * Adding bf16-mixed option for both training and inference Signed-off-by: taejinp <[email protected]> * Apply isort and black reformatting Signed-off-by: tango4j <[email protected]> * Adding bf16-mixed option for e2e_diarize_speech.py Signed-off-by: taejinp <[email protected]> * Apply isort and black reformatting Signed-off-by: tango4j <[email protected]> --------- Signed-off-by: taejinp <[email protected]> Signed-off-by: tango4j <[email protected]> Co-authored-by: tango4j <[email protected]> * Replace texterrors with kaldialign library (#14775) * replace texterros with kaldialign for f-score computation Signed-off-by: andrusenkoau <[email protected]> * replace texterros with kaldialign for asr confidence Signed-off-by: andrusenkoau <[email protected]> * replace texterrors with kaldialign for ASR_Confidence_Estimation.ipynb Signed-off-by: andrusenkoau <[email protected]> * replace texterrors with kaldialing for ASR_Context_Biasing.ipynb Signed-off-by: andrusenkoau <[email protected]> * Apply isort and black reformatting Signed-off-by: andrusenkoau <[email protected]> * decrease kaldialign version Signed-off-by: andrusenkoau <[email protected]> --------- Signed-off-by: andrusenkoau <[email protected]> Signed-off-by: andrusenkoau <[email protected]> Co-authored-by: andrusenkoau <[email protected]> * Update prune-distill notebooks to Qwen3 + simplify + mmlu eval (#14785) * Update prune-distill notebooks to Qwen3 + simplify Signed-off-by: Keval Morabia <[email protected]> * address comments Signed-off-by: Keval Morabia <[email protected]> * Add readme.rst Signed-off-by: Keval Morabia <[email protected]> --------- Signed-off-by: Keval Morabia <[email protected]> * ci: Automodel deprecation warning (#14787) * add deprecation notice Signed-off-by: Alexandros Koumparoulis <[email protected]> * add deprecation notice Signed-off-by: Alexandros Koumparoulis <[email protected]> * add deprecation warning Signed-off-by: Alexandros Koumparoulis <[email protected]> * remove import Signed-off-by: Alexandros Koumparoulis <[email protected]> * move code Signed-off-by: Alexandros Koumparoulis <[email protected]> * add more notices Signed-off-by: Alexandros Koumparoulis <[email protected]> * Apply isort and black reformatting Signed-off-by: akoumpa <[email protected]> * Remove automodel cicd Signed-off-by: Dong Hyuk Chang <[email protected]> * Add deprecation notice for Automodel Signed-off-by: Dong Hyuk Chang <[email protected]> --------- Signed-off-by: Alexandros Koumparoulis <[email protected]> Signed-off-by: akoumpa <[email protected]> Signed-off-by: Dong Hyuk Chang <[email protected]> Co-authored-by: Alexandros Koumparoulis <[email protected]> Co-authored-by: akoumpa <[email protected]> * Remove export-deploy, automodel, and eval tutorials (#14790) Signed-off-by: Charlie Truong <[email protected]> * Update gpt_oss.py (#14706) Signed-off-by: Chen Cui <[email protected]> * MXFP8 must only use E4M3 as dtype (#14793) Signed-off-by: Aditya Vavre <[email protected]> * fix: Use shutil.copy fallback to handle file metadata permission errors (#14639) * Add fallback for file copy to handle metadata errors Signed-off-by: vipnydav <[email protected]> * Add robust_copy for resilient file copy Signed-off-by: vipnydav <[email protected]> * Apply isort and black reformatting Signed-off-by: vipnydav <[email protected]> * remove imported Path from test_file.py Signed-off-by: vipnydav <[email protected]> * Move robust_copy method to util file Signed-off-by: vipnydav <[email protected]> * Apply isort and black reformatting Signed-off-by: vipnydav <[email protected]> * Fix lint Signed-off-by: vipnydav <[email protected]> --------- Signed-off-by: vipnydav <[email protected]> Signed-off-by: vipnydav <[email protected]> Co-authored-by: vipnydav <[email protected]> * OneLogger Integration (#13437) * feat: add callback group definition & callback ABC Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: "Zhengjiang Shao" <[email protected]> Signed-off-by: Zhengjiang Shao <[email protected]> * Apply isort and black reformatting Signed-off-by: PytLab <[email protected]> * feat: insert callback functions of CallbackGroup Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: "Zhengjiang Shao" <[email protected]> Signed-off-by: Zhengjiang Shao <[email protected]> * Apply isort and black reformatting Signed-off-by: PytLab <[email protected]> * chore: PR test for jiashang Signed-off-by: Jiashang Hu <[email protected]> * feat: use __init_subclass__ to cover all ModelPT subclasses Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: "Zhengjiang Shao" <[email protected]> Signed-off-by: Zhengjiang Shao <[email protected]> * Apply isort and black reformatting Signed-off-by: PytLab <[email protected]> * feat: Adding metadata config manager poc Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: "Saju Prasad" <[email protected]> Signed-off-by: Saju Prasad <[email protected]> * Apply isort and black reformatting Signed-off-by: sajup-oss <[email protected]> * feat: revert test changes. Signed-off-by: liquor233 <[email protected]> * fix: Updating metadata attributes Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: sajup <[email protected]> * Apply isort and black reformatting Signed-off-by: sajup-oss <[email protected]> * fix: Adding OneloggerCallback Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: sajup <[email protected]> * fix: Reverting changes in examples/multimodal/speech_llm/modular_audio_gpt_train.py Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: sajup <[email protected]> * Apply isort and black reformatting Signed-off-by: sajup-oss <[email protected]> * fix: update modular models and megatron GPT models Signed-off-by: Jiashang Hu <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * feat: add on_app_start and on_app_end Signed-off-by: Jiashang Hu <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * fix: Adding small test example for testing Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: sajup <[email protected]> * Apply isort and black reformatting Signed-off-by: sajup-oss <[email protected]> * fix: Fixing review comments as discussed with Jiashang Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: "Saju Prasad" <[email protected]> Signed-off-by: Saju Prasad <[email protected]> * Apply isort and black reformatting Signed-off-by: sajup-oss <[email protected]> * fix: updating nemo code to v2 Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: sajup <[email protected]> * Apply isort and black reformatting Signed-off-by: sajup-oss <[email protected]> * fix: updating wandb to get info from env Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: sajup <[email protected]> * Apply isort and black reformatting Signed-off-by: sajup-oss <[email protected]> * fix: fix som impl issue Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: liquor233 <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * fix: fix issue for exp manager. Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: liquor233 <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * feat: remove callback_group Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: liquor233 <[email protected]> * feat: fix timingtracker issue Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: liquor233 <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * feat: fix for startup callbcaks Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: liquor233 <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * feat: change to adapter Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: liquor233 <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * feat: use new nv-one-logger Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: liquor233 <[email protected]> * feat: add on_app_end Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: liquor233 <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * feat: make OneLogger configurable Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: liquor233 <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * feat: remove NeMocallback import Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: liquor233 <[email protected]> * feat: fix the enable_onelogger setting. Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: liquor233 <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * feat: clean the code. Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: liquor233 <[email protected]> * feat: enable onelogger Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: liquor233 <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * test: Adding few unit tests Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: "Saju Prasad" <[email protected]> Signed-off-by: Saju Prasad <[email protected]> * Apply isort and black reformatting Signed-off-by: sajup-oss <[email protected]> * feat: tmp fix for functional testing. Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: liquor233 <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * fix: add on_app_end for NeMov2 Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: liquor233 <[email protected]> * fix: typo. Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: liquor233 <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * fix: fix the get attributes Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: liquor233 <[email protected]> * fix: moving test test_meta_info_manager.py to tests/collections/common/ Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: "Saju Prasad" <[email protected]> Signed-off-by: Saju Prasad <[email protected]> * fix: fix format issue. Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: liquor233 <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * fix: fix lint errors Signed-off-by: liquor233 <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * Revert "Apply isort and black reformatting" This reverts commit de6994d7e6e12e4040a5819cd1375c7a22ee7e0a. Signed-off-by: Jiashang Hu <[email protected]> * Revert "fix: fix lint errors" This reverts commit 8e47ecd749a1583597e8b8253f4eee4b231dbdf6. Signed-off-by: Jiashang Hu <[email protected]> * fix: fix linting issues. Signed-off-by: Jiashang Hu <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * fix: fix linting issue Signed-off-by: Jiashang Hu <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * fix: add copyright info Signed-off-by: Jiashang Hu <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * fix: small fix. Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: liquor233 <[email protected]> * fix: fix small issues for t5 Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: liquor233 <[email protected]> * fix: fix dataloader issue. Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: liquor233 <[email protected]> * fix: remove dataloader setting. Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: liquor233 <[email protected]> * feat: update OneLogger. Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: liquor233 <[email protected]> * fix: fix hydra runner. Signed-off-by: Jiashang Hu <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * fix: start using partial config. Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: liquor233 <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * fix: fix the unused variables Signed-off-by: Jiashang Hu <[email protected]> * fix: change get_one_logger name Signed-off-by: Jiashang Hu <[email protected]> * fix: code clean up. Signed-off-by: Jiashang Hu <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * fix: import more specific to avoid circular dependency. (#14306) Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: Peiyuan <[email protected]> * fix: use ptl callback from ls Signed-off-by: Jiashang Hu <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * feat: fix meta info manager. Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: liquor233 <[email protected]> * fix: fix meta data issue. Signed-off-by: Jiashang Hu <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * fix: fix the lint issue Signed-off-by: Jiashang Hu <[email protected]> * fix: fix the unit tests. Signed-off-by: Jiashang Hu <[email protected]> * fix: fix minor metadata issue. Signed-off-by: Jiashang Hu <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * fix: fix some test issues Signed-off-by: Jiashang Hu <[email protected]> * fix: fix pytest issue for meta info manager Signed-off-by: Jiashang Hu <[email protected]> * fix: fix lint issues for optimizers. Signed-off-by: Jiashang Hu <[email protected]> * fix: fix pytest issues. Signed-off-by: Jiashang Hu <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * fix: fix CICD issues. Signed-off-by: Jiashang Hu <[email protected]> * fix: fix all pytests Signed-off-by: Jiashang Hu <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * chore: fix lint Signed-off-by: Jiashang Hu <[email protected]> * chore: fix unused import issues. Signed-off-by: Jiashang Hu <[email protected]> * chore: fix CICD issues. Signed-off-by: Jiashang Hu <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * fix: fix the CICD issues. Signed-off-by: Jiashang Hu <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * fix: fix the linting issue Signed-off-by: Jiashang Hu <[email protected]> * fix: fix CICD issues. Signed-off-by: Jiashang Hu <[email protected]> * fix: fix the circular import issue. Signed-off-by: Jiashang Hu <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * fix: fix some pytests. Signed-off-by: Jiashang Hu <[email protected]> * fix: revert some change. Signed-off-by: Jiashang Hu <[email protected]> * fix: error handling for init onelogger Signed-off-by: Jiashang Hu <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * chore: fix one_logger code. Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: liquor233 <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * chore: remove unused vars. Signed-off-by: Jiashang Hu <[email protected]> * fix: fix CICD for nemo Signed-off-by: Jiashang Hu <[email protected]> * chore: fix NeMo CICD. Signed-off-by: Jiashang Hu <[email protected]> * chore: renaming onelogger Signed-off-by: Jiashang Hu <[email protected]> * chore: fix some exception. Signed-off-by: Jiashang Hu <[email protected]> * chore: renaming. Signed-off-by: Jiashang Hu <[email protected]> * chore: resolve some comments. Signed-off-by: Jiashang Hu <[email protected]> * chore: remove duplicate init. Signed-off-by: Jiashang Hu <[email protected]> * chore: resolve some github comments. Signed-off-by: Jiashang Hu <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * chore: fix the linting issue. Signed-off-by: Jiashang Hu <[email protected]> * chore(callbacks): restore generic CallbackGroup and route telemetry v… (#14628) * chore(callbacks): restore generic CallbackGroup and route telemetry via group\n\n- Add BaseCallback and CallbackGroup with update_config and class init hook\n- Register OneLoggerAdapterCallback into group; merge config update into class\n- Replace direct OneLogger API usages with CallbackGroup across code\n- Ensure trainer attaches registered callbacks via group.update_config\n- Add nv-one-logger>=2.0.0 to base requirements\n\nSigned-off-by: Jiashang Hu <[email protected]> Signed-off-by: Jiashang Hu <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> * chore: renaming. * chore: revert the change to install nv-one-logger * chore: fix the linting issue Signed-off-by: Jiashang Hu <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> --------- Signed-off-by: Jiashang Hu <[email protected]> Signed-off-by: liquor233 <[email protected]> Co-authored-by: liquor233 <[email protected]>\nSigned-off-by: liquor233 <[email protected]> * Add tests for callback group (#14632) * chore: fix some circular dependency issues. * chore: move the files to utils. * chore: add unit tests * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> * chore: fix nv-one-logger tests * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> * chore: fix lint issue. * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> * chore: change the location. * chore: remaining fix. * chore: remaining changes. * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> * chore: fix the tests * chore: fix some lint. * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> * Revert prompt_encoder.py to c5ef26c (Jason Wang) to undo auto-formatting * pre-commit: exclude prompt_encoder.py from black/isort formatting * chore: undo lasst commit. * fix: fix some part for nemocallback. * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> * chore: fix some pytest * fix: verify the auto-hooked functions are called once Signed-off-by: Zhengjiang Shao <[email protected]> --------- Signed-off-by: liquor233 <[email protected]> Signed-off-by: Zhengjiang Shao <[email protected]> Co-authored-by: liquor233 <[email protected]> Co-authored-by: Zhengjiang Shao <[email protected]>\nSigned-off-by: liquor233 <[email protected]> * fix: fix the double init issue Signed-off-by: Jiashang Hu <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> Signed-off-by: liquor233 <[email protected]> * fix: fix the push Signed-off-by: Jiashang Hu <[email protected]> * Guarantee one logger on_app_end calls (#14691) * fix: guarantee on_app_end calls can be invoked finally Signed-off-by: Zhengjiang Shao <[email protected]> * feat: add context manager creator in CallbackGroup * Revert "feat: add context manager creator in CallbackGroup" This reverts commit 381f83de5c914f08707fecb22e4674e7b3f6b104. Signed-off-by: Zhengjiang Shao <[email protected]> --------- Signed-off-by: Zhengjiang Shao <[email protected]> * fix: remove meta info manager (#14689) * fix: remove meta info manager Signed-off-by: Jiashang Hu <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> --------- Signed-off-by: Jiashang Hu <[email protected]> Signed-off-by: liquor233 <[email protected]> Co-authored-by: liquor233 <[email protected]> * fix: fix some linting issues. * fix: fix unit tests. * chore: fix mcore Signed-off-by: Jiashang Hu <[email protected]> * fix: fix the installing problem Signed-off-by: Jiashang Hu <[email protected]> * fix: fix requirements Signed-off-by: Jiashang Hu <[email protected]> * fix: fix the mcore version. Signed-off-by: Jiashang Hu <[email protected]> * fix: fix the mcore version. Signed-off-by: Jiashang Hu <[email protected]> * fix: fix the mcore version. Signed-off-by: Jiashang Hu <[email protected]> * fix: fix the mcore version. Signed-off-by: Jiashang Hu <[email protected]> * fix: fix the mcore version. Signed-off-by: Jiashang Hu <[email protected]> * fix: fix the mcore version. Signed-off-by: Jiashang Hu <[email protected]> * fix: use correct global_step for async ckpt success event Signed-off-by: Zhengjiang Shao <[email protected]> * fix: fix unit tests Signed-off-by: Jiashang Hu <[email protected]> * fix: fix requirements Signed-off-by: Jiashang Hu <[email protected]> * fix: refactor the unit tests Signed-off-by: Jiashang Hu <[email protected]> * fix: insert callbacks in CallbackGroup before other PTL callbacks Signed-off-by: Zhengjiang Shao <[email protected]> * fix: fix call on app start flag Signed-off-by: Jiashang Hu <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> * fix: fix unit tests Signed-off-by: Jiashang Hu <[email protected]> * fix: bump nv-one-logger version Signed-off-by: Jiashang Hu <[email protected]> * fix: fix the unit tests Signed-off-by: Jiashang Hu <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> * fix: fix the cicd issues. * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> * fix: fix some lint issues Signed-off-by: Jiashang Hu <[email protected]> * fix: fix unused import Signed-off-by: Jiashang Hu <[email protected]> * fix: make oneloggernemocallback singleton Signed-off-by: Jiashang Hu <[email protected]> * fix: fix lint issues Signed-off-by: Jiashang Hu <[email protected]> * fix: make oneloggernemocallback singleton * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> * fix: keep the original callbacks order in CallbackGroup when merging with trainer.callbacks * fix: fix the unit tests Signed-off-by: Jiashang Hu <[email protected]> * fix: fix unit tests Signed-off-by: Jiashang Hu <[email protected]> * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> * fix: fix lint issues Signed-off-by: Jiashang Hu <[email protected]> * fix: fix the pickle issue. * Apply isort and black reformatting Signed-off-by: liquor233 <[email protected]> * fix: fix issue. * fix: fix callback Signed-off-by: Jiashang Hu <[email protected]> * fix: fix callback group Signed-off-by: Jiashang Hu <[email protected]> --------- Signed-off-by: Zhengjiang Shao <[email protected]> Signed-off-by: PytLab <[email protected]> Signed-off-by: Jiashang Hu <[email protected]> Signed-off-by: Saju Prasad <[email protected]> Signed-off-by: sajup-oss <[email protected]> Signed-off-by: liquor233 <[email protected]> Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: sajup <[email protected]> Signed-off-by: sajup <[email protected]> Signed-off-by: Saju Prasad <[email protected]> Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: liquor233 <[email protected]> Signed-off-by: Saju Prasad <[email protected]> Signed-off-by: Jiashang Hu <[email protected]>\nSigned-off-by: Peiyuan <[email protected]> Signed-off-by: liquor233 <[email protected]> Co-authored-by: PytLab <[email protected]> Co-authored-by: Jiashang Hu <[email protected]> Co-authored-by: Saju Prasad <[email protected]> Co-authored-by: sajup-oss <[email protected]> Co-authored-by: sajup <[email protected]> Co-authored-by: liquor233 <[email protected]> Co-authored-by: Saju Prasad <[email protected]> Co-authored-by: Saju Prasad <[email protected]> Co-authored-by: Peiyuan <[email protected]> Co-authored-by: Peiyuan Qi <[email protected]> * Disable blank Issues (#14788) Signed-off-by: Pablo Garay <[email protected]> * Add community label bot (#14796) Signed-off-by: Charlie Truong <[email protected]> * Add mistral small3 24B config and recipe (#14784) * Add mistral small3 24B config and recipe Signed-off-by: Joosung Yoon <[email protected]> --------- Signed-off-by: Joosung Yoon <[email protected]> Signed-off-by: Alexandros Koumparoulis <[email protected]> Signed-off-by: Alexandros Koumparoulis <[email protected]> Co-authored-by: Alexandros Koumparoulis <[email protected]> Co-authored-by: Alexandros Koumparoulis <[email protected]> * Update changelog for `r2.3.0` (#14812) * beep boop: Update changelog Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update changelog for 2.3.3 Signed-off-by: Charlie Truong <[email protected]> * Fix changelog for 2.3.3 Signed-off-by: Charlie Truong <[email protected]> --------- Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Charlie Truong <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Charlie Truong <[email protected]> * QWEN2.5-VL 7B FP8 Recipe (#14801) * QWEN2.5-VL FP8 Recipe Signed-off-by: Lifu Zhang <[email protected]> * Apply isort and black reformatting Signed-off-by: tomlifu <[email protected]> * add model configs Signed-off-by: Lifu Zhang <[email protected]> --------- Signed-off-by: Lifu Zhang <[email protected]> Signed-off-by: tomlifu <[email protected]> Co-authored-by: tomlifu <[email protected]> * disk space management: nemo install test (#14822) * Add Customization Capabilities to Cache-Aware Models (#14757) * Add Customization Capabilities to Cache-Aware Models Signed-off-by: Vladimir Bataev <[email protected]> * Unify params with other transcription scripts Signed-off-by: Vladimir Bataev <[email protected]> * Fix usage with manifests containing relative paths Signed-off-by: Vladimir Bataev <[email protected]> * Fix decoding config setup Signed-off-by: Vladimir Bataev <[email protected]> * Return back output_path Signed-off-by: Vladimir Bataev <[email protected]> * Raise not implemented error if batched beam search performed with partial hypotheses Signed-off-by: Vladimir Bataev <[email protected]> * Raise not implemented error if batched beam search in transducer performed with partial hypotheses Signed-off-by: Vladimir Bataev <[email protected]> * Fix after merge Signed-off-by: Vladimir Bataev <[email protected]> * Fix att_context_size param Signed-off-by: Vladimir Bataev <[email protected]> * Use optional for left_chunks Signed-off-by: Vladimir Bataev <[email protected]> * Apply isort and black reformatting Signed-off-by: artbataev <[email protected]> * Unify parameters with transcribe_speech Signed-off-by: Vladimir Bataev <[email protected]> * Fix docstring Signed-off-by: Vladimir Bataev <[email protected]> * Unify dtype selection Signed-off-by: Vladimir Bataev <[email protected]> * Fix unused variables Signed-off-by: Vladimir Bataev <[email protected]> * Enhance inline documentation. Set compute_dtype=float32 by default. Signed-off-by: Vladimir Bataev <[email protected]> --------- Signed-off-by: Vladimir Bataev <[email protected]> Signed-off-by: artbataev <[email protected]> Co-authored-by: artbataev <[email protected]> * Evo2 address rare over-masking in 1m context dataset (#14821) * Address problems where sometimes in 1m dataset there are very large masked segments Signed-off-by: John St John <[email protected]> * only flip the tag extra if the segment length is too long Signed-off-by: John St John <[email protected]> * Undo the change to the pre commit config Signed-off-by: John St John <[email protected]> * Add clarifying comments about the state flipping logic Signed-off-by: John St John <[email protected]> --------- Signed-off-by: John St John <[email protected]> * Update cherry-pick workflow to use version 0.63.0 (#14832) * Update cherry-pick workflow to use version 0.63.0 Signed-off-by: Pablo Garay <[email protected]> * Update cherry-pick workflow version tag Signed-off-by: Pablo Garay <[email protected]> --------- Signed-off-by: Pablo Garay <[email protected]> * docs: Removing automodel items (#14840) Signed-off-by: Andrew Schilling <[email protected]> * update docs per guidance (#14841) * Update changelog for `v2.4.1` (#14828) * beep boop: Update changelog Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Fix changelog for 2.4.1 Signed-off-by: Charlie Truong <[email protected]> --------- Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Charlie Truong <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Charlie Truong <[email protected]> * Fi…
ko3n1g
pushed a commit
that referenced
this pull request
Oct 18, 2025
* Add hybrid parakeet with target language ID modelssupport and offline inferance pipeline Signed-off-by: Enas Albasiri <[email protected]> * formatted Target Lang Parakeet model support and offline pipeline Signed-off-by: Enas Albasiri <[email protected]> * add example use for Parakeet AST hybrid transducer CTC Signed-off-by: Enas Albasiri <[email protected]> * PR revision integrated Signed-off-by: Enas Albasiri <[email protected]> * add sample config file to target lang ID Signed-off-by: Enas Albasiri <[email protected]> * add straming iferacne support for RNNT with target lang ID support Signed-off-by: Enas Albasiri <[email protected]> * update streaming_utils-- rebase Signed-off-by: Enas Albasiri <[email protected]> * modifed Parakeet with target lang to Parakeet with prompt Signed-off-by: Enas Albasiri <[email protected]> * added unit tests and modifed files to reflect revisions Signed-off-by: Enas Albasiri <[email protected]> * added transcribe function to the model and test for it Signed-off-by: Enas Albasiri <[email protected]> * added CI-CD run test and timestamps test Signed-off-by: Enas Albasiri <[email protected]> * Apply isort and black reformatting Signed-off-by: ealbasiri <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * fix CodeQL failing tests Signed-off-by: Enas Albasiri <[email protected]> * Fix empty f-string issue in audio_to_text_lhotse_prompt Signed-off-by: Enas Albasiri <[email protected]> * keep transcription.py without changes Signed-off-by: Enas Albasiri <[email protected]> * keep transcribe_speech no change Signed-off-by: Enas Albasiri <[email protected]> * add more robus to coda graph in model forward and forward unit test Signed-off-by: Enas Albasiri <[email protected]> * fixed failing ci test Signed-off-by: Enas Albasiri <[email protected]> * add documentation Signed-off-by: Enas Albasiri <[email protected]> * Support QwenVL for inference API (#14534) * Support QwenVL for inference engine * Apply isort and black reformatting Signed-off-by: meatybobby <[email protected]> * Remove comment out * Reformat * Skip pylint check * Add unit tests * Apply isort and black reformatting Signed-off-by: meatybobby <[email protected]> --------- Signed-off-by: meatybobby <[email protected]> Co-authored-by: meatybobby <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * Hyena: Allow to use unfused RMSNorm + TELinear to restore accuracy and some speed (#14542) Signed-off-by: Enas Albasiri <[email protected]> * Fix sequence packing loss calculation (#14437) * Fix sequence packing loss calculation Signed-off-by: Rayan Dasoriya <[email protected]> * Fix nemo2 path Signed-off-by: Rayan Dasoriya <[email protected]> * Skip pylint Signed-off-by: Rayan Dasoriya <[email protected]> --------- Signed-off-by: Rayan Dasoriya <[email protected]> Co-authored-by: Dmytro Pykhtar <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * [Audio]: added streaming mode to SpectrogramToAudio (#14524) * [Audio]: added streaming mode to SpectrogramToAudio Signed-off-by: Rauf <[email protected]> * added time buffer Signed-off-by: Rauf <[email protected]> * renamed Nf -> num_frames Signed-off-by: Rauf <[email protected]> * added AudioToSpectrogram and scale and magnitude power Signed-off-by: Rauf <[email protected]> * added multiple chunking support Signed-off-by: Rauf <[email protected]> * added properties _stream_initialized, _eps, got rid of _prev_spec_frame Signed-off-by: Rauf <[email protected]> * added hanning window Signed-off-by: Rauf <[email protected]> * Apply isort and black reformatting Signed-off-by: nasretdinovr <[email protected]> * added a docstring regarding streaming istft mode Signed-off-by: Rauf <[email protected]> --------- Signed-off-by: Rauf <[email protected]> Signed-off-by: nasretdinovr <[email protected]> Co-authored-by: nasretdinovr <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * fix: fix missing rope scaling in exporting llama embedding model (#14523) Signed-off-by: Zhiyu Li <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * Update evo2 defaults so converted checkpoints have the right parameters (#14514) * Update evo2 defaults so converted checkpoints have the right parameters Signed-off-by: John St John <[email protected]> * Fix line too long issue Signed-off-by: John St John <[email protected]> * Fix expected changes to configs that are locked into our tests Signed-off-by: John St John <[email protected]> --------- Signed-off-by: John St John <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * deprecate t0 scripts (#14585) Signed-off-by: dimapihtar <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * cfg typo correction (#14588) Signed-off-by: Malay Nagda <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * [Perf script] Add use_te_activation_func and activation_func_fp8_input_store flags (#14522) * Add use te activation func and save act input in fp8 flags Signed-off-by: Guyue Huang <[email protected]> * Fix field name Signed-off-by: Guyue Huang <[email protected]> * Update scripts/performance/vlm/finetune_qwen25vl_32b.py Co-authored-by: malay-nagda <[email protected]> Signed-off-by: Guyue Huang <[email protected]> --------- Signed-off-by: Guyue Huang <[email protected]> Signed-off-by: Guyue Huang <[email protected]> Co-authored-by: malay-nagda <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * Modify logging message to signal that RestoreConfig will be used (#14469) Signed-off-by: Enas Albasiri <[email protected]> * Bump TE and Mcore (#14568) * Bump TE and Mcore Signed-off-by: Charlie Truong <[email protected]> * Use Mcore 69b65 Signed-off-by: Charlie Truong <[email protected]> --------- Signed-off-by: Charlie Truong <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * Avoid host-device sync in PTL logging (#14489) * remove sync in logging Signed-off-by: qiyuw <[email protected]> * Apply isort and black reformatting Signed-off-by: WanZzzzzz <[email protected]> * add class and func docstrings in data_sampler.py for pylint Signed-off-by: qiyuw <[email protected]> * Apply isort and black reformatting Signed-off-by: WanZzzzzz <[email protected]> --------- Signed-off-by: qiyuw <[email protected]> Signed-off-by: WanZzzzzz <[email protected]> Co-authored-by: qiyuw <[email protected]> Co-authored-by: WanZzzzzz <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * Integrate implicit filter kernel with Hyena layer (#14621) * add 1b arclongcontextconfig Signed-off-by: Farhad Ramezanghorbani <[email protected]> * fix device mess Signed-off-by: Farhad Ramezanghorbani <[email protected]> * add implicit_filter support Signed-off-by: Farhad Ramezanghorbani <[email protected]> * use padded input Signed-off-by: Farhad Ramezanghorbani <[email protected]> * Apply isort and black reformatting Signed-off-by: farhadrgh <[email protected]> * Revert "add 1b arclongcontextconfig" This reverts commit 029969b. --------- Signed-off-by: Farhad Ramezanghorbani <[email protected]> Signed-off-by: farhadrgh <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * Fix kv_channels configuration for Gemma2 27b (#14590) * fix gemma2 27b kv dimension Signed-off-by: Ananth Subramaniam <[email protected]> * fix gemma2 27b kv dimension Signed-off-by: Ananth Subramaniam <[email protected]> --------- Signed-off-by: Ananth Subramaniam <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * [Flux] small fixes (#14333) * feat: print expert groups on megatron init (#13874) Signed-off-by: Alexander Zhipa <[email protected]> Co-authored-by: Alexander Zhipa <[email protected]> Signed-off-by: CarlosGomes98 <[email protected]> * set a different seed for each dp rank Signed-off-by: CarlosGomes98 <[email protected]> * calculate loss inside autocast Signed-off-by: CarlosGomes98 <[email protected]> * disable per token loss, grad acc fusion Signed-off-by: CarlosGomes98 <[email protected]> * add missing self.seed Signed-off-by: CarlosGomes98 <[email protected]> * black formatting Signed-off-by: CarlosGomes98 <[email protected]> * Apply isort and black reformatting Signed-off-by: gautham-kollu <[email protected]> --------- Signed-off-by: Alexander Zhipa <[email protected]> Signed-off-by: CarlosGomes98 <[email protected]> Signed-off-by: gautham-kollu <[email protected]> Co-authored-by: Alexander Zhipa <[email protected]> Co-authored-by: Alexander Zhipa <[email protected]> Co-authored-by: gautham-kollu <[email protected]> Co-authored-by: gautham-kollu <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * [Flux] Add MXFP8 Support (#14473) * [Flux] Add MXFP8 support. Signed-off-by: Wil Kong <[email protected]> * [Flux] Add current and block scaling. Signed-off-by: Wil Kong <[email protected]> --------- Signed-off-by: Wil Kong <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * use hf hub to download ckpt (#14638) Signed-off-by: Ao Tang <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * Fine-tune embedding models (E5-Large-V2 and LLaMA-3.2-1B) on the allnli triplet dataset with NeMo Framework (#14584) * Create E2E-Embedding-Finetuning Signed-off-by: Hemant Giri <[email protected]> * Update E2E-Embedding-Finetuning Signed-off-by: Hemant Giri <[email protected]> * Delete tutorials/llm/embedding/E2E-Embedding-Finetuning Signed-off-by: Hemant Giri <[email protected]> * Create README.md Signed-off-by: Hemant Giri <[email protected]> * Add files via upload Signed-off-by: Hemant Giri <[email protected]> * Add files via upload This is a notebook for E2E finetuning a embedding model Signed-off-by: Hemant Giri <[email protected]> * Update README.md Signed-off-by: Hemant Giri <[email protected]> * Update README.md Signed-off-by: Hemant Giri <[email protected]> * Delete tutorials/llm/embedding/E2E-Embedding-Finetuning/download_dataset.py Signed-off-by: Hemant Giri <[email protected]> * Delete tutorials/llm/embedding/E2E-Embedding-Finetuning/finetune_e5.py Signed-off-by: Hemant Giri <[email protected]> * Delete tutorials/llm/embedding/E2E-Embedding-Finetuning/finetune_llama1b.py Signed-off-by: Hemant Giri <[email protected]> * Delete tutorials/llm/embedding/E2E-Embedding-Finetuning/import_e5_large.py Signed-off-by: Hemant Giri <[email protected]> * Delete tutorials/llm/embedding/E2E-Embedding-Finetuning/import_llama1b.py Signed-off-by: Hemant Giri <[email protected]> --------- Signed-off-by: Hemant Giri <[email protected]> Co-authored-by: Ao Tang <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * [Perf script] Llama and GPT3 perf script use mlp cast fusion Signed-off-by: Guyue Huang <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * remove service launch scripts (#14647) Signed-off-by: dimapihtar <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * warning instead of error with chat template (#14641) Signed-off-by: jenchen13 <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * fix notebook (#14643) Signed-off-by: Chen Cui <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * [Audio]: fixed bug in conformet unet (#14626) Signed-off-by: Rauf <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * Delete tutorials/llm/llama/biomedical-qa directory (#14653) Signed-off-by: Chen Cui <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * Fix code checkout during test (#14658) Signed-off-by: Charlie Truong <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * Fix Flux seed as optional Arg (#14652) * fix flux seed as optional Signed-off-by: Ao Tang <[email protected]> * fix fluxcontrolnet Signed-off-by: Ao Tang <[email protected]> * Fix code checkout during test Signed-off-by: Charlie Truong <[email protected]> --------- Signed-off-by: Ao Tang <[email protected]> Signed-off-by: Charlie Truong <[email protected]> Co-authored-by: Charlie Truong <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * remove older TTS tutorials (#14660) Signed-off-by: Jason <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * Remove PEFT scheme condition from recipe (#14661) * Remove PEFT scheme condition from recipe Signed-off-by: Ali Taghibakhshi <[email protected]> * remove unnecessary peft conditioning 12b --------- Signed-off-by: Ali Taghibakhshi <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * Add gpt-oss lora exporter (#14589) * add gpt-oss lora exporter Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * update lora exporter for experts Signed-off-by: Chen Cui <[email protected]> * disallow exporting expert lora since nemo implementation is not equivalent to hf Signed-off-by: Chen Cui <[email protected]> * linting Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * address comment Signed-off-by: Chen Cui <[email protected]> --------- Signed-off-by: Chen Cui <[email protected]> Signed-off-by: cuichenx <[email protected]> Co-authored-by: cuichenx <[email protected]> Co-authored-by: Charlie Truong <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * Add NeMo Voice Agent (#14325) * update streaming ASR Signed-off-by: stevehuang52 <[email protected]> * add voice agent Signed-off-by: stevehuang52 <[email protected]> * update readme Signed-off-by: stevehuang52 <[email protected]> * update websocket Signed-off-by: stevehuang52 <[email protected]> * update Signed-off-by: stevehuang52 <[email protected]> * update Signed-off-by: stevehuang52 <[email protected]> * update readme Signed-off-by: stevehuang52 <[email protected]> * update Signed-off-by: stevehuang52 <[email protected]> * clean up Signed-off-by: stevehuang52 <[email protected]> * clean up Signed-off-by: stevehuang52 <[email protected]> * fix typo Signed-off-by: stevehuang52 <[email protected]> * fix codeQL Signed-off-by: stevehuang52 <[email protected]> * update cfg Signed-off-by: stevehuang52 <[email protected]> * remove unused Signed-off-by: stevehuang52 <[email protected]> * update readme Signed-off-by: stevehuang52 <[email protected]> * change default models Signed-off-by: stevehuang52 <[email protected]> * fix diar diable Signed-off-by: stevehuang52 <[email protected]> * fix diar diable Signed-off-by: stevehuang52 <[email protected]> * update ux Signed-off-by: stevehuang52 <[email protected]> * update tts Signed-off-by: stevehuang52 <[email protected]> * update readme Signed-off-by: stevehuang52 <[email protected]> * fix and update Signed-off-by: stevehuang52 <[email protected]> * fix asr Signed-off-by: stevehuang52 <[email protected]> * update readmme Signed-off-by: stevehuang52 <[email protected]> * update doc and llm dtype Signed-off-by: stevehuang52 <[email protected]> * refactor and add example prompts Signed-off-by: stevehuang52 <[email protected]> * update readme Signed-off-by: stevehuang52 <[email protected]> * update readme Signed-off-by: stevehuang52 <[email protected]> * clean up Signed-off-by: stevehuang52 <[email protected]> * clean up Signed-off-by: stevehuang52 <[email protected]> * update info on streaming sortformer Signed-off-by: stevehuang52 <[email protected]> * move code to 'nemo/agents/voice_agent' Signed-off-by: stevehuang52 <[email protected]> * update doc Signed-off-by: stevehuang52 <[email protected]> * clean up Signed-off-by: stevehuang52 <[email protected]> * refactor Signed-off-by: stevehuang52 <[email protected]> * update doc Signed-off-by: stevehuang52 <[email protected]> * remove the unnecessary streaming state conversion and import it from sortformer_modules, remove PostProcessingParams Signed-off-by: Weiqing Wang <[email protected]> * Apply isort and black reformatting Signed-off-by: weiqingw4ng <[email protected]> * update doc Signed-off-by: stevehuang52 <[email protected]> * clean up Signed-off-by: stevehuang52 <[email protected]> * fix for llama-nemotron template, and refactor Signed-off-by: stevehuang52 <[email protected]> * fix tts separator Signed-off-by: stevehuang52 <[email protected]> * fix for llama-nemotron Signed-off-by: stevehuang52 <[email protected]> * update cfg Signed-off-by: stevehuang52 <[email protected]> * refactor and update doc Signed-off-by: stevehuang52 <[email protected]> * change default llm to qwen Signed-off-by: stevehuang52 <[email protected]> * update doc Signed-off-by: stevehuang52 <[email protected]> --------- Signed-off-by: stevehuang52 <[email protected]> Signed-off-by: Weiqing Wang <[email protected]> Signed-off-by: weiqingw4ng <[email protected]> Co-authored-by: Taejin Park <[email protected]> Co-authored-by: Kunal Dhawan <[email protected]> Co-authored-by: Weiqing Wang <[email protected]> Co-authored-by: weiqingw4ng <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * Update get_tensor_shapes function whose signature was refactored (#14594) * Update get_tensor_shapes function whose signature changed and wasn't refactored Signed-off-by: Asha Anoosheh <[email protected]> * Bump Mcore commit to latest on 0.14.0 branch Signed-off-by: Charlie Truong <[email protected]> * Bump Mcore Signed-off-by: Charlie Truong <[email protected]> * Set flux fsdp test to optional Signed-off-by: Charlie Truong <[email protected]> * Fix flux test to skip Signed-off-by: Charlie Truong <[email protected]> --------- Signed-off-by: Asha Anoosheh <[email protected]> Signed-off-by: Charlie Truong <[email protected]> Co-authored-by: Charlie Truong <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * fixing kernel restarting when transcribing (#14665) * fixing kernel restarting when transcribing Signed-off-by: Weiqing Wang <[email protected]> * fixing the same issue for tutorials/asr/ASR_with_NeMo.ipynb Signed-off-by: Weiqing Wang <[email protected]> * remove the change caused by IDE Signed-off-by: Weiqing Wang <[email protected]> --------- Signed-off-by: Weiqing Wang <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * Skip trt-llm and vllm install in install test (#14663) Signed-off-by: Charlie Truong <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * Canary tutorial fix (#14673) Signed-off-by: Nune <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> * added links to docs/false_positives.json Signed-off-by: Enas Albasiri <[email protected]> * added functional_tests/ASR_dev_run_Speech_to_Text_Hybrid_RNNT_CTC_Prompt Signed-off-by: Enas Albasiri <[email protected]> * updated file paths in functional test Signed-off-by: Enas Albasiri <[email protected]> --------- Signed-off-by: Enas Albasiri <[email protected]> Signed-off-by: ealbasiri <[email protected]> Signed-off-by: meatybobby <[email protected]> Signed-off-by: Rayan Dasoriya <[email protected]> Signed-off-by: Rauf <[email protected]> Signed-off-by: nasretdinovr <[email protected]> Signed-off-by: Zhiyu Li <[email protected]> Signed-off-by: John St John <[email protected]> Signed-off-by: dimapihtar <[email protected]> Signed-off-by: Malay Nagda <[email protected]> Signed-off-by: Guyue Huang <[email protected]> Signed-off-by: Guyue Huang <[email protected]> Signed-off-by: Charlie Truong <[email protected]> Signed-off-by: qiyuw <[email protected]> Signed-off-by: WanZzzzzz <[email protected]> Signed-off-by: Farhad Ramezanghorbani <[email protected]> Signed-off-by: farhadrgh <[email protected]> Signed-off-by: Ananth Subramaniam <[email protected]> Signed-off-by: Alexander Zhipa <[email protected]> Signed-off-by: CarlosGomes98 <[email protected]> Signed-off-by: gautham-kollu <[email protected]> Signed-off-by: Wil Kong <[email protected]> Signed-off-by: Ao Tang <[email protected]> Signed-off-by: Hemant Giri <[email protected]> Signed-off-by: jenchen13 <[email protected]> Signed-off-by: Chen Cui <[email protected]> Signed-off-by: Jason <[email protected]> Signed-off-by: Ali Taghibakhshi <[email protected]> Signed-off-by: cuichenx <[email protected]> Signed-off-by: stevehuang52 <[email protected]> Signed-off-by: Weiqing Wang <[email protected]> Signed-off-by: weiqingw4ng <[email protected]> Signed-off-by: Asha Anoosheh <[email protected]> Signed-off-by: Nune <[email protected]> Signed-off-by: Enas Albasiri <[email protected]> Co-authored-by: Enas Albasiri <[email protected]> Co-authored-by: ealbasiri <[email protected]> Co-authored-by: meatybobby <[email protected]> Co-authored-by: meatybobby <[email protected]> Co-authored-by: Anton Vorontsov <[email protected]> Co-authored-by: Rayan Dasoriya <[email protected]> Co-authored-by: Dmytro Pykhtar <[email protected]> Co-authored-by: nasretdinovr <[email protected]> Co-authored-by: nasretdinovr <[email protected]> Co-authored-by: Zhiyu Li <[email protected]> Co-authored-by: John St. John <[email protected]> Co-authored-by: malay-nagda <[email protected]> Co-authored-by: Guyue Huang <[email protected]> Co-authored-by: Bruno Alvisio <[email protected]> Co-authored-by: Charlie Truong <[email protected]> Co-authored-by: Qiyu Wan <[email protected]> Co-authored-by: qiyuw <[email protected]> Co-authored-by: WanZzzzzz <[email protected]> Co-authored-by: Farhad Ramezanghorbani <[email protected]> Co-authored-by: Ananth Subramaniam <[email protected]> Co-authored-by: Carlos Gomes <[email protected]> Co-authored-by: Alexander Zhipa <[email protected]> Co-authored-by: Alexander Zhipa <[email protected]> Co-authored-by: gautham-kollu <[email protected]> Co-authored-by: gautham-kollu <[email protected]> Co-authored-by: Wil Kong <[email protected]> Co-authored-by: Ao Tang <[email protected]> Co-authored-by: Hemant Giri <[email protected]> Co-authored-by: Jenny Chen <[email protected]> Co-authored-by: Chen Cui <[email protected]> Co-authored-by: Jason <[email protected]> Co-authored-by: Ali Taghibakhshi <[email protected]> Co-authored-by: cuichenx <[email protected]> Co-authored-by: He Huang (Steve) <[email protected]> Co-authored-by: Taejin Park <[email protected]> Co-authored-by: Kunal Dhawan <[email protected]> Co-authored-by: Weiqing Wang <[email protected]> Co-authored-by: weiqingw4ng <[email protected]> Co-authored-by: Asha Anoosheh <[email protected]> Co-authored-by: Weiqing Wang <[email protected]> Co-authored-by: nune-tadevosyan <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Important
The
Update branchbutton must only be pressed in very rare occassions.An outdated branch is never blocking the merge of a PR.
Please reach out to the automation team before pressing that button.
What does this PR do ?
Add NeMo voice agent pipeline that fuses NeMo ASR, diarization, TTS and any LLM together. Please see the README file https://github.com/NVIDIA/NeMo/blob/heh/nemo_voice/examples/voice_agent/README.md for details
Collection: [asr,voice_agent]