-
Notifications
You must be signed in to change notification settings - Fork 3.4k
Closed
Labels
ASRPoRMajor feature to be highlighted in release notesMajor feature to be highlighted in release notesfeaturerequest/PR for a new featurerequest/PR for a new feature
Milestone
Description
PR: #14561
PIC: @ealbasiri
Adds support for Hybrid RNNT-CTC BPE Model with Prompt Feature (EncDecHybridRNNTCTCBPEModelWithPrompt), enabling flexible ASR and AST tasks through prompt-based conditioning.
Key Features
- Architecture: Hybrid RNNT-CTC model with prompt vector conditioning
- Prompt vector (one-hot encoded) is concatenated to ASR embeddings from FastConformer
- Concatenated vector is fed into decoder for prompt-aware processing
- Tasks: model supports both ASR and AST
- Inference Modes: Supports both buffered streaming and offline inference
- Scalable Design: Can support multilingual ASR and AST tasks
Prompt-Based Conditioning
- Target language prompt: Required input that conditions the model behavior
- Source language detection: Not required - model automatically handles source language
- Task determination:
- Same source/target → ASR (transcription)
- Different source/target → AST (translation)
- Supported languages: multilingual ASR/AST
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
ASRPoRMajor feature to be highlighted in release notesMajor feature to be highlighted in release notesfeaturerequest/PR for a new featurerequest/PR for a new feature