Skip to content

Hybrid RNNT-CTC Prompted Parakeet Model support #14954

@nithinraok

Description

@nithinraok

PR: #14561

PIC: @ealbasiri

Adds support for Hybrid RNNT-CTC BPE Model with Prompt Feature (EncDecHybridRNNTCTCBPEModelWithPrompt), enabling flexible ASR and AST tasks through prompt-based conditioning.

Key Features

  • Architecture: Hybrid RNNT-CTC model with prompt vector conditioning
    • Prompt vector (one-hot encoded) is concatenated to ASR embeddings from FastConformer
    • Concatenated vector is fed into decoder for prompt-aware processing
  • Tasks: model supports both ASR and AST
  • Inference Modes: Supports both buffered streaming and offline inference
  • Scalable Design: Can support multilingual ASR and AST tasks

Prompt-Based Conditioning

  • Target language prompt: Required input that conditions the model behavior
  • Source language detection: Not required - model automatically handles source language
  • Task determination:
    • Same source/target → ASR (transcription)
    • Different source/target → AST (translation)
  • Supported languages: multilingual ASR/AST

Metadata

Metadata

Assignees

No one assigned

    Labels

    ASRPoRMajor feature to be highlighted in release notesfeaturerequest/PR for a new feature

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions