Add Smallest AI STT and TTS service integrations#3897
Open
harshitajain165 wants to merge 6 commits intopipecat-ai:mainfrom
Open
Add Smallest AI STT and TTS service integrations#3897harshitajain165 wants to merge 6 commits intopipecat-ai:mainfrom
harshitajain165 wants to merge 6 commits intopipecat-ai:mainfrom
Conversation
added 5 commits
March 10, 2026 13:35
- STT: Update model from lightning to pulse with new API URL - STT: Add SmallestRealtimeSTTService using Pulse WebSocket API for low-latency streaming transcription - TTS: Add lightning-v3.1 model and set as default - stt_latency: Add SMALLEST_TTFS_P99 constant Made-with: Cursor
Migrate STT/TTS services from deprecated set_model_name()/set_voice() to the
new ServiceSettings pattern (STTSettings/TTSSettings). Add default voice_id
("sophia") for TTS services, fix voice references, and include two foundational
example scripts showing WebSocket and HTTP usage.
Made-with: Cursor
Made-with: Cursor
harshitajain165
force-pushed
the
pipecat-smallest-stt-tts-integration
branch
from
f144845 to
6c67264
Compare
markbackman
reviewed
Mar 10, 2026
Contributor
markbackman
left a comment
markbackman
left a comment
There was a problem hiding this comment.
Thanks for the submission!
Is it worth considering a simpler implementation where there's only one service implementation per module? The goal would be to select the implementation best suited for real-time use cases. I think this would make sense for STT, especially as a websocket implementation is superior to HTTP. The same can be true of TTS.
This was referenced Mar 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
smallestoptional dependency group inpyproject.toml(httpx,numpy,soundfile,websockets)SMALLEST_TTFS_P99latency constant instt_latency.pyChanges
src/pipecat/services/smallest/__init__.pyDeprecatedModuleProxysrc/pipecat/services/smallest/stt.pySmallestSTTService) and WebSocket (SmallestRealtimeSTTService) STT implementationssrc/pipecat/services/smallest/tts.pySmallestTTSService) and HTTP (SmallestHttpTTSService) TTS implementationssrc/pipecat/services/stt_latency.pySMALLEST_TTFS_P99constantpyproject.tomlsmallestextras groupuv.lockexamples/foundational/07zl-interruptible-smallest.pyexamples/foundational/07zl-interruptible-smallest-http.pyTest plan
ruff checkandruff format)