Skip to content

feat(tts): add includes_inter_frame_spaces flag to word-timestamp API#4330

Merged
filipi87 merged 1 commit into
pipecat-ai:mainfrom
inworld-ai:ian/tts-inter-frame-spaces
Apr 23, 2026
Merged

feat(tts): add includes_inter_frame_spaces flag to word-timestamp API#4330
filipi87 merged 1 commit into
pipecat-ai:mainfrom
inworld-ai:ian/tts-inter-frame-spaces

Conversation

@ianbbqzy

@ianbbqzy ianbbqzy commented Apr 17, 2026

Copy link
Copy Markdown
Contributor

Some TTS providers return verbatim tokens where spaces and punctuation are already embedded (e.g. Inworld). Joining these with an extra space produces hello , world instead of hello, world.

Changes:

  • TTSService.add_word_timestamps / _add_word_timestamps / _WordTimestampEntry gain an opt-in includes_inter_frame_spaces: bool = False flag, threaded through to every emitted TTSTextFrame. Defaults to False — no change for existing services.
  • InworldTTSService passes includes_inter_frame_spaces=True and returns tokens verbatim from _calculate_word_times.
  • Tests in test_tts_frame_ordering.py cover HTTP and WebSocket paths: verbatim text, PTS ordering, text-before-audio ordering, and the Inworld punctuation-token scenario.

@codecov

codecov Bot commented Apr 17, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 55.55556% with 4 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/pipecat/services/inworld/tts.py 0.00% 3 Missing ⚠️
src/pipecat/services/tts_service.py 83.33% 1 Missing ⚠️
Files with missing lines Coverage Δ
src/pipecat/services/tts_service.py 75.40% <83.33%> (+3.66%) ⬆️
src/pipecat/services/inworld/tts.py 24.95% <0.00%> (ø)

... and 15 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ianbbqzy ianbbqzy changed the title fix(tts/rtvi): support inter-frame spacing for verbatim word-alignmen… feat(tts): thread includes_inter_frame_spaces flag through word-timestamp API Apr 17, 2026
Some TTS providers (e.g. Inworld) return verbatim tokens where spaces and
punctuation are already embedded in the token text. When downstream consumers
join these tokens with an extra space they produce "hello , world" instead of
"hello, world".

Add an opt-in `includes_inter_frame_spaces: bool = False` parameter to
`add_word_timestamps` / `_add_word_timestamps`. The flag is threaded through
`_WordTimestampEntry` and stamped onto every emitted `TTSTextFrame`.
Defaults to `False` — no behaviour change for existing services.

`InworldTTSService` passes `includes_inter_frame_spaces=True` and stops
pre-processing tokens in `_calculate_word_times`, returning them verbatim.

Tests added to `test_tts_frame_ordering.py` covering both HTTP and WebSocket
delivery paths: verbatim text preservation, PTS ordering, text-before-audio
ordering, and the Inworld punctuation-token scenario.

Made-with: Cursor
@ianbbqzy ianbbqzy force-pushed the ian/tts-inter-frame-spaces branch from 58a0708 to b435ddf Compare April 18, 2026 19:03
@ianbbqzy ianbbqzy changed the title feat(tts): thread includes_inter_frame_spaces flag through word-timestamp API feat(tts): add includes_inter_frame_spaces flag to word-timestamp API Apr 18, 2026
@ianbbqzy ianbbqzy marked this pull request as ready for review April 20, 2026 01:53

@filipi87 filipi87 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @ianbbqzy, these changes look good.

I have a follow-up PR where I’m making includes_inter_frame_spaces optional so we can keep the default frame behavior.

I’m also bumping to small-webrtc-prebuilt 2.5.0 to fix karaoke highlighting.

Thank you for doing this.

@ianbbqzy

Copy link
Copy Markdown
Contributor Author

Regarding making includes_inter_frame_spaces optional. From my perspective, it already is because it defaults to false, so the default behavior won't change. But feel free to make it explicit if that aligns with Pipecat's code styles better, thanks!

@filipi87 filipi87 merged commit b435ddf into pipecat-ai:main Apr 23, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants