-
Notifications
You must be signed in to change notification settings - Fork 1.8k
Description
Environment
- pipecat version: 0.0.102 (and current
mainat commit c682a44) - Python version: 3.13.5
- Operating System: macOS 15.4 (Darwin 25.2.0)
Issue description
Commit 5f64dae0c ("Filter RTVIObserver to downstream frames only") re-added a direction != FrameDirection.DOWNSTREAM: return guard to RTVIObserver.on_push_frame(). This was intended to prevent duplicate RTVI messages for frames that are broadcast in both directions (e.g. UserStartedSpeakingFrame, FunctionCallResultFrame).
However, GeminiLiveLLMService pushes TranscriptionFrame only upstream (FrameDirection.UPSTREAM) — it is never broadcast downstream. As a result, the RTVIObserver silently drops every user transcription, and RTVI clients never receive user-transcription messages.
The same issue likely affects OpenAIRealtimeLLMService and any other realtime/multimodal LLM service that pushes transcription frames upstream only.
Relevant code:
RTVIObserver.on_push_framedirection guard:src/pipecat/processors/frameworks/rtvi.py(line ~1226)GeminiLiveLLMService._push_user_transcriptionpushing upstream:src/pipecat/services/google/gemini_live/llm.py(line ~1615)
# RTVIObserver — drops ALL upstream frames including TranscriptionFrame
if direction != FrameDirection.DOWNSTREAM:
return# GeminiLiveLLMService — pushes TranscriptionFrame upstream only
await self.push_frame(
TranscriptionFrame(text=text, user_id="", timestamp=time_now_iso8601(), result=result),
FrameDirection.UPSTREAM,
)Reproduction steps
- Create a bot using
GeminiLiveLLMServicewith any pipeline (e.g. the26b-gemini-live-function-calling.pyexample) - Connect an RTVI client (e.g.
@pipecat-ai/client-react) - Speak to the bot
- Observe that the bot responds with audio, but the RTVI client never receives
user-transcriptionmessages
Expected behavior
RTVI clients should receive user-transcription messages containing the user's speech transcribed by the Gemini Live API (via input_audio_transcription).
This worked correctly before commit 5f64dae0c when the direction filter was not present in RTVIObserver.
Actual behavior
RTVIObserver.on_push_frame() returns early for all upstream frames. Since TranscriptionFrame from Gemini Live is only pushed upstream, it is never processed, and no user-transcription RTVI message is ever sent to the client.
The Gemini Live service does receive and process transcriptions from the API (visible in logs as [Transcription:user] [...] at DEBUG level), but the RTVIObserver discards them before they can be forwarded.
Logs
# Transcription IS received from Gemini API:
2026-02-14 19:06:28.685 | DEBUG | pipecat.services.google.gemini_live.llm:_handle_msg_input_transcription:1683 - [Transcription:user] [maybe problem solve neighbor yeah.]
# But NO corresponding RTVI user-transcription message is ever sent to the client
Suggested fix
Either:
- Broadcast
TranscriptionFramein both directions fromGeminiLiveLLMService(and other realtime LLM services), so the downstream copy reaches RTVIObserver — OR - Allowlist
TranscriptionFrameandInterimTranscriptionFramein the RTVIObserver's direction check, so upstream-only transcription frames are still processed