[UtkarshUsername]

Problem

Native streaming can fall behind while dictating on some machines, as reported in #32050. Web and desktop currently choose native streaming whenever the selected model supports it, with no way to use that same model after recording stops.

Change

Add Streaming transcription under Settings → Voice → Advanced, enabled by default. Turning it off retains the recording locally until stop and sends it through the existing authorized batch path. The choice is captured for the recording, including retries. Availability follows the selected transcription destination and model; unsupported models disable the control without changing the saved device preference. Non-native chunked previews and cloud transcription keep their existing behavior.

The native worker now plans batch audio from the actual run mode, so a streaming-capable model receives the existing bounded chunks, silence handling, and batch error recovery. The shared web voice controller covers composer dictation, shortcuts, text fields, comments/annotations, and model tests. Desktop uses that same web implementation; no mobile UI is added.

Scope and approval

This is focused configuration of the existing native-streaming and batch transcription capability. It keeps today's default and the selected model. Related to #32050 as an opt-out/workaround, without claiming to resolve the entire performance issue. No engine update, GPU Auto policy, feed-interval tuning, pre-roll change, or benchmark/recommendation system is included.

Verification

• vp run check:changed: format, lint, knip, GraphQL validation, typechecks of 37 affected packages including mobile/desktop consumers, and 161 focused tests passed.
• A native-worker regression test first failed on the original implementation with Model input limit exceeded, then passed with bounded offline chunks and no streaming session.
• Browser behavior tests cover default-on settings, actual batch PCM retention and authenticated destination requests, captured preferences during preparation and retry, cancellation, native versus buffered capabilities, translation locale, and saved preferences across destination capability changes.
• Isolated Windows client: Moonshine Streaming Tiny Q8_0 on CPU with the first 8 seconds of the public transcribe.cpp/samples/whole-earth.wav sample injected as microphone input. ON opened one native session and showed evolving text while recording. OFF reached the real POST /api/voice/transcribe batch endpoint and returned HTTP 200. A direct production-worker run of the same sample also returned the expected sentence with silence filtering enabled.

Before

After

Implemented with GPT-6.1 Sol through the Codex harness.

[macroscopeapp]

Approvability

Verdict: Approved at ca98bde

Macroscope's review found this PR approvable — Adds a device-local option to transcribe after recording stops while preserving streaming by default. It reuses the existing consent-checked batch path and selected model, with limited, reversible impact.

^{You can add or adjust custom eligibility rules. Learn more.}

[coderabbitai]

No actionable comments were generated in the recent review.

Walkthrough

A saved client setting now controls native streaming transcription. The browser voice-input flow uses the setting when choosing between native streaming and buffered transcription. Server audio planning and silence-model selection use the effective mode for each operation.

Changes: Streaming transcription preference

Layer / File(s)	Summary
Preference contract and settings control packages/contracts/src/settings.ts, packages/contracts/src/settings.test.ts, apps/web/src/components/settings/VoiceSettingsPanel.tsx, apps/web/src/components/settings/VoiceSettingsPanel.browser.test.tsx, apps/web/src/components/settings/settingsSearch.ts, docs/user/voice-input.md	Client settings add speechStreaming, which defaults to true and accepts patches. Voice settings expose the preference for supported models, add search support, and document how to disable streaming. Browser tests cover persistence and model support.
Browser mode selection apps/web/src/speech/browserVoiceInput.ts, apps/web/src/speech/useEnvironmentSpeechInput.ts, apps/web/src/speech/browserVoiceInput.browser.test.ts	The browser voice-input platform reads the latest setting and selects native streaming or buffered transcription based on the setting and environment support. Tests cover mode selection, retry, and cancellation.
Server processing by operation mode apps/server/src/speech/native.ts, apps/server/src/speech/SpeechService.ts, apps/server/src/speech/native.test.ts	Native audio creation and VAD eligibility use the operation’s effective streaming mode. Silence-model selection receives the operation mode. A test checks bounded offline transcription chunks when the model supports streaming.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant ClientSettings
  participant useEnvironmentSpeechInput
  participant createBrowserVoiceInputPlatform
  participant TranscriptionMode
  ClientSettings->>useEnvironmentSpeechInput: Store speechStreaming preference
  useEnvironmentSpeechInput->>createBrowserVoiceInputPlatform: Provide the current setting
  createBrowserVoiceInputPlatform->>TranscriptionMode: Select native streaming or buffered transcription

Loading

Possibly related issues

• pingdotgg/t3code#32673 — The browser tests cover retaining audio after a failed batch transcription and retrying it.
• pingdotgg/t3code#32653 — The added batch path retains recorded audio through transcription, though the issue’s draft-change handling is not addressed.
• pingdotgg/t3code#32050 — The new preference allows users to disable native live transcription and use batch transcription instead.
• pingdotgg/t3code#33197 — The batch path retains audio until recording stops, but thread-owner cancellation behavior is not changed.

Possibly related PRs

• pingdotgg/t3code#30623 — Adds the environment voice-input flow and client settings that this change extends.
• pingdotgg/t3code#30625 — Adds the server batch and native-streaming paths whose mode selection this change updates.
• pingdotgg/t3code#32674 — Changes streaming-versus-batch recovery and audio retention used by the newly selectable batch path.

Suggested reviewers: juliusmarminge, t3dotgg

Merge Risk: Minimal · up to ca98b

This adds an opt-out toggle for streaming transcription that is on by default, so existing behavior is unchanged unless users turn it off. No actionable merge-blocking risk was found.

Pre-merge checks: 4

Check name	Status	Explanation
Title check		The title clearly and concisely identifies the main change: adding a native streaming transcription toggle. It follows the repository’s conventional commit style.
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.
Description check		The description follows the required Problem, Change, Scope and approval, and Verification sections. It explains the behavior, scope, affected components, tests, manual checks, screenshots, limitation…

Full details: Description check

Explanation

The description follows the required Problem, Change, Scope and approval, and Verification sections. It explains the behavior, scope, affected components, tests, manual checks, screenshots, limitations, and agent details. It is complete enough to pass.

Finishing Touches

Generate unit tests (beta)

• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Recent review info

Run configuration

• Source mode: Auto
• Repository: pingdotgg/t3code
• Branch: main
• File: .coderabbit.config.ts
• Commit: d2c2595
• Layers: Repository TS config
• Review profile: CHILL
• Plan: Advanced
• Run ID: 3c2d338b-ddee-470b-90a8-2bf35f4ead57

Commits

Reviewing files that changed from the base of the PR and between d2c2595 and ca98bde.

Files selected for processing: 12

• apps/server/src/speech/SpeechService.ts
• apps/server/src/speech/native.test.ts
• apps/server/src/speech/native.ts
• apps/web/src/components/settings/VoiceSettingsPanel.browser.test.tsx
• apps/web/src/components/settings/VoiceSettingsPanel.tsx
• apps/web/src/components/settings/settingsSearch.ts
• apps/web/src/speech/browserVoiceInput.browser.test.ts
• apps/web/src/speech/browserVoiceInput.ts
• apps/web/src/speech/useEnvironmentSpeechInput.ts
• docs/user/voice-input.md
• packages/contracts/src/settings.test.ts
• packages/contracts/src/settings.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

[maria-rcks]

Note

Written by claude-opus-5-5 on behalf of Maria

Verified in a real dev environment

Head ca98bdec7a, compared with main at 0629735d37. I used vp run dev on Linux with a seeded dev database and added a demo-repo git project through Add project → Local folder. I selected this environment under Transcribe with and downloaded Moonshine Streaming Tiny from Voice settings. I drove headless Chrome with Playwright. Its fake microphone played the public whole-earth.wav sample, and I dictated into the new-thread composer with Start voice input and Finish voice input.

Before (main): Advanced settings have no streaming option. Dictating with this model always opens /ws/voice and shows live preview text while recording.

After (PR): Streaming transcription appears under Advanced. It is on by default.

• ON (default): /ws/voice opens, live text appears from about 3 s into the recording, and the final text lands in the composer.
• OFF: no /ws/voice socket opens and no preview text appears during a 9 s recording. On Finish voice input, one POST /api/voice/transcribe returns 200 about 3 s later. The composer then gets "It was an amazing publication called The Whole Earth Catalog, which was one of the Bibles of my generation. It was created by".
• OFF, 45 s recording: one POST /api/voice/transcribe returns 200 with the full ~45 s passage. There was no Model input limit exceeded error, so the batch path chunks the streaming model correctly.
• Persistence: OFF stays saved after a fresh browser launch. Turning it back ON restores /ws/voice with live text.
• Unsupported model: after selecting Moonshine Tiny, which does not stream, the switch is disabled and says "This model doesn't support native streaming." The saved OFF value is kept. Dictating with that model still uses its existing chunked preview over /ws/voice and transcribes correctly. Switching back to Moonshine Streaming Tiny shows the switch enabled and still OFF.
• Settings search: searching "streaming" lists Streaming transcription · Voice and opens Voice settings. Clicking the result opens Voice at the top without scrolling to the row. The existing Transcription hardware and model unloading result does the same, so this is not new in this PR.

Streaming OFF: no live text while recording, and the transcript appears after Finish:

Video: dictation with streaming off (16 s mp4)

Streaming ON (default): live preview while recording, as on main:

Video: dictation with streaming on (29 s mp4)

Not verified

• Desktop (Electron) shell: not run. It uses the same web voice controller.
• GPU inference: this VM is CPU only.
• Whether OFF helps on machines where live transcription falls behind: that slowdown did not reproduce here.
• Other streaming models (Parakeet Unified, Nemotron, Voxtral) and remote/relay environments: not run.
• Mobile: not run. The PR adds no mobile UI.