Skip to content

SSE: emit periodic keepalive heartbeats so idle streams survive proxy timeouts #226

Closed

Description

@EricAndrechek

Area: streaming (api) · ops — reliability gap · found via WaveHouse-Stats dogfooding (reproduced live)

Expected: a long-lived SSE stream survives through a CDN/proxy even with no events flowing.

Actual: the handler writes : connected once (internal/api/stream.go:67) then sends nothing until an event arrives — the select loop (:116-145) has only context-done and incoming-event cases, no heartbeat ticker. On a quiet stream an intermediary (Cloudflare's edge idle-timeout, ~2 min) resets the idle HTTP/2 stream with INTERNAL_ERROR. Browser EventSource auto-reconnects so it's masked there, but curl/server-side consumers drop repeatedly and each reconnect re-runs gap-fill.

Impact (Stats): curl against /v1/stream?table=gh_events dies ~every 2 min while the table is quiet; we wrap it in a reconnect loop.

Scope: periodic SSE comment heartbeat on a ticker, added to the select loop.

Related: SSE per-subscriber channel buffer sizing #152 (same handler).


From WAVEHOUSE-FEEDBACK.md (WaveHouse-Stats dogfooding); validated by code-read against 0f8826c on 2026-06-04.

Activity

  1. added
    bugSomething isn't working
    area/observabilityMetrics, logs, traces, health, profiling
    area/apiHTTP handlers, routing, middleware
    on Jun 4, 2026
  2. coderabbitai commented on Jun 4, 2026

    @coderabbitai
    🔗 Related PRs

    #124 - chore(api)!: drop hub wildcard fan-out; SSE/WS use ?table= (closes #100) [merged]
    #174 - refactor(table names): handle unsafe table names [merged]


    📝 Issue Planner

    Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.

    • Create Plan

    🧪 Issue enrichment is currently in open beta.

    You can configure auto-planning by selecting labels in the issue_enrichment configuration.

    To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

    issue_enrichment:
      auto_enrich:
        enabled: false

    💬 Have feedback or questions? Drop into our discord!

  3. EricAndrechek commented on Jun 4, 2026

    @EricAndrechek
    MemberAuthor

    Reproduced live (Eric, 2026-06-04) on the public demo, server-side:

    curl -N "https://stats.wavehouse.dev/v1/stream?table=gh_events"
    : connected
    curl: (92) HTTP/2 stream 1 was not closed cleanly: INTERNAL_ERROR (err 2)
    (after 2m 5s, EC2 → Cloudflare Tunnel)
    

    Confirms the ~2-minute idle reset — and note it's via a Cloudflare Tunnel here, not just the CDN edge, so the keepalive is needed for tunneled/proxied server-side consumers generally (not one specific CDN). Browser EventSource masks it via auto-reconnect; curl/server-side consumers don't. Trivial fix (a :\n\n heartbeat ticker in the select loop) and it's visible to anyone curling the demo, so it's a cheap launch-polish win.

  4. moved this from Backlog to Ready in WaveHouse Task Boardon Jun 9, 2026
  5. self-assigned this
    on Jun 17, 2026
  6. moved this from Ready to In progress in WaveHouse Task Boardon Jun 17, 2026
  7. added a commit that references this issue on Jun 26, 2026
    725fdee
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area/apiHTTP handlers, routing, middlewarearea/observabilityMetrics, logs, traces, health, profilingbugSomething isn't working

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions