Skip to content

Latest commit

 

History

History
488 lines (400 loc) · 273 KB

CHANGELOG.md

File metadata and controls

488 lines (400 loc) · 273 KB

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

Unreleased

Added

  • Named pipes invalidate their cached results on writes to the tables they read, via versioned namespaces and a refresh-time cascade (internal/discovery/{discovery,explain}.go, internal/api/pipes.go, internal/pipes/pipes.go, internal/cache/{cache,local,version_manager}.go, internal/chsql/nats.go (moved from internal/query/ident.go), cmd/wavehouse/main.go, docs/src/content/docs/{pipes.mdx,architecture.md}, plus unit + live-ClickHouse tests): closes #178. A pipe's table dependencies are resolved by asking ClickHouse — EXPLAIN QUERY TREE on the bound query the first time it runs (discovery.ResolveTables, reading the table list off ClickHouse's own analysis), cached per bound query (parameter combination). WaveHouse never parses pipe SQL itself and never rejects a pipe. EXPLAIN QUERY TREE is the pre-optimization tree, so it keeps a table even when the query is metadata-answered (SELECT count() FROM t still tracks t) or reads a currently-empty table — both of which system.query_log and the post-optimization EXPLAIN PLAN would drop. It also tracks reads that aren't plain FROM clauses: a joinGet() of a Join-engine table and a dictGet() of a dictionary (mapped via system.dictionaries to the ClickHouse table backing it). Table-name extraction is robust to EXPLAIN-rendering quirks the unit fixtures didn't originally cover: a FINAL/SAMPLE modifier appends fields after the name (table_name: db.t, final: 1) and a string literal can itself contain the text table_name: — neither leaks into, nor fabricates, a tracked table — and a dictGet against a not-yet-loaded dictionary force-loads it first, since ClickHouse reports an empty source for an unloaded dict that would otherwise drop its backing table. A build-tagged differential fuzzer (internal/discovery/fuzz_deps_test.go, run with -tags fuzzdeps against a local ClickHouse) cross-checks the resolved set for hundreds of nested / UNION / parameter-gated queries against a construction-based ground truth. Each resolved table is folded into the cache key as its own versioned namespace (a per-name lookup, no per-request graph walk), so a write to it misses the key. Views and materialized views are versioned namespaces too: the schema registry resolves each view's definition (also via EXPLAIN) once per schema change into a cascade (base table → the views reading it, transitively), pushed into the cache, so a write to a base table also bumps every view over it and evicts a pipe reading the view; a view redefined in place (and the views built on it) is evicted at the next refresh. A materialized view's TO target rides the same cascade: CREATE MATERIALIZED VIEW mv TO target makes target an ordinary base table that only inserts to the MV's source ever populate, so a source write also bumps the target — and chained targets, and the views over them — evicting a pipe that reads the target table directly (the target is parsed off the rendered create_table_query, since system.tables grew first-class target columns only in ClickHouse 26.6, and propagated along the dependencies_table trigger edges — a join or dictGet in the MV's definition never fires it). Resolution is precise per parameter binding, and otherwise over-resolves — never under: for a parameter-gated UNION, ClickHouse folds the bound predicate so a constant-false arm is pruned (source=web depends only on web_events, not mobile_events; resolved and cached per bound query, and emitting the wavehouse_pipe_dep_tables_pruned_total metric), while a query ClickHouse can't analyze (a write/DDL pipe, an unreachable server, a missing table) falls back to a single database-wide version that every write bumps, so any write evicts it — O(1) per request, and it holds even before the schema registry's first successful refresh. The pruning is conservative — an arm is dropped only when its filter is provably constant-false, so anything unproven stays tracked (over-resolve, never stale). Schema refresh is atomic (rebuilt only when content changes) and re-resolves pipes (PipesHandler.ClearResolvedDeps). Pipes run the author SQL as-is with no policy row/column filtering (the per-pipe allowed_roles gates execution), so every allowed caller shares one cached entry. Unsupported, but bounded (run, not blocked): write/ingest pipes fail EXPLAIN and take the database-version fallback (a parameter as a whole table name, FROM {{tbl}}, binds to a quoted string and errors at execution and resolution alike, so nothing caches — while a name merely assembled from a parameter, FROM events_{{year}}, resolves per binding like any bound query); an external read — a table function (s3()/numbers(), detected as a TABLE_FUNCTION node in the query tree), a cross-database table (direct or a joinGet/dictGet target), or a dictionary without a local backing table — can never be evicted by a write, so its result is TTL-capped (~10s) instead of cached at full TTL. Single-database (the configured clickhouse.database). The version-folded cache key is snapshotted before the query executes and reused for its Set (Cache.Key), so a write landing mid-query orphans the entry rather than re-homing the pre-write result under the post-bump key — neither cached read path (pipes or structured queries) can serve a result older than the write that should have evicted it. Degraded modes fail toward expiry, not staleness: a structured query on an unfoldable view and a dependency set that was dead-branch pruned are TTL-capped (cache.UnresolvedDepsTTLCap — the pruned cap is a belt-and-suspenders bound on a hypothetical wrong prune, with an integration test pinning that pruning still fires on the shipped ClickHouse version), and a transient EXPLAIN failure (an unreachable server, a canceled request) is retried on the pipe's next execution instead of memoized as permanent fallback. Schema refreshes are serialized end-to-end so overlapping refreshes can't install an older cascade over a newer one, and a refresh that lands mid-resolution wins over the in-flight result (generation-gated memo insert). The degraded modes are metered: wavehouse_pipe_dep_resolution_fallback_total and wavehouse_pipe_dep_unresolved_total count degraded executions (cache hits included — the share of pipe traffic on the fallback / TTL-capped), and wavehouse_pipe_dep_memo_evictions_total counts resolution-memo evictions past its fixed 50k cap (sustained growth = a high-cardinality pipe parameter).

  • Non-JWT operator key for full-access admin + break-glass recovery (internal/auth/auth.go, internal/auth/context.go, internal/auth/auth_test.go, internal/api/router.go, internal/api/router_test.go, internal/config/config.go, internal/config/config_test.go, cmd/wavehouse/main.go, config.yaml, deployments/compose/standalone.yaml, tests/e2e/fixtures/config.yaml, docs/src/content/docs/configuration.mdx, docs/src/content/docs/access-control.mdx, docs/src/content/docs/api.md, docs/src/content/docs/deployment.md, docs/src/content/docs/development.md, docs/src/content/docs/architecture.md, docs/src/content/docs/reverse-proxy.mdx, AGENTS.md, SECURITY.md): closes #240; partially advances the auth-hardening epic #228 and the management/data-plane split #359. Adds an optional auth.operator_key (WH_AUTH_OPERATOR_KEY): a request presenting it — via an Authorization: Operator <key> header (forwarded verbatim by proxies, no collision with Bearer JWTs) or the X-Operator-Key alias — is authorized as a full-access platform operator — the whole data plane and the /v1/admin/* management surface — without minting a JWT and independently of the token verifier, giving the person running the deployment a role-free credential for bootstrap and break-glass. The middleware checks it before the Bearer token with a constant-time comparison (crypto/subtle) and stamps two things into the request context: the live admin_role (so the policy evaluator's admin bypass grants unrestricted data-plane access while a policy exists) and a platform-operator bit that RequireAdmin honors even when the policy is nil/deleted — the one HTTP path that can restore a wiped policy, which previously required SSH access and a reboot. Empty (the default) disables it, so existing deployments are unchanged; treat it as an admin secret (load from a secret store, serve only over TLS). A successful operator authentication is audit-logged at INFO; a request presenting a non-matching operator key is logged at WARN and counted by a new wavehouse_auth_operator_key_failures_total counter (a probing/brute-force signal on the most privileged credential in the system) before falling through to the normal token/default path — the middleware still never rejects. The Authorization auth-scheme is matched case-insensitively (RFC 7235) via a shared authScheme helper, which also makes the existing Bearer JWT scheme case-insensitive (previously it required the canonical Bearer casing). Explicitly out of scope, tracked in #359: scoping the operator credential to the management surface only, and capability-scoped admin permissions.

  • Missing-dedupe-id observability + optional strict mode (internal/api/ingest.go, internal/api/ingest_test.go, internal/config/config.go, cmd/wavehouse/main.go, config.yaml, deployments/compose/standalone.yaml, docs/src/content/docs/configuration.mdx, docs/src/content/docs/api.md, docs/src/content/docs/architecture.md): closes #219. With dedupe enabled, a row missing the configured id_field can't be deduped — previously it was published with idempotency silently disabled and no log or metric, so a producer bug that dropped the id turned off the guarantee for those rows unnoticed. Now every such row is logged at WARN and counted by a new wavehouse_ingest_dedupe_missing_id_total counter (labeled by table), making the loss observable server-side. A new opt-in dedupe.require_id (WH_DEDUPE_REQUIRE_ID, default false) turns that signal into enforcement: a row missing the id is rejected (400 for a single insert; a per-record failure in a batch) instead of published — a tripwire for producers that must guarantee the id (complements the client-side #202). Default behavior is unchanged.

  • "Durability & Storage" operations guide (docs/src/content/docs/durability.md (new), docs/src/config/sidebar.ts, docs/src/content/docs/reverse-proxy.mdx, docs/src/content/docs/configuration.mdx, docs/src/content/docs/deployment.md): documents #84. A new Operations page making the embedded-JetStream durability contract explicit before the docs site publishes: a 200 from POST /v1/ingest means the event has been fsync'd to disk on the node (the server runs with SyncAlways: true in internal/mq/embedded.go), which makes the storage substrate's fsync tail the ingest latency floor. Covers the contract (and how it differs from JetStream's default page-cache-then-periodic-sync mode), why a slow fsync tail manifests as create stream: ... context deadline exceeded and 503 backpressure, a where-it's-cheap-vs-expensive substrate table (managed cloud block storage and PLP NVMe vs. ZFS-without-SLOG / qcow2-on-ext4 / spinning disks), an fio recipe + verdict bands to measure your own storage (with the macOS F_FULLFSYNC honesty caveat), and the symptom checklist. Forward-references the configurable group-commit interval (mq.sync_interval, #139) and the planned wavehouse storage-check preflight (#84) without claiming either exists yet. Cross-linked from Configuration (Message Queue), Deployment (Persistent Storage), and the Ingest Pipeline's worker-side ack section; no code changes.

  • "Behind a reverse proxy" deployment guide (docs/src/content/docs/reverse-proxy.mdx (new), docs/src/config/sidebar.ts, docs/src/content/docs/deployment.md, docs/src/content/docs/configuration.mdx, docs/src/content/docs/api.md, internal/api/stream.go, internal/api/stream_test.go): closes #241. A new Operations page for the common "WaveHouse behind nginx / Caddy / Cloudflare Tunnel" setup, since several behaviors only matter behind a proxy and weren't documented together. Covers: TLS termination (WaveHouse serves plain HTTP and manages no certs); the request-body size limits and the division of responsibility (WaveHouse ships fixed in-code memory-safety backstops — 1 MiB control / 16 MiB ingest — while the proxy is the tunable outer limit, so a missing/loose proxy limit can't OOM the server); Server-Sent Events buffering + idle-timeout tuning (WaveHouse sends a : connected comment on open plus a periodic : keepalive comment so quiet streams survive proxy idle timeouts, #226); the ?token= / since / Last-Event-ID forwarding streams need; X-Forwarded-For trust (don't expose :8080 directly — it's honored, so a direct client could spoof it); and which health paths to expose (/livez//readyz internal-optional, /v1/health must stay public). Ships full example nginx, Caddy, and Cloudflare-Tunnel configs, and is cross-linked from Deployment, Configuration, and the API reference. One small code change lands with it: the SSE endpoint (GET /v1/stream) now sets X-Accel-Buffering: no so nginx-class proxies stream events without buffering out of the box (nginx strips the header before the client sees it; Caddy/Cloudflare ignore it). The health-probe guidance is also upgraded from "optional" to a recommendation — keep the bare /livez//readyz//healthz paths internal (a public /readyz turns each hit into a ClickHouse Ping) and expose only /v1/health publicly.

  • Coverage publishing — a self-hosted Go coverage README badge and GitHub Code Quality PR comments (.github/workflows/ci.yml, .github/actionlint.yaml (new), scripts/cov/main.go, scripts/ci/publish-badge.sh (new), .testcoverage.yml, go.mod/go.sum, README.md, AGENTS.md, docs/src/content/docs/development.md): closes #133, now that the repo is public. Two published surfaces, both non-gating — make cov's thresholds stay the only merge gate. (1) README badge: a new cov badge subcommand renders a shields.io endpoint JSON for the merged Go total using the exact number threshold.total gates (same .testcoverage.yml excludes), and a new non-gating badge job — the sole holder of contents:write, running only on trusted main — publishes it to an orphan badges branch via scripts/ci/publish-badge.sh, which the README reads over raw.githubusercontent.com (unrestricted for a public repo). (2) PR comments: the coverage job converts the merged Go profile to Cobertura (go tool gocover-cobertura, a new pinned Go tool dependency, with -ignore-dirs mirroring the YAML's global excludes) and uploads it to GitHub Code Quality via actions/upload-code-coverage (code-quality: write); the github-code-quality[bot] posts the aggregate + per-file diff-vs-main comment. The upload is continue-on-error so this public-preview GitHub feature can never red CI, and fork PRs skip it (no code-quality token, per GitHub's own guard). actionlint doesn't recognize the preview code-quality permission scope yet, so a new .github/actionlint.yaml suppresses only that one message. Requires the repo's Settings → Code quality enablement for the comments to render. Full design in .github/workflows/README.md §"Coverage publishing".

  • Mermaid diagrams export as PNG, with Copy/Download in the zoom lightbox (docs/src/integrations/diagram-png.mjs (new), docs/src/components/MermaidZoom.astro, docs/astro.config.mjs, docs/src/content/docs/development.md): diagrams render as inline SVG (great for reading — selectable text, screen-reader semantics, theme-reactive colors) but can't be right-click-copied or dropped into a slide deck. A new astro:build:done integration now also rasterizes every diagram to dist/diagrams/<slug>/<index>-<theme>[-transparent].png — light and dark, 2× DPI, in two variants per theme (a WYSIWYG surface card and a transparent-background version for slide decks) — reusing the Playwright Chromium rehype-mermaid already needs, with a content-hash cache (node_modules/.cache/wh-diagram-png) so unchanged diagrams skip Chromium and the whole pass is wrapped so it can never fail the build. The zoom lightbox (open by clicking any diagram) gains Copy (async-clipboard image, falling back to download where unsupported) and Download buttons that act on the PNG for the current theme, plus a background toggle (checkerboard icon) that flips Copy/Download between the solid and transparent variant and previews transparency as a checkerboard on the zoom stage; inline diagrams stay button-free. The upstream @wave-rf/astro-themed-mermaid plugin is intentionally untouched — it's color-agnostic (CSS-var placeholders resolved at runtime), so a WYSIWYG light/dark PNG must be rasterized post-build where global.css + a theme apply. PNGs are gitignored build artifacts, regenerated in CI/deploy.

  • Site header override — a top-level "Docs" nav link, the search box kept expanded, and a splash-width header (docs/src/components/Header.astro (new), docs/src/components/Logo.astro (new), docs/src/components/WaveMark.astro (deleted), docs/src/components/SiteTitle.astro, docs/src/components/Footer.astro, docs/src/assets/branding/lockup-themed.svg (new), docs/src/config/sidebar.ts, docs/src/env.d.ts, docs/src/styles/global.css, docs/scripts/branding/generate.sh, docs/astro.config.mjs): Starlight has no config hook for header links, so we override its Header component through the sanctioned components map. The custom header re-renders every built-in control (SiteTitle, Search, ThemeSelect, SocialIcons, LanguageSelect) through its virtual:starlight/components/* import — so their overrides and Pagefind search keep working untouched — and adds a <nav> with a single top-level Docs link (→ /getting-started), the way into the doc tree from the sidebar-less splash homepage. The link target lives in a new headerNav in sidebar.ts, colocated with the sidebar as one navigation source of truth, and the nav is hidden below the md (50rem) breakpoint where, on doc pages, the sidebar drawer already carries navigation (on the drawerless splash, the hero CTAs carry it) — the mobile bar keeps the full lockup + search button so it doesn't read as empty. Search keeps Starlight's expanded "Search … ⌘K" box on every page at ≥50rem (including the splash) so the keyboard shortcut stays discoverable, and the primary-nav "Docs" link is baseline-aligned to the WaveHouse wordmark. The logo is a reusable Logo.astro that inlines ONE themeable lockup SVG (lockup-themed.svg, derived from the brand kit by generate.sh): the mark takes currentColor/--wh-accent and the wordmark --wh-ink, so it themes light↔dark purely from CSS — no pre-baked image variants and no four-image download — rendering the full lockup at every width (the mark-only collapse below 30rem left the mobile header and footer looking bare; the lockup is ~8.3rem wide at header size, fitting even a 320px bar), with SiteTitle.astro reduced to a thin home-link wrapper and the footer brand block reusing the same <Logo/> (labelled via a title prop) so header and footer render one wordmark; the lockup now covers every logo use, so the standalone-mark WaveMark.astro component is deleted. The generated lockup-themed.svg ships without its former leading comment: XML forbids -- inside comments (as in --wh-ink), so the comment made browsers/GitHub reject the standalone file (PR #368 review) — the generated/do-not-hand-edit note lives in Logo.astro instead. Because the wordmark is now real inline SVG, Header.astro aligns "Docs" to it with align-items: baseline plus one documented geometry ratio instead of a hand-tuned pixel nudge, so alignment can't drift when fonts or sizes change. On the splash homepage the header's inner content is held in a centered 68rem column (deliberately narrower than the hero's 80rem band — the inset look settled in review), while doc pages stay full-bleed; since the header is preserved across view-transitions, that width now tweens between the two states (a global .wh-header rule in global.css, since the selector needs body:has(.wh-hero) — outside any component's scope — and must stay applied to the persisted header to tween) rather than snapping. env.d.ts gains type declarations for the built-in component modules the override re-renders so astro check passes.

Changed

  • Live SSE events are now projected and serialized once per role instead of once per subscriber (internal/stream/hub.go (new), internal/stream/{subscriber,bucket,heartbeat,metrics,doc}.go, internal/api/stream.go, internal/api/hub.go + internal/api/transform.go (both removed — the broadcast hub moves to internal/stream, and the orphaned test-only transformForClient is dropped), cmd/wavehouse/main.go, docs/src/content/docs/architecture.md, AGENTS.md, plus tests in internal/stream/{hub,filter,subscriber,bucket,heartbeat}_test.go and internal/api/{stream,transform,router,errors}_test.go): the first PR of the SSE delivery-path throughput epic (#294), building on the internal/stream primitives from #346. The broadcast hub moves into internal/stream as Hub: subscribers register under (topic, role), and Broadcast decodes each event once, applies each subscribed role's column policy once, builds one SSE frame per role, and fans it to every member of that role's Bucket. Previously every connection independently ran json.Unmarshal → policy.Evaluate → filterEventColumns → json.Marshal (plus a second unmarshal just to read the id: timestamp) on the same event in its own read loop — byte-identical work repeated N times. For a single-role audience (the public dashboard, every viewer public) that collapses N re-projections to 1, moving the measured ~2 270 deliveries/s ceiling toward an events/s ceiling. The (topic, role) key is sufficient and claims-independent: column visibility derives only from the role+table policy entry, and the stream path applies no row-level filter (a documented invariant — if row-level filtering is ever added to streaming, the key must take claims into account). The handler's two select cases (keepalive vs. per-subscriber event) collapse into one byte-pump over a single Subscriber.Frames() queue carrying typed Frames; the subscriber queue grows from cap 1 (keepalive-only) to 64 so live events buffer while the handler is mid-write. Gap-fill replay and Last-Event-ID/?since= resumption are unchanged (replay stays per-connection via the shared stream.ReplayFrame; live frames carry the same id: <received_timestamp>). Slow-consumer drops, silent before, now increment wavehouse_sse_dropped_frames_total; an inert Subscriber.Evicted() seam is wired for the eviction follow-up. The per-delivery OpenTelemetry span (another #294 item) was already removed in #346. Deferred to follow-ups: active slow-consumer eviction (#94) and right-sizing the subscriber buffer + broadcast lock cost (#152).

  • CI is now a job DAG instead of one monolithic job, and the docs deploys no longer expose the Cloudflare token to PR-authored code (.github/workflows/ci.yml, .github/workflows/housekeeping.yml, .github/actions/setup-env/action.yml, Makefile, docs/wrangler.jsonc, AGENTS.md, docs/src/content/docs/development.md, docs/src/content/docs/claude-code.md, CONTRIBUTING.md, scripts/lint-pr-title.sh, .claude/hooks/agent-bash-gate.sh): closes #305. The single make ci job becomes parallel jobs over the same Makefile targets (local make ci stays the dev mirror): lint, unit, integration, e2e (builds its own SDK dist + cover binary via make -j test-e2e on a warm per-suite cache and runs the suite exactly like a local run), coverage (a dedicated job that merges every suite's coverage-<suite> fragment and applies every threshold gate via make cov — like local make ci's final step, so the gate is decoupled from the e2e suite; it's needs: changes only and polls for the fragments rather than needs-ing the suites, so its setup overlaps them and the merge fires ~10s after the last suite instead of serializing ~50s of setup onto the critical path), and docs-build (make build-docs, docs-affecting changes only, uploads the docs dist artifact the preview/deploy jobs consume) — public-repo runners are free and 4-core, so the pipeline spreads horizontally instead of queueing in one process. The architecture is documented once, in .github/workflows/README.md (DAG diagram, design invariants, cache key policy, add-a-job recipe, and the measured-but-deferred optimizations — e2e sharding among them), and the workflow's logic lives in shellcheck-gated scripts (scripts/ci/ — classify-changes.sh, check-pr-title.sh, docs-preview-comment.sh, timing-summary.sh, wait-artifact.sh; over the shared, dependency-free path classifier scripts/classify-paths.sh, unit-tested by scripts/classify-paths.test.sh via make test-classify-paths and reused by the pre-push git hook so a docs/prose-only push requires only make verify, not a full make ci — the same suites CI skips for those changes) rather than inline YAML; caches are owned end-to-end by setup-env via nested actions/cache (automatic post-job saves — the per-job save-step boilerplate is gone); a non-gating Timing summary job writes a per-job wall-clock table to every run's Summary page; and make verify gains two leaves that gate the new surface area — lint-sh (shellcheck v0.11.0, checksum-verified install via scripts/install-shellcheck.sh) and lint-gha (actionlint v1.7.12) — so the CI plumbing is linted like any other source. The workflow also handles merge_group events (full suite against the merge-group ref), enabling a merge queue on main: the queue re-tests each PR against current main at landing time, which replaces the ruleset's "require branches to be up to date" rule — no more manual branch updates after every sibling merge. A new aggregator job named CI is the ruleset's sole required status check (it fails on any failed/cancelled job and counts skipped jobs as passing), so docs-only PRs skip the Go suites without orphaning the gate and future job changes never require ruleset edits. The PR-title (Conventional Commits) gate moves into the PR title job under that aggregator, validated by the same scripts/lint-pr-title.sh from a trusted main checkout; PR housekeeping (pull_request_target) drops to non-required and keeps what needs fork-PR write access — path labels, the sticky title-explainer comment, and a new nudge that re-runs the failed PR title job when a title edit fixes it (the job re-reads the title from the API, so no new push is needed). The #305 fix: docs previews/production deploys run in dedicated docs-preview/docs-deploy jobs that check out trusted main (wrangler, worker source, and config never resolve from the PR tree), consume only the static docs/dist artifact, and are the only jobs that reference CLOUDFLARE_* secrets; previews now publish right after docs-build instead of waiting on the full test pipeline, and the docs-preview deploy is non-gating — it's not in the CI aggregator's needs (only docs-build gates), so a slow or failed Cloudflare preview reports its own "Docs preview" check but never delays or reds the required check; production (docs-deploy, on the post-merge main push) still requires everything green. Per-job least-privilege permissions replace the old workflow-wide contents: write, and the Go build cache is partitioned per job (unit/integration/e2e compile with different flags) so each suite stays warm.

  • Brand kit gains branding/avatar.png — a 512×512 white tile for square/rounded avatar slots (docs/scripts/branding/generate.sh, docs/public/branding/avatar.png): apple-touch-icon.png is only 180×180 and goes soft when a consumer blows it up to a GitHub org/team avatar (or a Slack/social profile picture). generate.sh gains a section 5b that emits a 512px tile — the same solid-bg + centered-mark composition as the touch icon, but on a white background with the brand-blue mark (so it reads as a profile picture rather than a black square in light UIs), rendered from the source mark.svg (vector → crisp at any size) with the mark centered and padded so it clears GitHub's rounded-square crop. Both tiles share one solid_tile_svg helper so the composition is defined once. No code or runtime surface changes; regenerate with make branding-docs.

  • Dependabot npm updates moved to the pnpm workspace root, fixing recurring ERR_PNPM_OUTDATED_LOCKFILE CI failures on dependency PRs (.github/dependabot.yml, docs/src/content/docs/development.md, SECURITY.md): the three per-member npm update configs (directory: /docs, /clients/ts, /tests/e2e/sdk) are replaced by a single config at the workspace root (directory: /, group npm-deps, prefix deps). The repo has one root pnpm-lock.yaml (the #190 consolidation), and Dependabot only updates a lockfile co-located with the manifest it targets — so a per-member config bumped a member's package.json without regenerating the root lockfile, and every such PR (e.g. #211, #337) then failed CI's pnpm install --frozen-lockfile with ERR_PNPM_OUTDATED_LOCKFILE; no rebase could fix it, because a rebase never regenerates the lockfile. Pointing at the root lets Dependabot read pnpm-workspace.yaml, walk every member, and update the one lockfile within the PR, and as a bonus brings the root package.json's own devDeps (biome, markdownlint, nyc) under Dependabot. Trade-off: one combined weekly npm PR with a single deps: prefix instead of three per-area PRs (docs: / deps(sdk): / deps(tests):).

  • Go toolchain requirement bumped to 1.26.5 (go.mod): the go directive moves from 1.26.4 to 1.26.5 so local builds (via GOTOOLCHAIN=auto) and CI's setup-go (which reads go.mod via go-version-file) install a Go whose standard library clears the govulncheck findings GO-2026-5856 (crypto/tls) and GO-2026-4970 (os), both fixed in 1.26.5 — those findings were failing make verify's vulncheck leaf (and with it the pre-commit hook) on every tree. Patch-level toolchain bump only — no source changes — and the released binaries pick up the patched stdlib too.

Security

  • Policy _in is now enforced on both the row-filter and insert-check paths, closing a fail-open row-security gap (internal/policy/policy.go, internal/api/ingest.go, docs/src/content/docs/access-control.mdx, plus tests in internal/policy/policy_test.go, internal/api/ingest_test.go): closes #224. The Filter schema accepted _in but the engine never read it: on the row-filter/SELECT path resolveFilters had no _in branch, so a row-security filter like tenant_id: { _in: … } produced no WHERE predicate and the role saw every row instead of its tenant subset (a fail-open, same family as #223); on the check/INSERT path only _eq was honored, silently dropping any other operator. _in now takes a single claim that resolves to a JSON array (the multi-tenant case — a token's tenant_ids list) and emits col IN (?, …) with one bound param per element; a scalar claim is a one-element set, and an empty/absent claim matches no rows (fail-closed) rather than widening to all of them. On the insert path an _in check requires the column be present and one of the set — there is no single value to auto-inject as _eq does, so an omitted column is rejected (403 check failed). The comparison operators are enforced on filter (_eq/_neq/_gt/_lt/_in all produce predicates now, so nothing is rejected there) and, on check, _neq/_gt/_lt become a loud config-load rejection (no insert-time semantics; check honors _eq + _in). The _in value stays a single templated string in the wire schema (Go Filter.In, SDK PolicyFilter._in), matching the established "set = array" shape of the caller-query in operator.

  • denied_aggregations is now enforced case-insensitively against the caller-supplied function name, closing a policy bypass (internal/policy/policy.go, plus tests in internal/policy/policy_test.go): closes #318. IsAggregationAllowed lower-cased the aggregation name only after the deny-list loop and the empty-allow-list early return, so the deny check compared a lower-cased deny entry against the raw caller input — a denied aggregation slipped past simply by changing case (SUM bypassed a sum deny entry, and with an empty allow list the call was then permitted). isValidAggFn already accepts any casing and the SQL builder emits the function name verbatim, so the denied aggregation actually executed. The case fold now happens once, before the deny check, so deny wins regardless of caller casing — matching the case-insensitive contract the access-control docs already specified.

  • A role's structured-query resource caps are now enforced server-side by ClickHouse, so a public read can't outrun its budget during a server-side scan/merge/aggregation phase (internal/api/ch_settings.go (new), internal/api/structured_query.go, internal/policy/policy.go, internal/policy/scalars.go (new), internal/config/config.go, internal/query/builder.go, cmd/wavehouse/main.go, config.yaml, clients/ts/src/types.ts, docs/src/content/docs/{access-control.mdx,configuration.mdx}, plus tests in internal/api/ch_settings_test.go, internal/policy/{policy,scalars}_test.go, internal/config/config_test.go, internal/query/builder_test.go, tests/integration/query_limits_test.go (all new/expanded), tests/e2e/sdk/query.test.ts): closes #316. The native ClickHouse connection passed no per-query Settings, so a read's policy caps bound it only client-side — a Go context deadline (which cancels only while the client is reading result blocks) plus a SQL LIMIT. Memory and rows scanned were never bounded server-side, so a heavy aggregation could allocate gigabytes of state or scan an entire table well within the time budget; and because clickhouse-go derives a max_execution_time setting from the context deadline only for deadlines > 1s, a sub-second time cap reached ClickHouse with no server-side time bound at all. The structured-query path now attaches per-query Settings derived from the role's resolved permissions: max_execution_time (fractional seconds, emitted explicitly so the sub-second case is enforced), max_result_rows + result_overflow_mode=throw (defense-in-depth behind the SQL LIMIT), max_rows_to_read + read_overflow_mode=throw, and max_memory_usage — so a query that exceeds its budget is rejected by the server (ClickHouse codes 158 / 241) rather than running to completion. Boundary: WaveHouse owns the dynamic, per-role caps (sent as per-query settings); the global, static backstop — which applies to every query including named pipes and raw admin SQL — is configured in ClickHouse's own settings profiles and quotas (documented in configuration.mdx), composes with the per-role caps, and holds even against a WaveHouse bug. Schema (pre-launch): the per-role policy fields are human-readable in / numeric out — max_execution_time_ms (int) → max_execution_time (set as a duration string "5s" or a bare ms number; read back as ms), the new max_memory_usage (set as a size string "4GiB" — IEC/SI respected via dustin/go-humanize, so 4GB ≠ 4GiB — or a bare byte number; read back as bytes), and the new max_rows_to_read (int), backed by two small Millis/ByteSize types in the policy package. The formerly hard-coded query.DefaultMaxRows = 10000 result-LIMIT becomes the documented, tunable query.default_max_rows config knob (Build takes it as a parameter). Raw admin SQL remains unbounded by WaveHouse (governed by ClickHouse). Verified RED before the fix: the handler-level integration test confirmed a capped read returned the full result set when the settings weren't sent.

  • The structured-query column allowlist is now a hard cap on every column a query references — closing a fail-open data-exposure family on the primary read path (internal/query/builder.go, internal/query/errors.go (new), internal/policy/policy.go, internal/discovery/discovery.go, internal/api/structured_query.go, docs/src/content/docs/access-control.md, docs/src/content/docs/api.md, plus tests in internal/query/builder_test.go, internal/api/structured_query_test.go, internal/policy/policy_test.go, internal/discovery/discovery_test.go, tests/e2e/sdk/query.test.ts): closes #223. POST /v1/query previously authorized only the columns a caller explicitly listed, so several inputs bypassed a role's allow_columns/deny_columns entirely:

    • Omitting columns (or sending [] / ["*"]) made the builder emit SELECT *, returning every column — the reported leak (an unauthenticated {"limit":2} returned columns hidden from the public role, including raw webhook payload).
    • group_by on a denied column (SELECT count(*) … GROUP BY salary) enumerated that column's distinct values wholesale; a filter on a denied column turned the returned row count into a value-inference oracle; an order_by on a denied column leaked ordering. None of these clauses were policy-checked — only the SELECT list and aggregation arguments were.

    Column authorization now lives in the one place that already enumerates every column reference — the SQL builder (query.Build, made policy-aware) — so the projection, aggregation arguments, filters, group_by, order_by, and time_range are all checked against the role's allowlist in a single pass, and no current or future clause can silently skip the check. A full-row read is now requested explicitly with "select_all": true, which for a column-restricted role expands to exactly that role's allowed columns, so denied columns never reach ClickHouse; unrestricted roles and admin keep SELECT *; a role allowed the table but no columns fails closed with 403. Omitting columns now returns nothing (an empty result) rather than expanding to a projection — so a hidden column can't leak by being left out at all. Because the generated SQL is now role-specific, the read cache key partitions by column visibility for free — one role can no longer be served another's cached columns. The structured-query and live-stream paths now share the same per-column decision (policy.IsColumnAllowed, plus new AllowedProjection/RestrictsColumns helpers), so the two read surfaces enforce identical column visibility and can't drift apart. New unit, handler, and end-to-end (SDK → WaveHouse → ClickHouse) tests cover each bypass shape as explicit red-team cases. One intended behavior change falls out of authorizing order_by/group_by/filter: a column-restricted role can no longer order, group, or filter by a column it cannot read — those clauses must reference only readable columns (unrestricted roles and admin are unaffected). The SDK's bare .fetch() maps to select_all, so it still returns a full row set (the same-release #270 fix already dropped its implicit ORDER BY received_timestamp default).

  • Structured queries accept any identifier ClickHouse accepts (bring-your-own schemas), with every identifier backtick-quoted to stay injection-safe (internal/chsql/ (new), internal/query/builder.go, internal/query/ast.go, internal/policy/policy.go, internal/api/structured_query.go, clients/ts/src/query-builder.ts, clients/ts/src/types.ts, docs/src/content/docs/access-control.md, docs/src/content/docs/api.md, plus tests in internal/query/{builder,ast}_test.go, internal/api/structured_query_test.go, internal/policy/policy_test.go, internal/chsql/chsql_test.go, tests/integration/identifier_roundtrip_test.go (new), tests/e2e/sdk/{query,cache,ingest}.test.ts): table names, column names, and aggregation aliases may now contain dots, spaces, unicode, SQL keywords, even embedded backticks/backslashes — anything ClickHouse permits in a quoted identifier. The builder and the policy row-filter path route every identifier through one shared chsql.QuoteIdent that escapes exactly as ClickHouse's own backQuote() does (\ → \\, ` → \`, verified against SHOW CREATE TABLE on a live server). This replaces the prior safe-identifier regex, which rejected legitimate columns in existing customer schemas. It also subsumes the earlier aggregation-alias injection hardening: an alias such as n FROM secrets -- is now neutralized by quoting (it becomes an inert column label) rather than rejected, so callers keep arbitrary alias names while the reparent/statement-break/subquery payloads still cannot break out. Column and table references stay bounded by schema membership (the injection boundary for those) plus the per-column allowlist (#223); aliases carry no membership but are fully contained by quoting. The lone refusal is an identifier literally containing ? — clickhouse-go's positional value-binder counts every ? in the query text, even inside a quoted identifier, so such a name is rejected fail-closed rather than risk shifting value parameters (tracked in #279). Two related API changes fall out of treating names literally: in a query, * is now a literal column name (quoted, gated by schema membership), not a wildcard — the all-columns read moved to an explicit select_all boolean (columns and select_all are mutually exclusive; omitting both selects nothing), and the SDK gains .selectAll() with a bare .fetch() mapping to it. count(*) still means count-all, and allow_columns: ["*"] is still the policy wildcard. The columns field also now accepts a single string as shorthand for a one-element list. Values are unchanged (positional ?, driver-escaped). New real-ClickHouse round-trip tests cover weird column/table/alias names (including embedded backticks and backslashes), injection containment, and the */select_all contract; server-side {name:Identifier} params were evaluated and rejected because clickhouse-go cannot mix them with positional value binding.

  • Pipe parameters can no longer inject SQL through a non-scalar value (internal/pipes/pipes.go, plus tests in internal/pipes/pipes_test.go, internal/api/pipes_test.go, tests/e2e/sdk/admin.test.ts, and docs docs/src/content/docs/{pipes.mdx,api.md,architecture.md}): closes #317. Pipes bind parameters by inlining escaped literals into the SQL template (no positional ? binds), and formatParamValue's fallback branch emitted whatever Go's fmt.Sprintf("%v") produced — so a JSON array or object supplied as a parameter value reached ClickHouse raw and unescaped. In a string-context placeholder an array element's inner ' terminated the literal, giving a clean UNION/boolean injection that bypassed the per-column allowlist the structured-query path enforces. The execute route (GET/POST /v1/pipes/{name}) is authorized only by the pipe's allowed_roles (commonly [public]), so this was reachable unauthenticated. formatParamValue is now recursive and fails closed: every scalar leaf — including each element of an array — is escaped, a JSON array renders as a parenthesized (v1, v2, …) list (the same IN (…) shape the structured-query builder emits, so list parameters such as WHERE id IN {{ids}} now work safely), and a value with no scalar SQL form — a JSON object, or an empty array that would render as the invalid IN () — is rejected with a 400 rather than emitted raw. A numeric-looking query-string value still renders bare (so ?limit=100 works as a number) via a strict numeric-literal check; any other string is single-quote-escaped, so it can never become invalid or injectable bare SQL. The declared parameter type stays advisory metadata (documenting intent for callers and the SDK), with binding keyed off the runtime value as before. New unit tests cover the array/object/empty/nested-list cases plus the original injection payloads as red-team regressions, new handler tests cover the array-binds and object-rejected paths, and an end-to-end SDK test executes an IN-list pipe against real ClickHouse with a quote-bearing element to exercise escaping.

  • The public read endpoints now bound the request body they decode, closing a single-request OOM vector (internal/api/query.go, internal/api/structured_query.go, internal/api/pipes.go, internal/api/policy.go, docs/src/content/docs/api.md, plus tests in internal/api/structured_query_test.go, internal/api/pipes_test.go, and internal/api/policy_test.go): closes #315. POST /v1/query and GET/POST /v1/pipes/{name} called json.NewDecoder(r.Body).Decode(...) with no http.MaxBytesReader, so a large JSON array — e.g. a giant in-list filters[].value — decoded whole. JSON amplifies roughly an order of magnitude (~13×) bytes→live-heap into Go's in-memory representation (interface boxing, slice-growth doubling, GC headroom), so a single ~50 MiB request inflated to hundreds of MiB and a few concurrent ones OOM-killed the box — a memory, not throughput, vector that a per-IP rate cap doesn't mitigate. Both decoders now wrap the body in a new 1 MiB maxControlBodyBytes cap (a query/pipe-parameter body is bounded by nature, far under 1 MiB even with a large in-list) and return 413 request body exceeded 1048576 bytes via the same writeMaxBytesError mapping the ingest/admin handlers already use — bringing the public read path to body-cap parity with the previously-hardened /v1/ingest and /v1/admin/query (16 MiB, the data-plane cap). The same cap is extended to the remaining body-decoding handlers in the package so none is left as an OOM gap — the admin pipe-definition PUT /v1/admin/pipes/{name} and the admin policy decoders PUT /v1/admin/policy + POST /v1/admin/policy/validate (a pipe or policy document is a bounded description). The pipe Execute parameter path keeps its lenient fallback for a malformed-but-within-cap body (parameters may come from the query string alone) and only hard-stops with 413 on an over-cap body. The cap is a fixed in-code backstop, not a tuning knob — operators set their own outer limit at the reverse proxy; see the new deployment guide in this release.

  • Dropped chi's deprecated middleware.RealIP from the router (internal/api/router.go, docs/src/content/docs/reverse-proxy.mdx): closes #281. It rewrote r.RemoteAddr from spoofable forwarded headers (X-Forwarded-For / X-Real-IP / True-Client-IP) on every request, whether or not a trusted proxy set them — the IP-spoofing GHSAs chi deprecated it for (SA1019; GHSA-3fxj-6jh8-hvhx / GHSA-rjr7-jggh-pgcp / GHSA-9g5q-2w5x-hmxf), which also blocked the weekly go-deps dependabot group on the chi bump (#209). Nothing in WaveHouse read r.RemoteAddr (no per-IP logic — rate limiting is the reverse proxy's job), so removing it is behavior-preserving and drops the spoofing vector; r.RemoteAddr is now the honest immediate peer (the proxy, when one fronts WaveHouse). Capturing the real client IP safely — trusted-proxy-aware and surfaced in traces and logs, plus gating trace-context propagation on the same trust boundary — is tracked as a follow-up (#333).

Fixed

  • Live demo hero feed renders in event_ts order instead of SSE arrival order (docs/src/components/LiveDemo.astro): the landing-page live activity feed prepended each streamed row to the top in the order the SSE stream delivered it, but a producer's webhook burst (a single merge-queue cycle fires ~20 events) arrives in no guaranteed order and the stream relays it in ingest order — so a late or out-of-order delivery landed above newer rows (e.g. a pushed 12m ago sitting on top of reviewed a pull request 9m ago). addRow now keeps the feed sorted by event_ts descending — it slots each row in before the first strictly-older sibling rather than blind-prepending — so the live tail matches the already-sorted gh_activity_recent backfill. The zone-less-SSE-timestamp normalization the sort relies on (normTs) was already in place; equal-second rows keep arrival order (gh_events.event_ts is only second-granular for CI/checks, so there's no finer tiebreak), and dedup + the MAX_ROWS trim are unchanged. Surfaced in dogfooding on wavehouse.dev; the client-side analog of the ingest-order reality the SSE stream can't reorder.

  • SSE streams emit a periodic keepalive comment so quiet connections survive proxy and tunnel idle timeouts (internal/stream/ (new package: subscriber.go, bucket.go, heartbeat.go + tests), internal/api/{stream,stream_test}.go, internal/config/{config,config_test}.go, cmd/wavehouse/main.go, config.yaml, docs/src/content/docs/{reverse-proxy.mdx,api.md,configuration.mdx,architecture.md,deployment.md}): closes #226. The stream handler (internal/stream/metrics.go, internal/api/{hub,hub_test}.go also touched) wrote a single : connected comment on open and then sent nothing until an event arrived, so on a quiet table an intermediary's idle timeout reset the connection — Cloudflare's edge (and a Cloudflare Tunnel) dropped quiet streams about every two minutes in dogfooding, and every curl/server-side reconnect re-ran NATS gap-fill (browser EventSource masked it by auto-reconnecting). A single shared Heartbeater goroutine now drives keepalives for every live connection: connections are spread across a ring of buckets and one bucket is pushed a minimal : SSE keepalive comment per tick — the writes don't all fire at the same instant, and the per-connection period comes from one timer instead of a time.Ticker per connection. The user-facing knob is stream.keepalive_interval (WH_STREAM_KEEPALIVE_INTERVAL, default 30s) — the longest a quiet stream goes without a write — chosen to clear the common 55–60s idle windows (nginx/ingress-nginx proxy_read_timeout, AWS ALB, Heroku) with ~2× margin while still clearing Cloudflare's ~120s edge; stream.keepalive_buckets (WH_STREAM_KEEPALIVE_BUCKETS, default 3) is an advanced load-spreading knob and the wheel ticks every keepalive_interval ÷ keepalive_buckets, so one rotation always spans exactly the interval regardless of bucket count. The wheel lives in a new internal/stream package (groundwork for #294, which will move the broadcast hub in alongside it) and exposes a Bucket interface (push a byte slice to a set of connections) so the #294 delivery-path throughput work can reuse it to serialize once per (role, table) rather than per subscriber. The keepalive is a standard SSE comment (ignored by EventSource and spec-compliant parsers; curl just prints it), and a failed keepalive write doubles as a liveness check that ends the handler once a connection has gone away. The reverse-proxy guide gains a per-provider/per-software idle-timeout reference table (with source links and how-to-change notes, plus the absolute-cap exceptions a keepalive can't fix, e.g. Envoy's 15s route timeout), and -race tests cover the concurrent connect/disconnect teardown path (the wheel pushing to a connection that is mid-teardown); raising the proxy idle timeout for /v1/stream is now optional rather than required. Streams are observed through metrics rather than per-event traces: the handler records wavehouse_sse_active_streams, wavehouse_sse_stream_duration_seconds, and wavehouse_sse_frames_sent_total / wavehouse_sse_bytes_sent_total (labeled by kind: keepalive, event, replay), and the per-event SSE.PushEvent span was dropped — the router already excludes /v1/stream from HTTP tracing, and a span per delivered event per subscriber is high-volume, low-value, and existed only to read the hub's trace_headers envelope, now collapsed so the hub broadcasts the raw event bytes instead of base64-wrapping them.

  • Docs mobile rendering overhaul — tables, diagrams, chrome, and wrapped headings no longer break at phone widths (docs/src/styles/global.css, docs/src/components/{ScrollHints (new),MermaidZoom,Footer,ReadingProgress}.astro): a 15-agent visual audit of every page × 320–1440px × both themes (driven by the PR author's report that "many pages look broken on mobile") found four systemic defects, all shipped-broken before this branch and some made worse by it. (1) Tables crushed instead of scrolling: the fixed-layout + overflow-wrap: anywhere treatment kept dense tables inside the column by shredding content — at 390px reference tables wrapped identifiers a few characters at a time (WH_P/ROME/THEU/S_PO/RT), and even at 1440px fixed equal columns split tokens mid-word. Tables now scroll instead of crushing: a JS wrapper (ScrollHints.astro) gives each table a framed scroll container with true auto layout (prose wraps first, scroll only when unbreakable columns can't fit), with a GitHub-style display:block scroll fallback when JS is off. (2) Mermaid diagrams scaled to thumbnails on phones: standalone diagrams shrank to fit the column (≈3–7px labels — unreadable); they're now wrapped in the same scroller and keep a readable size with horizontal pan (no-JS keeps scale-to-fit), the zoom lightbox uses safe centering so a natural-size diagram's left/top edge is reachable on phones (plain centering made overflow start-side unscrollable), and coarse-pointer devices get a visible ⤢ zoom affordance (cursor: zoom-in means nothing on touch). (3) Translucent fixed chrome double-exposed on mobile: the glass header and Starlight's sticky "On this page" bar ghosted scrolled content through themselves; both are opaque below 50rem (glass stays on desktop), and the hand-written -webkit-backdrop-filter twin was removed because it made LightningCSS treat prefixing as author-managed and drop the standard backdrop-filter from the build entirely (Firefox had no blur). (4) Wrapped headings drew their section rule through the text: Starlight renders content headings display: inline inside .sl-heading-wrapper (for the anchor link), so backgrounds/borders fragment per line — the h2 rule struck through every wrapped line on mobile (the pre-branch border-bottom had the same bug) and heading block margins silently no-oped; rhythm + rule now live on the wrapper. Plus: every horizontal scroller (code blocks, tables, diagrams, the zoom lightbox's pannable stage) gets a right-edge fade scrim while more content is hidden (cut-off code read as truncated, not scrollable), code ligatures are disabled site-wide (!= rendered as ≠ in tables documenting exact operator syntax), the footer keeps three columns at tablet widths, and the reading-progress bar regains visibility (0.9 opacity). A second 15-agent verification round over a fresh corpus then confirmed the fixes and caught the stragglers: scrolling tables gained per-column minimum widths (without floors, auto layout squeezed off-screen prose columns to ~100px whose 10–20 wrapped lines left near-blank viewport-tall rows — later scoped to tables measured as genuinely overflowing, see the Layout bullet below), table code chips no longer break at hyphens (--url read as two flags; later refined to wrap between tokens, same bullet), diagrams pin to their Mermaid-stamped natural width inside the pan scroller (a 540px floor still let the widest shrink ~33% to ~5px labels; later scoped to phone widths), aside titles flow as block text so a title's code chip can't detach from its sentence (Starlight's flex title made the chip its own flex item), and the ingest-pipeline state-machine's edge labels were shortened to stop dagre overlapping them at every width.

  • sdk.md's Live Queries example no longer chains the query builder off from() directly (docs/src/content/docs/sdk.md, docs/src/content/docs/index.mdx): wh.from(table) returns a TableRef, which has no .where()/.orderBy()/.limit()/.liveQuery() — those live on QueryBuilder, entered only via .select(...)/.selectAll(). The documented example failed to type-check and, copied as-is, threw TypeError: wh.from(...).where is not a function; it now starts the chain with .selectAll(), and the landing-page SDK showcase (added on this branch, which had faithfully inherited the broken pattern from this example) gets the same fix. Surfaced by the branch's pre-push review; no e2e test exercises that chain shape, which is why it had gone uncaught.

  • Docs search and the mobile menu survive in-page navigation again — restored via the supported astro-vtbot Starlight integration, which keeps the SPA view transitions rather than dropping them (docs/package.json, pnpm-lock.yaml, docs/src/components/Head.astro): the design-token rebuild (#142) added a bare <ClientRouter/> (Astro View Transitions) to Head.astro, but Starlight does not support a bare router — its lead maintainer warns it breaks search and the mobile menu (withastro/starlight#2823). The SPA router swapped the whole header on every navigation, so (a) the previous page's window ⌘K handler pointed at a now-detached <dialog> and the next ⌘K threw InvalidStateError: Failed to execute 'showModal' … The element is not in a Document, and (b) Pagefind's one-shot DOMContentLoaded init never re-ran, so the search box reopened empty; the mobile menu broke by the same header-swap. Rather than give up the SPA model (instant navigation, preserved sidebar scroll) for browser-native cross-document transitions, Head.astro now wraps Starlight's Head in astro-vtbot's components/starlight/Base.astro — the integration #2823 names as the supported path. Base supplies the <ClientRouter fallback="swap"/> plus a <ReplacementSwap/> that swaps only the main content frame and preserves the rest of the header, so the search <dialog> and its ⌘K listener are never detached, and a <StarlightConnector/> that re-runs Starlight's own per-navigation init — it closes the mobile menu, updates the current-page marker, and keeps the existing sidebar DOM when its links are unchanged, preserving scroll position. The docs components keep the astro:page-load/astro:after-swap re-init they have on main (navigations are still Astro View Transitions), so nothing else changes. Verified on a production build (astro build + the Cloudflare Worker) with Playwright: SPA navigation preserves the JS context (no full reload); ⌘K search opens and returns results after navigation with no showModal error and exactly one <dialog> in the document; and the mobile menu opens, auto-closes on navigation, then re-opens afterward (the original breakage).

  • Mermaid node labels no longer clip their last glyph, and the comparison diagram pair stacks vertically instead of shrinking side-by-side (docs/src/config/mermaid-theme.mjs, docs/package.json, pnpm-lock.yaml, docs/src/styles/global.css, upstream astro-themed-mermaid → v0.3.0): the build-time Chromium measured each node's box at the default font-weight 400 while global.css displays labels at weight 500, so every box came out ~1px too narrow and the longest label clipped on the right (Buffer Consumer → Buffer Consume — present across all 116 nodes on the live preview). Fixed upstream via a new measurementCss option on astro-themed-mermaid that injects label metrics into the render page so Mermaid measures what the browser shows; mermaid-theme.mjs passes the node-label weight + letter-spacing (plus a 1.5px safety pad) and resets cluster titles to their default measurement (they render as overflow:visible pills sized separately by the plugin, so a weight bump there would only mis-center them). Selectors are bare (.nodeLabel p, not svg[aria-roledescription…] .nodeLabel p) because Mermaid measures the label before the flowchart <svg> wrapper exists. Separately, the .diagram-pair wrapper (the DIY-vs-WaveHouse comparison on why-wavehouse) dropped its ≥1500px flex-row variant: two detailed diagrams side-by-side each shrank to ~450px on a wide monitor, so they now always stack vertically at the full content width — matching the new AGENTS.md "Authoring Mermaid diagrams" house style (favour vertical/TB over LR).

  • Embedded NATS no longer installs its own process-wide SIGINT handler (internal/mq/embedded.go, internal/testutil/natsjs.go): closes #287. nats-server's Start() registers a signal handler unless NoSigs is set, and its SIGINT path is Shutdown(); WaitForShutdown(); os.Exit(0) — racing WaveHouse's own graceful shutdown on every Ctrl-C/SIGTERM-adjacent stop. Depending on who won: the NATS handler's os.Exit(0) skipped all of run()'s deferred cleanup (dedup/cache/ClickHouse close) while still exiting 0, or the two concurrent Server.Shutdown() calls panicked (close of nil channel in shutdownEventing, nats-server v2.14.0 events.go:1907) with exit status 2 — and a panic exit also skips Go coverage flushing, which surfaced as the e2e suite suddenly reporting 0.0% coverage in make ci (the orchestrator SIGINTs the instrumented binary precisely to flush counters). Now NoSigs: true: WaveHouse's shutdown path is the single lifecycle owner and already stops the embedded server via EmbeddedNATS.Close(); the twin options literal in testutil.NewJetStream (pipes/policy KV tests) gets the same flag so a Ctrl-C during a local test run can't os.Exit(0) the test binary mid-suite. Removing the early-exit also exposed how long a clean graceful exit actually takes when no OTel collector is listening — ~15s of serialized exporter dial-backoff (filed as #288) — so the e2e orchestrator's SIGINT-to-kill budget rises 10s → 30s (scripts/orchestrator/main.go; fast exits are unaffected, the wait selects on process exit). Both shutdown paths now also WaitForShutdown() after Shutdown() — owning the lifecycle means waiting it out, so the process can't exit (or t.TempDir() cleanup race) while JetStream is still tearing down. Superseded in part — the 10s → 30s orchestrator-budget bump was reverted back to 10s once OTel provider shutdown became concurrent and context-bounded (internal/observability/provider.go), which caps a collectorless clean exit at ~3s instead of ~15s; the NoSigs lifecycle fix above is unaffected.

  • Docs corrections surfaced by this branch's review pass (docs/src/content/docs/{sdk,getting-started,development,api,configuration,deployment}.md, docs/src/content/docs/index.mdx): All ten POST /v1/ingest?table= curl examples quote their URLs — unquoted ? aborts on zsh (macOS default) with no matches found before curl runs, and the very first runnable command on the landing page was one of them. The Getting Started quickstart warns that the first ingest after creating a table can 404 unknown table for up to the 60-second schema-refresh interval (and that POST /v1/schema/refresh is admin-only — also tagged in configuration.md and deployment.md, since the quickstart's trial role can't call it). The e2e test-file lists in sdk.md and development.md gain the ndjson suite (#217). sdk.md stops presenting wh.dlq.stream() as working — there is no server-side DLQ stream yet (the SSE bridge only carries ingest.>), so the documented call connected and received nothing; it's now caveated against #197 in all three places it appears. The admin-only sweep reaches the rest of the reference: api.md's schema/DLQ sections get local admin callouts + 401/403 (and 500-on-refresh) error rows matching the gates in internal/api/router.go, wh.dlq joins wh.pipes/wh.policy/wh.schema in stating the role requirement, and the SDK README's codegen section warns that /v1/schema is admin-only (--auth <jwt>); the /v1/admin/query curl example gains the Authorization header it always needed (it was the only admin example that 403'd as written), and the JWT-testing section explains where change-me-in-production comes from and that the compose stack must set WH_AUTH_JWT_SECRET. (A pre-existing secretless-deployment auth-forgery vulnerability surfaced during review — empty-string-HMAC admin tokens validate against a no-secret instance — is tracked in #291 for a security-reviewed code fix; this PR removes the false "secretless ⇒ no token validates / pure public deployment" framing that masks it from every doc it ships — api.md, access-control.md, and configuration.md — leaving the legitimate tokenless-quickstart wording in getting-started.md intact.)

  • time_range.since/until accept day/week suffixes (7d, 2w) and reject unparseable values with a 400 instead of an opaque ClickHouse error (internal/query/builder.go, internal/query/builder_test.go, docs/src/content/docs/api.md, docs/src/content/docs/sdk.md): closes #285. Go's time.ParseDuration stops at hours, so a documented "7d" (sdk.md) failed to parse and fell through to ClickHouse as a raw DateTime literal → opaque Cannot read DateTime error (surfaced as a retryable 500, per #271). resolveTimeValue now pre-expands Nd→N*24h / Nw→N*168h (Prometheus/ClickHouse-style) before parsing, and any value that is neither a relative duration nor an RFC3339 timestamp now fails closed — the builder maps it to a 400 with a clear message rather than handing the string to ClickHouse.

  • development.md & CONTRIBUTING.md no longer describe the pre-consolidation CI / review automation (docs/src/content/docs/development.md §CI & review automation, CONTRIBUTING.md, Makefile comments): the contributor docs still referenced pr-title.yml/Validate, Check/Build status checks, project-orchestrator.yml, and label.yml — all replaced or deleted by the May 2026 workflow consolidation (housekeeping.yml) and #115 — and described card movement and reviewer re-requests the orchestrator no longer performs. Rewritten against the live config: required checks are CI (one job running make ci), PR housekeeping (Conventional Commits title + the previously undocumented 72-char cap, file-path labels via the labeler step, reviewer assignment on open/ready), and Admin approval; board placement is native Projects v2 workflows; the stale "Lint/Test not required (#57)" note is gone (those suites run inside the required CI check). Also corrects a misattribution this branch had propagated: starlight-links-validator does not use Chromium — only rehype-mermaid does (fixed in the prose and two Makefile comments), and make clean's help text/targets table now mention docs/.dev-dist/. Same sweep, code-comment side: admin-approval.yml's header claimed the ruleset's required_approving_review_count was 0 — the live ruleset says 1 (any collaborator), with the workflow's status check being what makes one of those approvals specifically an admin's — and AGENTS.md's make clean one-liner now lists the docs build outputs.

  • Theme-adaptive favicon that actually displays (docs/astro.config.mjs, docs/src/components/Head.astro, docs/scripts/branding/generate.sh, docs/public/branding/favicon-{light,dark}.svg (new)): the CSS-swap favicon.svg has carried a prefers-color-scheme flip since the branding pipeline landed, but no browser ever showed it — the docs final pass (#193) dropped the explicit sizes="any" SVG icon link, so Chromium selected the static .ico, and Chromium rasterizes an SVG favicon only once regardless (crbug.com/1208277). The head now declares the numeric-sized .ico + sizes="any" SVG pair, the brand kit gains statically-colored favicon-{light,dark}.svg variants (two new copy-out assets), and a Head.astro inline script live-swaps them on prefers-color-scheme changes, GitHub-style: replacing the link node each repaint (in-place href mutation is honored only transiently) and removing the .ico from the live candidate list, since Chromium's favicon scorer otherwise commits an exact-size .ico over a sizes="any" SVG (verified against the browser's profile Favicons DB rather than blog lore). No-JS consumers keep the full SSR markup (CSS-swap favicon.svg + .ico); Safari ≤18, which can't use SVG favicons, falls back to the root /favicon.ico auto-probe that the brand pipeline keeps in place. Supersedes the favicon wiring described in the branding-pipeline entry below (favicon: option + .ico/apple-touch head tags only).

  • TS SDK no longer hardcodes ORDER BY received_timestamp DESC as the default query order, so wh.from(table).fetch() works on any schema (clients/ts/src/query-builder.ts, clients/ts/src/query-builder.test.ts, tests/e2e/sdk/query.test.ts, docs/src/content/docs/sdk.md): closes #270. The query builder previously injected ORDER BY received_timestamp DESC as both the default query order (_buildAST) and the cursor-pagination column (_fetchNext), so a bare .fetch() against a bring-your-own-schema table lacking that column emitted invalid SQL → ClickHouse Unknown expression identifier received_timestamp → HTTP 500. The SDK now emits an ORDER BY only when the caller sets one via .orderBy(), and sends no implicit order otherwise. Soft behavior change: .fetch() still reports hasMore honestly from the row count, but cursor pagination's result.next is now undefined (rather than always defined when hasMore is true) unless the query has an explicit .orderBy(); callers paginating via await result.next!() must guard on result.next (the sdk.md / README examples already do). This also sidesteps the #175 default-cursor tie bug — there is no longer a default cursor to tie on. A backend-owned, per-table default sort order (designed together with server-side pagination, which would subsume #175) is tracked as a follow-up (#274). Stream dedup in clients/ts/src/stream/live-query.ts / stream/sse.ts still uses received_timestamp and is intentionally out of scope (the live ingest stream always carries that column). Builds on #199 (query-builder structure).

Changed

  • Read-cache invalidation cut over to namespace-versioned keys, replacing the flat per-namespace version map (internal/cache/cache.go, internal/cache/local.go, internal/cache/version_manager.go, internal/api/structured_query.go, internal/api/pipes.go, internal/ingest/worker.go, internal/testutil/mocks.go, plus tests in internal/cache/{local,version_manager}_test.go and internal/ingest/worker_test.go): the Cache interface is now keyed by what a result depends on rather than one opaque string. Get/Set take a sha (hash of SQL+params) plus the []Namespace ({table, scope} pairs) the result reads — one for a structured query, several for a (future) pipe — and QueryKey folds the table version and per-namespace version of every dependency into the key, so a write to any dependency misses it. Invalidation is two-level: BumpTable(table) advances a table version that is embedded in every namespace key, invalidating all scopes of that table in O(1); BumpNamespace(table, scope) advances one scope plus the whole-table view, leaving other scopes' cached entries valid. The old GetCacheKey/GetVersion/IncrementVersion/InvalidateCache flat-versions path is removed, and the structured-query handler and ingest worker are wired onto the new path via Cache.Invalidate([]Namespace) (empty scope → BumpTable, scoped → BumpNamespace); the ingest worker collapses a single-table batch to one whole-table bump before calling Invalidate (a whole-table bump subsumes every per-scope bump for that table), keeping Invalidate a simple one-pass bump. Pipes currently pass no dependencies (TTL-only invalidation) until they can report the tables they read (#178).

  • Docs site polish: navigation + reading progress bars, branded search/asides/404, and client-side error tracking (docs/src/components/{Head,Footer,PostHog,ProgressBar (new),ReadingProgress (new)}.astro, docs/src/content/docs/404.md, docs/src/styles/global.css): builds on the astro-vtbot SPA integration above. Slow navigations get a thin accent progress bar — a new self-hosted ProgressBar.astro on vtbot's loading() hook, because vtbot's own <ProgressBar/> imports Swup's plugin from unpkg.com at runtime in every visitor's browser (a third-party CDN script in prod, against the repo's supply-chain bar; verified present in the built bundle before replacing). Doc pages additionally get a reading-progress bar (ReadingProgress.astro, rendered from the sidebar footer branch): a 2px accent line under the header filling with the scrolled-past fraction of the article body — the "how far through this page am I" signal the right-rail TOC gives desktop but mobile had nothing for; it spans only the content column (offset past the sidebar once that exists, hidden entirely ≥72rem where the TOC takes over), ignores footer chrome, hides on articles shorter than the viewport and under the open mobile menu, and updates via one rAF-throttled passive listener that survives view transitions. (A direction-aware page slide + title morph were also tried here and ripped out on review — too much motion for docs; a reduced-motion media query zeroing all view-transition animations stays as a belt-and-braces guard.) The search dialog — previously stock Pagefind chrome — now sits on the design system (surface/border/radius frame with a gentle reduced-motion-aware pop, code-surface input with accent focus, card result groups with a header band + surface-tint row hover replacing Starlight's bare bright outline, accent <mark> hits, branded clear/load-more controls, blurred backdrop), as do asides (the stock blue/purple/orange/red tokens were never remapped, so the first :::note authored would have landed off-palette), prose kbd/mark, heading-anchor hover, inner-surface scrollbars, and the mobile menu button (stock inverted to a white circle in dark mode). With asides on the design system, nineteen titled note-blockquotes across api/pipes/access-control/sdk/development became real :::note/:::caution asides (titles promoted to aside labels; the bodies are unchanged — short untitled clarification quotes stay blockquotes), and the sidebar gained a "Reference" group (API Reference + TypeScript SDK out of "Guides" — lookup material, not narrative; no URLs changed). Getting Started gains a "Troubleshooting first runs" section — the five first-session trip-ups (schema-refresh 404, pre-flush empty query, fail-closed 403 on non-demo tables, port conflicts, cold-start timing), each already documented behavior, now answered where a new user actually hits them — and its SDK pointer became a :::tip aside. A fresh-eyes screenshot audit then tightened the loose ends: Starlight's stock tab underline (a box-shadow that survived the segmented restyle as floating arcs under inactive tabs), a bottom scrim where search results hard-clipped against the dialog's rounded edge, an 88% glass header (scrolled body text bled through at 78%), the mobile menu button joining the search trigger's bordered-surface control family in both states, and full-width stacked hero CTAs on small phones. The landing page gains an SDK showcase — Ingest / Query / Live-updates tabs of @wavehouse/sdk snippets (verified against sdk.md's documented API) under a "Query it like a database. Subscribe to it like a socket." section — with Starlight's <Tabs> restyled as a segmented control, and the lone "How it ships" card folded into the quickstart prose. The 404 page replaces its bare title + bullet list with a "signal lost" treatment: the landing's grid + glow atmosphere, a mono 404, an EKG-style trace (heartbeat decaying into a flatline, drawn by a looping monitor-sweep dash animation over a live→dead gradient, with a blinking sample dot; static full trace for reduced motion), and the card-style quick links — its custom h1 replaces the hidden stock title panel so the page keeps exactly one heading. PostHog gains capture_exceptions: true, surfacing broken docs JS (LiveDemo SSE, copy buttons, theme scripts) in error tracking; the SPA pageview capture (capture_pageview: 'history_change') was verified against vtbot's lifecycle docs and is unchanged.

  • Docs "Last updated" footer shows each page's real last-edit date, and robots.txt declares open AI content signals (.github/workflows/ci.yml, docs/public/robots.txt): Starlight's lastUpdated reads git history, but CI checked out with the default shallow (depth-1) clone, so every page silently fell back to the build time — the live site showed the same "Last updated" on every page regardless of when it actually changed. CI now checks out with fetch-depth: 0. Separately, robots.txt declares open Content-Signals (search=yes, ai-input=yes, ai-train=yes): WaveHouse is open source, so agents are welcome to read, index, answer from, and train on these docs. (A markdown-twin Link header — the other half of the agent-discoverability pass — is being upstreamed to cloudflare-md-router instead, where the twin concept lives.)

  • Mermaid diagrams are click-to-zoom, with a top-level house style to author them vertically (docs/src/components/MermaidZoom.astro (new), docs/src/components/Footer.astro, docs/src/styles/global.css, AGENTS.md): wide flowchart LR diagrams shrink to ~36% in the content column and are hard to read inline. Clicking any diagram now opens it in a lightbox — small diagrams scale up to fill the viewport (capped 2.5×), wide ones show at natural size with pan-scroll for full detail; Esc / backdrop / button closes, keyboard-focusable with Enter. View-transition-safe (one delegated document listener guarded by a window flag, re-enhanced on astro:page-load). The clone is re-id'd so its id-scoped inline <style> (font, themed fills) and url(#…) marker refs keep working. The fit calculation sizes against the stage's content box — subtracting the stage's own padding and flooring fractional pixels — because sizing against raw viewport fractions overshot by the padding, so the "fully zoomed" state always rendered both scrollbars inside the sheet. The modal now always FITS the diagram (oversized ones scale down rather than panning by default — no scrollbars, rounded corners intact), with a 1:1 toggle appearing for diagrams larger than the stage: natural size + pan with the two-sided edge fades; pinch-zoom works natively on touch. Every inline diagram also carries a persistent (dimmed, hover-brightening) corner zoom chip — cursor:zoom-in only ever spoke to a pointer that happened to hover. AGENTS.md §"Authoring Mermaid diagrams" adds the directive: default to top-down (TB/TD), reserve LR for short chains, never sit two large diagrams side-by-side, keep labels short.

  • The docs-preview PR comment shows who/what/when it's built on, and reports upload failures (.github/workflows/ci.yml): the sticky <!-- docs-preview-comment --> comment now leads with a status line (live / upload-failed / URL-unparsed) and a compact bulleted summary — the head commit (subject, linked), its author(s) linked to their GitHub profiles (including Co-authored-by trailers, and a distinct committer when someone else applied the patch), the commit's own timestamp in its original timezone offset, and a separate deploy time in the maintainer team's zone (America/Detroit, DST-aware EST/EDT) — so a stale, mis-zoned, or late-pushed preview is obvious at a glance. Commit dates are read from git (the REST API normalizes them to UTC, dropping the commit's timezone); GitHub @-logins, which aren't in git, come from a single commits-API call used only when it succeeds, with a noreply-email fallback. The step now also runs when the upload fails (or succeeds but its URL can't be parsed from the wrangler output) and updates the same comment in place to say so, instead of leaving a stale "live" comment. All user-controlled text (commit subject and author names) is passed as printf %s args so a % can't be read as a format directive.

  • Site-wide UI consistency pass: one radius scale, one button shape, AA-correct light theme, branded Starlight chrome, and a wide-screen layout that uses the space (docs/src/styles/global.css, docs/src/components/{Hero,LiveDemo}.astro, docs/astro.config.mjs): a measured audit (every page, both themes) found the docs reading as "a polished landing bolted onto stock Starlight." Fixes:

    • Radius — collapsed ~11 ad-hoc values (3.2 / 6 / 6.4 / 7.2 / 9 / 10 / 13 / 15984px …) to one scale (--radius-xs/sm/md/lg/pill = 4/8/12/16/999px). All buttons are now 8px, all cards/panels/code-frames/tables 12px (Expressive Code's borderRadius set so the frame and its inner corners share one value — no more 1px lip), chips/inline-code 4px, pills/dots 999px.
    • Buttons — the same CTA used to render three ways (hero "Get started" was an 8px/weight-600/#06B0BF rectangle; the closer-band one was a 56px weight-400 #1DC9D8 pill). Unified a.sl-link-button to the hero language (8px rectangle, weight 600, single accent fill); the hero's "View on GitHub" finally renders its ↗ icon (Starlight parses icon: external into an object, so the old string compare silently never matched); the informational stat strip no longer fake-lifts on hover (it isn't clickable). Dead .action.* duplicate block removed.
    • Light theme — split an --wh-accent-text role (dark #1DC9D8, light #086D77) so accent-as-text passes AA in light (was the bright #06B0BF at ~2.6:1); added light overrides for the status hues (amber/emerald/rose, previously dark-only and failing as labels); stepped the page bg one shade darker (#FAFAF9 → #F1F2F5) so white cards/panels read as distinct layers (was ~1.04:1, visually flat); darkened light --wh-ink-subtle and lifted dark --wh-ink-subtle/--wh-terracotta-text to clear 4.5:1 on the surfaces they sit on; :focus-visible uses the AA-safe role and no longer squares element corners.
    • Layout — settled on Starlight's stock geometry after a measured detour: an earlier round widened the content track in steps (50rem on laptops, 58rem ≥100rem), sub-capped prose at 46rem, and capped + centered the .main-frame shell at 100rem. The widths left text, tables, and code ending at three different right edges, and the ultrawide cap broke outright — auto margins on a flex item size it to fit-content (the column measured 544px instead of 720px at 1680px) while the position: fixed left sidebar stayed glued to the viewport edge as the frame centered away from it. One stock 45rem track at every width keeps every block on the same edges and the right-rail TOC at its designed share of the leftover. Within that column nothing scrolls sideways: code blocks wrap (expressiveCode.defaultProps.wrap, hanging indent preserved), table code wraps BETWEEN tokens (at spaces, and after slashes in paths, via per-token nowrap spans + <wbr> — never mid-token, so --url can't shred into two flags), Mermaid diagrams scale-to-fit on desktop with click-to-zoom for detail (the natural-size pin that forced them into horizontal scrollers everywhere is now phone-only, where panning beats 3px labels), and a table becomes its own scroller only when measured as genuinely unfittable (the per-column min-width floors apply only then — applied unconditionally they were CREATING the scroll on tables that fit). The configuration reference also stacks each setting's YAML key over its env var in one column (the two were the widest columns in every table on the page). Verified zero page-level horizontal overflow across 14 pages × 12 widths (320–2560) × both themes; at desktop widths every table and code block fits the column except two tables in the ~90px window at the exact 72rem breakpoint, where stock Starlight squeezes the pane below 45rem. A third (and fourth) round refined the breakout on review feedback: tables now EARN their width (a table whose prose wraps comfortably at 45rem stays at prose width — stretching its sentences to 60rem line lengths reads worse, not better; only tables that overflow or crowd at prose width take the wide track, capped 2.5rem short of the rails), diagram scrollers get an explicit min(natural-width, pane) width (fit-content can't measure a width:100% SVG child — wide diagrams were pinned at prose width while small ones already sat at natural size), the page footer (edit link / last-updated / prev-next) stays at prose width inside the un-capped container, and the edge-fade threshold rose to 8px so devicePixelRatio rounding can't paint a "more to scroll" fade on blocks with nothing to scroll — with a matching LEFT fade when scrolled (the right-only mask read as content amputated behind you). The width-earning then got a natural-width sense: each table's max-content width is measured in a batched reflow, and a table whose natural width is at or near the breakout track (≤1.3×) widens to stop wrapping entirely (the why-wavehouse matrices), while prose-heavy tables (natural width = whole unwrapped sentences) keep the reading measure; ≥4-column tables under ~12.5rem average also widen. Mermaid cluster strokes stopped being sliced by the scroller's overflow-y (2px block padding), svgs sitting directly inside a .diagram-pair get their own scroller each (the pair container itself was being classed as one scroller — two diagrams shared one zoom chip and one natural width, and one diagram's highlight box clipped), and the fade threshold rose again to 12px after a phantom fade reproduced at devicePixelRatio 2. A follow-up round then let wide content earn width back without disturbing that geometry: the markdown body becomes a 3-track grid (the classic full-bleed pattern — prose capped in the centered middle track at the stock 45rem; the page title and prose share a left edge via --sl-content-margin-inline: auto), and table/code/diagram blocks span all three tracks sized fit-content — a short block stays at prose width with aligned edges, a wide one grows centered up to the main pane, whose width is a function of the viewport alone, so the right-rail TOC never moves and the page still can't scroll sideways. Table headers gained real emphasis (0.8125rem/650 on full ink over the elevated band, stronger underline), and the wrap-vs-scroll call became a crowding heuristic: a ≥4-column table averaging under ~8.5rem per column reads worse wrapped than panned, so it scrolls instead of wrapping every other word.
    • Landing atmosphere (re-architected) — the grid + glow lived inside .wh-hero, whose overflow: hidden clipped them to a centered ~1080px box (flat margins on wide screens, a hard rectangular edge, the glow sliced into an abrupt colored corner, and nothing under the navbar or below the hero). They're now two page-level fixed layers scoped via body:has(.wh-hero) — a continuous full-viewport "signal-field" grid (teal-tinted, soft-cornered vignette, runs edge-to-edge under the translucent navbar and all the way down the page) and a soft, breathing teal hero glow with pure radial falloff (no clipped edges) — so the background is genuinely full-bleed, top-anchored, and continuous in both themes. The hero's own clip + orb elements are gone; dead .wh-grid-bg/.wh-glow-orb/.wh-glass utilities removed. (Both page-level layers sit at negative z-index, which Starlight's layered opaque body background painted over — so the landing body is set transparent, letting html's base color be the canvas and the grid/glow paint above it.)
    • Hero CTAs — the pnpm add @wavehouse/sdk hint became a click-to-copy command chip (faux terminal prompt, copy→check feedback, clipboard-API with a selection-range fallback), and a live ★ N stars chip joins the hero meta — fed by the same demo pipe() data as the panel, so the hero carries live social proof and reinforces the real-time story.
    • Live "wow" metrics in the demo panel — the panel now proves the real-time claim other tools can't make: an ⚡ ingested → live in ~N ms readout computed in the browser from each SSE event's received_timestamp (WaveHouse's server ingest clock; ingest → broadcast → browser, clamped for clock skew — Wave-RF/WaveHouse-Stats#33), a live events/min sparkline (new gh_events_per_minute pipe, bumped per streamed event — Stats#34), and a lifetime N tracked total (gh_summary.total_events). Each degrades to a placeholder when the stream is quiet or a pipe is unavailable.
    • Starlight chrome — pagination prev/next now mirror the link-card treatment (hover lift + glow), tables get a card container + uppercase header (inline code in headers kept case-exact) and later a readability pass (14px top-aligned body cells, zebra striping so the eye tracks across wide rows), blockquotes an accent rail + tint, the search trigger brand colors, and the sidebar gains uppercase group-label eyebrows + a real hover state + a rounded current-item (the old group-label override was a dead selector matching 0 elements). The never-wired data-section accent system was removed (it always resolved to teal anyway).
  • The landing hero is the live demo: real GitHub-activity data through @wavehouse/sdk in the visitor's browser, replacing the static terminal mock + wave-mesh animation (docs/src/components/LiveDemo.astro (new), docs/src/components/Hero.astro, docs/src/content/docs/index.mdx, docs/src/styles/global.css, docs/scripts/screenshot.mjs, docs/package.json, Makefile, pnpm-lock.yaml): the hero's visual column now renders a live panel fed by the public read-only role on stats.wavehouse.dev (Wave-RF/WaveHouse-Stats, our own-WaveHouse dogfood deployment): every headline number from one cached gh_summary pipe call (Wave-RF/WaveHouse-Stats#19 — it also carries reconciler-observed open_issues/open_prs, live for any future [data-wh-live] placement), the activity feed backfilled from the server-curated, server-deduped, cached gh_activity_recent pipe (Wave-RF/WaveHouse-Stats#18 — its (event_type, action) pair list and the component's verb map must stay in sync; both sides carry the note), and a live SSE tail via stream(). The SSE leg streams all gh_events rows and is filtered in-browser (type-only, because live rows are raw producer payloads where an action IN-filter could drop rows whose action is absent; the verb allowlist in addRow is the final arbiter), a composite-key dedup absorbs webhook redeliveries and the backfill/stream overlap, and zone-less producer timestamps on live rows are normalized to UTC before parsing (backfill rows come back RFC3339; the raw SSE payload is ClickHouse-style and would otherwise read as local time). The pre-pipe reads — per-number pipes and a structured-query backfill (168h) — remain as runtime fallbacks, so a stats-stack rollback degrades the panel to slower, not blank. A star CTA ("Star Wave-RF/WaveHouse — and watch it land here, live") and a full-demo link close the panel, and the closer band gains a live stargazer chip fed by the same fetch ([data-wh-live] targets, hidden until data arrives). The panel server-renders a layout-stable skeleton (fixed-height feed, "—" counts) so the page is complete without JS and degrades to placeholders + an "offline" pill if the demo instance is unreachable; the script re-initializes per Astro view-transition mount and tears down (SSE close(), timers) when navigating away; styles are is:global because the feed rows are runtime-created DOM that Astro's scoped selectors would skip. It is fluid down to ~320px: the hero's single-column mobile track, the .wh-hero__visual flex wrapper, and the 3-up stat grid all carry min-width: 0 / minmax(0, …) so the panel shrinks to the viewport instead of being floored at the stat grid's ~325px min-content and clipped by the hero's overflow: hidden (the feed-row title ellipsis-truncates, stat labels stay single-line). @wavehouse/sdk becomes a docs workspace dependency — the docs site is now a real SDK consumer — with check-docs and dev-docs gaining a build-ts prerequisite (astro check and the browser bundle resolve clients/ts/dist/). docs/scripts/screenshot.mjs switches networkidle → load (the SSE stream never goes idle) and waits on the visual column's entrance animationend instead of the deleted terminal lines, deliberately independent of live data so screenshots can't hang on the demo instance. Follow-ups: Wave-RF/WaveHouse-Stats#18–21 (curated pipe, summary pipe, embed capacity, stats-page polish), #286 (query playground page + live SDK examples in docs).

  • make dev-docs is now production-faithful: rebuild-on-save through the real Worker instead of astro dev (docs/scripts/dev.mjs (new), docs/package.json, docs/astro.config.mjs, Makefile, AGENTS.md, .gitignore, .vscode/launch.json, docs/scripts/screenshot.mjs, docs/src/content/docs/development.md): the Astro dev server skips everything the Cloudflare Worker adds in production — cloudflare-md-router content negotiation (.md twins), the pagefind search index, and the starlight-llm-tools outputs only exist in real builds — so the de-facto docs workflow had become a manual make build-docs && make preview-docs cycle, restarted by hand per change. pnpm run dev (and therefore make dev-docs) now drives docs/scripts/dev.mjs: a debounced full astro build on every save to src/, public/, or a root build input (astro.config.mjs, tsconfig.json, package.json, .env* — but not worker/index.ts/wrangler.jsonc, which wrangler dev hot-reloads itself, and not pnpm-lock.yaml, which changes on git pull before pnpm install has run), with saves landing mid-build coalescing into one follow-up build, staged into a gitignored docs/.dev-dist/ and synced in-place into dist/ only on success (plain node fs, prune-then-copy — no rsync or any other external tool, so minimal WSL/container setups work) — a failed build prints a red banner + terminal bell and keeps serving the last good site rather than the emptied outDir a direct in-place build would leave — while wrangler dev --live-reload on :4321 (DOCS_PORT overrides; same default as before, so docs/scripts/screenshot.mjs and muscle memory still work; one-off preview-docs stays on :8787) picks up the asset changes live, new routes included, and refreshes the browser over its websocket. Each rebuild is a real production build with one deliberate exception: the loop sets WAVEHOUSE_DOCS_WATCH=1, under which astro.config.mjs drops starlight-links-validator — mid-edit prose always has dangling links, and failing every rebuild over them would mean never seeing the change being made; rendered output is identical, CI / make build-docs still enforce link validity, and DOCS_WATCH_STRICT=1 keeps the validator on in watch builds for those who want broken links to fail loudly at edit time (the development.md tooling notes now spell out that split; build errors themselves — broken Mermaid, bad frontmatter, MDX errors — always fail the watch build loudly either way). The raw HMR dev server remains available as pnpm --filter wavehouse-docs run start for fidelity-insensitive iteration (e.g. CSS tweaking) — the VS Code launch config gains a second entry for it, with the existing pnpm dev entry relabeled to match its new behavior.

  • Docs Mermaid diagrams render through a per-diagram disk cache (docs/astro.config.mjs, docs/package.json, pnpm-lock.yaml; the cache itself ships in astro-themed-mermaid v0.2.0): diagram SSR runs headless Chromium via rehype-mermaid and re-rendered all 17 diagrams on every docs build — including builds that touched no diagram, i.e. almost all of them. The config now uses the package's new mermaid.rehypeMermaid export, which wraps rehype-mermaid with a content-addressed cache under node_modules/.cache/astro-themed-mermaid/ (keyed on diagram source + render options + package versions, so theme and toolchain changes self-invalidate; entries are re-id'd to content-derived mermaid-c<hash> ids so cross-build entries can't collide on a page). A build whose diagrams all hit skips Chromium — and even the rehype-mermaid/playwright import — entirely: measured warm rebuilds drop from ~6–7s to ~3.7s, a single-diagram edit re-renders only its page (~5s), and output was verified byte-identical modulo SVG ids. Cold builds (fresh clone, CI, make clean-tools) render once and harvest; cache I/O is best-effort and degrades to a normal render, never a failed build.

  • Docs-site analytics rebuilt: PostHog routed through the first-party relay, with CI-safe config and soft-nav-correct capture (docs/src/components/PostHog.astro (new), docs/src/components/HomepageCtaTracking.astro (new), docs/src/components/{Head,Hero,Footer}.astro, docs/src/content/docs/index.mdx, docs/astro.config.mjs, docs/.gitignore, docs/posthog-setup-report.md (new), .claude/skills/integration-astro-view-transitions/ (new, vendored), scripts/docs-prose.sh, docs/src/content/docs/claude-code.md): the hardcoded PostHog head snippet in astro.config.mjs (pointed straight at us.i.posthog.com) is replaced by a PostHog.astro component in the Starlight Head override, initialized behind a window.__posthog_initialized guard with capture_pageview: 'history_change' so ClientRouter soft navigations both survive re-init and capture pageviews (verified headless: $pageview fires with navigation_type: pushState). Analytics now route through the managed first-party reverse proxy t.wave-rf.com — the shared *.posthog.com ingest domains sit on ad-blocker lists — with ui_host keeping toolbar/app links on us.posthog.com. Configuration reads PUBLIC_POSTHOG_PROJECT_TOKEN / PUBLIC_POSTHOG_HOST (gitignored docs/.env) and falls back to committed values: CI builds the site with no PostHog env, so env-only wiring shipped a dead snippet to production — the committed phc_ token is the public client token that ships in every served page by design, not a secret. Three click events are instrumented — hero_cta_clicked (Hero.astro), footer_link_clicked (Footer.astro), homepage_cta_clicked (HomepageCtaTracking.astro, extracted to a component because MDX parses { as JSX expression syntax and a raw <script is:inline> body fails astro build) — all idempotent across hard loads and astro:page-load re-binds. Local-dev noise (localhost was ~85% of recent pageviews) is filtered PostHog-side via project test-account filters on $host, not client guard code. The wizard's vendored agent skill lands at .claude/skills/integration-astro-view-transitions/ (inventoried in claude-code.md) and its frozen setup report at docs/posthog-setup-report.md, denylisted from the docs-prose review set as an internal artifact.

Removed

  • Gemini Code Assist integration and the on-repo Claude PR review workflow (.gemini/config.yaml, .gemini/styleguide.md, .github/workflows/claude-review.yml deleted; .dockerignore, AGENTS.md, .github/prompts/pr-review.md, .claude/agents/pre-push-reviewer.md, .claude/skills/pr-review-locally/SKILL.md, .claude/hooks/agent-bash-gate.sh, docs/src/content/docs/development.md, docs/src/content/docs/claude-code.md updated). Dropped the Gemini Code Assist app configuration (.gemini/) and the GitHub Actions Claude PR review workflow (claude-review.yml — the manual @claude / /review / workflow_dispatch reviewer) outright. Local Claude Code tooling under .claude/ is unchanged: the pre-push-reviewer subagent still runs the canonical .github/prompts/pr-review.md locally before pushes, and that prompt file is retained for it. Living docs, agent/skill/hook references, and the bot-re-trigger tables were scrubbed of both reviewers; CodeRabbit and Copilot remain the advisory PR reviewers, and Admin approval + the main branch protection ruleset remain the actual merge gate. Earlier CHANGELOG entries that mention either reviewer are left as historical record.

Added

  • Browser-first SDK distribution: an IIFE global build, CDN entry points, a wavehouse-codegen bin, and a Node 18 floor (clients/ts/tsup.config.ts, clients/ts/package.json, clients/ts/README.md, docs/src/content/docs/sdk.md, docs/src/content/docs/development.md, pnpm-workspace.yaml): @wavehouse/sdk already shipped browser-ready ESM/CJS (zero deps, native fetch/EventSource) but documented only the npm install + bundler path. The build now also emits a minified, self-contained IIFE bundle (dist/index.global.js) that defines a WaveHouse global, wired to new unpkg/jsdelivr package fields — so <script src="https://cdn.jsdelivr.net/npm/@wavehouse/sdk"></script> then WaveHouse.createClient({ … }) works on a no-build, FTP-deployed page — and the SDK README + sdk.md gain a "No build step (CDN)" section covering both the ESM-CDN (<script type="module"> + esm.sh) and global forms. The codegen CLI now ships as a wavehouse-codegen bin (bundled to dist/cli/codegen.js, shebang preserved); consumers run npx wavehouse-codegen instead of the prior npm run codegen / tsx src/cli/codegen.ts, which only worked inside this repo since files ships dist/ only. engines.node is relaxed from >=22 to >=18 — the runtime needs only fetch (stable since Node 18); streaming under Node < 22 still uses the SDK's existing "provide an EventSource polyfill" prompt — so the published package no longer warns or fails to install in projects on Node 18/20. tsup.config.ts becomes an array of three build targets and dist/ is cleaned by the build npm script rather than tsup's clean, so the configs can't race wiping each other's output. The SDK docs' CDN section documents version resolution (a bare URL serves the latest published release; version pins, @N ranges, and the dev channel), and development.md gains a Releasing the SDK section covering the dev-snapshot vs. sdk-v*/latest publish flow and the first-release latest promotion.

  • Forgiving multi-format ingest on POST /v1/ingest (single object, JSON array, or NDJSON), plus SDK insert([...]) / insertNDJSON() (internal/api/ingest.go, internal/api/record_reader.go (new), internal/api/ingest_test.go, clients/ts/src/{http,types,table,index}.ts, clients/ts/src/{http,table}.test.ts, tests/e2e/sdk/ndjson.test.ts (new), tests/e2e/sdk/tables.ts, docs/src/content/docs/{api,sdk}.md, AGENTS.md): closes #195 (part of #194). The ingest endpoint now accepts a single flat JSON object, a JSON array of objects (any length, even one), or an NDJSON batch — the format is auto-detected from the body, with Content-Type only a hint (the first non-whitespace byte decides; an explicit application/x-ndjson selects line-framing unless the body starts with [). Each record is validated, authorized, deduplicated, and published independently, so one malformed or rejected record never blocks the rest of the batch; a batch (array/NDJSON) returns a per-record summary {total, succeeded, failed, duplicates, results:[{index, ok|duplicate|error}]} with 200 whenever the body was readable (branch on failed/results, not the status), while a lone object keeps its {"ok":true} / {"duplicate":true} response byte-for-byte. Whole-request conditions abort with a non-200: a table-level insert denial (403, checked once up front), a JSON-array syntax error or oversized NDJSON line (400), the new 16 MiB body cap (413, parity with /v1/admin/query), backpressure (503 + Retry-After), and publish/dedup-backend errors (500). Formats sit behind a pull-based recordReader seam (object / array / line readers + a content sniffer) so CSV and response-streaming can drop in later without reworking the handler; per-record vs whole-request handling stays in the shared processRecord helper.

    • SDK. from(t).insert(rows[]) now serializes to a single NDJSON request instead of firing one unbounded concurrent POST per row (which also silently returned only the first error). Per-record outcomes surface on InsertResult (total/succeeded/failed/duplicates/results), and ok is true only when every record succeeded; the call's top-level error is reserved for whole-request failures. A new from(t).insertNDJSON(source) sends NDJSON you already have (string / Uint8Array / Blob/File / ReadableStream). An empty array is a no-op (no request). New exported types InsertRecordResult and NDJSONSource.
    • Deferred to #196: bounded-concurrency chunking of very large arrays + backpressure-aware retry. insert([...]) currently sends one request regardless of size.
  • npm publishing for @wavehouse/sdk via OIDC trusted publishing (.github/workflows/publish-npm.yml (new), .github/workflows/publish-dev.yml, clients/ts/package.json, clients/ts/LICENSE (new)): closes #227. A dedicated publish-npm.yml is the single npm trusted publisher for the SDK (npm allows one trusted workflow file per package), with two jobs mirroring the Go/Docker release model: dev publishes 0.0.0-dev.<sha> under the dev dist-tag on every push to main that touches clients/ts/ (the npm analog of the :dev image), and release publishes a tagged version under latest/alpha/beta/rc/next (derived from the version) plus a GitHub Release on sdk-v* tags — decoupled from the server's v* tags. Auth is GitHub OIDC trusted publishing (no NPM_TOKEN secret); --provenance was deferred until the repo went public — now enabled (see the build-provenance entry below). clients/ts/package.json gains author/homepage/bugs and a normalized repository.url, and the package ships a bundled LICENSE. publish-dev.yml gains only a pointer comment here (its :dev image attestation lands in the build-provenance entry below).

  • Signed build-provenance attestations for every released artifact — npm, Go binaries, and the GHCR image (.github/workflows/publish-npm.yml, .github/workflows/release.yml, .github/workflows/publish-dev.yml, SECURITY.md, README.md, clients/ts/README.md, docs/src/content/docs/deployment.md): now that the repo is public, all release artifacts ship signed Sigstore build provenance. @wavehouse/sdk publishes with npm publish --provenance on both the dev and tagged channels (via the existing OIDC id-token, no new secret); release.yml attests the GoReleaser binaries (one attestation over dist/checksums.txt) and the multi-arch ghcr.io/wave-rf/wavehouse image manifest digest (push-to-registry: true, stored as an in-registry referrer); publish-dev.yml attests the rolling :dev image the same way (dev binaries aren't distributed). Uses actions/attest-build-provenance (SHA-pinned) with id-token/attestations: write scoped to the publish jobs, and guards the digest-resolve step (set -euo pipefail + non-empty check) so a registry hiccup fails the release loudly instead of attesting an empty digest. Consumers verify with gh attestation verify (image + downloaded release archives) or the npm provenance badge / npm audit signatures (SDK); provenance covers the distributed binaries and image, not go install (which builds from source). Refs #149.

  • CORS support for the SSE streaming endpoint (internal/api/router.go, internal/api/stream.go, internal/api/router_test.go, docs/src/content/docs/api.md): closes #215. GET /v1/stream already inherited the router-level corsMiddleware (Access-Control-Allow-Origin echo + OPTIONS preflight), but the SSE resumption header Last-Event-ID — which the handler reads to gap-fill from the client's last seen event — was not in Access-Control-Allow-Headers, so a cross-origin fetch-based SSE client resuming via that header failed its preflight. Last-Event-ID is now allow-listed, the SSE handler carries a comment pointing at the middleware, and new end-to-end tests pin CORS on /v1/stream (preflight advertises Last-Event-ID, the streaming GET echoes an allowed origin, a disallowed origin gets no header) so the behavior can't silently regress. Browser EventSource clients still pass the JWT via ?token= since they can't set an Authorization header.

  • Direct-to-cloud OTLP — TLS + auth headers via the standard OTEL_EXPORTER_OTLP_* env vars, no sidecar required (internal/observability/provider.go, internal/config/config.go, cmd/wavehouse/main.go, internal/testutil/otlp.go, tests/integration/otel_test.go, config.yaml, docs/src/content/docs/{configuration,deployment}.md, AGENTS.md): closes #97. WaveHouse exports traces/metrics/logs straight to a cloud OTLP gateway (Honeycomb, Grafana Cloud) with no sidecar to terminate TLS or inject auth. The OTLP destination — endpoint, TLS (an https:// endpoint selects system root CAs), a custom/private CA (OTEL_EXPORTER_OTLP_CERTIFICATE), mutual TLS (OTEL_EXPORTER_OTLP_CLIENT_CERTIFICATE/_CLIENT_KEY), and per-RPC auth headers (OTEL_EXPORTER_OTLP_HEADERS) — is configured through the OpenTelemetry SDK's standard OTEL_EXPORTER_OTLP_* environment variables; InitProvider passes no endpoint or header options, so custom-CA + mutual TLS work for traces and metrics. The gRPC logs exporter (otlploggrpc v0.19/v0.20) is the exception: it loads OTEL_EXPORTER_OTLP_CERTIFICATE / client-cert vars but never wires them into its dial (only the programmatic WithTLSCredentials is honored), so a custom/private CA and mutual TLS do not apply to the logs signal — public-CA/system-roots TLS and plaintext still work. This is an upstream bug (open-telemetry/opentelemetry-go#6661) that WaveHouse deliberately does not work around in app code; route logs through a local collector if you need a private-CA gateway. A malformed OTEL_EXPORTER_OTLP_HEADERS entry is logged and skipped by the SDK (fail-soft), the same way the SDK handles every other OTEL_EXPORTER_OTLP_* value. The earlier otel.addr/WH_OTEL_ADDR and otel.headers/WH_OTEL_HEADERS config keys are removed in favor of the spec vars — set OTEL_EXPORTER_OTLP_ENDPOINT instead (default is the SDK's localhost:4317). Datadog has no public direct-to-cloud OTLP endpoint; its supported path remains the local DDOT Collector embedded in the Datadog Agent, reached as a plaintext receiver (see the deployment guide). Integration tests pin the TLS dial path for traces and metrics — trusting an in-process self-signed receiver via OTEL_EXPORTER_OTLP_CERTIFICATE, the same path a real operator uses — and header propagation across all signals.

  • GET /version build-info endpoint (internal/api/version.go (new), internal/api/version_test.go (new), internal/api/router.go, internal/api/router_test.go, cmd/wavehouse/main.go, docs/src/content/docs/api.md): a public, unauthenticated endpoint returning the binary's version, git_commit, and build_time — the same values injected via -ldflags (Makefile, .goreleaser.yaml) and logged at startup — plus go_version from runtime.Version(). Completes issue #44: the ldflags injection and startup logging already shipped; this exposes the same metadata over HTTP so an operator can confirm exactly which build is running on a deployed instance. Wired alongside /livez and /readyz, outside the /v1 auth tree; the values are non-sensitive and already in the logs, so there is no auth gate.

  • Configurable RBAC: a single admin role, public access driven by the policy, and authentication decoupled from authorization (internal/auth/, internal/policy/, internal/api/, internal/config/config.go, cmd/wavehouse/main.go, config.yaml, docs/src/content/docs/configuration.md, docs/src/content/docs/api.md, AGENTS.md):

    • No auth.enabled / auth.dev_mode flags. The JWT middleware (new internal/auth package) always runs. A request with no token, or an invalid/expired/malformed one, falls back to an empty role that policy.ResolveRole maps to the policy default_role; the token error is recorded in the request context so a gate that ultimately denies fails loud (401 "invalid/expired token") rather than a bare 403. Elevated access requires a valid token. With neither jwt_secret nor jwks_url configured, no token validates and every request is the default_role — a pure public deployment.
    • Configurable admin_role (policy field, exact case-sensitive match, "admin" by default). The hard-coded admin/service string checks are gone and service is no longer a privileged role. A single policy.IsAdmin is the source of truth, shared by Evaluate, ResolveRole, Validate, the /v1/admin gate (RequireAdmin, reads admin_role live from the store), and pipe authorization (policy.RoleAllowed). ResolveRole/Validate no longer ToLower/TrimSpace the role — roles are opaque, exact-match strings.
    • Public (no-token) read access via a usable default_role: setting it opens public access, removing it closes it, live on a policy PUT/delete. Setting default_role equal to the admin_role is permitted and makes every roleless request admin (including /v1/admin/*) — a local/dev-only convenience that is not refused by ResolveRole or Validate, but the store logs a loud warning on every node that adopts such a policy (policy.DefaultRoleGrantsAdmin). Roles do not inherit the default's permissions.
    • Schema discovery and DLQ stats are now admin-only (RequireAdmin), replacing the prior token-only RequireAuthenticated gate (removed). The "*" any-role wildcard remains removed from policy.Evaluate and pipe allowlists (column allow-lists unaffected).
    • Fail-closed by structure, not a flag. Evaluate(nil)/IsAdmin(nil) deny everyone — including the admin role — so deleting the policy from KV is a total lockout on its own (a fresh or emptied deployment is bootstrapped from the policy file, not an implicit admin grant); the Store.SetFailClosed retention hack and the allowAnon admit-decision plumbing are removed.
    • Every authorization denial emits a structured slog WARN (internal/api/errors.go). One shared writeAuthzDenied path logs reason, role_observed vs role_resolved, roles_allowed, the chi route pattern (low-cardinality, no concrete path params), method, status, and a gate tag that names the check that denied — admin (the RequireAdmin gate, including /v1/schema and /v1/dlq/stats, which carry no /admin prefix), policy (the per-table evaluator, which also logs the table + action), or pipe (which also logs the pipe name). The denial gates (RequireAdmin and the ingest/structured-query/pipes handlers) take an injected *slog.Logger (wired from main, like the policy/pipes stores) rather than reaching for slog.Default(), and log with the request context — so the record inherits the process logger's format/destination and, under the OTel logger, its trace_id/span_id, making a misconfigured role or policy visible without reproducing the request.
  • First-boot trial policy for the standalone Docker compose stack (deployments/compose/dev-policy.yaml (new), deployments/compose/standalone.yaml, docs/src/content/docs/configuration.md): the docker compose quickstart now seeds a permissive public policy on first boot — mounted read-only from dev-policy.yaml and wired via WH_POLICY_FILE_PATH=/app/policy.yaml — so clone → ingest → query → stream works tokenless out of the box. The grant is deliberately narrow: public read/write on the two demo tables (clicks, events) only, not admin — no raw SQL, no policy/pipe CRUD, no schema/DLQ admin, no other tables — so the file escalates nothing if it ever rides into a real deployment (those demo tables won't exist there). WaveHouse stays fail-closed (no policy = every request denied, including admin); this only seeds NATS KV once, after which KV is authoritative and edits go through PUT /v1/admin/policy. Tune it — or drop the default_role to require auth — before production (see access-control.md). Also documents the existing clickhouse.http_scheme key (default http) in the configuration reference.

Changed

  • Reviewer assignment is now fully GitHub-native — removed housekeeping.yml's reviewer-request, the assign-and-request-review composite, and the ADMINS list (.github/workflows/housekeeping.yml, .github/actions/assign-and-request-review/ (deleted), .github/board-config.env, AGENTS.md, docs/src/content/docs/development.md, docs/src/content/docs/claude-code.md): the required_reviewers ruleset rule requests the @Wave-RF/wavehouse-admins team on every PR, and the team's code-review assignment (configured on the team) auto-assigns + load-balances a specific member — so the workflow no longer picks/requests a reviewer or sets an assignee. housekeeping.yml is now just path-labeling + PR-title-lint. With its last consumer gone, ADMINS is dropped from board-config.env (which now only carries Task Board IDs for triage.yml), removing the team-membership-vs-ADMINS drift.
  • Retired the custom Admin approval status-check workflow for the native required_reviewers ruleset rule; de-automated Dependabot (.github/workflows/admin-approval.yml (deleted), .github/workflows/dependabot-automerge.yml (deleted), .github/workflows/housekeeping.yml, .github/board-config.env, AGENTS.md, docs/src/content/docs/development.md; main branch protection ruleset): the admin-review gate is now the ruleset's required_reviewers rule — an approval from the @Wave-RF/wavehouse-admins team — replacing the 227-line admin-approval.yml that drove a commit-status gate (which produced a per-commit red-X / yellow-dot on every PR until approval, and kept ADMINS inlined in the workflow and in board-config.env). The admin bypass mode is now pull_request (no direct pushes to main for anyone; admins still self- / emergency-merge their own PR), require_last_push_approval is on, and a separate tag ruleset restricts v* / sdk-v* creation to admins. dependabot-automerge.yml (auto-approve + auto-merge of patch/minor bumps) is removed — Dependabot PRs now get a human admin review like any PR. Required status checks drop from three to two (CI + PR housekeeping).
  • Pre-push review gate is now reviewer-count-agnostic, driven by a single source of truth (scripts/pre-push-reviewers.sh (new), .claude/hooks/agent-bash-gate.sh, .claude/hooks/review-marker.sh, .claude/commands/prepush.md, .claude/commands/docs-review.md, .claude/agents/pre-push-reviewer.md, .claude/agents/docs-reviewer.md, AGENTS.md, docs/src/content/docs/claude-code.md): the two mandatory reviewers (pre-push-reviewer, docs-reviewer) were hardcoded in both hooks, /prepush, AGENTS.md, and the docs, so adding a third (e.g. a security reviewer) meant editing all of them. Now scripts/pre-push-reviewers.sh lists the gating subagents — one name per line — and is the only place the set is defined: agent-bash-gate.sh requires a marker for every listed reviewer (fail-closed if the list is missing, empty, or corrupt), review-marker.sh writes the marker for whichever listed reviewer returns VERDICT: ship_it, and /prepush reads the list to launch the relevant reviewers in parallel. Marker names are now derived from the reviewer name — tmp/<name>-passed-<HEAD-sha> (tmp/review-passed-* → tmp/pre-push-reviewer-passed-*, tmp/docs-review-passed-* → tmp/docs-reviewer-passed-*); markers live in gitignored tmp/, so the rename needs no migration. Adding a reviewer is now two steps with no hook edits — create .claude/agents/<name>.md, then add <name> to the list (AGENTS.md §"Adding a pre-push reviewer").
  • Pre-push reviewers run only where relevant — trivial changes skip the rest, on the record (scripts/skip-pre-push-review.sh (new), .claude/commands/prepush.md, .claude/skills/pr-review-locally/SKILL.md, .claude/hooks/agent-bash-gate.sh, AGENTS.md, docs/src/content/docs/claude-code.md): /prepush now judges which reviewers a change actually needs (the code reviewer is pointless on a docs-only typo; the docs reviewer on a test-only change) and runs those in parallel, skipping the rest via scripts/skip-pre-push-review.sh <name> "<reason>" — a sanctioned, logged skip that writes the reviewer's marker (so the push proceeds) and appends the reason to tmp/review-skips-<HEAD>.log. The skip command refuses unknown reviewer names and prints a ⚠️ when a skip looks wrong (skipping docs review while docs files changed, or code review while non-docs files changed); agent-bash-gate.sh echoes the skip log at push time so skips are auditable. This deliberately trades a little safety for speed — the orchestrator self-certifies skips — so the rule is bias to running; when unsure, run it (or /prepush all to force the full set). Separately, the pr-review-locally skill (auditing someone else's PR) now runs all the relevant reviewers in parallel, not just the code reviewer, surfacing their combined findings.
  • PR titles are validated by one shared rule — locally before gh pr create and in the required PR housekeeping check (scripts/lint-pr-title.sh (new), .github/workflows/housekeeping.yml, .claude/hooks/agent-bash-gate.sh, CONTRIBUTING.md, docs/src/content/docs/development.md, docs/src/content/docs/claude-code.md): the Conventional-Commits title rule (12 types, lowercase-first subject, no trailing period, ≤ 72 chars — the title becomes the squash-merge subject on main) previously lived only as a regex inside housekeeping.yml, so a malformed title was discovered after the PR existed, when the required check failed — the recurring "title too long / wrong format" round-trip. The rule now lives in scripts/lint-pr-title.sh (single source of truth): housekeeping.yml calls it for the required check, agent-bash-gate.sh lints quoted --title/-t values on gh pr create / gh pr edit --title so agents catch a bad title before the PR exists (fail-open when no quoted title is parseable), and humans can validate with scripts/lint-pr-title.sh "<title>". PR_TITLE_SKIP_LENGTH=1 exempts Dependabot's long grouped-update titles from the length cap; PR_TITLE_MAX_LEN overrides it.
  • AGENTS.md gains craftsmanship + review-cooperation rules (AGENTS.md): a new §Craftsmanship codifies three cross-cutting habits for every change — comment the why not the what (concise, matching the file's density), DRY / one-source-of-truth, and leave-it-neater-than-you-found-it (small safe cleanups in passing; large or uncertain ones become a tracking issue). §Review Response now requires addressing every reviewer finding — fixing small valid out-of-scope ones cooperatively, opening (or linking an existing) tracking issue for ones too large or uncertain to do in-PR, and never silently dropping a flag — and Operating Rules #2/#4 are reworded to match (the push needs every reviewer, not "both"; address-and-resolve, not just resolve).
  • docs-reviewer is now a mandatory, always-on pre-push gate, run in parallel with pre-push-reviewer (scripts/docs-prose.sh (new), .claude/hooks/review-marker.sh, .claude/hooks/agent-bash-gate.sh, .claude/agents/docs-reviewer.md, .claude/agents/pre-push-reviewer.md, .github/prompts/docs-review.md, .github/prompts/pr-review.md, .claude/commands/docs-review.md, AGENTS.md, docs/src/content/docs/claude-code.md): the docs-prose review was previously advisory (no marker, soft-nudged by a pre-push-reviewer [SHOULD]). It now emits a VERDICT: ship_it|iterate|block line in its default (branch) scope; the review-marker.sh SubagentStop hook writes tmp/docs-reviewer-passed-<HEAD-sha> on ship_it, and agent-bash-gate.sh requires both that marker and the existing tmp/pre-push-reviewer-passed-<HEAD-sha> before any push to a PR (non-main) branch with commits ahead of main — unconditionally, even for code-only changes, because the reviewer also checks code↔docs sync (code that changed but whose docs didn't, per §Documentation Sync). Scope widened from "Starlight site + top-level *.md" to a denylist resolved by scripts/docs-prose.sh: every tracked .md/.mdx except .claude/**, .github/**, CHANGELOG.md, AGENTS.md, CLAUDE.md, *.draft.md/*.old.md, PERF-CLAIMS-REVIEW.md — so the README (incl. the SDK's clients/ts/README.md), CONTRIBUTING.md, SECURITY.md, CODE_OF_CONDUCT.md, and SUPPORT.md are now covered and new docs are picked up automatically (CODE_OF_CONDUCT.md/SUPPORT.md deep-reviewed only on change). An explicit /docs-review <path> or /docs-review all stays advisory (no VERDICT:, no marker). The pre-push-reviewer docs-prose soft-gate is removed (the hard gate replaces it); the soft-gate nudge in .github/prompts/pr-review.md is retained only for the markerless cloud PR-review path.
  • Lint/format coverage extended to the docs package + repo-wide Markdown (biome.json, package.json, .markdownlint.json, .markdownlint-cli2.jsonc, Makefile, AGENTS.md): Biome's files.includes now covers docs/** JS/TS/JSON (astro config, mermaid theme, sidebar, worker, dev scripts) on top of the SDK and E2E harness — .astro templates and Markdown remain out of Biome's scope by design. Markdown across the whole repo is now linted by markdownlint-cli2 (new root devDependency), wired into make lint / make fix next to Biome; rules live in .markdownlint.json (so editor extensions stay in sync) and file globs in .markdownlint-cli2.jsonc. MD013 (line length), MD033 (inline HTML — the README's theme-aware <picture>), and MD060 (table column style) are disabled as ill-fitting here; MD024 (duplicate headings) is exempted in CHANGELOG.md only, since Keep a Changelog repeats ### Added / Changed / Fixed per version. .mdx is not linted (markdownlint parses CommonMark, not MDX's JSX).
  • Relicensed from MIT to Apache-2.0 (LICENSE, clients/ts/LICENSE (new), NOTICE (new), README.md, CONTRIBUTING.md, SUPPORT.md, clients/ts/README.md, clients/ts/package.json, deployments/Dockerfile, deployments/Dockerfile.goreleaser, docs/src/components/Hero.astro, docs/src/content/docs/index.mdx, docs/src/content/docs/why-wavehouse.md): the whole project moves from the MIT License to the Apache License 2.0 — a permissive→permissive change (Wave RF owns the copyright) that adds an explicit patent grant. Both LICENSE files carry the canonical Apache-2.0 text, a top-level NOTICE asserts Copyright 2026 Wave RF, and every textual reference is flipped: the README badge + text, CONTRIBUTING, SUPPORT, both Dockerfiles' OCI licenses label, the docs site's hero/stat card, the JSON-LD schema.org license URL, and the comparison table.
  • Ingest worker batches and flushes per table (internal/ingest/worker.go, internal/ingest/worker_test.go, docs/src/content/docs/ingest-pipeline.md (new), docs/src/config/sidebar.ts, tests/e2e/sdk/* , tests/e2e/fixtures/): closes #191. The worker previously kept a single shared batch across all tables, so a high-volume table tripping the 500-row size trigger could strand a low-volume table's rows in a batch that then waited out the 5s timer (and vice-versa) — the root cause of the batching.test.ts CI flake.
    • Per-table batching. A single JetStream consumer now feeds a dispatchLoop that parses just enough to route each message to a per-table goroutine (tableLoop), lazily spawned on first sight of a table. Each table has its own batch, deadline timer, and size trigger, so one table's traffic never delays another's. State is single-owner/lock-free (no mutexes on the hot path); flushes hand the goroutine a private row snapshot so the race detector stays clean.
    • Pipelined flushes with a bounded in-flight count (coalescing). At most one insert runs per table at a time; the next batch accumulates while the current one is in flight, and a size/timer trigger that fires during a flush is latched and runs the moment the slot frees. Overflow coalesces into a larger insert when a table is hot — bigger batches under load, which is exactly what avoids ClickHouse "too many parts" without a flush-rate floor. maxBatch/maxWait are now struct fields (overridable in tests); ackWait/maxAckPending/pullMaxMessages invariants are documented. Background DoubleAcks are tracked on a separate ackWg so the fsync-bound acks stay off the insert path, and shutdown drains bottom-up (table loops → acks) under a single deadline.
    • New contributor doc docs/src/content/docs/ingest-pipeline.md (linked in the sidebar): a deep-dive on the goroutine/channel/timer topology, the tableBatcher state machine, the three contexts, shutdown ordering, backpressure knobs, and the path to multi-instance scale.
    • E2E table isolation. Each e2e test file now owns its own ClickHouse tables (clicks_<suite>/events_<suite>/users_<suite>, generated from tests/e2e/sdk/tables.ts), bootstrapped by setup.ts; the static tests/e2e/fixtures/00*_*.sql files are removed (DDL now lives in tables.ts). Cross-file data contamination is structurally impossible, and the dlq.test.ts 5s sleep workaround is gone. Running the files in parallel (dropping maxWorkers: 1) remains deferred (tracked in #214 — per-table policy storage) — see the "Deferred" section of the ingest-pipeline doc — because several files still read-modify-write the single global policy document.
  • Health endpoints aligned to the current Kubernetes convention — /livez + /readyz canonical, /healthz a permanent alias, /health + /ready deprecated — plus a new public GET /v1/health SDK liveness ping (internal/api/router.go, internal/api/router_test.go, internal/api/health.go, internal/api/health_test.go, internal/api/boot_chain_test.go, internal/config/config.go, internal/config/config_test.go, cmd/wavehouse/main.go, cmd/wavehouse/health.go, cmd/wavehouse/health_test.go, internal/discovery/discovery.go, internal/discovery/discovery_test.go, scripts/orchestrator/main.go, deployments/Dockerfile, deployments/Dockerfile.goreleaser, tests/integration/boot_resilience_test.go, tests/e2e/sdk/setup.ts, tests/e2e/sdk/admin.test.ts, tests/e2e/sdk/auth.test.ts, clients/ts/src/{sys,types,index,errors,errors.test,namespaces.test,http.test}.ts, docs/src/content/docs/{api,deployment,architecture,development,sdk}.md). Addresses #144 (part 1).
    • Canonical names. Liveness is now GET /livez and readiness GET /readyz — the split the kube-apiserver adopted when it deprecated the older conflated /healthz (one endpoint can't distinguish "restart me" from "stop routing to me"). /healthz is kept as a permanent alias of /livez (it's the most widely-recognized name); /health and /ready are deprecated aliases for the v0.1.x line, scheduled for removal in v0.2.0. Handlers and response bodies are unchanged: /livez returns {"status":"ok"} (or {"status":"degraded",…} + 503 while boot-time schema discovery is still failing), /readyz returns {"status":"ready"} (or {"status":"not ready","error":…} + 503); kubelet only reads the status code, the bodies are for humans.
    • New GET /v1/health (public, content-free). A liveness ping returning a bodiless 200/503 (no JSON to encode, nothing to cache) with the same boot-aware logic as /livez but no ClickHouse check. It's deliberately a /v1 API route rather than a Kubernetes probe path: operators may filter /livez//readyz//healthz out at the reverse proxy, so the SDK relies on /v1/health staying public for its "is the server online" check. Readiness-based load balancing remains the proxy/LB's job (via /readyz), not the client's — so no /v1/ready was added.
    • Tooling repointed. WaveHouse's own probes target the canonical /livez so they don't ride a removable alias: the distroless health subcommand (and its Docker HEALTHCHECK), the E2E orchestrator's startup wait, and the SDK E2E setup probe. The prometheus.path reserved-collision check now covers /livez//readyz plus the three aliases, and the tracing span-skip middleware excludes all five probe paths and /v1/health.
    • SDK (breaking). The sys namespace now exposes only a content-free server-online check. wh.sys.health() calls GET /v1/health and returns Result<void> (was Result<Health> against /health) — branch on error, not data. wh.sys.ready() was removed: readiness is a per-call ClickHouse query that belongs to the load balancer / reverse proxy, not the client, and /readyz may be filtered at the proxy — probe /readyz directly for k8s/LB. The now-unused Health and Ready types were dropped from the public exports.
    • SDK DX. Result<T> gained an ok: boolean discriminant so callers can branch with if (result.ok) (mirroring fetch's Response.ok) instead of checking error === null — cleaner for content-free results like health(). Additive and backward-compatible: data/error are unchanged, so existing checks keep working.
    • Deferred (issue #144 part 2): per-dependency drill-down probes (NATS, ClickHouse, schema, DLQ) and the richer aggregate /readyz body were intentionally left out of this release to keep the change small.
  • Docs site auto-deploys to wavehouse.dev from the CI job, and is driven from the root Makefile through the pnpm workspace (Makefile, .github/workflows/ci.yml, docs/wrangler.jsonc, docs/scripts/branding/ (moved from scripts/branding/), AGENTS.md, docs/src/content/docs/development.md, docs/scripts/screenshot.mjs, .github/actions/setup-env/action.yml). Three threads, one PR:
    • Driven from the root Makefile via the pnpm workspace. docs/ (package wavehouse-docs) is a pnpm workspace package built and served straight from the root Makefile with pnpm --filter wavehouse-docs run <script> — no sub-Makefile. Root targets are verb-first and slot into the normal make help sections: build-docs, dev-docs, preview-docs, branding-docs, clean-docs, plus a hidden install-playwright-docs helper that lazily fetches the Chromium build rehype-mermaid + starlight-links-validator need (the --with-deps apt step gated on $CI). The SDK (clients/ts/) and E2E harness (tests/e2e/sdk/) are workspace packages too, driven the same way (build-ts, test-ts, …), and make ci / make tools / make build-all call these targets directly.
    • Branding pipeline relocated. scripts/branding/{mark.svg,lockup.svg,og.template.svg,generate.sh} → docs/scripts/branding/ since it's a docs-only concern; generate.sh now resolves the repo root via git rev-parse --show-toplevel instead of a hard-coded relative climb, so future moves don't re-break it. make branding-docs is the canonical entry point from root.
    • Auto-deploy from the CI job. Cloudflare Workers Builds can't build this site — rehype-mermaid renders diagrams to themed SVG at build time via headless Chromium, and the Workers Builds image has no browser (and no root to apt-install one), so astro build fails there. Since make ci already builds docs/dist/ on a runner with a cached Chromium, the deploy runs as tail steps of .github/workflows/ci.yml, reusing that artifact and gated on the whole pipeline passing: push to main runs wrangler deploy (production → wavehouse.dev); PR branches run wrangler versions upload, which publishes a per-version preview URL of the form <version-prefix>-wavehouse-docs.wave-rf.workers.dev and posts it as a sticky PR comment. Deploys skip when no docs-affecting files changed and on fork PRs. Requires CLOUDFLARE_API_TOKEN + CLOUDFLARE_ACCOUNT_ID repo secrets; Cloudflare Workers Builds must stay disconnected from the wavehouse-docs Worker so it doesn't double-deploy. docs/wrangler.jsonc gains the routes block for the wavehouse.dev custom domain, workers_dev: true + preview_urls: true so the per-version preview URLs keep working, observability: { enabled: true, head_sampling_rate: 1 }, upload_source_maps: true, and a $schema pointer for editor validation; compatibility_date bumped from 2025-01-01 to 2025-11-01. The worker (docs/worker/index.ts, delegating to cloudflare-md-router) deploys alongside the static assets so Accept: text/markdown content negotiation works in production, not just under wrangler dev. AGENTS.md §"Repository Automation" documents the deploy mechanism so future maintainers know where to look.
  • Go toolchain requirement bumped to 1.26.4 (go.mod): the go directive moves from 1.26.3 to 1.26.4 so CI's setup-go (which reads go.mod via go-version-file) installs a Go whose standard library clears the govulncheck findings GO-2026-5037 / GO-2026-5038 / GO-2026-5039 (fixed in 1.26.4); local builds were already on 1.26.4. Patch-level toolchain bump only — no source changes — and the released binaries pick up the patched stdlib too.
  • Polish on the /v1/admin/query HTTP proxy from PR review (internal/api/query.go, internal/api/query_test.go, docs/src/content/docs/api.md, docs/src/content/docs/architecture.md, docs/src/content/docs/sdk.md, docs/src/content/docs/configuration.md, docs/src/content/docs/deployment.md): tightening fixes from a clean-context review on PR #164.
    • Status mapping split. ClickHouse 4xx responses now map to HTTP 400 (caller-fault, the SQL was bad) and 5xx responses map to HTTP 502 (gateway-fault, the upstream had a problem), instead of every non-200 collapsing to 500. Admin tooling that retries on 5xx-but-not-4xx and dashboards that classify SLO violations get back the signal they need.
    • Content-Type pass-through on inline FORMAT. The fallback "ClickHouse returned a non-JSON body" branch (triggered by FORMAT CSV / FORMAT Pretty / etc., which override the URL-level default_format=JSON) now passes through the upstream Content-Type instead of stamping application/json. Confirmed empirically that the inline FORMAT wins at the protocol level; SDK consumers doing await response.json() would previously have crashed on the mislabeled CSV.
    • Single source of truth for the deadline. Dropped the duplicate http.Client{Timeout: 30s} — the per-request context.WithTimeout(r.Context(), 30s) already bounds the whole exchange including the body read, and the client-level Timeout silently capped any longer inbound context.
    • Response-size cap (64 MiB). Wrapped io.ReadAll(resp.Body) in an io.LimitReader(_, maxCHResponseBytes+1) with a 64 MiB cap, returning 502 on overflow. Admin-only endpoint, but a runaway SELECT * could still pin RAM on the API server, and a clear error beats a silent OOM.
    • Inbound-body cap (16 MiB). Wrapped r.Body in http.MaxBytesReader(w, _, maxRequestBodyBytes) for symmetry with the response cap. Oversize returns 413 with "request body exceeded N bytes" — distinct from "invalid json" so admin scripts can tell "you sent garbage" from "you sent too much."
    • Cache-Control: no-store and X-Content-Type-Options: nosniff set unconditionally at handler entry, so error responses carry both headers too — not only the 200 path.
    • Test allocation pruned. Added unexported maxResponseBytes / maxRequestBytes override fields on QueryHandler so size-cap tests can pin the overflow path with a tiny body instead of allocating tens of MiB per run.
    • Tests: TestQueryHandler_ForwardsCHError becomes table-driven across 4xx/5xx → 400/502 mapping; TestQueryHandler_ForwardsRawBodyAndContentType (renamed) now also asserts the upstream MIME passes through; TestQueryHandler_ResponseSizeCap and TestQueryHandler_RequestBodyCap are new and pin the 64 MiB / 16 MiB safety nets. A shared assertSecurityHeaders helper checks Cache-Control: no-store and X-Content-Type-Options: nosniff on every response path (200, 4xx CH error, 5xx CH error, 413 oversize-request, 502 oversize-response, FORMAT pass-through).
  • BREAKING: POST /v1/admin/query is now a thin HTTP proxy to ClickHouse, not a clickhouse-go Query/Exec wrapper (internal/api/query.go, internal/api/query_test.go, internal/api/clickhouse_exec.go (new), internal/api/clickhouse_exec_test.go (new), cmd/wavehouse/main.go, clients/ts/src/sql.ts, clients/ts/src/client.ts, clients/ts/src/namespaces.test.ts, tests/integration/query_test.go, docs/src/content/docs/api.md, docs/src/content/docs/architecture.md). The handler now POSTs the SQL string verbatim to <scheme>://<host>:<http_port>/?default_format=JSON&date_time_output_format=iso&database=<db> with X-ClickHouse-User / X-ClickHouse-Key auth headers (matching the ingest worker's convention), reads ClickHouse's response, and forwards just the data array (or [] for mutations that return an empty body, or a JSON error envelope of shape {"error":"<trimmed ClickHouse message>"} for non-200 responses — the upstream plain-text body is wrapped inside the standard error envelope, not forwarded raw). Closes the isMutation / mutationVerbs / containsMutationVerbAtTopLevel / stripLeadingSQLComments / transformRow ~150-LOC machinery that lived in query.go to classify verbs and route between driver.Exec and driver.Query. Those primitives are still needed by the structured-query and pipes handlers (clickhouse-go's native Query() errors on no-result-set statements), so they moved into a new internal/api/clickhouse_exec.go and are called via executeCHQuery(ctx, conn, sql, params) — pipes and structured-query no longer instantiate a temporary QueryHandler just to borrow executeQuery. Three wins: (a) any ClickHouse-accepted SQL works, including multi-statement input (SELECT 1; TRUNCATE t) and arbitrary DDL/DML/SYSTEM verbs (current and future) without WaveHouse code changes; (b) ClickHouse's own error messages reach the admin verbatim — what they want from an escape hatch; (c) the handler is ~50 LOC of forward-and-shape-response with no SQL-parsing surface. Breaking implications: the request body no longer accepts a params array (ClickHouse's HTTP interface uses named {name:Type} binding, not positional ?). Inline literals into the SQL string, or use the structured query endpoint for safe binding from user inputs. The TypeScript SDK's sql() method drops its params?: unknown[] argument; callers of client.sql(sql, params) need to inline. NewQueryHandler signature changes from (driver.Conn) to (endpoint, username, password, database string). The response also gains a Cache-Control: no-store header so no downstream layer caches raw-SQL results. New tests in clickhouse_exec_test.go keep isMutation + Exec/Query routing + the transformRow UUID/time conversions covered for the structured-query and pipes paths that still need them; new tests in query_test.go pin the proxy contract (SQL forwarded verbatim, data array extracted, empty-body → [], ClickHouse error forwarded, security headers set on every response path, request context cancel propagates).
  • POST /v1/admin/query no longer caches, singleflights, or bypasses cache for mutations (internal/api/query.go, internal/api/query_test.go, internal/api/cache_key.go (new), internal/api/cache_key_test.go (new), cmd/wavehouse/main.go, docs/src/content/docs/api.md, docs/src/content/docs/architecture.md). Note (largely superseded — see the HTTP-proxy entry above): the structural details below (QueryHandler struct shape, NewQueryHandler(conn) signature, in-handler verb classifier, X-Cache: MISS header) describe the state immediately after the cache-strip commit. The subsequent HTTP-proxy refactor changed NewQueryHandler to (endpoint, username, password, database string), dropped the driver.Conn field, moved the verb classifier to internal/api/clickhouse_exec.go for the structured-query/pipes consumers that still need it, and uses Cache-Control: no-store in place of X-Cache. The cache-strip motivation below still stands; the historical implementation specifics follow. With the endpoint now an admin-only escape hatch (see the Security entry below), the L1/L2 + singleflight + mutation-cache-bypass machinery added in the original raw-SQL handler is dead weight: admin calls are infrequent, often mutations, often one-shot, so the cache hit rate is effectively zero and singleflight has no concurrent-request herd to collapse. QueryHandler shrinks to { CHConn driver.Conn }; Handle is parse → executeQuery → marshal → write. NewQueryHandler(conn) is the new signature (was NewQueryHandler(conn, *cache.TieredCache, time.Duration)). The X-Cache: HIT/MISS response header is gone from this endpoint — every response is a fresh ClickHouse round-trip, no header needed to disambiguate. The verb-classification machinery (isMutation, mutationVerbs, the comment-aware CTE parser) STAYS; it's correctness, not performance — clickhouse-go's driver.Query() errors on statements that return no result set, so we still route mutations through driver.Exec. The shared queryCacheKey helper moved to a new internal/api/cache_key.go since it's now consumed only by the structured query (/v1/tables/{table}/query) and pipes (/v1/pipes/{name}) handlers, which keep their caching + singleflight intact — those are the high-QPS read paths where the machinery earns its keep. Two now-obsolete tests removed: TestQueryCacheKey_Deterministic (moved to a fresh cache_key_test.go since the helper still has callers) and TestQueryHandler_MutationBypassesCache (no cache to bypass). The mutation-routing tests (TestExecuteQuery_MutationRoutesToExec, TestExecuteQuery_SelectRoutesToQuery) stay — they pin the still-required Exec-vs-Query classification.
  • JWT auth middleware polish from PR review (internal/auth/auth.go, internal/auth/auth_test.go, docs/src/content/docs/api.md, docs/src/content/docs/sdk.md): docs + test follow-ups from review of the RBAC work; no behavior change.
    • Documented that the Authorization header takes precedence over the ?token= query param: the query param is only a fallback for header-less clients (browser EventSource), so a token in that more log-leakable URL never overrides an explicit header credential, and it is stripped from the URL after extraction (api.md). sdk.md gains a "How the token is transmitted" note — header on REST, ?token= on SSE, server prefers the header when both are present.
    • Replaced the parse-path TODO with an in-code comment: the middleware returns no error by design (authentication is decoupled from authorization). A failed/invalid/expired parse — and the effectively-unreachable non-MapClaims branch — all fall through to the single fail-safe that records the token error via WithAuthError, leaving the request roleless for a downstream gate to surface as 401 vs 403.
    • Added TestMiddleware_JWKSEmptyKeySet_TokenDoesNotAuthenticate: an empty {"keys":[]} set constructs at boot but validates no token at request time, so every token falls back to the roleless default_role (fail-closed). It signs an EdDSA token so the token passes the JWKS algorithm allowlist and actually reaches the empty key set, rather than being rejected by the alg gate first.

Added

  • SUPPORT.md + SECURITY.md are now one click from every public surface; GitHub Discussions is the new home for usage questions (SUPPORT.md (new), README.md, CONTRIBUTING.md, .github/ISSUE_TEMPLATE/bug_report.md, .github/ISSUE_TEMPLATE/feature_request.md, docs/src/content/docs/index.mdx, docs/src/components/Footer.astro, docs/src/config/sidebar.ts, AGENTS.md). Closes #154, #155. New SUPPORT.md at the repo root sets honest alpha-stage expectations: best-effort response, 1–2 business days for an initial response on bugs/features/usage questions; security reports prioritized per SECURITY.md (48-hour acknowledge / 5-business-day initial assessment); no auto-close on stale threads. Usage questions ("how do I…") route to GitHub Discussions → Q&A (Discussions was just enabled on the repo); bug-report Issues filed for usage questions get redirected. No Discord/Slack today. README gains a Security section + a Support section near the bottom; CONTRIBUTING gains a top-of-file pointer; both issue templates gain a footer pointing at SUPPORT.md, Discussions, and the private SECURITY channel; the docs landing page cross-links SECURITY/SUPPORT from its closing call-to-action band; the Starlight footer's Project column adds Discussions + Support + Security so they're reachable from every page on wavehouse.dev; the sidebar's Contributing group adds Discussions + Support next to the existing Security Policy link. AGENTS.md §"Governance Files" documents the SLA so internal expectations match the public statement.

  • Biome for TypeScript lint + format (biome.json, root package.json, all clients/ts and tests/e2e/sdk sources): one fast formatter + linter replaces ad-hoc TS formatting, wired into the dev and CI flow (make targets, biome check --error-on-warnings). Includes a repo-wide format pass.

  • Cross-suite coverage enforcement (.testcoverage.yml, scripts/cov/, clients/ts/vitest.config.ts, tests/e2e/sdk/vitest.config.ts, tests/e2e/fixtures/config.yaml): make ci now gates coverage across the Go unit/integration/e2e and TypeScript unit/e2e suites (stitched by scripts/cov), and the previously-disabled SSE public-events streaming test is re-enabled (closes #153). Continues the SDK coverage work tracked in #176.

Removed

  • BREAKING: the ingest pipeline is now insert-only; the action field on ingest.<table> envelopes is gone from the wire format (internal/ingest/bento.go, internal/ingest/bento_test.go, tests/integration/ingest_test.go, tests/integration/dlq_test.go, docs/src/content/docs/architecture.md, docs/src/content/docs/api.md, AGENTS.md): the worker accepts {table_name, scope, received_timestamp, data} envelopes (the existing EventMessage shape) and bulk-INSERTs them. All non-insert mutations (DELETE/UPDATE/TRUNCATE/DROP/ALTER/REPLACE/…) must go through POST /v1/admin/query under the admin role (the raw-SQL endpoint moved under /v1/admin/* in a follow-up commit on this same Unreleased section — see the Security entry below; the RawSQL: true policy field is gone too). The rationale: our policy engine authorizes mutations by evaluating row/column rules against the payload of an operation, which works for inserts but not for predicate-driven mutations (we can't prove a WHERE clause is satisfiable only for rows the caller is allowed to touch, and WHERE 1=1 would otherwise nuke a table). Rather than ship a partial enforcement story, the pipelined surface narrows to the shape policy can authorize today; /v1/admin/query becomes the single sanctioned admin-equivalent surface for everything else. The embedded NATS server runs with DontListen: true (internal/mq/embedded.go), so the only Publishers reachable on ingest.> are in-process Go code (today, only the HTTP /v1/ingest/{table} handler), and EventMessage carries no action field so the legitimate producer never sets one. The delete-envelope DLQ shape and the Wave-DLQ-Type NATS header it required are gone with this — dlq.<table> now carries only insert-failure payloads (the inner data object), so DLQ consumers no longer need to discriminate. The jsInput.chConn and jsInput.js fields are dropped along with the chConn driver.Conn parameter on StartIngestWorker. Re-introducing pipelined / structured mutations is deferred until the policy engine can authorize predicates — track separately if/when scheduled.
  • Hub wildcard subscriptions (internal/api/hub.go, internal/api/hub_test.go): dropped the NATS-style * / > pattern matching from Hub.Broadcast, the wildcard pattern loop, the sent dedup map, the matchTopic helper, and the eight wildcard tests (plus TestMatchTopic). After the #89 MVP cuts every producer publishes a concrete ingest.<table> subject and the SDK only ever subscribes to one concrete subject, so the wildcard fan-out was unused machinery. Closes #100 (part of #87). Net −210 lines (mostly tests).
  • project-orchestrator.yml workflow + its three composite-action artifacts (.github/workflows/project-orchestrator.yml, .github/actions/board-upsert-status/, .github/actions/set-linked-issues-status/, .github/scripts/board-fetch-item.sh, AGENTS.md, CHANGELOG.md): −887 lines net. The orchestrator was the largest single source of cross-trigger complexity on this repo (3-4 workflow_run-chained runs per PR push, statusCheckRollup GraphQL perms quirks, integration-token NONE for private-org members) for behaviour that is mostly either provided natively by GitHub or a one-click manual operation on a 4-person team. Replaced by: reviewer-assign step in housekeeping.yml that fires once on pull_request_target: opened / ready_for_review (not per-synchronize, so it doesn't re-spam after dismiss_stale_reviews_on_push), plus GitHub's native Projects v2 workflows (Auto-add to project, Item added, Pull request merged) configured in the project UI. Trade-offs explicit in the PR body: drafts no longer auto-flip on bot-clean, CHANGES_REQUESTED doesn't auto-move the board card, linked-issue card mirroring is dropped. AGENTS.md §"Governance Files" + §"Task Board state machine" + §"Review tooling reference" all rewritten to match. dependabot-automerge.yml trimmed in parallel: no more board-upsert step (native handles placement), PROJECT_BOARD_TOKEN guard removed (no longer used in this workflow), reviewer list sourced from board-config.env's ADMINS via replace(), major-bump comment uses the marker-comment upsert pattern from housekeeping.yml.
  • STATUS_* and old ADMINS consumers in board-config.env — STATUS option IDs had only orchestrator-side consumers and are now unreferenced. ADMINS was restored to board-config.env after the initial orchestrator-removal commit dropped it (Gemini and Claude both flagged the resulting drift across three inlined copies); both housekeeping.yml and dependabot-automerge.yml now load ADMINS from board-config.env. admin-approval.yml keeps its own inline copy with the documented latency-avoidance reasoning.
  • WebSocket streaming transport — SSE is now the sole real-time transport (clients/ts/src/stream/ws.ts, clients/ts/src/stream/ws-manager.ts, internal/api/stream_ws.go deleted; internal/api/stream_sse.go → internal/api/stream.go; the transport option is gone from ClientConfig / StreamOptions; the streaming route is now /v1/stream, previously /v1/stream/ws + /v1/stream/sse). WebSocket support was built earlier in this same Unreleased cycle and is removed before any tagged release, so there is no user-facing break. Rationale: one transport keeps the surface small, and SSE behaves more consistently across corporate proxies and networks that block WebSocket upgrades. The SDK streaming roadmap that motivated this cleanup (live-vs-replay, multi-subscription replay, header-based SSE auth, NDJSON upload, and more) is tracked under #194.

Changed

  • Config and policy examples in every supported format, plus reference-page chrome odds and ends (docs/src/content/docs/{configuration.mdx,access-control.mdx} (renamed from .md), docs/src/content/docs/{architecture,development,pipes,claude-code}.md, docs/src/styles/global.css): the example config file is now YAML / Environment synced tabs (syncKey="cfg"; every WH_* name cross-checked against internal/config/config.go — the config FILE is YAML-only by struct tags, so no JSON tab there), and the complete policy example is YAML / JSON synced tabs (syncKey="policy"; the JSON twin machine-verified equal to the YAML). Code frames that represent real files gained title= headers (config.yaml, policy.yaml, .env, ~/.claude.json, …) and every frame — terminals included — now shows a quiet language badge at its top-right (inside the title bar where one exists; ScrollHints copies pre[data-language] up to the frame since CSS attr() can't cross elements; plain-text-family frames stay unlabeled, and the badge yields to the hover copy button on headerless frames). A follow-up round, on review feedback: every substantive policy fragment on the access-control page is now a YAML/JSON tab group (7 groups total, each JSON twin machine-parsed equal to its YAML), the configuration tables returned to separate YAML Key / Env Var columns (the stacked merge predated wide-table support; 4-column tables now ride the breakout track), code-format tab groups DOCK onto their code frame (flush bottom edge, shared corner — a floating segmented control above an unrelated box read as disconnected), external-link ↗ markers extend to prose and footer links (absolute http(s) hrefs are off-site by construction), and the diagram zoom chip became a proper masked stroke icon on a translucent blurred chip instead of a ⤢ text glyph. A fifth round (review feedback + an automated content/visual sweep) then: scoped the cluster-stroke padding to diagram scrollers only (on table cards it had opened a page-bg seam between the card border and the header band), cleared the zoom stage's leftover edge-fade attributes on every open (a prior 1:1 pan session painted a phantom fade on the next fitting diagram), settled the docked-group chrome (no title=/filename comments inside format tabs — the lead-in prose names the file, the tab names the format; the configuration Environment tab became dotenv-style ini so it stops rendering as a titled terminal), added the missing cfg tab groups (access-control's policy-bootstrap fragment, pipes' .sql-seeding fragment — pipes.md → .mdx), pretty-printed the wide single-line JSON wire examples (api.md responses; the SSE data: lines deliberately stay wire-accurate single lines and wrap visually), manually wrapped >100ch comment/log lines across deployment/architecture/sdk pages (the WARN/ERROR log examples now break with hanging indents under wrap=false), excluded prev/next pagination cards and the footer brand sentence from the external-link ↗ (a second glyph inside the Next card read as broken), and made the slim footer wrap deliberately below ~86rem (links become their own left-aligned row; the squeezed pane can't fit the one-line bar there). A sixth, dedicated pass killed the recurring "shadow with nothing to scroll" artifact by construction rather than by threshold: a WRAPPED code block (pre.wrap) can never meaningfully scroll sideways (pre-wrap + break-word guarantee every line fits — anything scrollWidth reports past clientWidth is hanging wrap-point whitespace or zoom/font-timing rounding), so the edge-fade updater now hard-strips both fade attributes from wrapped pres unconditionally; classifications also re-derive once document.fonts.ready resolves, so fallback-font metrics can't go stale. Code blocks then adopted the tables' earn-the-width rule (the PR author's "if it's going to wrap anyway, no point being so wide" observation): a block widens only when its longest line actually FITS the wide track (measured batched, nowrap), or when it's wrap=false and scrolls — a block whose longest line exceeds even the full pane keeps the prose width instead of stranding two words on a continuation line at maximum width. Long trailing comments that previously forced the wrap moved above their lines across access-control/deployment/development/sdk examples. A dedicated verification agent then proved the artifact dead: 42,696 element scans — 18 pages × 141 widths (320–2560, step 16, resize-triggered re-classification), every tab option clicked at 3 widths, the full zoom lifecycle (fit / 1:1 / pan / toggle-back / next-diagram) on 16 diagrams at 2 widths, soft-navigation chains, and blocked- and delayed-webfont variants — zero violations of the invariant (fade attribute ⇒ real overflow > 12px, and never on a wrapped pre). A seventh round, on CodeRabbit review: code-ONLY tab groups (the YAML/JSON/env switchers) now earn the wide track as a unit — classifyCode() previously scanned only direct .expressive-code children, so tabbed samples could never receive data-wh-wide; digging in exposed a second, older hole — the natural-width probe set white-space: pre on the pre, but EC applies wrapping on the descendant code spans (pre.wrap .ec-line .code), so the probe measured the WRAPPED width and every default-wrap block was unwidenable (only wrap=false blocks ever took the wide track); the wrap class itself now comes off for the batched measurement, which also lets the long-lined wrap blocks across the api/development/sdk pages widen as the sixth-pass rule always intended (verified: 19 pages × 5 widths, zero overflow / fade-on-wrap / stale-fade violations); all panels are measured (inactive ones get a momentary inline display:block — layout never reaches paint mid-batch) and the group pins to its widest panel’s natural width so the frame doesn’t resize across tab switches (groups with prose in any panel keep the reading measure, and a tab switch now refreshes the edge fades — a pre revealed from hidden measured 0×0 when fades were last stamped). The bare table selector left ScrollHints’ scroller list (after wrapping, tables scroll on their .wh-table-wrap — a fade stamped on the non-scrolling <table> itself could never clear), the phone Mermaid wrapper-scroller rule extends to class/state diagrams (they had the 540px no-JS min-width floor without the matching overflow-x: auto), and the search-dialog backdrop dropped a reintroduced hand-written -webkit-backdrop-filter twin (the same LightningCSS author-managed-prefix trap documented at the glass header — the build emits the twin itself). Bare issue references in prose became links (#97, #223, #224, #279). The mobile icon-only search trigger joins the menu button's circular control family (it rendered as an unstyled square), and external sidebar links (Contributing Guide, Discussions, …) carry a ↗ marker.
  • SDK docs split into six topic pages, with package-manager-synced install tabs (docs/src/content/docs/sdk/{index.mdx,queries.md,streaming.md,pipes.md,admin.md,reference.md} (new), docs/src/content/docs/sdk.md (deleted), docs/src/config/sidebar.ts, docs/src/components/Hero.astro): the 806-line monolith becomes /sdk (overview: install, quick start, client setup, Result<T>, section cards) plus /sdk/{queries,streaming,pipes,admin,reference}, under a nested "TypeScript SDK" sidebar group (which also keeps the llms.txt page ordering coherent — the LLM outputs read the sidebar). Topic-first URLs are the multi-language plan on record: when a second SDK language lands, the shared usage pages grow <Tabs syncKey="lang"> code tabs and each language gets its own setup/caveats page, so the topic URLs never churn. Installation became <Tabs syncKey="pkg"> (pnpm / npm / yarn / bun / deno / CDN), and the landing hero's click-to-copy install chip now follows that synced choice — Starlight persists the selected tab label in localStorage under its syncKey, the hero script maps it to the matching one-liner (CDN and anything unknown keep the pnpm default). Every inbound link from other pages already pointed at bare /sdk (still the overview); the three intra-page #streaming anchors became /sdk/streaming links, and starlight-links-validator gates the rest. A follow-up adds an "Imports & Runtimes" section to the overview — import-line tabs synced to the same pkg choice (the CDN reader sees the esm.sh import everywhere, including a fully-tabbed Quick Start), Node CJS require() and the WaveHouse IIFE global documented from the shipped exports map, and a verified runtime-support note (streaming needs an EventSource polyfill in Node — the global is still flag-gated as of Node 26, checked empirically; non-streaming works on Node 18+). Each topic page points CDN readers at the section.
  • Pre-push CI marker is tree-keyed, not commit-keyed (Makefile, .githooks/pre-push, .claude/hooks/agent-bash-gate.sh, AGENTS.md). The make ci → git add → git commit → git push flow no longer false-positive-blocks — committing advances HEAD but doesn't change the tree CI validated. make ci writes tmp/ci-passed-tree-<TREE-SHA> computed against a throwaway GIT_INDEX_FILE (so untracked files are captured without touching the real index); pre-push and the agent-bash-gate resolve each pushed ref's ^{tree} and match. Skipped on CI runners (they don't push). review-passed stays commit-keyed — a new commit needs a new review.
  • Worktrunk seeds new worktrees from main (.config/wt.toml). Added wt step copy-ignored to post-start ahead of make tools, so wt switch --create <branch> copies .bin/ and the three node_modules/ trees (~1 GB combined) from main instead of cold-starting pnpm install --frozen-lockfile × 3. [step.copy-ignored] exclude keeps tmp/, data/, and vitest's coverage/ out.
  • Claude PR review is now manual-only (.github/workflows/claude-review.yml, AGENTS.md): dropped the pull_request trigger (was firing on opened/synchronize/reopened/ready_for_review) so the workflow only runs on @claude or /review comments from trusted reviewers, or workflow_dispatch. "Claude review" was never a required status check (the ruleset gates on Admin approval / CI / PR housekeeping), so this changes who pays the action budget — every push no longer triggers a review automatically; reviewers explicitly opt in via mention after they've looked at the diff themselves. Falls out: the workflow header's pull_request_target OIDC paragraph dropped (only applied to the auto-trigger path), concurrency's cancel-in-progress is now unconditionally false (manual triggers should always queue, never cancel an in-flight review), the workflow-level if: loses its pull_request branch + dependabot-author exclusion, and the gate-step case drops the pull_request) arm + its two PR_EVENT_* env vars (gh pr view --jq .headRefOid is hoisted out of the case since both remaining paths need it identically). The trust gate stays — it's more load-bearing under manual-only, since issue_comment runs with full repo secrets and a trusted reviewer commenting @claude on a fork PR would otherwise check out untrusted fork code with write tokens. AGENTS.md synced in three places: §"Review Response" step 6 (Claude moved from synchronize-rerun to PR-comment-mention), the review tooling reference table (Claude row now "Manual-only" with "No" in the re-runs column), and §"Repository Automation" §"Code review" entry (rewritten to lead with manual-only).
  • BREAKING: the SSE streaming endpoint replaced ?topic= with ?table= (issue #100, option B). The NATS subject convention no longer leaks into the public HTTP API; the server builds ingest.<table> internally. SSE returns 400 Bad Request when ?table= is missing or fails the ^[a-zA-Z_][a-zA-Z0-9_]*$ safe-identifier check (the same regex internal/ingest and internal/query use for SQL identifiers — crucially this rejects NATS wildcard characters * and > before they reach the gap-fill FilterSubject).
  • TypeScript SDK (@wavehouse/sdk) updated to the new contract: SSEOptions carries table instead of topic. The high-level SDK API (wh.from('clicks').stream()) is unchanged.
  • Doc-sync for the ?topic= → ?table= rename: docs/src/content/docs/getting-started.md quick-start now shows ?table=clicks (the previous ?topic=ingest.clicks example would 400 against the new contract).
  • Boot-time schema discovery no longer crashes the binary (cmd/wavehouse/main.go, internal/api/health.go, internal/discovery/discovery.go, docs/src/content/docs/api.md, docs/src/content/docs/deployment.md). Previously, any error from the initial SchemaRegistry.Refresh — connection-refused, missing database, transient network blip — caused cmd/wavehouse to call os.Exit(1). The supervisor would restart the process every ~10s in an unbounded loop, port 8080 never bound, and operators got connection refused on probes even when ClickHouse was otherwise healthy. Now the first Refresh failure is non-fatal: a new api.BootState is set with the diagnostic, the server still binds :8080, and a background goroutine calls the new SchemaRegistry.RetryRefresh(ctx, 2s, 60s, onAttempt) with exponential backoff until success or shutdown. While BootState.Err() is non-nil, /health returns 503 with {"status":"degraded","error":"…"} so an operator can curl /health to learn why the gateway isn't ready to serve traffic instead of grepping a restart-loop log; /ready returns 503 with {"status":"not ready",…} for the same reason. Once BootState.Set(nil) fires (either from the initial sync Refresh or from SchemaRegistry.RetryRefresh success), /health stays at 200 OK for the rest of the process lifetime — that flip is sticky and reflects "boot completed once." /ready, by contrast, remains conditional on current ClickHouse reachability: it can flip back to 503 on transient runtime blips after boot, since readiness is a probe of "can I serve traffic right now," not "did I ever boot." Resolves #95.
  • Frontend toolchain pinned: pnpm 10.33 → 11.1.3, Node 22 LTS via .nvmrc (.nvmrc, .github/actions/setup-env/action.yml, clients/ts/package.json, clients/ts/pnpm-workspace.yaml, docs/package.json, docs/pnpm-workspace.yaml, tests/e2e/sdk/package.json, tests/e2e/sdk/pnpm-workspace.yaml, AGENTS.md, README.md, docs/src/content/docs/development.md). Closes #160. Three TS workspaces (clients/ts, tests/e2e/sdk, docs) plus the setup-env composite action bump from pnpm@10.33.0 to pnpm@11.1.3. pnpm 11 deprecated the pnpm field in package.json — onlyBuiltDependencies moved into a new per-workspace pnpm-workspace.yaml and was renamed to allowBuilds: with a map-of-booleans syntax (each workspace ships its own file even though they're not a single monorepo — having them in-tree shadows any stray ancestor pnpm-workspace.yaml, which is how pnpm 11 walks the directory tree). clients/ts and tests/e2e/sdk allow esbuild; docs additionally allows sharp (Astro image pipeline) and workerd (Cloudflare Workers runtime). New repo-root .nvmrc pins Node 22 LTS — matches the setup-env action's existing node-version: "22" — and docs/package.json#engines.node bumps from >=20.0 to >=22.0 so corepack / volta warn local devs before CI does. The original issue's open question about electron-winstaller resolved to a contamination from a stray pnpm-workspace.yaml in the developer's home directory, not a real WaveHouse transitive — correctly omitted from allowBuilds. The Node 26 Vitest crash mentioned in the issue is no longer reproducible after the recent vitest@^4 bump in clients/ts (#103); the .nvmrc still lands because pinning has independent value (CI parity + Node-major-surprise guard).
  • pnpm workspace consolidation: a single root pnpm-workspace.yaml + lockfile (pnpm-workspace.yaml, pnpm-lock.yaml, root package.json; removed clients/ts/pnpm-workspace.yaml, docs/pnpm-workspace.yaml, tests/e2e/sdk/pnpm-workspace.yaml and their per-project pnpm-lock.yaml files): the three TS projects (clients/ts, docs, tests/e2e/sdk) are now real workspace members under one root install with a shared catalog: for common dev dependencies. The in-repo root workspace file still shadows any stray ancestor pnpm-workspace.yaml, and the minimumReleaseAge supply-chain guard is preserved at the root.
  • Single root Makefile; removed the per-subproject docs/Makefile (Makefile, docs/Makefile deleted): one top-level Makefile drives Go plus all TS workspaces with verb-first targets and a first-class SDK target; docs build/lint/format now run from the repo root instead of make -C docs.
  • engines.node normalized to >=22 across every workspace package.json (package.json, clients/ts/package.json, docs/package.json, tests/e2e/sdk/package.json): the repo root and clients/ts still declared >=18 even though CI, .nvmrc, and docs already required Node 22 — the declared floor now matches the .nvmrc pin everywhere (tests/e2e/sdk previously declared none). The redundant packageManager field was dropped from docs/package.json since the root package.json is authoritative for the workspace.

Security

  • BREAKING: raw SQL moves from POST /v1/query to POST /v1/admin/query and is gated on the admin role (internal/api/router.go, internal/api/router_test.go, internal/api/query.go, internal/api/query_test.go, internal/policy/policy.go, internal/policy/policy_test.go, cmd/wavehouse/main.go, docs/src/content/docs/api.md, docs/src/content/docs/architecture.md, docs/src/content/docs/getting-started.md, docs/src/content/docs/development.md, docs/src/content/docs/why-wavehouse.md, clients/ts/src/sql.ts, clients/ts/src/types.ts, clients/ts/src/client.test.ts, clients/ts/src/namespaces.test.ts, tests/integration/query_test.go, tests/e2e/sdk/admin.test.ts, tests/e2e/sdk/setup.ts, tests/e2e/sdk/query.test.ts, internal/ingest/bento.go, README.md, SECURITY.md, AGENTS.md, .gemini/styleguide.md, .github/prompts/pr-review.md). Previously, raw SQL was mounted at /v1/query under the /v1 auth middleware with the authorization decision deferred to the handler — any caller with a policy.RolePermissions.raw_sql: true grant on any table could submit raw SQL and slip through as a non-admin. The route now lives under the /v1/admin/* group, so its URL telegraphs the access level, and it shares the surrounding RequireAdmin gate with the rest of that tree (policy CRUD, pipes CRUD). Raw SQL has no per-statement scope check (we cannot authorize predicates without a full SQL parser), so the admin gate is the entire authorization story; a separate, tighter gate just for raw SQL would be redundant, since the whole /v1/admin/* tree already requires the admin role. The policy.RolePermissions.raw_sql (RawSQL) field is removed outright from Policy/ResolvedPermissions and from the Evaluate return value — operators with raw_sql: true in policy.yaml will see a YAML-load warning for the unknown field but the field is otherwise ignored; the equivalent capability is now expressed only as "issue the JWT with the admin role." The in-handler PolicyStore plumbing and the if h.PolicyStore != nil { … } raw-SQL check inside QueryHandler.Handle are deleted along with the field. A new router-level test (TestNewRouter_RawSQLAdminGate) pins the contract: admin reaches the handler, while service, viewer, and no-role requests are all 403 (service is no longer privileged). The four obsolete handler-level policy tests (TestQueryHandler_Policy*, TestQueryHandler_NoPolicyAllowsAll) are removed. The shared /v1/admin/* gate is declared once as a requireAdmin (RequireAdmin) local at the top of the /v1 route closure. The normal surfaces for non-admin callers — POST /v1/ingest?table={table}, POST /v1/query?table={table}, GET/POST /v1/pipes/{name} — are unchanged.
  • Per-pipe allowed_roles now fails closed on an empty role (internal/api/pipes.go, internal/api/pipes_test.go, docs/src/content/docs/api.md): PipesHandler.Execute enforced a pipe's allowed_roles allowlist only when the request carried a non-empty role, so an empty or absent role skipped the check and the restricted pipe was served. Per-pipe allowed_roles is the only authorization gate on the execute path (GET/POST /v1/pipes/{name} sit outside the /v1/admin/* RequireAdmin gate), so any roleless request reached a restricted pipe unchecked — triggered whenever a request carried no token, or a token missing the configured auth.role_claim (either way the resolved role is empty). Removed the if role != "" guard so an empty role flows into the scan, matches nothing, and returns 403; empty allowlist entries are skipped so a stray "" can't authorize an empty role. The gap that hid this — every prior role test set a non-empty role — is closed by consolidating the four standalone role tests into a table-driven TestPipesHandler_Execute_RoleAuthorization matrix that pins the empty/absent-role rows, plus a focused TestPipesHandler_Execute_RestrictedPipe_EmptyRoleDenied regression. Closes #159.
  • Access-control policies reject empty role names and never match a roleless request to an empty role key (internal/policy/policy.go, internal/policy/policy_test.go): the #159 step-3 audit (cross-check internal/policy/ against empty roles) surfaced the policy-side twin of the empty-allowed_roles-entry footgun. Evaluate did a direct rolePerms[role] lookup, so a stray "" role key in policy.yaml would have authorized any request whose resolved role is empty (no token, or a JWT missing auth.role_claim) on the policy-gated paths (POST /v1/query?table={table}, POST /v1/ingest?table={table}, the SSE stream). Two complementary guards: Validate now rejects empty/whitespace role keys at write/bootstrap time, and Evaluate skips the direct key lookup for an empty role so a roleless request can only be authorized by the configured admin_role (or a default_role that resolves to a real listed role) — fail-closed even if a malformed policy reaches the engine from KV. Regression tests pin both guards. Part of #159.
  • minimumReleaseAge: 10080 (7 days) on every pnpm workspace (clients/ts/pnpm-workspace.yaml, docs/pnpm-workspace.yaml, tests/e2e/sdk/pnpm-workspace.yaml): pnpm 11 will refuse to install any package published in the last seven days, giving npm and security researchers time to flag a compromised release before it lands in our lockfile. Existing locked versions are grandfathered. For a one-off override on an urgent hotfix release, list the package under minimumReleaseAgeExclude: in the same file. Part of #160.
  • Structured-query / pipes cache no longer collides on string params containing NUL bytes or on string-vs-int values with the same textual rendering (internal/api/cache_key.go, internal/api/cache_key_test.go). The prior queryCacheKey framing wrote "\x00%v" between params, so ("SELECT 1", ["foo\x00bar"]) and ("SELECT 1", ["foo", "bar"]) produced the same SHA-256 digest — a cached row from one query could be served for the other (cross-query data exposure if either tuple could be reached by a caller of the structured-query or pipes endpoints with user-controllable string inputs). A separate construction reached the same boundary by ending the raw SQL with bytes that mimicked a param frame. Reframed each section with a 1-byte type marker plus 8-byte big-endian length, and JSON-marshalled each param as {type, value} so the type tag also separates "42" (string) from 42 (int). Table-driven TestQueryCacheKey covers the NUL-collision regression, the SQL-mimics-a-param-frame regression, and the type-distinct regression. Net effect on deploy: a one-time cold cache for /v1/tables/{table}/query and /v1/pipes/{name} consumers; raw-SQL has no cache so no impact there. Reported by CodeRabbit on PR #164 with empirical digest reproduction.

Fixed

  • policy.NewStore fails loud on a broken bootstrap, and policy.file_path no longer has a default (internal/policy/store.go, internal/policy/store_test.go, internal/config/config.go, internal/config/config_test.go, config.yaml, docs/src/content/docs/configuration.md, docs/src/content/docs/deployment.md, tests/e2e/fixtures/policy.yaml (new), scripts/orchestrator/main.go). The old NewStore swallowed every bootstrap failure — missing file, malformed YAML/JSON, Validate rejection, KV Put failure — as a Warn log, leaving the cache nil. With the RBAC work elsewhere in this section making Evaluate(nil)/IsAdmin(nil) deny everyone (including the admin role, no implicit grant), that masked path turned into a silent total lockout: the process bound :8080 and /health returned 200, but every authenticated request 403'd. The masking test TestStore_NewStore_BootstrapMissingFileIsNotFatal codified the silent swallow as if it were intentional (require.NoError + assert.Nil(store.Get())), which is why the bug shipped — internal/api/, tests/integration/, and most other call-sites use policy.NewMemoryStore and bypass NewStore entirely, so only the e2e suite (the one place an actual NewStore boot has to succeed) could ever notice, and it didn't until the RBAC change closed the implicit-admin escape hatch. Three changes ship together: (1) NewStore now load-first from KV (KV is authoritative; the file is a one-shot seed for fresh deployments), distinguishes jetstream.ErrKeyNotFound from other KV errors via errors.Is (a transient KV read failure no longer downgrades to "treat as empty + bootstrap"), and propagates every bootstrap failure as a fatal error so a misconfigured deployment refuses to boot instead of silently denying every request — when bootstrapPath == "" and KV is empty the store still comes up but logs a loud operator-facing warning so a deliberate "I'll seed via API" choice is distinguishable from a silent lockout in stdout scraping. (2) BREAKING: policy.file_path has no default anymore — the prior env-default:"policy.yaml" quietly opted every deployment into a policy.yaml lookup in CWD that the operator never asked for, and combined with (1) that would refuse boot on any image that didn't ship that exact file. Set WH_POLICY_FILE_PATH / policy.file_path explicitly to opt into bootstrap; leave empty and seed via PUT /v1/admin/policy. The e2e orchestrator (scripts/orchestrator/main.go) already set the env var explicitly so it's unaffected; cfg.Policy.FilePath now defaults to "" in TestLoad_Defaults. (3) New tests/e2e/fixtures/policy.yaml — a minimal seed (no default_role, viewer perms on clicks/events/users to mirror tests/e2e/sdk/setup.ts) — so the orchestrator-launched stack boots cleanly under the strict contract; setup.ts still PUTs the test policy on top of it before any test runs. Tests: the misleading IsNotFatal test is renamed to BootstrapMissingFileIsFatal and now asserts the error; new tests cover invalid YAML, invalid JSON, invalid policy (Validate failure on bootstrap), and KVTakesPrecedenceOverFile (the post-first-run restart case where the seed file may have been moved or deleted — KV still wins, missing file is no longer fatal); the EmptyKV test also asserts the operator-facing warning so a future silent-lockout regression is caught. internal/policy unit coverage rises to 93.6%.
  • isMutation no longer misclassifies CTE-prefixed reads as mutations when the SQL contains an identifier that collides with a mutation-verb name (internal/api/clickhouse_exec.go, internal/api/clickhouse_exec_test.go). containsMutationVerbAtTopLevel previously checked every depth-0 identifier against mutationVerbs, so for WITH x AS (SELECT 1) SELECT * FROM system.tables the database name system matched the SYSTEM verb and routed the read through Exec — executeCHQuery returned an empty [] instead of the actual rows, silently. The same false-positive class trips for CTE aliases whose own name collides with a mutation verb (WITH set AS (...), WITH alter AS (...), etc.) and for table aliases (SELECT * FROM cte AS set). Two-part fix: (a) identifiers whose next non-whitespace token is AS or ( are recognised as CTE definition names and skipped without checking the keyword sets, which handles the CTE-alias-collides-with-verb case; (b) among the remaining identifiers, the scanner stops on the FIRST that's a known statement keyword (mutation OR read-class — a new nonMutationVerbs set covers SELECT/SHOW/DESCRIBE/DESC/EXPLAIN/EXISTS/CHECK) and decides based on mutationVerbs membership. Affects callers of executeCHQuery (structured-query and pipes handlers). Regression tests added: system.tables / system.columns reads, CTE aliases named set/alter/drop, a multi-CTE case with two mutation-verb-name aliases, and a control case that reads system.tables inside a CTE body but actually issues an INSERT (must still be classified as mutation). Reported by CodeRabbit on PR #164.
  • The raw-SQL endpoint returns HTTP 200 with [] for mutation/DDL statements instead of HTTP 500 (internal/api/query.go, internal/api/query_test.go). Note (largely superseded — see the HTTP-proxy entry above): the in-handler verb classifier + cache-bypass approach described below was the state at this commit. The subsequent HTTP-proxy refactor moved the verb classifier to internal/api/clickhouse_exec.go (still used by the structured-query and pipes handlers) and the /v1/admin/query path now lets ClickHouse classify natively and forwards mutations' empty body as []. The mutation-cache-bypass machinery is moot — there's no cache on that path at all anymore. The HTTP 200 + [] contract for successful no-result mutations is preserved, just delivered through the proxy. executeQuery previously routed every statement through driver.Query, which expects a result set. ClickHouse DDL/DML like TRUNCATE, DROP, DELETE, ALTER returns no result set, and the clickhouse-go driver surfaced that as an internal error that bubbled out as HTTP 500. The handler now classifies by the leading SQL verb (INSERT/UPDATE/DELETE/TRUNCATE/DROP/ALTER/CREATE/RENAME/OPTIMIZE/REPLACE/GRANT/REVOKE/ATTACH/DETACH/KILL/SET/USE/SYSTEM → Exec; everything else → Query) and marshals [] on Exec success — the same "no rows" shape an empty SELECT returns. Leading whitespace and -- / /* */ SQL comments are stripped before the verb is classified. Mutations also bypass the TieredCache + singleflight path entirely in QueryHandler.Handle: caching [] under the SQL cache key would let a second identical TRUNCATE/INSERT/etc. hit the cache and skip ClickHouse within DefaultTTL (silent data loss on the second request); singleflight collapsing concurrent identical mutations would drop one of two writes. X-Cache: MISS is set on every mutation response since the cache is never consulted. Closes #118. Now that the raw-SQL endpoint is the only sanctioned surface for non-insert mutations (see Removed above), this response-shape and cache-bypass fix is part of locking down that surface. (Originally landed at /v1/query; the endpoint moved to /v1/admin/query later in this same Unreleased section — see the Security entry below.)
  • Ingest worker no longer infinite-retries permanent delete errors (internal/ingest/bento.go, internal/ingest/bento_test.go, docs/src/content/docs/architecture.md, AGENTS.md): jsInput.Read's action: "delete" block called m.Nak() on every chConn.Exec failure, which JetStream interprets as "redeliver immediately." A delete whose error was deterministic (syntax error, unknown table, malformed identifier) would loop forever — clogging the buffer consumer, burning CPU, and spamming logs with the same message. Phase 1 of issue #91: every delete-Exec error is now treated as permanent. The original NATS envelope is published to dlq.<table> (reusing the existing bentoDLQDropped counter when the DLQ publish itself fails) and the message is DoubleAck'd so it leaves the main queue. Issue #91 stays open after this lands as the Phase 2 tracker for transient-vs-permanent error classification (timeouts and network errors should still Nak() for retry); Phase 1 alone is the stopgap, Phase 2 makes the trade-off acceptable in production. Superseded by the insert-only lock in Removed above — both the delete branch and the DLQ delete-envelope shape it required are gone.
  • CORS middleware: spec-compliant wildcard, no credentials, no header decoration on same-origin (internal/api/router.go, internal/api/router_test.go, AGENTS.md, config.yaml, docs/src/content/docs/configuration.md, docs/src/content/docs/deployment.md): closes #29 and bookends #30. Three behavior changes, one rationale. (1) Dropped Access-Control-Allow-Credentials: true entirely — WaveHouse is a Bearer-token API (Authorization: Bearer <jwt>), cookies are never used (verified: no http.Cookie / SetCookie anywhere in the Go tree, TS SDK sends no credentials: 'include'), so credentials mode is unnecessary AND the previous combination of Allow-Credentials: true with Allow-Origin: * violated the CORS spec — browsers reject that pairing, which silently broke any client that ever set credentials: 'include'. (2) Requests with no Origin header (same-origin browser navigation, server-to-server, curl) now skip the CORS decoration entirely instead of unconditionally stamping Allow-Methods/Allow-Headers/Allow-Credentials on every response. (3) A preflight from a disallowed origin still returns 204 but with no CORS headers, which the browser treats as preflight failure — same outcome as before but without leaking the methods/headers list to origins that aren't allowed. Allowlist mode sets Vary: Origin on both hits and rejects so shared caches can't memoize a headerless reject under the URL alone and replay it to a later allowed-origin request. Test coverage in router_test.go pins each branch: wildcard echoes *, allowlist hit echoes origin + Vary: Origin, allowlist miss gets no Allow-Origin but still gets Vary: Origin (both for regular and OPTIONS), no-Origin requests pass through clean, and a table-driven test asserts Allow-Credentials is never emitted across wildcard / empty-allowlist / allowlist-hit. Posture is documented as AGENTS.md §"Key Design Decisions" item 16 so future contributors don't reintroduce credentials or cookie auth without a design discussion. Config sample updated with the dev recipe (point at http://localhost:3000 etc. instead of * once a frontend is built).
  • OTel shutdown no longer hangs process exit when the collector is unreachable (cmd/wavehouse/main.go): the defer otelShutdown(context.Background()) was unbounded, and the OTel SDK's batch processors don't fully honor the shutdown context against an unreachable gRPC endpoint (the dial retries with backoff continue past the deadline). Bounded the shutdown to 5s. Discovered while writing tests/integration/otel_test.go TestOTel_UnreachableEndpoint_DoesNotBlockStartupOrEmits. Superseded — the flat 5s bound in cmd/wavehouse/main.go is now 3s, and the bounding itself moved into internal/observability/provider.go, which shuts the trace/metric/log providers down concurrently and returns the instant the caller's context deadline passes (the experimental logs SDK's BatchProcessor.Shutdown ignores ctx and would otherwise block ~10s in gRPC backoff against an unreachable collector). Regression-guarded by TestInitProvider_ShutdownParallelBounded.
  • Coverage badge step no longer fails on every main push (.testcoverage.yml, AGENTS.md): the profile: field pointed at tmp/coverage/unit/coverage.txt (~73% — ./internal/... only) but threshold.total was 80, so vladopajic/go-test-coverage's own gate failed unconditionally even after make ci had passed its per-suite + merged gates. Switched the action's profile to tmp/coverage/total/coverage.txt — the merged unit+integration+e2e profile scripts/cov merge produces at the end of make ci (~81% across the same files) — so the 80% gate applies to the project-wide number the badge advertises, not to a single suite. Per-suite gates in the YAML's suites: block continue to run inside make ci itself against the per-suite profiles before the badge step ever fires. AGENTS.md updated to state 80% project-wide minimum + per-suite minima. Superseded — #129 removed this vladopajic/go-test-coverage badge step entirely, and the Added entry at the top of this section (#133) replaces it with a self-hosted cov badge shields.io badge + GitHub Code Quality PR comments. The profile: tmp/coverage/total/coverage.txt setting introduced here still stands, now consumed by scripts/cov.
  • Dependabot auto-merge no longer slipped past CI on consolidated-pipeline PRs (main branch protection ruleset; .github/workflows/project-orchestrator.yml; .github/workflows/claude-review.yml; AGENTS.md): on commit 93b3206 the previously six-job CI was collapsed into a single job named CI, but three places still referenced the old multi-job names — the ruleset's required_status_checks (Validate + Admin approval only — CI not listed), the bot-clean check in project-orchestrator.yml (looking for Build/Validate/Lint/Test/Integration Tests/SDK Tests), and REQUIRED_CHECKS in claude-review.yml (same six). Net effect: a Dependabot PR opened at T+0 would have its Validate and Admin approval checks satisfied within ~30s (PR-title workflow + the auto-approval workflow), gh pr merge --auto would fire, GitHub would see all required checks green, and the PR would squash-merge before ci.yml even started running. Pre-consolidation runs (PR #90 etc.) didn't expose this because GitHub's auto-merge waits for in-flight checks as a courtesy, and the multi-job CI was producing checks before auto-merge fired — but with one collapsed CI check that didn't start until later, the courtesy wait disappeared. Fix: added CI to the ruleset's required-checks (now Admin approval / CI / Validate) via the gh api PUT /repos/Wave-RF/WaveHouse/rulesets/15353356 round-trip, and updated both bot-clean lists to look for CI + Validate. AGENTS.md note about the required-check set updated to match. Diagnosis steps recorded inline as code comments in both workflow files so future-me knows where to look if the check name changes again.

Added

  • Team-wide Claude Code configuration + agent PR discipline gates (.claude/agents/pre-push-reviewer.md, .claude/hooks/agent-bash-gate.sh, .claude/hooks/review-marker.sh, .claude/hooks/gofumpt-on-save.sh, .claude/settings.json, .claude/skills/pr-review-locally/SKILL.md, .claude/skills/pr-sync-with-main/SKILL.md, .claude/commands/cover.md, .githooks/pre-commit, .githooks/pre-push, .config/wt.toml, AGENTS.md §"Agent PR Discipline" + §"Local-First Validation", docs/src/content/docs/claude-code.md). Two distinct enforcement layers: universal git hooks (.githooks/, installed by make tools via git config core.hooksPath .githooks) run make verify on commit and require tmp/ci-passed-<HEAD-sha> before push for everyone (humans + agents); agent-only gates layer on top in .claude/hooks/agent-bash-gate.sh (PreToolUse Bash) and block: --no-verify on git push/commit; gh pr create without --draft; gh pr ready, --approve, --request-changes (humans-only state transitions); gh pr edit --add-reviewer/--add-assignee and gh api .../requested_reviewers POST/PUT/PATCH (humans-only reviewer assignment — bot reviewers re-triggered via PR comments instead); and git push to an open-PR branch without both ci-passed and review-passed markers. The pre-push-reviewer subagent runs the canonical .github/prompts/pr-review.md in fresh context against the branch's full merge-base diff (git diff main...HEAD) plus latest commit, open PR comments + reviews, CI status, and linked-issue acceptance criteria, then emits a parseable VERDICT: ship_it|iterate|block consumed by review-marker.sh (SubagentStop) which writes tmp/review-passed-<HEAD-sha> only on ship_it. ship_it is gated to zero findings at any severity — any [MUST]/[SHOULD]/[MAY] forces iterate and the orchestrator loops review → fix → review until clean before pushing. Marker writes are NOT regex-blocked beyond the cheap settings.json deny patterns (Bash(touch tmp/ci-passed:*), Write(tmp/ci-passed-*), Edit(tmp/ci-passed-*), and review-passed equivalents); the rest is an explicit honest-agent rule in AGENTS.md ("you do not write a marker file by any other means — period") because Bash can write a file by a dozen paths and regex enforcement is a porous game of whack-a-mole. Humans retain --no-verify for explicit intentional bypass; .config/wt.toml ships worktrunk hooks so parallel-agent worktrees install .githooks/ correctly. docs/src/content/docs/claude-code.md is the contributor-facing page that explains the layering, quick-setup, and discipline rules.
  • Astro / Starlight documentation site at wavehouse.dev (docs/, Makefile, .github/dependabot.yml, .github/workflows/ci.yml, .gitignore, .vscode/launch.json): full content + tooling for the public docs (Getting Started, Why WaveHouse, Architecture, API Reference, TypeScript SDK, Configuration, Deployment, Development). Build-time mermaid via rehype-mermaid (inline-svg strategy, no client-side JS), LaTeX via remark-math + rehype-katex, image zoom via starlight-image-zoom, expressiveCode with github dark/light + env/dns Shiki language aliases. PostHog frontend telemetry (key is public by design — embedded in every visitor's browser). editLink pointed at main; lastUpdated on. Sidebar lives in docs/src/config/sidebar.ts as the single source of truth, consumed by both Starlight rendering and the LLM-friendly outputs below. Cloudflare Workers + Static Assets deployment via docs/wrangler.jsonc. Make targets stay minimal: dev-docs (Astro dev server on :4321), build-docs, preview-docs — the last serves the production build through wrangler dev, so the Cloudflare Worker's content negotiation (Accept: text/markdown / LLM-bot User-Agent → .md twin) is exercised end-to-end the same way it will be in production. (Auto-deploy wire-up and the docs-subproject Makefile split land in a later [Unreleased] entry below.)
  • Per-page .md twin + llms.txt family via starlight-llm-tools (docs/astro.config.mjs, docs/worker/index.ts, docs/wrangler.jsonc): every doc is also served as raw markdown at <path>.md with a navigation header (Section / Subpages / Related / HTML version pointers), plus three concatenated views — /llms.txt manifest, /llms-full.txt (every page, sidebar-ordered), /llms-small.txt (overview pages only). Page-title slot gains a Copy Markdown button and Open with AI dropdown (Claude / ChatGPT / Cursor). Content negotiation is handled by a Cloudflare Worker that re-exports the cloudflare-md-router package's default handler in one line — Accept: text/markdown or known LLM-bot User-Agent (GPTBot, ClaudeBot, PerplexityBot, Applebot-Extended, etc.) gets the .md twin transparently, with a fall-through to the HTML response when the twin doesn't exist.
  • Three extracted plugin packages, MIT-licensed: github.com/Wave-RF/cloudflare-md-router (Workers handler + createMdRouter() factory + extensible bot-UA regex; consumed in docs/worker/index.ts as a one-line re-export), github.com/Wave-RF/starlight-llm-tools (Starlight plugin that auto-injects the four routes, the two components, and the PageTitle override; optionally calls into starlight-glossary/transform when present), and github.com/Wave-RF/astro-themed-mermaid (color-agnostic Mermaid pipeline — a remark plugin that injects centralized classDefs, the rehype-mermaid options, and an Astro integration; the diagram palette lives once in docs/src/styles/global.css as --wh-mermaid-* and is mapped in docs/src/config/mermaid-theme.mjs, so the package itself carries no colors). Adopting astro-themed-mermaid replaced the previous inline rehype-mermaid { strategy: "inline-svg" } wiring in astro.config.mjs and dropped the direct unist-util-visit devDependency. All pinned via github:Wave-RF/... in docs/package.json (astro-themed-mermaid to the #v0.1.1 tag, the other two to the default branch); lockfile pins the resolved commit SHAs. Pre-npm-publish state — switch to normal version specifiers when the packages land on npm.
  • Custom WaveHouse mark + favicon / OG card / inline nav mark / README header, fed by a single-source make branding-docs pipeline (docs/src/assets/branding/{mark.svg,lockup.svg,og.template.svg}, docs/scripts/branding/generate.sh, Makefile, docs/astro.config.mjs, docs/public/branding/{favicon.svg,favicon.ico,apple-touch-icon.png,og.png,mark-light.svg,mark-dark.svg,lockup-dark.svg}, docs/public/favicon.ico, docs/src/components/WaveMark.astro, README.md): replaces the default Astro sparkle favicon and the 🏖️ emoji in the README header with the clickwave mark — four vertical sine-curve columns + a filled cursor square on the right, on a 100-unit viewBox. The columns echo ClickHouse's column-store grammar; the cursor square reads as both the "click" in ClickHouse and a "live cell" indicator for the streaming/real-time story. Brand colors live as design tokens: #086D77 deep teal on light backgrounds, #06B0BF electric blue (Wave RF parent brand) on dark — both also exposed in docs/src/styles/global.css as --wh-accent / --wh-accent-hi. docs/src/assets/branding/mark.svg is the single source of truth (one multi-element SVG using stroke="currentColor" for the wave columns and fill="currentColor" for the cursor square), and the brand colors come from the --brand-* palette in docs/src/styles/global.css (no hex lives in generate.sh — it greps each hue out of the stylesheet so the live site and the generated assets can't drift); make branding-docs extracts the mark's inner content + viewBox and emits seven derived brand-kit artifacts under docs/public/branding/ (plus a copy of favicon.ico at the public/ root) via rsvg-convert + ImageMagick: a CSS-swap browser favicon (:root{color} block flips via prefers-color-scheme:dark and the inner currentColor attributes inherit, covering both stroke and fill in one declaration), a multi-resolution legacy favicon.ico (16/32/48), a 180×180 iOS home-screen tile (apple-touch-icon.png; electric-blue mark on near-black #0A0B0E background, scale rebased to 1.24 for the new 100-unit viewBox, solid because transparent apple-touch icons render a white halo on iOS), a 1200×630 OG card rendered from og.template.svg (gradient background + faint accent grid + corner radial glow + canonical lockup (mark + "WaveHouse" wordmark as a designed 480×100 unit from lockup.svg, with the mark in accent cyan and the wordmark in off-white via a two-color substitution so the visual hierarchy reads as "logo + headline" instead of one monochrome bar) + two-line tagline with ClickHouse highlighted in the accent color + footer row pairing A Wave RF open-source project (left) with wavehouse.dev (right), lockup spliced into a __LOCKUP_GROUP__ placeholder via sed), and three copy-out brand assets — mark-light.svg / mark-dark.svg (the mark in the light / dark accent) and lockup-dark.svg (mark + wordmark for dark backgrounds), used by the README header and as a downloadable kit. Wave-column centers + cursor square shifted left by 4 units inside the viewBox so left and right padding match — without the shift the cursor square sits flush against the right edge while the leftmost wave has ~7 units of breathing room. make branding-docs is not wired into build-docs or any CI workflow — derived artifacts are committed so contributors don't need librsvg / imagemagick on their PATH to build docs, only someone iterating on the mark itself does. astro.config.mjs sets Starlight's favicon: "/branding/favicon.svg" — there is no logo: block, because the nav mark is rendered by the inline <WaveMark/> component (Astro inlines mark.svg at build time so its currentColor strokes stay theme-aware, unlike an SVG loaded through <img> where the media queries wouldn't fire) — and adds the .ico + <link rel="apple-touch-icon"> head tags pointing at /branding/*; the legacy favicon.ico still ships at the public/ root so older clients (IE/Edge, some RSS readers) that can't load SVG favicons auto-probe /favicon.ico from the site root by convention — no explicit <link> tag needed. README H1 uses a <picture> element with <source media="(prefers-color-scheme: dark)"> so GitHub renders the right variant per visitor theme. Future edits: tweak mark.svg (or a --brand-* hue in docs/src/styles/global.css, or the OG composition in og.template.svg), run make branding-docs, commit the diff.
  • Documentation site design-system rebuild with custom Hero / Header / SiteTitle / Footer components (docs/src/styles/global.css, docs/src/components/{Hero,Footer,SiteTitle,Head}.astro, docs/src/content/docs/index.mdx, docs/astro.config.mjs, docs/package.json, Makefile, docs/scripts/screenshot.mjs): replaces the previous custom.css baseline with a full Tailwind v4 token system in global.css — accent / ink / surface / border tokens for both light and dark themes, plus Starlight --sl-color-* overrides so the existing Starlight chrome picks up the WaveHouse palette without per-page overrides. New Hero.astro powers the splash landing page (index.mdx) with an animated terminal preview, stats grid, and CTA actions; Footer.astro renders one of two brand-aware footers keyed on hasSidebar (from Astro.locals.starlightRoute) — a compact footer on doc pages (which already carry the fixed left sidebar) and a full multi-column marketing footer (Docs / Operations / Project link columns + sign-off row) on splash / landing pages that have room to breathe; SiteTitle.astro replaces Starlight's default header title with the clickwave mark + wordmark and a small hover transform; Head.astro adds theme-color meta tags for both color schemes. New runtime deps: @astrojs/starlight-tailwind, @tailwindcss/vite, tailwindcss@4, @fontsource-variable/inter, @fontsource-variable/jetbrains-mono. docs/scripts/screenshot.mjs is a new Playwright-based visual-regression script (pnpm exec node scripts/screenshot.mjs from docs/, with make dev-docs running in another shell on :4321) that captures landing / getting-started / API / architecture pages in both themes at desktop and mobile widths, used for design iteration and golden-image diffing. Chromium is fetched on-demand by make build-docs / make dev-docs (via a dedicated install-playwright-docs Make target) so rehype-mermaid's SSR diagram pipeline (used by architecture.md / why-wavehouse.md) and starlight-links-validator can run in CI without an out-of-band Chromium install step; Go-only contributors running make tools skip the ~130 MB download.
  • In-process OTLP gRPC test receiver + integration tests for the OTel pipeline (internal/testutil/otlp.go, tests/integration/otel_test.go): testutil.NewFakeOTLP(t) binds a gRPC server on 127.0.0.1:0, registers the trace/metric/log Export services, and captures every received payload — exposing SpanCount(), MetricCount(), LogCount(), and LogCountAtLevel() for assertions. Test cases verify trace sampling (1.0 → all, 0.5 → ~half within binomial tolerance, 0.0 → none), the WARN+ERROR floor (logs at sample_rate=0.0 still export 100% of WARN/ERROR), per-signal gating (traces-only, logs-only), and the unreachable-endpoint invariant (InitProvider succeeds on lazy gRPC dial, emits don't block on failed export). Avoids spinning a full SigNoz stack in CI — the receiver is ~150 LOC and the whole suite runs in <5s.
  • OpenTelemetry pipeline is now opt-in via [otel] config block with per-signal gates and configurable sampling (internal/config/config.go, cmd/wavehouse/main.go, internal/observability/provider.go, internal/observability/logger.go, config.yaml, docs/configuration.md, docs/deployment.md): otel.enabled (WH_OTEL_ENABLED, default false) is the master switch, and the OTLP gRPC collector endpoint is set via the standard OTEL_EXPORTER_OTLP_ENDPOINT env var (default localhost:4317; see the Direct-to-cloud OTLP entry above). Each signal is then individually gated: otel.traces.{enabled,sample_rate} (head-based, default 1.0 — matches the OpenTelemetry SDK default), otel.metrics.enabled (no sampling — pre-aggregated), otel.logs.{enabled,sample_rate} (DEBUG/INFO at default 1.0 — WARN/ERROR always export at 100% as a non-configurable safety floor). All sampling knobs apply only to the OTLP push path; stdout always emits 100% of records (scraping-style pipelines like Promtail/Alloy → Loki own their own ingest policy). All sample rates are validated to [0.0, 1.0] at config load. Stdout logging is always active (the logger fans out to both stdout and OTLP), so logs never disappear regardless of collector state. gRPC exporters are lazy, so an unreachable collector does not block startup — startup was previously fatal on OTel init failure, now it is not. WH_LOG_LEVEL (DEBUG/INFO/WARN/ERROR, case-insensitive) now drives the slog LevelVar for both stdout and the OTLP log exporter. provider.go adds the otlploggrpc log exporter alongside the existing trace/metric exporters and registers the global OTel logger provider. InitProvider now takes a ProviderConfig struct AND returns a (shutdown, promHandler http.Handler, err) triple (breaking signature change within the internal package); NewLogger gains a otlpSampleRate float64 parameter.
  • Prometheus exposition is its own top-level [prometheus] config block, independent of OTel (internal/config/config.go, internal/observability/provider.go, cmd/wavehouse/main.go, config.yaml, docs/configuration.md, docs/deployment.md): prometheus.{enabled,path,port} (env vars WH_PROMETHEUS_{ENABLED,PATH,PORT}, defaults false / /metrics / 0) drives a Prometheus-format /metrics endpoint. Lives at the top level — not nested under [otel.metrics] — so operators using a scrape pipeline (Grafana Alloy / Mimir / vanilla Prometheus) do not need to enable or even understand [otel] to expose metrics. Three modes are first-class: OTLP push only (otel.enabled=true, prometheus.enabled=false), Prometheus only (otel.enabled=false, prometheus.enabled=true), or both. When both are on the underlying OTel MeterProvider feeds both readers — same Meter() API records to both; OTel→Prometheus name translation is automatic (dots/dashes → underscores, counters get _total). InitProvider's MetricsEnabled and PrometheusEnabled fields are independent; main.go runs init whenever either is wanted. port=0 mounts on the API router; non-zero spins a sidecar listener for firewall-isolated production posture. Disabled by default — explicit opt-in since enabling adds an unauthenticated endpoint. Validated at config load: port range [0, 65535], must not collide with server.port, path must start with /. This unblocks Grafana Cloud / Mimir / Loki / Tempo via Alloy (stdout scrape + Prometheus scrape + OTLP traces) — see docs/deployment.md for the wiring patterns. deployments/signoz/ is a new self-contained compose setup for running SigNoz locally (ClickHouse + query service + collector); credentials are env-substituted with a .env.example template so the literal default:password no longer ships in source-controlled YAML.
  • .dockerignore rewritten for completeness (.dockerignore): the previous version excluded .git, bin, tmp, etc. but missed node_modules (~100MB across clients/ts + tests/e2e/sdk), the entire clients/ and tests/ trees, .bin/ (golangci-lint), and editor cruft (.vscode/, .idea/, *.swp). Build context for make docker was uploading content the Go binary build never reads. New version is grouped by purpose with comments explaining what each block does and why it's safe to exclude (the clients/ and tests/ trees are the big wins — Go only compiles files in the package tree it's targeting, so excluding test source from the context doesn't change the binary).
  • OCI image labels in source-build Dockerfile (deployments/Dockerfile, deployments/Dockerfile.goreleaser): added the static org.opencontainers.image.{title,description,url,source,licenses,vendor} labels to both Dockerfiles. Per-build dynamic labels (revision, version, created) are still injected at build time by goreleaser's dockers_v2.labels — for dev images the title/description templates branch on WAVEHOUSE_DEV to surface the (dev) suffix and dev-build description. The static set is the floor: even if the dynamic labels are missing, docker inspect shows project identity.
  • HEALTHCHECK in both Dockerfiles via a built-in health subcommand (cmd/wavehouse/health.go, cmd/wavehouse/main.go, deployments/Dockerfile, deployments/Dockerfile.goreleaser): distroless has no shell, no curl/wget, so the standard wget --spider HEALTHCHECK pattern doesn't work. Solution is the canonical distroless approach (used by Cilium, Trivy, Coredns, etc.): embed the probe in the same binary the container runs. New runHealthCheck() does an HTTP GET http://127.0.0.1:$WH_SERVER_PORT/health with a 2-second timeout and exits 0 on 200 OK, 1 otherwise. Argv router in main.go dispatches wavehouse health to that function before falling through to the normal server path. Both Dockerfiles get HEALTHCHECK --interval=10s --timeout=3s --start-period=15s --retries=3 CMD ["/app/wavehouse", "health"]. The probe targets /health (liveness — no external deps) rather than /ready (which requires ClickHouse) so a transient ClickHouse blip doesn't mark the container unhealthy and trigger a restart. docs/deployment.md Health Checks section gains a Docker HEALTHCHECK subsection plus compose depends_on: condition: service_healthy example and a K8s probe note (kubelet probes are external and don't go through HEALTHCHECK).
  • VOLUME ["/app/data"] declaration in both Dockerfiles: if an operator runs the container without an explicit volume or bind mount at /app/data, Docker now auto-creates an anonymous volume — keeping JetStream/Pebble state off the writable overlay layer regardless of operator forgetfulness. Doesn't override or interact with explicit -v / compose volumes:; those still take precedence. Pipes (/app/pipes) is intentionally NOT a VOLUME since it's a read-only bootstrap source, not state. Catches the same persistence foot-gun as the runtime WarnIfFreshDataDir check, but earlier in the lifecycle.
  • GoReleaser snapshot validation in CI (.github/workflows/goreleaser-validate.yml): runs goreleaser release --snapshot --clean on PRs that touch .goreleaser.yaml, the deployment Dockerfiles, go.mod/go.sum, or release-related workflow files. Snapshot mode implies --skip=publish, so this never pushes anything — it's a pure dry-run that catches goreleaser config syntax errors, cross-compile breakage (across the 8 GOOS/GOARCH combos in the build matrix), and Dockerfile/dockers_v2 issues at PR-time instead of at tag-time when they'd block a release. Path-filtered to keep the day-to-day PR CI minute budget tight.
  • Rolling :dev images on GHCR for every push to main (.github/workflows/publish-dev.yml): runs the same goreleaser pipeline as release.yml against a synthetic local-only tag (v0.0.0-dev.<short-sha>, never pushed to origin) with WAVEHOUSE_DEV=1 set. The env var switches .goreleaser.yaml into dev mode: image tags become :dev-<full-sha> (immutable, point-in-time) + :dev (rolling, tracks HEAD of main), OCI labels gain the dev suffix, and release.disable evaluates true so no GitHub Release is created. Linux amd64 + arm64 are produced natively via Go's cross-compiler (no QEMU, no buildx setup action — goreleaser handles its own buildx invocation against Dockerfile.goreleaser, which is binary-COPY only). One Dockerfile, one config, one tool drives both dev and tagged releases. Concurrency group publish-dev cancels in-flight builds on new pushes so :dev tracks the latest commit, not a slower racing build.
  • Weekly GHCR cleanup workflow (.github/workflows/cleanup-ghcr.yml): cron-scheduled (Monday 03:00 UTC) prune of :dev-<sha> tags older than 30 days, but always keeping the newest 5 regardless of age. Tagged releases (latest, vX.Y.Z) and the rolling :dev pointer are never touched. Uses gh api + jq directly rather than a third-party action so the logic is auditable inline (consistent with the supply-chain-hygiene memo in AGENTS.md feedback).
  • go install documentation in README.md: WaveHouse is now installable via go install github.com/Wave-RF/WaveHouse/cmd/wavehouse@latest. Module path in go.mod already matched the repo URL — only docs needed to change. README quick-start now has three flavours (Docker Compose, prebuilt GHCR image, go install/go run) and explicitly notes that internal/ packages are not importable as a library — applications use the binary, container, or TypeScript SDK.
  • Single data_dir config + fresh-state startup warning + permission-error remediation hint (internal/config/config.go, internal/config/persistence.go, cmd/wavehouse/main.go): collapsed WH_MQ_EMBEDDED_DIR and WH_DEDUPE_EMBEDDED_DIR into a single WH_DATA_DIR (yaml: data_dir, default ./data). NATS lives at <data_dir>/nats; Pebble (when dedupe is enabled) at <data_dir>/pebble. Subdirectory names are convention, not config — one knob, one volume mount. Replaced the previous mountinfo-based persistence preflight (relative-path + overlay-fs detection) with a much simpler check: WarnIfFreshDataDir calls os.ReadDir on the configured subdirectory and logs a WARN if it's missing or empty. On a first-ever run the warning is expected; on every subsequent run it should be silent — so when it does fire after a redeploy, that's the most direct possible signal that the persistent volume isn't actually persisting. Closes the silent-data-loss foot-gun reported on 2026-05-06. NATS / Pebble init failures also get a LogStorageInitError wrapper that detects os.ErrPermission and appends a hint= field with the exact chown -R 65532:65532 remediation operators need — covers the dominant Docker-bind-mount footgun where the host directory is owned by root and the distroless nonroot user can't write to it. New docs/deployment.md "Distroless Permission Traps" section explains the named-volume vs bind-mount asymmetry (named volumes copy-up ownership automatically; bind mounts require manual host-side chown).
  • Unit tests for previously-untested packages (internal/observability, internal/mq, internal/dedupe) and new KV-backed tests for internal/policy.Store + internal/pipes.Store. Also adds testutil.NewJetStream(t) — a shared helper that spins up an in-process NATS + JetStream for tests that need a real jetstream.JetStream handle. Per-package coverage: observability 84%, mq 78%, policy 89.6%, pipes 85.2%, dedupe 61.2% (the ScyllaDB paths in distributed.go and schema.go remain integration-only).
  • Composite actions for board operations (.github/actions/board-upsert-status, .github/actions/assign-and-request-review, .github/actions/set-linked-issues-status): reusable building blocks for idempotent "upsert item on the Task Board + set Status," "assign PR reviewers + request their review as a pair," and "mirror status to all linked issues" across workflows. Replaces ~200 lines of duplicated GraphQL/gh-project bash between project-orchestrator.yml and dependabot-automerge.yml, and gives the new board-state-sync.yml a single place to debug board interactions.
  • Bidirectional board state sync (.github/workflows/board-state-sync.yml): when a human manually moves a card's Status on the Task Board UI, mirrors the change to the linked PR/Issue. Uses the projects_v2_item: edited event + changes.field_value payload to detect Status changes specifically; guards against ping-pong loops by no-oping when the target counterpart is already in the expected mirrored state. Mapping: PR Ready ↔ Issue In review, PR In review ↔ Issue Ready, both Done together. In progress on one side doesn't clobber the other (manual mid-review state, preserved).
  • Dependabot major-bump Task Board placement (.github/workflows/dependabot-automerge.yml): on a held major-version bump, the workflow now adds the PR to the board with Status=Ready and assigns both admins + requests review — parity with the reeval path for non-Dependabot PRs, so major bumps don't vanish into the PR list.
  • workflow_dispatch manual trigger on ci.yml: lets us re-run CI on demand for any branch via gh workflow run CI --ref <branch>. Added after PR #79 hit a GitHub-side anomaly where pull_request events stopped firing for a single PR's branch.

Changed

  • Breaking: mq.Message.Ack() and mq.Message.Nak() now return error (previously no return value) and no longer accept a context.Context.
  • Breaking: Added mq.Message.DoubleAck(ctx context.Context) error for synchronous, server-confirmed acknowledgments. Use this for critical ingest paths (ClickHouse writes).
  • Ingest worker now uses NATS DoubleAck for explicit server-side confirmation before finalizing batches.
  • Ingest worker now strictly enforces insert and delete actions; unrecognized actions are safely rejected to the DLQ.

Fixed

  • board-state-sync.yml guard exit 0 didn't stop subsequent steps: the step's exit 0 ends only the step's shell, not the job, so downstream steps would run with an empty GH_TOKEN and fail noisily on gh api calls. Converted to the SKIP_BOARD=true + conditional-if: pattern used in project-orchestrator.yml. Makes the "not configured → clean no-op" path actually quiet.
  • project-orchestrator.yml reviewer assignment silently skipped when PROJECT_BOARD_TOKEN absent: the Assign + request review step gated on steps.pr-status.outputs.promote == 'true', but the pr-status step itself was gated on env.SKIP_BOARD != 'true', so when the board token was missing, promote was never set and the assign step silently didn't fire — even though assignment uses GITHUB_TOKEN and doesn't need the board token. Condition extended to (pr-status.outputs.promote == 'true' || env.SKIP_BOARD == 'true') so the non-board path still assigns/requests review.
  • dependabot-automerge.yml major-bump comment lied when board placement was skipped: the comment step posted "Added to the Task Board" unconditionally but ran before the board-add / token-guard steps, so an install without PROJECT_BOARD_TOKEN would see a false claim. Reordered (guard + board-add + assign now run before the comment) and the comment wording is now conditional on SKIP_BOARD — accurate in both configured and unconfigured states.
  • admin-approval.yml required-check staying red after a valid admin approval (.github/workflows/admin-approval.yml): two independent causes, both fixed.
    • concurrency.cancel-in-progress: true was producing CANCELLED check runs when a second event (e.g., a bot comment) fired while the approval-triggered run was still executing. Cancelled check runs aren't SUCCESS or FAILURE; the ruleset treated them as not-passing. Changed to cancel-in-progress: false so runs queue and complete sequentially — an extra few seconds of latency is far cheaper than a stuck merge gate.
    • Added a fast-path: when the triggering event is an admin's APPROVED review, trust github.event.review.state == 'approved' + review.user.login directly instead of round-tripping through GET /pulls/{n}/reviews. The reviews API has observable lag after pull_request_review.submitted — the API can take 1–2 seconds to reflect a new review, and the workflow was sometimes polling before that. The slow path (reviews API) retries up to 3× with 2-second backoff for events that aren't themselves a review submit.
  • Restored received_timestamp injection in Bento ingest pipeline: Fixed a regression where the API-receipt timestamp was lost during the transition to the custom Bento worker. The jsInput now captures received_timestamp from the NATS envelope and the clickhouseOutput safely injects it into the JSON payload via byte-splicing before ClickHouse insertion, ensuring time-series query accuracy.

Changed

  • BREAKING: removed WH_MQ_STREAM_NAME / mq.stream_name — JetStream stream names are now hardcoded in internal/mq/mq.go. The previous configurable stream name was a half-implemented multi-tenancy story from the external-NATS era: cfg.MQ.StreamName was threaded into mq.NewEmbedded, api.EnsureDLQStream, ingest.StartIngestWorker, and api.NewDLQHandler, but the Sweeper, SSE handler, and WS handler all called the hardcoded mq.StreamName() function (returning "WAVEHOUSE") instead. Setting WH_MQ_STREAM_NAME=PROD would have created the stream as PROD but left the Sweeper looking up WAVEHOUSE — silent state-sync drift. Since WaveHouse is now fully embedded (each instance has its own NATS process), there's no other tenant to namespace against, and the config option was the inconsistent outlier next to the already-hardcoded WAVEHOUSE_PIPES (KV) and WAVEHOUSE_POLICY (KV) buckets. New mq.DLQStreamName() mirrors mq.StreamName() for symmetry. mq.NewEmbedded, api.EnsureDLQStream, ingest.StartIngestWorker, and api.NewDLQHandler all dropped their streamName / baseStreamName parameters; signature changes ripple through cmd/wavehouse/main.go, tests/integration/setup_test.go, and internal/api/dlq_test.go. TestValidate_DefaultsStreamNameWhenEmpty removed; TestDLQStreamName added. Removes the corresponding TODO entry in TODO.md.
  • BREAKING: replaced WH_MQ_EMBEDDED_DIR / WH_DEDUPE_EMBEDDED_DIR with single WH_DATA_DIR. NATS lives at <data_dir>/nats, Pebble (when dedupe is enabled) at <data_dir>/pebble. The two old env vars and yaml keys (mq.embedded_dir, dedupe.embedded_dir) are removed — anything setting them needs to switch to data_dir / WH_DATA_DIR. Default is ./data. Compose env (deployments/compose/standalone.yaml, tests/e2e/compose.yaml, clients/ts/playground/compose.yaml) updated to WH_DATA_DIR=/app/data with a wavehouse-data:/app/data volume; scripts/orchestrator/main.go updated to set WH_DATA_DIR=tmp/data. Both Dockerfile and Dockerfile.goreleaser pre-create only the parent dirs (/app/data and /app/pipes) owned by nonroot:nonroot — the binary mkdirs nats/ and pebble/ subdirectories itself when NATS/Pebble open their stores, so deeper pre-creation would buy nothing. Dockerfile.goreleaser uses a tiny alpine:3 layout stage to do this since distroless has no shell.
  • BREAKING: renamed WH_PIPES_DIRECTORY → WH_PIPES_DIR and changed default from ./pipes to empty. Empty default skips the bootstrap entirely — most users will create pipes via the API. The pipes directory is now documented as read-only at runtime: it's a seed loaded into the NATS KV WAVEHOUSE_PIPES bucket on startup, not authoritative storage. Compose snippets in docs/deployment.md show the :ro bind-mount pattern (./my-pipes:/app/pipes:ro). internal/pipes/pipes.go already handled directory == "" by skipping the load — only the config and docs needed to change.
  • project-orchestrator.yml refactored to the set-once-assignees + mirrored-status model. Assignees are now treated as per-card identity (Issue = implementer, PR = reviewer) rather than rotated across state transitions; statuses mirror across linked PR↔Issue pairs. Specific changes:
    • On bot-clean → Ready: linked issues now promoted to In review (was missing; previously only re_requested path moved issues).
    • Assignee and GitHub review-request are now applied together via a single composite action — board queue and GitHub notification stay in sync.
    • Lint, Test, Integration Tests, SDK Tests joined Check/Build/Validate in the bot-clean required-checks set, matching the updated ruleset.
    • Promotion guard ("don't yank In progress / In review / Done back to Ready") preserved.
    • ~250 lines of duplicate GraphQL/item-edit shell collapsed into composite-action calls. Full state-machine spec in AGENTS.md §Task Board state machine.
    • Re-eval trigger set fixed. The original check_suite: completed trigger silently never fired on internal PRs — GitHub suppresses check_suite events when the suite was created by a GITHUB_TOKEN-triggered workflow (standard case for our CI). Replaced with workflow_run watching the CI workflow, which is the documented chain-off-another-workflow mechanism and doesn't have the suppression gotcha. Also broadened the pull_request_review: submitted handler so COMMENTED reviews (which is what bots like Gemini post) also trigger a bot-clean re-eval — gives the state machine three independent paths (push, bot review, CI completion) to re-eval instead of effectively one.

Fixed

  • API error responses now always set Content-Type: application/json (internal/api/errors.go, all internal/api/*.go handlers, internal/api/router.go): error paths that constructed JSON-shaped bodies via http.Error left the response Content-Type at the stdlib default text/plain; charset=utf-8, which broke strict SDK clients that branch on the response media type. New writeJSONError(w, status, message) helper sets Content-Type: application/json (no charset, matching the success-path handlers and RFC 8259) and X-Content-Type-Options: nosniff to prevent MIME sniffing, then writes a JSON {"error": "..."} body. Every http.Error callsite that wrote a JSON body — across ingest.go, query.go, structured_query.go, pipes.go, policy.go, schema.go, router.go, middleware.go, dlq.go, stream_sse.go — now routes through the helper. The chi router's default handlers were also wired through the helper: r.NotFound (unknown URL) and r.MethodNotAllowed (unsupported method) now emit JSON, and chi/middleware.Recoverer was replaced with a custom jsonRecoverer that logs the panic with slog.Error (err, stack, path, method) and returns a JSON 500 to the client. The recoverer wraps the response writer with chi/middleware.NewWrapResponseWriter and only emits the JSON 500 when neither headers nor body bytes have been written yet — if the handler had already flushed a partial response before panicking, the JSON body is suppressed (Go's net/http model can't retract committed headers, so appending after them would corrupt the response). /health and /ready were brought into the same header contract: both now set X-Content-Type-Options: nosniff alongside the existing Content-Type: application/json. The one remaining text/plain error surface — failed WebSocket upgrades on /v1/stream/ws — sits below the application contract at the HTTP-101 negotiation layer and is documented as a caveat in docs/api.md. Closes #72. Test coverage: internal/api/errors_test.go adds direct unit tests on the helper plus a strict shared assertJSONErrorResponse assertion that every handler-level error path now invokes; router_test.go adds tests for the NotFound, MethodNotAllowed, panic-recovery, no-panic-passthrough, and http.ErrAbortHandler-re-panic paths.
  • TestDLQIntegration flake — ClickHouse readiness race (tests/integration_test.go): the testcontainer wait strategy only checked that the 9000/tcp port was listening, but ClickHouse opens that port well before it's ready to accept native-protocol queries. The next chConn call could hit connection reset by peer mid-handshake. Fix: wait on both the port AND GET /ping returning 200 (HTTP signal that the server is fully initialized), then explicitly chConn.Ping(ctx) with retries before any Exec. clickhouse.Open is lazy, so without the explicit Ping the first real query would still be the one to meet a half-ready server. Closes the dominant flake mode tracked in #70.
  • golangci-lint config verify external-schema flake (.github/workflows/ci.yml): the action ran golangci-lint config verify by default, which fetches https://golangci-lint.run/jsonschema/golangci.v2.11.jsonschema.json and times out when that service blips — failing Lint for reasons unrelated to our Go code. Pass verify: false to skip the schema-validate pre-flight; the actual linter run is unaffected.
  • project-orchestrator.yml board-item lookup crashed with accepts 1 arg(s), received 4 (.github/workflows/project-orchestrator.yml): gh api --jq accepts a single jq expression — it can't forward --arg to the embedded jq. The six callsites that used --jq --arg n "$N" '...' passed four positional tokens to gh api and exploded. Surfaced on PR #71 going ready_for_review: Taite was assigned (preceding step), but the board add/edit never ran. Fix: pipe raw JSON to a standalone jq invocation at all six sites.
  • deployments/Dockerfile pinned golang:1.25-alpine against go.mod requiring 1.26.2 — when we bumped go.mod to 1.26.2, the Dockerfile base image wasn't bumped. make build in CI uses the host Go toolchain (1.26.2), so the mismatch stayed invisible; make docker / the E2E suite's compose-built image went straight into go: go.mod requires go >= 1.26.2 (running go 1.25.9; GOTOOLCHAIN=local). The new e2e job surfaced it. Fix: bump base image to golang:1.26-alpine.

Changed

  • Restored unit-test coverage threshold to 70% in .testcoverage.yml (was temporarily lowered to 60% in #66) and removed the exclude.paths entries for internal/dedupe, internal/mq, and internal/observability now that those packages have _test.go files. internal/testutil remains excluded — it is test-only helper code. Measured total is 71.3%, up from 65% with the old exclusions applied. CI/Makefile/AGENTS.md commentary updated to reflect the stable 70% bar. Closes #67.
  • pr-title.yml and label.yml now use only pull_request_target. The dual-trigger (pull_request + pull_request_target) introduced in #69 was a transition: pull_request_target is on main and confirmed firing, so the pull_request half is dropped. Stops running each check twice on internal PRs and removes the sticky-comment TOCTOU race. Concurrency-group event_name suffix reverted with the trigger removal — only one event fires now.
  • claude-review.yml waits for required CI to settle and skips on red. New first step polls the check-runs API for the PR's required checks (Check, Build, Validate, Lint, Test, Integration Tests) until they reach a terminal state, then short-circuits if any of them failed/cancelled/timed-out. Stops burning Claude review tokens on PRs the human will bounce back as "fix CI first." Re-pushing green CI re-triggers the review normally. (Polls named checks rather than using gh pr checks --watch — the latter would wait on every check including the Review job itself and deadlock.)
  • CI now runs SDK + E2E test jobs. sdk-test runs make test-sdk (vitest unit tests in clients/ts) on every PR — gap surfaced by #63 where a Dependabot SDK bump ran zero TypeScript tests. e2e runs make test-e2e (vitest in tests/sdk, which spins up the full ClickHouse + WaveHouse compose stack via setup.ts) gated on pull_request.draft == false so WIP pushes don't pay Docker-build time. Both use SHA-pinned actions/setup-node@v6.4.0 and pin Node 22 (the E2E streaming test exercises native WebSocket, which Node 20 doesn't provide). The e2e job is marked continue-on-error: true pending follow-up on two pre-existing bugs surfaced by wiring it in (a raw-SQL query that races Bento's ingest-batch flush, and the WebSocket polyfill already addressed by the Node-version bump).
  • Reviewer assignment, admin approval, and Task Board orchestration moved from CODEOWNERS to workflows. Removed .github/CODEOWNERS; admin approval on every PR is now enforced by .github/workflows/admin-approval.yml as a required status check (fails unless Eric or Taite has an APPROVED review, Dependabot PRs bypass via dependabot-automerge.yml). Reviewer assignment + Task Board state orchestration handled by .github/workflows/project-orchestrator.yml: when a PR becomes bot-clean (required checks green + all review threads resolved), the workflow adds it to the Task Board (project #7), sets Status=Ready, and assigns the non-author admin to the PR's assignees field (not as a GitHub review-request — the board card + assignment is the single signal). Subsequent events drive a full state machine: CHANGES_REQUESTED review moves PR→In review and linked issue→Ready; author re-requesting review moves PR→Ready and issue→In review; merge handled by existing project automation. One manual step intentionally preserved: reviewer moves PR card Ready→In progress when they start reviewing. Ruleset updated: require_code_owner_review: false (no CODEOWNERS), Admin approval added to required_status_checks.

Removed

  • .github/workflows/board-state-sync.yml: bidirectional manual-move sync (added earlier in this Unreleased band) is gone. In 71 runs the workflow never received a single projects_v2_item: edited event from GitHub — delivery for this event on private-org → private-repo is unreliable and unfixable from our side. Every push to a feature branch was also producing a phantom failure run with 0 jobs (GHA quirk for workflows whose only trigger is project-related). The "manual board moves auto-mirror to the linked counterpart" feature is now manual: move both cards yourself. Composite actions (board-upsert-status, set-linked-issues-status) and board-config.env stay in place; restore the workflow from git history if/when GitHub fixes event delivery. AGENTS.md §Task Board state machine updated.

Fixed

  • Hub.Broadcast duplicate delivery (internal/api/hub.go): a channel subscribed to both an exact topic and a matching wildcard pattern (e.g. "ingest.clicks" + "ingest.>") received every broadcast twice because the exact-match loop never populated the sent set that the wildcard loop consulted. The one-line fix populates sent on the exact-match path. Regression covered by TestHub_WildcardNoDuplicateDelivery.
  • Ingest worker nil-deref on context cancellation (internal/ingest/bento_test.go): bentoMockMsg embedded jetstream.Msg as a nil interface, so m.Headers() / m.Subject() — called by observability.ExtractNATS at the top of jsInput.Read — panicked before any test even reached the cancellation path. Mock now provides real Subject() and Headers() methods backed by struct fields. Unblocks TestJsInput_Read_DeleteCancelledCtx and every other TestJsInput_* case.
  • Two Hub wildcard tests asserted impossible conditions: TestHub_WildcardStarNoMultiToken broadcast "ingest.clicks" (which does match "ingest.*") while asserting no delivery, and TestHub_WildcardDoesNotMatchExact broadcast "ingest.clicks" against "ingest.>" while asserting the > should not match. Corrected the broadcast subjects to actually exercise the wildcard no-match cases ("ingest.clicks.subpath" and "ingest" respectively).
  • Pebble dedup used != for ErrNotFound (internal/dedupe/embedded.go): switched to errors.Is so a future wrapped error doesn't slip past the not-found branch. Also bounded m.WAL.Size before the int64 conversion to satisfy gosec G115.
  • Hardcoded service name in cmd/wavehouse-api/main.go: OTEL provider was being initialized with the literal "wavehouse-standalone" in the clustered API binary. Now uses the serviceName variable so the clustered binary registers as wavehouse-clustered-api.
  • agent label trigger was unreachable AND under-gated: .github/workflows/claude-agent.yml's labeled-trigger branch gated on github.event.sender.author_association, which doesn't exist on the sender User object in issues.labeled webhook payloads — it's only set on content objects like comment / review / issue. The check silently evaluated false, making the agent label path never fire (flagged in Claude's post-merge review of #55). Fix: removed the unreachable association check AND added a first-step permission verification that calls the collaborators/permission API and requires admin / maintain / write before proceeding — closing the privilege-escalation gap Gemini flagged where a triage-role user could otherwise trigger the agent.
  • CONTRIBUTING.md — formatting guidance said gofmt; corrected to gofumpt to match what make fmt runs and make fmt-check enforces in CI. gofmt-formatted code would fail CI. Flagged in Claude's post-merge review of #55.

Changed

  • All three cmd/ binaries use a run() int pattern (cmd/wavehouse, cmd/wavehouse-api, cmd/wavehouse-worker, tests/cmd/bento_pub): main() now calls os.Exit(run()) so deferred OTEL shutdown and resource Close() calls actually fire on error exit — previously every os.Exit(1) inside main() silently skipped them. All HTTP servers gained a ReadHeaderTimeout: 10s to close gosec G112 (Slowloris).
  • Errcheck sweep across cmd/ and internal/: wrapped deferred Close / Shutdown calls in func() { _ = x.Close() }() and acknowledged ignored errors on json.Encoder.Encode, http.ResponseWriter.Write, fmt.Fprintf, rows.Close, and NATS m.Ack / m.Nak. jsInput.Read's Ack/Nak paths now log a warning on failure so lost acks are visible.
  • Narrowed revive ruleset in .golangci.yml to an explicit list of semantically useful rules (error-return, context-as-argument, var-naming, etc.). Dropped the defaults exported, package-comments, and unused-parameter which together produced ~50 findings that were pure comment-style noise for internal-only packages. Easy to re-enable per-rule later if the team wants to require doc comments.
  • Unit test coverage threshold temporarily lowered 70% → 60% in .github/workflows/ci.yml and Makefile, and added .testcoverage.yml with exclude.paths for packages that have no _test.go files (internal/dedupe, internal/mq, internal/observability, internal/testutil). Those packages get linked into test binaries and so appeared in the coverage profile at 0%, dragging the measured total to 54.7% even though the tested surface was at 64.9%. After the exclusions, measured total is 64.9% vs a 60% threshold. Restoring the 70% target + deleting the exclusions is tracked in #67.
  • Debug fmt.Println / fmt.Printf removed from cmd/*/main.go — replaced with logger.Info / logger.Error lines or folded into the existing startup log. Prevented unformatted plaintext lines from appearing interleaved with JSON logs in production (flagged by Claude's re-review of #66). Also dropped the hardcoded 172.18.240.1:4317 WSL-gateway default in cmd/wavehouse-worker/main.go — now defaults to 127.0.0.1:4317 like the other two binaries.
  • goreleaser-action CLI version pin: release.yml now passes version: "~> v2" to goreleaser-action instead of version: latest. Picks up patch / minor GoReleaser bumps automatically but breaks loudly on a v3 major bump rather than silently changing release binaries. Flagged in Claude's post-merge review of #55.
  • dependabot-automerge.yml permissions kept at contents: write + pull-requests: write: initially considered dropping contents: write per Claude's post-merge review of #55 (citing that gh pr review + gh pr merge --auto only need pull-requests: write). Copilot's review of #64 caught the edge case — when required checks are already green at the moment gh pr merge --auto runs, it performs an immediate squash merge, which needs tree-write. Keeping the broader scope to avoid silent failures in the all-green edge case.

Added

  • Repository governance files: CLAUDE.md and .gemini/styleguide.md pointers to AGENTS.md so Claude Code and Gemini Code Assist pick up project conventions automatically. (Note: .github/CODEOWNERS was originally added here but is removed later in this same Unreleased cycle — see the Changed entry at the top of this section.)

  • PR title linting: .github/workflows/pr-title.yml validates that PR titles follow Conventional Commits. Enforced as a required status check (Validate) in the main branch protection ruleset — PRs with non-conforming titles cannot merge. CONTRIBUTING.md documents the full accepted type list (feat, fix, docs, refactor, test, chore, ci, deps, build, perf, revert, style), with optional scope and optional ! breaking-change marker.

  • Issue triage automation: .github/workflows/triage.yml classifies new/edited issues via GitHub Models (gpt-4o-mini) and applies area/*, security, and breaking-change labels. When a PROJECT_BOARD_TOKEN secret with project scope is configured, also writes the Priority field on Task Board project #7. The step soft-fails if the secret is missing so label-only triage still works out of the box.

  • Claude Code agent workflow: .github/workflows/claude-agent.yml invokes anthropics/claude-code-action when an OWNER, MEMBER, or COLLABORATOR mentions @claude in an issue / PR / review / comment, or applies the agent label to an issue. Requires a CLAUDE_CODE_OAUTH_TOKEN secret (generated via claude setup-token); without it the workflow exists but cannot authenticate.

  • Claude PR review workflow: .github/workflows/claude-review.yml auto-reviews PRs opened or updated by OWNER/MEMBER/COLLABORATOR authors (Dependabot and drafts skipped). Runs alongside Gemini Code Assist; both are advisory — the Admin approval required status check + the main branch protection ruleset are the merge-gate. Fork PRs from first-time contributors are not auto-reviewed; a maintainer can invoke Claude by commenting @claude.

  • Dependabot auto-merge workflow: .github/workflows/dependabot-automerge.yml auto-approves and enables auto-merge on patch / minor dependency bumps (via dependabot/fetch-metadata). Major bumps post a comment and stay held for human review. Dependabot PRs bypass the Admin approval required check entirely, so all patch/minor bumps — including workflow-touching ones — merge hands-off after CI passes. Trust model: bot-approval + CI + fetch-metadata classification.

  • File-path auto-labeling: .github/workflows/label.yml (via actions/labeler) applies area/*, dependencies, github_actions, go, and documentation labels to PRs based on the files they change. Sync-mode: labels follow the current changed-file set. .github/labeler.yml is the path-to-label map — update it when adding a new internal/<pkg>/ package alongside the matching area/* label.

  • PR-title sticky comment: .github/workflows/pr-title.yml now posts (and auto-updates / auto-removes) a sticky comment when a PR title doesn't match Conventional Commits, explaining the format with examples. Moved from pull_request to pull_request_target so fork PRs also get the helpful comment. Still required by the ruleset as the Validate check.

  • Aggressive Gemini config: .gemini/config.yaml sets comment_severity_threshold: LOW so Gemini surfaces nit-level findings too; .gemini/styleguide.md rewritten with [MUST]/[SHOULD]/[MAY] classification, WaveHouse-specific security checks (SQL-injection in ClickHouse paths, JWT claim handling, raw-SQL permission gate), documentation-sync enforcement, and explicit Correct/Incorrect examples. Dialog via @gemini-code-assist mention or /gemini slash command.

  • docs/development.md — CI & review automation section: documents the Conventional Commits title format, the main branch protection ruleset's required checks, squash-merge behavior (PR title / body authoritative for commit message), the three AI reviewers (Claude / Gemini / Copilot) with their trigger patterns, how to invoke each (@claude, @gemini-code-assist, /gemini <cmd>, the Copilot re-request button), the review-response discipline for agents, issue triage, PR auto-labeling, and the workflow checklist for adding a new internal/<pkg>/ package. Updated Dependabot subsection with the new SDK and E2E npm ecosystems plus the patch/minor auto-merge flow.

Changed

  • Claude PR review prompt upgraded: .github/workflows/claude-review.yml prompt restructured to follow Anthropic's canonical pr-review-comprehensive template (Code Quality / Security / Performance / Testing / Documentation) with an explicit "rigorous, skeptical staff engineer" tone directive, OWASP Top 10 walk for the security section with severity rubric (CRITICAL/HIGH/MEDIUM/LOW/NONE), WaveHouse-specific concerns (Go concurrency, ClickHouse SQL-injection surfaces, AGENTS.md doc-sync rules), and Anthropic's noise-reduction filter ("post only the feedback you also deem noteworthy, keep feedback concise").
  • Issue template config: .github/ISSUE_TEMPLATE/config.yml disables blank issues and adds a Security Policy contact link that routes vulnerability reports away from public issues.
  • Repo labels — area/* rationalization: Added area/api, area/ingest, area/query, area/cache, area/dedupe, area/policy, area/pipes, area/sdk, area/docs, area/infra. Renamed Observability → area/observability (preserves label on 17 historical issues). Added agent, security, breaking-change. Deleted unused hardware-org leftovers (Hardware, Software, firmware).

Changed

  • Supply-chain hardening — SHA-pinned GitHub Actions: All uses: references in ci.yml and release.yml now pin to full-length commit SHAs with version comments (actions/checkout, actions/setup-go, golangci/golangci-lint-action, vladopajic/go-test-coverage, docker/login-action, goreleaser/goreleaser-action). Protects against tag-retargeting attacks on third-party actions. Dependabot tracks updates and will open PRs that bump both the SHA and the comment.
  • SECURITY.md refreshed: JWT section now documents JWKS support (previously only mentioned HMAC), adds role-based access control detail, distinguishes raw-SQL vs structured-query permissions, and notes the supply-chain posture (SHA-pinned actions, govulncheck, Dependabot).
  • AGENTS.md extended: New sections for repository automation tiers (triage / Gemini / Claude agent), governance files (CODEOWNERS intent, pointer-file discipline), and the workflow rule that adding an internal/ package requires adding a matching area/* label and extending triage.yml's area enumeration.
  • Hub wildcard support: Broadcast() now matches NATS-style topic wildcards (* for one token, > for one-or-more remaining tokens) so SSE/WS clients subscribing to patterns like ingest.> receive events for all matching subjects. Includes dedup to prevent double-delivery.
  • Auth ?token= query parameter fallback: JWT can now be passed via ?token=<jwt> query parameter for WebSocket and SSE connections where custom headers are not possible. The Authorization header takes precedence. Token is stripped from URL after extraction.
  • SSE id: field and Last-Event-ID reconnection: SSE events now include an id: field set to received_timestamp, enabling native EventSource automatic reconnection. The Last-Event-ID request header overrides ?since= for seamless gap-fill on reconnect.
  • WebSocket multiplexing: WebSocket connections now support in-band JSON commands ({"action":"subscribe","topic":"..."} / {"action":"unsubscribe","topic":"..."}) for dynamic multi-topic subscriptions over a single connection. Outbound messages are wrapped in {"topic":"...","data":{...}} envelopes. Backward compatible with ?topic= query parameter.
  • DLQ stats table filter: GET /v1/dlq/stats now accepts optional ?table= query parameter to filter stats to a specific table.
  • TypeScript SDK SharedWSManager: Single multiplexed WebSocket per client with ref-counted subscriptions, auto-reconnect with exponential backoff, and client-side NATS-style wildcard dispatch.
  • TypeScript SDK LiveQuery: Stream-first backfill orchestrator — subscribes to the stream immediately, buffers events, fetches historical data, deduplicates by timestamp, then resumes live updates. Available via queryBuilder.liveQuery(subscriber, opts?).
  • TypeScript SDK FilteredStreamController: Streams created from query builders with active filters/columns now apply client-side filtering (eq, neq, gt, gte, lt, lte, in, like, not_like) and column projection.
  • TypeScript SDK AbortSignal support: stream({ signal }) option wires an AbortSignal to close the stream when aborted.
  • TypeScript SDK SSE connection counter: Warns when opening more than 5 concurrent SSE connections (browser limit).
  • TypeScript SDK default query limit: QueryBuilder.DEFAULT_LIMIT = 1000 applied when no explicit limit is set, preventing unbounded result sets.
  • TypeScript SDK unit tests: Comprehensive Vitest test suite for @wavehouse/sdk — errors.test.ts, http.test.ts, query-builder.test.ts, table.test.ts, pipes.test.ts, namespaces.test.ts (sql, schema, policy, DLQ, sys), client.test.ts (factory, namespace wiring, transport selection), stream/controller.test.ts (subscribe, unsubscribe, async iterator, ref counting).
  • TypeScript SDK codegen CLI: npm run codegen -- --url <url> --out <file> introspects /v1/schema and generates a TypeScript Database interface with ClickHouse-to-TypeScript type mapping (String, numeric, DateTime, Array, Map, Nullable, LowCardinality, etc.).
  • TypeScript SDK playground: Three runnable scripts (playground:public, playground:auth, playground:admin) demonstrating unauthenticated queries/SSE, JWT auth/WebSocket streaming, and admin workflows (schema, policy, pipes, DLQ, raw SQL). Includes Docker Compose file and setup/seed script.

Removed

  • Dead code: BufferConsumer — Removed BufferConsumer, coerceValue, insertTableBatch, flushBatch, and sendToDLQ from internal/ingest/buffer.go (replaced by Bento pipeline in bento.go). The EventMessage struct and BufferConsumerName constant were preserved in a new types.go.

Added

  • Unit tests — Bento pipeline: bento_test.go covering safeIdentifierRe (15 subtests), dlqOutput.WriteBatch (3 subtests), jsInput.Read (6 subtests), and jsInput.Close.
  • Unit tests — Sweeper: sweeper_test.go covering sweep() (8 subtests), findGapSequence() (7 subtests), and Start() context cancellation.
  • CORS origin allowlist: New server.cors_allowed_origins config field (WH_SERVER_CORS_ALLOWED_ORIGINS) controls which origins receive CORS headers. Defaults to * (allow all). When specific origins are listed, only matching requests get CORS headers with Vary: Origin.
  • Config validation: config.Validate() runs at startup, catching invalid mode, out-of-range port, negative timeouts, missing auth credentials, zero schema refresh interval, and negative cache/gap-window values.
  • Default query limit: Structured queries are capped at DefaultMaxRows (10,000) to prevent unbounded result sets. Explicit limits exceeding the cap are silently reduced.
  • WebSocket origin enforcement: WebSocket upgrades respect the same cors_allowed_origins config as HTTP CORS middleware.
  • Pipes Store nil-safe Put/Delete: pipes.Store.Put() and Delete() gracefully handle nil KV backing (memory-only mode), enabling seamless unit testing with NewMemoryStore.
  • Unit tests — security & robustness: Config validation tests (11 scenarios), coerceValue tests (20 subtests), builder DefaultMaxRows tests, policy Validate/Evaluate edge cases (nil policy, negative MaxRows/MaxExecutionTime, service role, nil claims, multiple templates), ingest policy check clause match/auto-inject/deny-columns/admin bypass, discovery validation (Tuple/Enum16/Decimal/IPv4/IPv6/unknown type, nil data, all-defaults), tiered cache singleflight dedup verification, pipes handler Put/Delete success paths, router wiring verification, middleware default role claim and non-bearer token tests.
  • Test infrastructure: Expanded internal/testutil/ with shared mocks (MockPublisher, MockCache, MockDeduplicator, MockSubscriber in mocks.go), JWT helpers (MakeJWT, MakeExpiredJWT in jwt.go), schema helpers (NewTestSchemaRegistry), and response assertions (AssertJSONResponse, AssertJSONContains).
  • Policy test helper: policy.NewMemoryStore(p) for in-memory policy testing without NATS.
  • Schema test helper: discovery.NewSchemaRegistryFromMap(tables) factory for schema-aware tests without ClickHouse.
  • Unit tests — critical path: middleware_test.go (12 scenarios: auth disabled, dev mode, JWT validation, role extraction, claim extraction), policy_test.go (Evaluate, IsColumnAllowed, IsAggregationAllowed, navigateClaims, resolveTemplate), config_test.go (defaults, YAML loading, env overrides, invalid YAML), ingest_test.go (valid payload, missing/unknown table, invalid JSON, schema validation, dedup, publish errors, policy enforcement).
  • Unit tests — core features: hub_test.go, health_test.go, schema_test.go, transform_test.go, router_test.go, builder_test.go, tiered_test.go, pipes_test.go.
  • Unit tests — handlers & cache: policy_test.go (handler: Get nil/populated, Put/Validate invalid JSON, Validate valid), query_test.go (missing SQL, invalid JSON, policy forbids/allows raw SQL, cache key determinism), structured_query_test.go (missing/unknown table, invalid JSON, policy forbidden/column/aggregation checks), pipes_test.go (handler: List, Get found/not-found, Execute not-found/role-forbidden/allowed/wildcard/missing-param/query-params, Put invalid JSON), stream_test.go (SSE/WS applyStreamPolicy: no-policy passthrough, column filtering, forbidden table, non-event JSON, invalid JSON), local_test.go (Get miss, Set and Get, expired key, overwrite, zero TTL).
  • Pipes test helper: pipes.NewMemoryStore(queries...) for in-memory pipe testing without NATS.
  • Makefile targets: tools, check-tools, build-debug, fmt-check, lint-fix, fix, coverage-enforce, mod-tidy-check, vulncheck, security, deadcode, audit-cgo, size-report, size-tree, size-treemap, dep-graph, dep-why, dep-cut, binary-analysis, release-test.
  • Linters: Added bodyclose, noctx, errorlint, tparallel to .golangci.yml.
  • Agent instructions: Testing conventions section in AGENTS.md, testing rules (6, 7) in copilot-instructions.md.
  • Dependabot: .github/dependabot.yml with weekly grouped PRs for Go modules and GitHub Actions.
  • Vulnerability scanning: govulncheck ./... runs in CI (check job) on every push and PR.
  • Air install check: make dev now checks for air in $PATH and prints install instructions if missing (same pattern as make lint for golangci-lint).
  • .air.toml modernized: Added #:schema header, replaced deprecated bin with entrypoint, added exclude_regex, exclude_unchanged, stop_on_error, send_interrupt, [color], and [misc] sections.
  • Build tags support: TAGS variable in Makefile for conditional compilation (e.g., make build TAGS="scylla dynamodb").
  • VS Code workspace config: .vscode/settings.json with gopls build flags (tools, integration tags), schema overrides, and search exclusions. .vscode/extensions.json with recommended extensions.
  • .markdownlint.json: Disables MD024 (duplicate headings) and MD060 (table column style) for CHANGELOG and doc compatibility.
  • DLQ unit tests: internal/ingest/buffer_test.go — tests for sendToDLQ publish routing, flushBatch DLQ failover on insert failure, DLQ-disabled ack behavior, and mixed-table batching.
  • DLQ handler unit tests: internal/api/dlq_test.go — tests for Stats endpoint (empty stream, populated counts, single table) and EnsureDLQStream idempotency, using embedded NATS.
  • DLQ integration tests: tests/integration_test.go — end-to-end tests with ClickHouse testcontainer + embedded NATS: DLQ stats empty on fresh start, DLQ populated when Bento insert fails on non-existent table, successful ingest produces no DLQ entries.
  • Shared test utilities: internal/testutil/ package with NopLogger() for silencing embedded NATS output in tests.

Changed

  • Generated artifacts moved to tmp/: Coverage output now goes to tmp/coverage/, analysis artifacts (size-map.*, graph.*) to tmp/analysis/. Repo root is no longer cluttered with generated files. Updated Makefile, .gitignore, CI workflow, and all docs.
  • CGO scoped to build targets only: Removed global export CGO_ENABLED=0 from Makefile. CGO_ENABLED=0 is now set per-command on go build invocations only. Test targets use the system default (CGO_ENABLED=1), which is required for -race on Linux CI runners. Tests continue to work on macOS (which has a pure-Go race detector) and now also work on Linux CI.
  • CI jobs parallelized: lint, test, build, and check jobs now run fully in parallel. Previously lint waited for check and integration waited for build. Integration tests now gate only on test (unit tests must pass first).
  • GoReleaser config: Added ids filter to dockers_v2 (only linux builds go into Docker image) and OCI labels for image metadata.
  • make vulncheck: Default output is now a summary (-show=summary). Use V=1 make vulncheck for full details.
  • make clean: Simplified to rm -rf bin/ tmp/ data/ dist/ — all generated files now live under tmp/.
  • CI workflow: Refactored .github/workflows/ci.yml to use Makefile targets (make mod-tidy-check, make fmt-check, make vulncheck, make coverage, make test-integration). Pinned golangci-lint to v2.1.6.
  • filterEventColumns(): Returns filtered copy instead of mutating shared event data (prevents cross-client data corruption during SSE/WS broadcast).
  • Hub bridge subscription: cmd/wavehouse-api/main.go now fails startup on Subscribe error instead of suppressing it.
  • make ci: Expanded to 7 steps: tidy + fmt + lint + vulncheck + build + unit tests + integration tests.
  • Pinned dev tools: gotestsum, gofumpt, and goimports are tracked in go.mod via native tool directives (Go 1.24+). The Makefile uses go run — no manual tool installation needed. Eliminates @latest non-determinism.
  • Race detector enabled by default: All test targets (make test, make test-integration, make test-all, make ci) now run with -race. Critical for catching data races in WaveHouse's concurrent NATS/cache/streaming code.
  • Integration tests bypass cache: -count=1 added to make test-integration to ensure tests always run against fresh Docker containers.
  • Makefile overhaul: Added make setup, make fmt (gofumpt + goimports), make ci (full check: fmt + lint + tests). Replaced make check with make ci. Verbose output via V=1 make test. Colored output with @-prefixed commands.
  • CI workflow hardened: Added formatting check (gofumpt -l) and module tidiness check (go mod tidy + git diff --exit-code). Integration tests now run with -race -count=1. All test steps use gotestsum. Go version read from go.mod (no hardcoded version). Replaced Codecov with native go tool cover + GitHub step summary.
  • .golangci.yml v2 format: Uses version: "2" with default: none for explicit linter control. Added goimports linter.
  • Lint binary check: make lint and make ci now check for golangci-lint in $PATH and print install instructions if missing.
  • Code formatting: Switched from gofmt to gofumpt (strict superset). Added gofumpt to .golangci.yml linters.
  • Embedded NATS logger: mq.NewEmbedded() now accepts an optional *slog.Logger parameter to control server log output.
  • Test structure standardized: Moved bento_pub smoke test to tests/cmd/bento_pub/. Consolidated Makefile test targets with ARGS pass-through.
  • Analysis Make targets rewritten: size-tree uses gsa --format text --hide-sections for a clean package-size table (was broken goda weight). size-treemap outputs text + SVG + interactive HTML treemap, auto-opens HTML outside CI. dep-graph suppresses graphviz cluster warnings, CI-aware auto-open. dep-cut filters to InDegree ≤ 3, strips github.com/ prefix, respects LIMIT var (default 30). audit-cgo adds informational context about CGO_ENABLED=0. New LIMIT variable for analysis row limits. Analysis artifacts added to make clean and .gitignore.

Fixed

  • RequireRole fail-closed: Deny requests with no role when auth.enabled is true, preventing a missing JWT claim from inadvertently granting admin access.
  • Cursor pagination with DateTime64 filters: coerceFilterValue now returns ClickHouse-compatible formatted strings (e.g. 2026-04-02 16:25:50.297) preserving sub-second precision, instead of time.Time which the driver formats with second-only precision via toDateTime(). Fixes cursor pagination returning 0 rows on subsequent pages when DateTime64 columns are used as cursor keys.
  • Pipes template inline substitution: BindParams now inlines parameter values directly into the SQL string (with proper escaping) instead of using ? positional placeholders. String values that look numeric are inlined bare (for LIMIT etc.), other strings are single-quote escaped. Fixes {{limit:10}} templates causing ClickHouse syntax errors.
  • WebSocket double "connecting" status: StreamController.onStatus now deduplicates redundant status transitions and SharedWSManager.subscribe() defers initial status notification to _doConnect() when opening a new connection. Fixes duplicate Status: connecting events.
  • SQL injection in Bento ingest: Table names are now validated against ^[a-zA-Z_][a-zA-Z0-9_]*$ regex before use in SQL queries, preventing injection via crafted NATS subject metadata.
  • CORS origin reflection: Replaced permissive origin-echoing CORS middleware with allowlist-based validation. Non-matching origins receive no CORS headers.
  • os.Exit in library code: ingest.StartIngestWorker now returns (*service.Stream, error) instead of calling os.Exit(1), enabling proper error handling by callers.
  • Policy nil claim injection: resolveTemplate() returns "" for unresolvable JWT claim templates instead of leaking raw template text into SQL filters.
  • Data loss on shutdown: Buffer ticker goroutine now calls flush() before returning on context cancellation, ensuring in-flight batches are written to ClickHouse during graceful shutdown.
  • Hub channel leak: Hub.Unsubscribe() now calls close(ch) after removing the channel, preventing goroutine leaks in SSE/WS handlers waiting on the channel.
  • Ristretto race on Close: LocalCache.Close() now calls cache.Wait() before cache.Close(), ensuring all async admission goroutines have finished before teardown. Added LocalCache.Wait() for test use.
  • Test race in tiered cache: memCache test helper now uses sync.RWMutex to prevent concurrent map read/write during singleflight tests.
  • Docs — configuration: Added missing clickhouse.http_port field, fixed policy.file_path default to policy.yaml, fixed pipes.directory default to ./pipes.
  • Docs — deployment: Fixed cluster.yaml → clustered.yaml Docker Compose file references.
  • Docs — development: Updated Makefile targets table (removed nonexistent build-all/compose-cluster, added all new targets), added missing linters to list, added missing packages to project structure, updated vulnerability scanning to use make vulncheck.
  • config.yaml: Fixed policy.file_path and pipes.directory defaults to match config.go struct tags.
  • DLQ stats per-subject counts: GET /v1/dlq/stats now passes WithSubjectFilter(">") to NATS Stream.Info(), fixing empty per-table breakdown in the response.
  • Bento DLQ subject routing: Bento ingest worker now sets table_name metadata on messages so the DLQ output routes to dlq.<table> instead of dlq.unknown.
  • JWKS authentication: New auth.jwks_url config for public key validation via JWKS endpoint. JWKS is tried first, falling back to HMAC secret. Powered by keyfunc/v3.
  • Role-based access control: JWT role extraction from configurable claim path (auth.role_claim). Built-in admin/service roles with full access; other roles governed by policies.
  • Hasura-style access control policies: Per-table, per-role column and row-level permissions with JWT claim templating ({{ jwt.path }}). Stored in NATS KV (WAVEHOUSE_POLICY) with file-based YAML/JSON bootstrap and cluster-wide sync via KV Watch.
  • Policy admin API: GET/PUT /v1/admin/policy for CRUD, POST /v1/admin/policy/validate for dry-run validation.
  • Structured query endpoint: POST /v1/tables/{table}/query accepts a type-safe query AST (columns, aggregations, filters, group by, order by, limit, time range). Validated against schema, permissions enforced, converted to parameterized SQL.
  • Timestamp bucketing: Structured queries truncate time ranges to configurable buckets (cache.timestamp_bucket_seconds, default 60s) to improve cache hit rates.
  • Named query pipes: Pre-defined SQL templates with parameter binding, role restrictions, and caching. GET/POST /v1/pipes/{name} for execution. Admin CRUD at /v1/admin/pipes/*. Bootstrap from .sql files via pipes.directory.
  • Ingest permission enforcement: When policies are active, ingest checks insert permission, validates allowed columns, enforces check rules, and auto-injects claim-derived values.
  • Stream permission filtering: SSE and WebSocket streams filter events per role — denied columns are removed and unauthorized tables are skipped.
  • Dev mode: auth.dev_mode skips JWT validation and treats all requests as admin (development only).
  • internal/policy/ package: Policy types, evaluation engine, and NATS KV store.
  • internal/pipes/ package: Named query types and NATS KV store with .sql file bootstrap.
  • internal/query/ package: Structured query AST types, SQL builder with schema validation, permission injection, and timestamp bucketing.
  • TypeScript SDK (clients/ts/): @wavehouse/sdk — zero-dependency client with type-safe query builder, real-time SSE streaming, live queries with smart aggregation updates (incrementable/decomposable/poll), and codegen CLI for generating typed interfaces from ClickHouse schemas.
  • Schema discovery: New internal/discovery/ package introspects ClickHouse system.columns to build a live schema registry. Schemas are cached and auto-refreshed on a configurable interval (schema.refresh_interval / WH_SCHEMA_REFRESH_INTERVAL).
  • Schema validation: Ingest payloads are validated against discovered ClickHouse schemas — unknown fields, type mismatches, and non-nullable violations are rejected with descriptive 400 errors.
  • Schema API endpoints: GET /v1/schema (list all tables), GET /v1/schema/{table} (single table), POST /v1/schema/refresh (force refresh).
  • Dead Letter Queue (DLQ): Failed batch inserts are published to a separate NATS stream (WAVEHOUSE_DLQ) instead of being silently lost. Controlled by dlq.enabled / WH_DLQ_ENABLED.
  • DLQ stats endpoint: GET /v1/dlq/stats returns pending message count and consumer info.
  • Optional authentication: JWT auth is now opt-in via auth.enabled / WH_AUTH_ENABLED (defaults to false). When disabled, all /v1/* routes are open.
  • Optional deduplication: Dedup is now opt-in via dedupe.enabled / WH_DEDUPE_ENABLED (defaults to false). When enabled, specify the dedup key field with dedupe.id_field / WH_DEDUPE_ID_FIELD.
  • Table-based ingest routing: Ingest endpoint is now POST /v1/ingest/{table} — the table name comes from the URL path.

Changed (breaking)

  • BREAKING: JWT middleware signature: JWTAuthMiddleware now takes AuthConfig struct instead of (secret, enabled) pair to support JWKS, role claims, and dev mode.
  • Raw SQL restriction: Non-admin/service roles must have raw_sql: true in their policy to use POST /v1/query.
  • Bento ingest worker: Replaced Go channel bridge (dataChan) with direct JetStream pull via consumer.Messages(). Eliminates the 1000-message buffer, ensures NATS acks happen immediately after ClickHouse writes, and removes all package-level mutable state. The custom Bento input plugin is now registered at runtime with the JetStream consumer captured via closure instead of using init() and globals.
  • BREAKING: Dropped multi-tenancy — Removed tenant_id from JWT claims, middleware, ClickHouse schema, dedup keys, query filtering (CTE injection), and all API request/response formats. WaveHouse is now a single-tenant gateway.
  • BREAKING: Dropped schemaless typed maps — Removed the str_data/num_data/bool_data Map columns, Flatten()/Unflatten() functions, and the fixed events table. WaveHouse now writes to user-defined ClickHouse tables with real columns.
  • BREAKING: New ingest format — Body is now a flat JSON object (e.g., {"page": "/home", "score": 42}) posted to POST /v1/ingest/{table}. The old {"id", "table_name", "data"} envelope is removed.
  • BREAKING: Bring Your Own Schema — WaveHouse no longer auto-creates ClickHouse tables. Users must create tables before ingesting. Removed clickhouse.auto_migrate / WH_CH_AUTO_MIGRATE and dedupe.auto_migrate / WH_DEDUPE_AUTO_MIGRATE config options.
  • BREAKING: Query endpoint is direct passthrough — SQL is forwarded to ClickHouse as-is. No CTE injection, no tenant scoping.
  • BREAKING: Auth disabled by default — Previously JWT was always required. Now it's opt-in.
  • BREAKING: Dedup disabled by default — Previously dedup was always on. Now it's opt-in with configurable ID field.
  • NATS subjects changed from ingest to ingest.<table> (e.g., ingest.clicks).
  • DLQ subjects use dlq.<table> pattern.
  • SSE/WS default topic changed from ingest to ingest.> (wildcard for all tables).
  • BufferConsumer now groups events by table and performs per-table dynamic INSERTs using only provided columns (omitted columns use ClickHouse defaults).
  • BufferConsumer coerces JSON float64 values to the correct Go integer/float types before appending to ClickHouse batches.
  • Query cache key now includes both SQL and parameters — different parameter values no longer share caches.
  • Dedup CheckAndMark signature changed from (ctx, tenantID, eventID) to (ctx, eventID).
  • ScyllaDB dedup table simplified to PRIMARY KEY (event_hash) (removed tenant_id column).

Removed

  • internal/schema/ package (Flatten, Unflatten functions) — replaced by internal/discovery/.
  • internal/ingest/schema.go (EnsureSchema function) — no more auto-migration.
  • TenantIDFromContext() and tenant middleware context key.
  • injectTenantFilter() CTE query rewriting.
  • tenant_id from all ClickHouse INSERT/SELECT operations.
  • Fixed events table schema and auto-migration DDL.
  • EventMessage.TableName envelope field (table now comes from URL).
  • clickhouse.auto_migrate and dedupe.auto_migrate config options.