Skip to content

Benchmarks: 'scheduled' run shape + cron infrastructure (restart-survival) #232

Closed

Description

@thomas-stegemann

Goal

Add the scheduled run shape to Benchmarks — a saved envelope can be set to fire on a cron schedule, the workbench (or a CLI / embedded host) runs it without operator interaction, and results land in the saved-runs history. Survives a process restart (cron entry is persisted with the envelope, hosted service re-registers on boot).

Context

Split out from #131 (v2.0 trim for RC). v2.0 has no scheduling infrastructure at all; this is a fresh backend pillar, not a CSS polish, so it doesn't fit the v2.0 RC.

Fits the v2.5 — Continuous integration milestone because scheduled runs are how Bowire integrates into CI / synthetic-probe pipelines (publishes results via OTLP for the host's dashboards, see #29).

Acceptance

  • BenchmarkEnvelope carries an optional schedule: { cron, timezone, enabled } block
  • BenchmarkSchedulingHostedService (.NET IHostedService) reads enabled schedules on boot, registers cron triggers, fires the runner
  • Result of each scheduled run lands in envelope.runs[] (newest first) with triggeredBy: 'schedule'
  • Workbench Detail-Pane shows the next firing time + a "Pause schedule" toggle
  • Cron entries survive a process restart (no in-memory-only state)
  • CLI bowire bench schedule list reflects active entries (Phase 2 — optional for the initial ship)
  • OTLP / Bowire ActivitySource emits per scheduled run so an external dashboard can build a Lighthouse-style synthetic probe view (see Self-telemetry + Grafana dashboards #29 / Monitoring — scheduled probes + alerting (Postman Monitors / Synthetics analog) #102)

Shipped note (architecture): criteria 1 and 3 landed against a server-side schedule entity (.bowire/benchmark-schedules/), not against the browser-side BenchmarkEnvelope. A cron entry has to fire with no browser open, and a hosted service in the server process cannot read the workbench's localStorage — so the schedule carries its own target, load shape and thresholds, and its runs live in a sibling history file rather than in envelope.runs[]. Same capability, different home; the body above reflects the original sketch and this note the as-built.

Out of scope

  • Distributed multi-machine scheduling (one host owns its schedules; cluster-coordination is a different layer)
  • A "first-run-now" trigger when an operator saves a new schedule (separate manual Run button is enough)
  • Notifications on failure — wire into the existing toast / OTLP pipeline; no email / Slack here

Activity

  1. thomas-stegemann commented on Aug 20, 2026

    @thomas-stegemann
    MemberAuthor

    Shipped. A saved benchmark can now fire on a cron schedule with nobody watching, and the run lands in a history the workbench, the CLI and an OTLP dashboard all read.

    The one design decision everything else follows from. Envelopes live in the browser's localStorage, and a hosted service in the server process cannot read those. So a schedule is its own server-side entity under .bowire/benchmark-schedules/, carrying the target, the load shape and the thresholds next to the cron expression — see the shipped note on the body above. Pointing at an envelope id would have produced a scheduler that only works while a browser happens to be open, which is the opposite of the feature.

    Restart survival is not a separate mechanism — it is that persistence. BowireBenchmarkSchedulingHostedService holds no schedule state of its own; it re-reads the store on every tick. A bounce therefore loses nothing, and an operator adding, pausing or deleting a schedule takes effect without one.

    Due-ness is computed against the schedule's own last recorded run, not an in-memory timer. A missed window — host asleep, deploy in progress — still fires once when the process comes back, and a restart cannot re-fire an occurrence that already ran.

    Cron parsing uses Cronos rather than a hand-rolled parser, in the optional Benchmarking package (never Core, so an embedded host that doesn't reference the rail doesn't take the dependency). The schedule block carries a named timezone, and DST-correct arithmetic is exactly what a hand-rolled parser gets wrong. Proven in production, not only in a unit test:

    nightly   Nightly weather
        0 3 * * * [Europe/Berlin]   Weather/getCurrent @ http://127.0.0.1:6000
        next: 2026-08-20 01:00:00Z
    

    03:00 Europe/Berlin in August is CEST (UTC+2) → 01:00 UTC. The unit tests pin the January/July pair explicitly (02:00 vs 01:00 UTC for the same expression).

    Surfaces. CLI: bowire bench schedule add|list|pause|resume|remove. add validates before storing — a bad cron or unparseable threshold is rejected at the prompt with exit 64, because a schedule that can never fire is worse than a rejected one: nothing reports it later. HTTP: GET /api/benchmarks/schedules (with next firing time and the last run's headline inline) and POST .../{id}/enabled. Workbench: the Benchmarks detail pane lists each schedule with "in 4h 12m" (exact timestamp on hover), a Pause/Resume toggle, and the last run's verdict plus any breached budget.

    Deliberately no create endpoint. A schedule carries a target URL the server will then call unattended, so authoring stays on the CLI where the operator is explicit about it. The browser reads and pauses. An unattended outbound call should not be two clicks away in a browser.

    Observability. Every scheduled run opens an activity on the Kuestenlogik.Bowire.Benchmarking ActivitySource tagged with p95, error rate, throughput, the trigger and the pass flag, and sets an error status when a budget breaks — enough for the synthetic-probe view #29 / #102 describe.

    Defensive throughout: a bad cron, an unparseable threshold, an unloaded protocol or a failing run degrade that one schedule and leave the others running. A scheduler that dies on one bad entry stops them all.

    A real bug this shook out. The schedule handlers first took the store as a handler parameter. In a host that mounts MapBowire() without the scheduling service registration, minimal-API binding does not know that type, infers it as a body parameter, and throws while mapping — taking down the whole route group, index page included. Three MapBowire_* tests caught it as an HTTP 500 on the index. The endpoints now resolve their store themselves (DI when present, constructed otherwise). The lesson generalises: a rail package's endpoints must not assume the host ran its service registration — degrade, never break the pane.

    Tests: 84, in a new Kuestenlogik.Bowire.Benchmarking.Tests project — the package shipped without one, which is part of what #358 tracks. They cover the DST pair, restart survival through a fresh store instance over the same root, the no-double-fire rule, history ordering and its cap, path-hostile ids, and each degradation path.

    Commits: 7065467 + 29a8fb0 (the #360 engine this builds on), 1d62165 (schedule model, store, hosted service), a71d4f1 (CLI + API), fbaa7bc (workbench panel + CSS), 472dbbb (the binding fix). CI green.

    Out of scope as stated in the issue and still out: distributed multi-machine scheduling, a fire-now trigger on save, and failure notifications beyond the OTLP/toast pipeline.

    Browser verification (the panel, clicked rather than asserted): the Benchmarks pane shows

    Scheduled runs                                    [Refresh]
    Nightly weather              in 17h 27m           [Pause]
    0 3 * * * [Europe/Berlin] · Weather/getCurrent · 20× @ c1
    never run
    

    with the exact instant on hover. Clicking Pause flipped enabled to false in nightly.json on disk, the API dropped nextRun to null, and the row re-rendered as paused with the button now reading Resume; Resume restored the firing time — UI → endpoint → store → what the scheduler reads, end to end.

    That pass also caught a second real defect: the panel was initially invisible, because it hung off renderBenchmarksDetailMain's populated branch and that function returns early when no benchmark spec is selected. An operator with no saved specs would have seen "No benchmarks yet" while the host was firing schedules behind their back. Fixed in 4ccf5bfd. No unit test would have found it — the function returns a valid node either way.

  2. moved this from Backlog to Done in Bowireon Aug 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    • Status
      Done

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions