Skip to content

Report rollup across services — one view over the artefacts Bowire already writes #587

Closed

Description

@thomas-stegemann

Split out of #188, which stays as the hosted-platform epic (deployable, upload API, auth, multi-tenancy) and moved to v3.0.

Why this is small

Everything an org rollup needs to aggregate already exists as structured artefacts on disk:

Source Artefact
bowire lint findings JSON (rule id, severity, service, method, field)
bowire contract verify .bowire/contract-results/<consumer>__<provider>.json
bowire bench run / scheduled runs k6-summary JSON + .bowire/benchmark-schedules/<id>.runs.json
bowire scan SARIF
bowire test JUnit
the PR bot an assembled per-repo report

So "what's the state of our APIs?" across N services is a read-and-aggregate problem over files Bowire already writes — the same vertical as the contract matrix (#364): one engine, four surfaces, no new infrastructure.

Scope

  • bowire report rollup --from <dir|file…> — read every report it finds, emit a per-service rollup: findings by severity, contract pass/fail, latest p95, test pass-rate. Text table plus --json.
  • --fail-on <severity> so a platform pipeline can gate on the rollup, matching lint, contract matrix and bench run.
  • Workbench rail rendering the same rollup (rows = services, columns = the signals above), drilling into a service's underlying report.
  • bowire.report.rollup MCP tool returning the same shape, so an agent can answer the portfolio question.
  • Shared wire shape between all three surfaces — one projection, as in Contract testing: workbench matrix view (consumer × provider pass/fail) #364, so CLI JSON and API JSON cannot drift.

Explicitly out of scope (that is #188)

Acceptance

  • bowire report rollup --from <path…> reads lint / contract / bench / scan / test artefacts and prints a per-service rollup
  • --json emits the shared wire shape; --fail-on <severity> gates non-zero
  • Workbench rail shows the rollup and drills into a service's source report
  • bowire.report.rollup MCP tool returns the same shape
  • A missing or malformed report degrades that one row, never the rollup
  • Tests covering the readers, the aggregation and the shared shape

Related

Activity

  1. thomas-stegemann commented on Aug 20, 2026

    @thomas-stegemann
    MemberAuthor

    Shipped across all four surfaces.

      SERVICE      WORST    LINT (H/M/L)   CONTRACTS   TESTS       P95       LAST
      billing-api  HIGH     —              0/1         —           312ms     2026-08-20
      gateway      HIGH     —              —           —           —         —
                   scan: 1 finding(s) at error level
      orders-api   MEDIUM   0/1/1          1/1         42/42       —         2026-08-19
    
      3 service(s) · 2 at high · 0 clean · 1 file(s) skipped
    

    The decision the whole thing rests on: parse the file formats, not the producing packages' types. A rollup answers a question across services, so its inputs come from other repositories, other machines and other Bowire versions. Deserialising into ContractVerificationReport or BowireBenchmarkScheduleRun would have required those packages to be loaded in the process doing the reading — which defeats the purpose, and would break the moment an upstream field changed. Format detection goes by content rather than file name, because CI jobs name these things whatever they like and a results.json could be any of four shapes.

    An em dash is not a zero. Every count is nullable: — means there is no such report, 0/1 means the report exists and one contract failed. Flattening both to 0 would let a service nobody has ever linted read as a clean bill of health — the single most likely way a portfolio view lies to you. In JSON the same distinction is null versus a number.

    Attribution needs no configuration. In order: an explicit --service, then the report's own identity where the format has one — a contract files under its provider, the service under test, not the consumer — then the first path segment that isn't storage layout (so .bowire/contract-results/web__orders.json doesn't invent a service called contract-results), then the file name. A CI job that collects reports into reports/<service>/ just works.

    Worst-severity ranks regressions above style. A failing contract, a failing test or a scan error is high; a breached latency budget is medium; lint severities fall below. That ordering is what --fail-on gates on.

    Four surfaces, one projection. CLI (bowire report rollup, --json, --fail-on), POST /api/report/rollup, the Rollup rail, and the bowire.report.rollup MCP tool all emit the same JSON through BowireRollupPayload — the lesson from #364, where serialising the model directly made the CLI print an enum ordinal while the endpoint printed a string. The endpoint is POST rather than GET because the paths are input, and a rollup over an arbitrary directory tree does not belong in a URL that lands in a log or a browser history.

    Verified end to end, not only in tests: against a three-service fixture the CLI produced the table above, --fail-on high exited 1, the live endpoint returned the same shape, the rail rendered the same rows in the browser, and the foreign file was skipped rather than breaking the run.

    Tests: 27. Twenty over the reader — every format's shape, the aggregation rules, attribution including the mixed-report fallback, and that malformed or foreign files are skipped with a reason while the rest of the portfolio still reports. Seven over the rail, including one that pins the path input: its value is rendered from state on every render, so a keystroke that never reached state would be silently reverted. That test exists because the browser pass could not settle the question — the machine's automation layer froze first — and a test answers it permanently where a retry would not.

    Docs: docs/features/report-rollup.md (+ toc), the MCP tool list, and a release-note highlight.

    Commits: 6bdc543 (reader), 7adef56 (CLI, endpoint, rail, MCP), 4fd30b6 (rail tests), 9704513 (docs). CI green.

    Out of scope and unchanged: uploading reports to a shared store, retained history for trends, org login and admin actions — that is the hosted dashboard, #188, now in v3.0.

  2. moved this from Backlog to Done in Bowireon Aug 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    • Status
      Done

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions