Skip to content

bowire test CLI runner with JUnit / SARIF / TTY output #181

Closed

Description

@thomas-stegemann

Problem

Bowire is currently a UI tool. The "shift-left" / "build safer apps through continuous testing" positioning needs a headless surface that runs in CI and emits CI-friendly output (JUnit-XML, SARIF, GitHub annotations). Without it, the workbench stays a one-developer-at-a-time explorer.

Proposal

Add a bowire test CLI subcommand that:

  1. Loads a workspace file (or a specific collection / recording / suite)
  2. Runs every invocation
  3. Evaluates assertions (per the assertion DSL issue)
  4. Aggregates pass/fail/skipped
  5. Emits results in one of:
    • JUnit-XML (for Jenkins, Azure DevOps, generic CI)
    • SARIF (for GitHub Code Scanning panels)
    • GitHub annotations (::error file=...,line=...)
    • Plain TTY (human-readable summary)
  6. Exits non-zero on any failure (configurable threshold)
bowire test --workspace=./bowire.workspace.json --suite=smoke \
    --report=junit --output=junit.xml --fail-on=red

Workspace + collections persist as plain JSON already (#159 Export/Import) — the runner reuses that loader. Environment vars resolve via --env-file or inherited shell vars. Secrets via env vars only (never in checked-in files).

Acceptance

  • bowire test subcommand in the existing CLI
  • Loads a workspace / suite from disk
  • Runs every invocation + assertion
  • JUnit-XML output (works in any CI)
  • SARIF output (lights up GitHub Code Scanning tab)
  • GitHub-Action wrapper repo with example workflow
  • Exit code reflects pass/fail, threshold configurable
  • --env-file, --vars, --workspace-id flags
  • Headless: no UI, no browser, no network beyond the calls under test
  • Sample CI workflow added to docs/setup/

Related

  • v2.2 — Test pillar
  • Pairs with assertion DSL
  • Foundation for bowire scan (v2.3) which uses the same runner skeleton

Activity

  1. moved this to Backlog in Bowireon Jun 15, 2026
  2. thomas-stegemann commented on Jul 3, 2026

    @thomas-stegemann
    MemberAuthor

    Progress — the CLI surface of this ticket is now complete:

    Still open before closing:

    • --workspace-id input (load a workspace instead of a file)
    • GitHub-Action wrapper repo + example workflow
    • Sample CI workflow in docs/setup/
    • Configurable failure threshold (--fail-on)
  3. thomas-stegemann commented on Jul 3, 2026

    @thomas-stegemann
    MemberAuthor

    Shipped. The bowire test CLI is complete:

    • Subcommand, loads recording / flow from disk, runs invocations + assertions
    • JUnit XML (--junit), SARIF 2.1.0 (--sarif), GitHub ::error annotations (--annotations)
    • Exit code reflects pass/fail; threshold configurable via --fail-on any|never (82825f4)
    • --env / --vars / --env-file; --workspace <dir> runs a whole git-native workspace (82825f4)
    • Headless (no UI/browser)
    • GitHub Action wrapper — packaging/github-action/action.yml, usable as Kuestenlogik/Bowire/packaging/github-action@v2
    • Sample CI workflow — docs/setup/ci.md (GitHub Actions + GitLab + contract testing)

    --workspace-id (by-id against ~/.bowire storage) awaits the storage-flip → tracked in #365.

  4. moved this from Backlog to Done in Bowireon Jul 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    • Status
      Done

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions