Skip to content

[Maintenance] testing: Per-PR perf regression detection (BDN delta vs base branch) #144

Description

@Chris-Wolfgang

Scope

#184 / P2 (benchmark gh-pages chart) graphs the BDN trend on every push to main — a backward-looking signal that surfaces regressions after they merge. The deferred work is the forward-looking signal: compare the PR's BDN run against its own base branch and post a delta-table comment on the PR.

A PR that doubles allocations on a hot path should be caught before merge, not after.

Acceptance criteria

  • A pr-benchmarks.yaml workflow runs the benchmark suite on the PR's HEAD and on base...HEAD's parent, captures both BDN JSON outputs, computes the per-benchmark delta (time / allocations), and comments on the PR with a markdown table.
  • The comment is replaced on each push to the PR (not appended) so the latest delta is always at the top.
  • The workflow fails the PR if any delta exceeds a configurable threshold (e.g. 20% slower or 50% more allocations) AND the PR didn't tag itself perf-impact-acknowledged.
  • The workflow only runs when src/ or benchmarks/ files change (path filter).

Notes

Metadata

Metadata

Assignees

No one assigned

    Labels

    maintenance - testingMaintenance: coverage %, integration/smoke/mutation tests, fixturesmaintenance-taskA Maintenance sub-issue — actionable improvement workthorough-reviewDeferred deep-review tier — work on canonical Tier-1 issues first

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions