-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Labels
needs-lldRequires LLD before implementationRequires LLD before implementation
Description
Summary
Create a tools/test-gate.py wrapper script that enforces the Skipped Test Gate in CI, complementing the soft gate in CLAUDE.md.
Background
Issue #81 adds a CLAUDE.md rule requiring agents to audit skipped tests. This is a "soft gate" - it relies on agent compliance. A hard gate would programmatically block PRs with unaudited critical skips.
Proposed Implementation
tools/test-gate.py
# Usage
python tools/test-gate.py pytest tests/ -v
# Wraps pytest, parses output, enforces auditBehavior:
- Run pytest with provided arguments
- Parse output for skipped tests
- Check for SKIPPED TEST AUDIT block in output or separate file
- If skips exist without audit → exit 1 (fail CI)
- If critical skips are UNVERIFIED → exit 1
- Otherwise → exit with pytest's exit code
Integration
# In GitHub Actions
- name: Run tests with skip gate
run: python tools/test-gate.py pytest tests/ -vAcceptance Criteria
-
test-gate.pywraps pytest and captures output - Detects skipped tests from pytest output
- Requires SKIPPED TEST AUDIT block when skips present
- Fails if critical tests are UNVERIFIED
- Passes through pytest exit code when no issues
- Works with common pytest flags (-v, -x, --cov, etc.)
Related
- Skipped Test Gate: Mandatory Audit Before Claiming Tests Pass #81 - Soft gate (CLAUDE.md rule) - should be done first
- Add GitHub Actions CI workflow for automated testing #116 - GitHub Actions CI workflow
Complexity
Medium - requires pytest output parsing and audit format validation
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
needs-lldRequires LLD before implementationRequires LLD before implementation