Skip to content

Fix branch coverage tracking and expand test suite #444

Open
benber86 wants to merge 38 commits intomasterfrom
fix/coverage-refactor
Open

Fix branch coverage tracking and expand test suite #444
benber86 wants to merge 38 commits intomasterfrom
fix/coverage-refactor

Conversation

@benber86
Copy link
Collaborator

@benber86 benber86 commented Mar 2, 2026 •
edited
Loading

What I did

Branch coverage barely worked with conditional statements/loops/multiline statements as described in #327.

How I did it

  • Replaced ast-based coverage trace approximation with bytecode based one
  • Used fuzzer + mutation test based harness to get claude to find, fix and write tests for all edge-cases

How to verify it

python -m pytest tests/unitary/test_coverage

Description for the changelog

Fix branch coverage and expand test suite

Cute Animal Picture

image

@benber86
shut mypy up
2bc55f0
@benber86
fix branch coverage arc tracking
c54eba7
@benber86
add tests for branch coverage
986820d
@benber86
cleanup
7543a22
@benber86
expand coverage test suite
42a8d15
@benber86
clean up
2db2df6
@benber86
Add coverage integration test
bcd88ec
@benber86
fix regression on multi line if statements
37b0778
@benber86
fix: skip operator AST nodes in coverage _lines exclusion loop
5837b4a
@benber86
fix: correct branch coverage arc detection for if-in-loop and void fu…
ad72e8f 
…nction edge cases
@benber86
fix: handle _false_arc sibling lookup for if nodes in orelse blocks
f517ccc
@benber86
correct branch arc tracking for if-else in for loops and remove gap h…
4468fc0 
…euristic
@benber86
exclude keyword-only lines of multi-line statements from coverable lines
e2ff9ad
@benber86
expand testing for more edge cases (pass only function, functiondef, …
cafe0b0 
…keyword only line)
@benber86
align branch arc targets with tracer output for multiline statements
4b288a4
@benber86
expand tests to cover more multiline edge cases
7ced162
@benber86
Refactor and fix regressions
a6a3564
@benber86
refactor to move away from heuristics
d232ca4
@benber86
Replace heuristic branch resolution with JUMPI-anchored raw-trace cla…
345e05f 
…ssification
@benber86
clean up and break down tests
8755f55
@benber86
fix regression on blank retrun statements with conditional statement
d5425a3
@benber86
fix repeated-If suppression for internal calls
79ff9d0
@benber86
fix compound-condition JUMPI misselection for break/continue/bare-return
508b977
@benber86
fix false full coverage when If.test contains internal function calls
b881472
@benber86
exclude site-packages rom coverage
74a2171
@benber86
expand test suite + fixes from new cases found by fuzzer
b152acd
@benber86
more edge case fixes + tests surfaced from mutation matrix loop
93304b7
@benber86
mutation matrix surfaced edge cases
c9e6b27
@benber86
more tests and fixes from mutations
cf87df0
@benber86
last mutants and cleanup
8481ee0
benber86 and others added 8 commits March 2, 2026 21:00
@benber86
fix noop-body branch reporting and some phantom arcs
8e09b2a
@benber86
final cleanup
6dda72f
@benber86
handle api inconsistency in coverage
aa8ce49
simplify: replace bytecode heuristics with direct JUMPI→ast_map lookup
9989e68 
The compiler's pc_raw_ast_map already maps JUMPI PCs to their If AST
nodes. Instead of six JUMPI-finding functions (forward scan, backward
scan, noop backward, helper gap detection, path classification,
event building), just walk the raw trace and when we hit a JUMPI that
maps to an If node, check the next PC to determine direction.

coverage.py: 815 lines → 497 lines
- Remove _build_coverage_events, _find_jumpi_forward_scan,
  _find_jumpi_noop_backward, _find_jumpi_backward, _has_helper_gap,
  _classify_from_raw_trace, _resolve_jumpi_direction
- Replace CoverageCollector with BranchCollector
- Keep reporter improvements (noop branch exclusion, multiline
  handling, proper false target resolution)

environment.py: simplify _trace_computation to pass computation
directly to collector instead of building event segments
fix docstring: CoverageCollector → BranchCollector
5566d40
@benber86
Merge and fix issue with coverage warnings
8f7df9c
@benber86
Trim down tests
9a4aaa6
@benber86
skip tracing when no active coverage session and avoid unnecessary al…
08185e6 
…locations in skip_arcs path
@benber86
Copy link
Collaborator Author

benber86 commented Mar 3, 2026 •
edited
Loading

  • Forced disabling optimizations when using branch coverage (users get warning)
  • Disabling optimization allows to remove most of the yuge amount code that used heuristics to deal with branch removals/op code changes
  • Removed multiline statement handling (exclusion) logic (added complexity but only gives a slightly more accurate statement count in report, coverage still readable either way)
  • We now just walk the trace, if a PC is JUMPI we look up corresponding AST Node in pc_raw_ast_map, then walk AST to find branch targets (_branch_arcs_for_if). From the trace we infer which arc was executed by checking if next trace PC == next bytecode PC or not.
  • Trimmed down test suite to remove exotic edge cases

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@benber86