Skip to content

ci: run snip verify so the filters' inline tests are enforced - #139

Merged
edouard-claude merged 1 commit into
masterfrom
ci/run-snip-verify
Jul 27, 2026
Merged

ci: run snip verify so the filters' inline tests are enforced#139
edouard-claude merged 1 commit into
masterfrom
ci/run-snip-verify

Conversation

@edouard-claude

Copy link
Copy Markdown
Owner

Found while reviewing the #136 fixes.

The 132 bundled filters carry inline tests: blocks and 21 of them use it. Nothing runs them. ci.yaml has go test -race -cover, go vet, golangci-lint and govulncheck, and go test does not read filter YAML. A contributor could break any filter's own fixtures and CI would stay green.

That is not hypothetical here. #117, #121, #132, #135 and #136 each added or changed inline tests precisely so snip verify would guard the behaviour being fixed:

All of those were unenforced.

The change

One step in the test job. The binary is built first because verify runs the embedded filter set rather than the files on disk, so an edit under filters/ is only visible after a rebuild.

      - name: Verify filters
        run: |
          go build -o /tmp/snip ./cmd/snip
          /tmp/snip verify

Confirmed it can actually fail

Breaking one expected: block in filters/git-diff.yaml and rebuilding:

git-diff ............ FAIL (2/3 passed)
132 filters, 20 tested, 45 tests, 44 passed, 1 failed
$ echo $?
1

Restored, and snip verify is back to 45/45 with exit 0. Runtime is under a second, so it costs nothing in the pipeline.

The 132 bundled filters carry inline `tests:` blocks, and 21 of them use it.
Nothing ran them: ci.yaml has go test, go vet, golangci-lint and govulncheck,
and `go test` does not read filter YAML. A contributor could break a filter's
own fixtures and CI stayed green.

That is not hypothetical for this repo. #117, #121, #132, #135 and #136 all
added or changed inline tests specifically so `snip verify` would guard the
behaviour they fixed, and none of those guards was actually wired to anything.

The binary is built first because verify runs the embedded filter set, not the
files on disk, so an edit to filters/ is only visible after a rebuild.

Confirmed the step can fail: breaking one expected block in filters/git-diff.yaml
gives "git-diff FAIL (2/3 passed)" and exit status 1.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant