ci: run snip verify so the filters' inline tests are enforced - #139
Merged
Conversation
The 132 bundled filters carry inline `tests:` blocks, and 21 of them use it. Nothing ran them: ci.yaml has go test, go vet, golangci-lint and govulncheck, and `go test` does not read filter YAML. A contributor could break a filter's own fixtures and CI stayed green. That is not hypothetical for this repo. #117, #121, #132, #135 and #136 all added or changed inline tests specifically so `snip verify` would guard the behaviour they fixed, and none of those guards was actually wired to anything. The binary is built first because verify runs the embedded filter set, not the files on disk, so an edit to filters/ is only visible after a rebuild. Confirmed the step can fail: breaking one expected block in filters/git-diff.yaml gives "git-diff FAIL (2/3 passed)" and exit status 1.
This was referenced Jul 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found while reviewing the #136 fixes.
The 132 bundled filters carry inline
tests:blocks and 21 of them use it. Nothing runs them.ci.yamlhasgo test -race -cover,go vet,golangci-lintandgovulncheck, andgo testdoes not read filter YAML. A contributor could break any filter's own fixtures and CI would stay green.That is not hypothetical here. #117, #121, #132, #135 and #136 each added or changed inline tests precisely so
snip verifywould guard the behaviour being fixed:filters/tsc.yaml— the two tests updated in fix(filters): drop compact_path from all 28 bundled filters #117, the only automated guard thatcompact_pathdoes not come backfilters/git-diff.yaml,git-log.yaml,git-show.yaml— added in fix(filters): stop git filters answering a different question than the one asked #132 to pin that snip invents no file count and that the diffstat substitution is labelledfilters/rails-routes.yaml— added in fix(filters): keep the route table in rails-routes #135, and the only place that pins the fix, since the Go integration fixture is larger than the head cap and passes either wayAll of those were unenforced.
The change
One step in the
testjob. The binary is built first becauseverifyruns the embedded filter set rather than the files on disk, so an edit underfilters/is only visible after a rebuild.Confirmed it can actually fail
Breaking one
expected:block infilters/git-diff.yamland rebuilding:Restored, and
snip verifyis back to 45/45 with exit 0. Runtime is under a second, so it costs nothing in the pipeline.