"missing-metadata" aggregate rule should already trigger on one file

[ghmer]

The "missing-metadata" rule seems to be an aggregate rule that is only triggered if there is more than one policy file to lint.
As we are currently looking into onboarding applications providing their own rego policies, we'd like to trigger this rule even if there is only one policy file to lint, so that we can establish "proper annotations" right from the start.

We therefore kindly ask to either change this rule to allow for "single rego policy files", or advice on how to achieve checking for missing-metadata without having to rely on the aggregate rule.

Thanks!

[anderseknert]

Thanks for raising this 👍 We'll try to get an update in before the next release.

[anderseknert]

Hey, so I've spent some time looking into this yesterday, and it reminded me why we choose to have aggregate rules trigger only when a whole project/workspace is linted, and not for single file. So, the thing is, even when someone lints a single file, that file often is part of a project. And those other project files commonly provide the aggregated data needed for compliance for a given aggregate rule. So having aggregate rules enabled when listing single files easily results in situations like this:

> regal lint my/project
100 files linted. No violations found.

> regal lint my/project/policy.rego
1 file linted. 
[...long list of violations...]

I.e. the linter reporting no issues when pointed at the project, while simultaneously reporting (potentially many) issues when pointed at individual policies in the same project (it claimed to find no issues in). While there is evidently a risk that some user actually only has one file in their project, that should be a rather temporary problem, as the issue is solved as soon as another file is added. Whereas the opposite approach would mean the issue demonstrated above would manifest for the lifetime of the project.

So in order to solve this, we could either:

1. Provide a single-file override option, that'd enable aggregate rules even for single files getting linted. But who would remember to add a special flag when linting only a single file?
2. Try to be smarter about a single file getting linted, and even in that case try to check for surrounding files, Regal configurations, etc.. and if found, only then disable aggregate rules as we do now.
3. Something else I haven't thought of :)

Unless someone has an idea for option 3, the second option seems like the better idea. It would require more work than what I had hoped for though, and not likely something we can get in for the next release.

@ghmer let me know if you have any ideas for how we could approach this better with the issue mentioned above in mind.

[ghmer]

I see the issue, and while I don't have any great idea of handling this in a generic way for all aggregate rules, I wonder if this could be handled differently for the one dedicated "missing-metadata" rule, as this is (probably?) the only aggregate rule that makes sense to evaluate even with a single file?

Also, to be more specific: I am not looking especially into a way of linting only one file, it is more about linting the project and running the configured rules even if the project only consists of one rego file. Not sure if that makes a difference, tho.

A "hack" seems to be to provide another "empty" / bare minimal rego file. I wonder if this was something regal could automagically to? Like having an option on that particular rule to "simulate" another rego file existing in this project?
Hmm, reading this again sounds really quite hacky, but I am desparate 🥹

[anderseknert]

Agreed, the best short term workaround would be to simply add an additional file to the project, so that linting the workspace always will involve more than a single file. If you're using some template repo for policy projects, maybe having a "root" package for docs like we do in Regal makes sense: https://github.com/open-policy-agent/regal/blob/main/bundle/regal/regal.rego

Leaving this open as I do want to make this better... but will need some time to think about what it'd look like first.