Inconsistent (redundant?) bundle builds

[tamirkamara]

Describe the bug

It looks like porter decides to rebuild a bundle for no consistent reason.
I see this when calling publish right after a build which is rebuilding everything. I also see it with explain...

To Reproduce

Steps to reproduce the behavior:

1. Run the following script: porter build && for i in {1..9}; do porter explain -oyaml > $i.yaml; done
2. Review the files created and notice that some include the "build prints" which shouldn't be there if every explain command used an existing built bundle.

Expected behavior

1. All explain in the script above should use the initial built bundle
2. When calling publish right after a build, the bundle isn't rebuilt

Version

Copy the output of porter version below

porter v1.0.4 (e3a2a5e)

[joshuabezaleel]

I assume this issue is related to #2391 as well (?)

[tamirkamara]

Looks like the underlining JIT build issue is the same, although there's no build-arg involved here.

[carolynvs]

@tamirkamara This doesn't reproduce as easily as your reproduction steps show. Are there any characteristics about your bundles that rebuild at the wrong times? Do you pass any flags to the build command? Do you run out of disk space and sometimes prune your local Docker cache?

[tamirkamara]

@carolynvs Most of our bundles exhibit this behavior with porter v1.0.4. For example: https://github.com/microsoft/AzureTRE/tree/main/templates/workspaces/base
It uses Terraform & AZ cli mixin, plus installs a couple of utils via a custom dockerfile.

(porter v1.0.8 with autobuild disabled solves this issue for us so maybe we can close it?)

[carolynvs]

It's to know that 1.0.8 with autobuild disabled is working for you. Yes, we can close for now and I'll keep an eye out for if I see it reproduce locally. Thanks!

[carolynvs]

I was able to reproduce this while fixing a bug in today's release. Reopening!

The problem is that when we calculate the manifest digest on the bundle.json stamp, we don't consistently sort the mixins, so depending on how Go feels like sorting the mixin map, we can get a different digest, and trigger rebuild.

[carolynvs]

Sorry this took so long to track down and reproduce! This will be included in this week's 1.0.13 release