E2E: go tag main config-ldflags-noassets slsa3

[laurentsimon]

Repo: https://github.com/slsa-framework/example-package/tree/v14.0.44
Run: https://github.com/slsa-framework/example-package/actions/runs/2540306790
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.go.tag.main.config-ldflags-noassets.slsa3.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.go.tag.main.config-ldflags-noassets.slsa3.yml
Trigger: push
Branch: v14.0.44
Date: Wed Jun 22 06:33:04 UTC 2022

[ianlewis]

The error encountered on the test is
["binary-linux-amd64","binary-linux-amd64.intoto.jsonl"] == ["null","null"] :: there should be no assets

This seems very strange bc the release does have assets on it
https://github.com/slsa-framework/example-package/releases/tag/v14.0.44

But the only run that I can find that ran on v14.0.44 didn't execute the upload-assets job
https://github.com/slsa-framework/example-package/actions/runs/2540306790

The time frame for when this job ran and when the assets were uploaded is about the same time (both are 18 hours ago as of this writing)

[ianlewis]

This is caused by two tests conflicting with each other. This test is not adding the assets but it's another test that's doing it.

For example this release has two sets of artifacts attached to it, one with the version in the names and one without.
https://github.com/slsa-framework/example-package/releases/tag/v20.0.32

It seems to be that e2e.go.tag.main.config-ldflags-assets-tag.slsa3.yml and e2e.go.tag.main.config-ldflags-noassets.slsa3.yml both have conflicts but perhaps not with each other?

[ianlewis]

It seems that these runs conflicted with each other. Both ran on tag v14.0.44:

• verifier-e2e.go.tag.main.config-ldflags-assets.slsa3.yml: in run 2540306791 uploaded assets to the v14.0.44 release.
• e2e.go.tag.main.config-ldflags-noassets.slsa3.yml in run 2540306790 failed because the above run uploaded assets and this test expects there to be none.

It looks like verifier-e2e.go.tag.main.config-ldflags-assets.slsa3.yml runs on every tag but uploads assets when maybe it shouldn't?

[laurentsimon]

Thank you Ian for all this digging!

/cc @asraa

We can address this in slsa-framework/example-package#56

[laurentsimon]

I have disabled the verify workflow for the time being. They are only used at release time.

[laurentsimon]

Repo: https://github.com/slsa-framework/example-package/tree/13.0.45
Run: https://github.com/slsa-framework/example-package/actions/runs/2573546164
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.go.tag.main.config-ldflags-assets.slsa3.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.go.tag.main.config-ldflags-assets.slsa3.yml
Trigger: push
Branch: 13.0.45
Date: Tue Jun 28 03:39:20 UTC 2022

[laurentsimon]

Repo: https://github.com/slsa-framework/example-package/tree/v14.2.1
Run: https://github.com/slsa-framework/example-package/actions/runs/2574203733
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.go.tag.main.config-ldflags-noassets.slsa3.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.go.tag.main.config-ldflags-noassets.slsa3.yml
Trigger: push
Branch: v14.2.1
Date: Tue Jun 28 06:34:00 UTC 2022

[laurentsimon]

Repo: https://github.com/slsa-framework/example-package/tree/v13.0.46
Run: https://github.com/slsa-framework/example-package/actions/runs/2580551381
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.go.tag.main.config-ldflags-assets.slsa3.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.go.tag.main.config-ldflags-assets.slsa3.yml
Trigger: push
Branch: v13.0.46
Date: Wed Jun 29 03:50:18 UTC 2022

[laurentsimon]

Repo: https://github.com/slsa-framework/example-package/tree/v14.2.2
Run: https://github.com/slsa-framework/example-package/actions/runs/2581130209
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.go.tag.main.config-ldflags-noassets.slsa3.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.go.tag.main.config-ldflags-noassets.slsa3.yml
Trigger: push
Branch: v14.2.2
Date: Wed Jun 29 06:33:15 UTC 2022