Use same perf tests for benchcomp in CI #2429
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of changes:
The current benchcomp compares the two versions (pre/post-PR commits) of Kani on the corresponding two versions of benchmarks. It make no sense to compare their performance if the commit modified the performance benchmarks.
In this PR, we copy the post-PR benchmarks to the pre-PR folder to make sure benchcomp always run on the same set of benchmarks.
Here is the CI run result with this commit
https://github.com/qinheping/kani/actions/runs/4879281724/jobs/8705705561?pr=3
comparing to the current workflow
https://github.com/model-checking/kani/actions/runs/4866466752/jobs/8678021607?pr=2422
Resolved issues:
Resolves #2424
Related RFC:
Optional #ISSUE-NUMBER.
Call-outs:
Testing:
How is this change tested?
Is this a refactor change?
Checklist
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 and MIT licenses.