Conversation
| - name: Test | ||
| shell: bash | ||
| run: cabal test +RTS -N | ||
| run: cabal run tests |
There was a problem hiding this comment.
alternatively, since we have some time to look at this deeply, perhaps we should take a look at https://hackage.haskell.org/package/hspec-core-2.11.0.1/docs/Test-Hspec-Core-Spec.html#v:sequential and https://hackage.haskell.org/package/hspec-core-2.11.0.1/docs/Test-Hspec-Core-Spec.html#v:parallel when running these tests in the actual PactTests.hs file, rather than relying on CI to do "the right thing". I'll take a stab at this today and if nothing turns up, let's merge as is, but ideally we'd have these be reproducible at the local level via cabal test.
There was a problem hiding this comment.
How did that analysis turn out, Emily?
There was a problem hiding this comment.
Hard to say - i can't get anything to trigger locally. I'm tempted to just say "merge this" and then we can re-evaluate later. I don't want to block.
|
@rsoeldner Let's also make note of the time it takes to complete the tests on GitHub, making sure that running them serially doesn't slow CI down too much. edit: |
|
@emilypi , It looks like this branch passes CI, We run tests against master and the branch of interest, for each PR. This PR runs the failing tests against master https://github.com/kadena-io/pact/actions/runs/4863641863 (If you click at the action report, checkout repository step, you see master being checked out) and then for my PR https://github.com/kadena-io/pact/actions/runs/4863600190. |
Previously, we noticed failing tests of the
ReplSpecwhich rely on a pseudo-terminal. The major reason is the parallel execution of the test-suite (usingcabal tests).This PR (should) fix the behaviour. Test PR #1215