assorted test fixes & refactors #4240

boneskull · 2020-04-21T21:13:43Z

increase default timeout for wiggle room
specify watch-ignore in case we run our own tests in watch mode
reformat .mocharc.yml
integration/fixtures/uncaught/listeners.fixture.js: reduce number of listeners created to avoid max listener warning
integration/fixtures/options/watch.spec.js: do not use context object; be more specific about spawn options
integration/fixtures/diffs.spec.js: do not pass -C; the helper already does this
node-unit/mocha.spec.js: do not actually create files; it's a unit test after all.
node-unit/cli/config.spec.js: rewiremock fixes
node-unit/cli/options.spec.js: rewiremock fixes; increase timeout
unit/hook-timeout.spec.js: do not override default (configured) timeout
unit/runner.spec.js: leverage unexpected-eventemitter
unit/throw.spec.js: proper teardown: remove uncaught exception listeners
unit/timeout.spec.js: increase timeout to greater than default (configured) value

coveralls · 2020-04-21T22:18:03Z

Coverage increased (+0.002%) to 93.103% when pulling d5d0c98 on boneskull/issue/2839-test-fixes into 9c965c9 on master.

.mocharc.yml

test/integration/fixtures/uncaught/listeners.fixture.js

boneskull · 2020-04-24T18:55:38Z

test/integration/options/watch.spec.js

  describe('when enabled', function() {
-    this.timeout(10 * 1000);
-    this.slow(3000);
+    let tempDir;


this is just me being fussy. but most all of our tests use function scope instead of the context to store data.

boneskull · 2020-04-24T18:56:58Z

test/node-unit/cli/config.spec.js

  });

  describe('loadConfig()', function() {
+    let parsers;


this "unit" test was reading files, and it shouldn't need to read files to assert what's trying to assert.

test/unit/throw.spec.js

boneskull · 2020-04-24T18:59:07Z

timeout-related changes were variously because of test flake, and because I implemented this while implementing parallel mode, the test execution times have a larger variance.

I'm starting to wonder if test timeouts aren't really that helpful, because I spend most of my time increasing the timeouts due to slow CI runs instead of catching significant performance issues.

the same code should be in PR #4240

.mocharc.yml

test/integration/options/watch.spec.js

test/node-unit/cli/config.spec.js

test/node-unit/mocha.spec.js

test/unit/throw.spec.js

juergba · 2020-04-30T08:17:45Z

test/node-unit/cli/config.spec.js

      describe('when supplied a filepath with ".js" extension', function() {
        const filepath = 'foo.js';

        it('should use the JS parser', function() {


A test for the .cjs extension is missing.

ah, didn't notice! just added one.

still missing

test/node-unit/mocha.spec.js

test/unit/throw.spec.js

- increase default timeout for wiggle room - specify `watch-ignore` in case we run our own tests in watch mode - reformat `.mocharc.yml` - `integration/fixtures/uncaught/listeners.fixture.js`: reduce number of listeners created to avoid max listener warning - `integration/fixtures/diffs.spec.js`: do not pass `-C`; the helper already does this - `integration/options/watch.spec.js`: do not use context object; be more specific about spawn options. tweak timings - `node-unit/mocha.spec.js`: make more unit-test-like insofar as that's possible when testing w/ `require()` - `node-unit/cli/config.spec.js`: rewiremock fixes; assertion tweaks; add test for `.cjs` extension - `node-unit/cli/options.spec.js`: rewiremock fixes; increase timeout - `unit/hook-timeout.spec.js`: do not override default (configured) timeout - `unit/runner.spec.js`: leverage [unexpected-eventemitter](https://npm.im/unexpected-eventemitter) - `unit/throw.spec.js`: proper teardown: remove uncaught exception listeners - `unit/timeout.spec.js`: increase timeout to _greater than_ default (configured) value - `example/config/.mocharc.yml`: quote yaml strings BONUS - fixes a weird call to `Mocha.unloadFile()` from `Mocha#unloadFiles()` Ref: #4198

boneskull · 2020-04-30T21:19:34Z

@juergba I think this is ready for your approval again

juergba

The ".cjs" test is still missing.

juergba · 2020-05-01T13:18:07Z

test/node-unit/cli/config.spec.js

      describe('when supplied a filepath with ".js" extension', function() {
        const filepath = 'foo.js';

        it('should use the JS parser', function() {


still missing

boneskull · 2020-05-01T21:56:41Z

forgot to push, sorry. if this passes CI, I'll merge

* test helper improvements - enables `RawResult` (the result of `invokeMocha()` or `invokeMochaAsync()`) to check the exit code via `to have code` assertion - add passed/failing/pending assertions for `RawRunResult` (the result of `runMocha()` or `runMochaAsync()`) - expose `getSummary()`, which can be used with a `RawResult` (when not failing) - reorganize the module a bit - create `runMochaAsync()` and `runMochaJSONAsync()` which are like `runMocha()` and `runMochaJSON()` except return `Promise`s - better trapping of JSON parse errors - better default handling of `STDERR` output in subprocesses (print it instead of suppress it!) - do not let the `DEBUG` env variable reach subprocesses _unless it was explicitly supplied_ - add an easily copy-paste-able `command` prop to summary - add some missing docstrings Ref: #4198 * increase timeout in watch test for CI the same code should be in PR #4240

- increase default timeout for wiggle room - specify `watch-ignore` in case we run our own tests in watch mode - reformat `.mocharc.yml` - `integration/fixtures/uncaught/listeners.fixture.js`: reduce number of listeners created to avoid max listener warning - `integration/fixtures/diffs.spec.js`: do not pass `-C`; the helper already does this - `integration/options/watch.spec.js`: do not use context object; be more specific about spawn options. tweak timings - `node-unit/mocha.spec.js`: make more unit-test-like insofar as that's possible when testing w/ `require()` - `node-unit/cli/config.spec.js`: rewiremock fixes; assertion tweaks; add test for `.cjs` extension - `node-unit/cli/options.spec.js`: rewiremock fixes; increase timeout - `unit/hook-timeout.spec.js`: do not override default (configured) timeout - `unit/runner.spec.js`: leverage [unexpected-eventemitter](https://npm.im/unexpected-eventemitter) - `unit/throw.spec.js`: proper teardown: remove uncaught exception listeners - `unit/timeout.spec.js`: increase timeout to _greater than_ default (configured) value - `example/config/.mocharc.yml`: quote yaml strings BONUS - fixes a weird call to `Mocha.unloadFile()` from `Mocha#unloadFiles()` Ref: #4198

* test helper improvements - enables `RawResult` (the result of `invokeMocha()` or `invokeMochaAsync()`) to check the exit code via `to have code` assertion - add passed/failing/pending assertions for `RawRunResult` (the result of `runMocha()` or `runMochaAsync()`) - expose `getSummary()`, which can be used with a `RawResult` (when not failing) - reorganize the module a bit - create `runMochaAsync()` and `runMochaJSONAsync()` which are like `runMocha()` and `runMochaJSON()` except return `Promise`s - better trapping of JSON parse errors - better default handling of `STDERR` output in subprocesses (print it instead of suppress it!) - do not let the `DEBUG` env variable reach subprocesses _unless it was explicitly supplied_ - add an easily copy-paste-able `command` prop to summary - add some missing docstrings Ref: #4198 * increase timeout in watch test for CI the same code should be in PR #4240

boneskull added type: chore generally involving deps, tooling, configuration, etc. semver-patch implementation requires increase of "patch" version number; "bug fixes" labels Apr 21, 2020

boneskull self-assigned this Apr 21, 2020

boneskull mentioned this pull request Apr 21, 2020

[DO NOT MERGE] WIP: run tests concurrently; see #2839 #4198

Closed

22 tasks

boneskull force-pushed the boneskull/issue/2839-test-fixes branch from c4d9199 to 7c373a4 Compare April 22, 2020 19:13

This was referenced Apr 22, 2020

remove cruft from wallaby config #4243

Merged

fixes some signal handling #4239

Merged

add support for running tests in parallel #4245

Merged

boneskull requested review from Munter, craigtaub, juergba and outsideris April 24, 2020 18:50

boneskull commented Apr 24, 2020

View reviewed changes

.mocharc.yml Show resolved Hide resolved

boneskull commented Apr 24, 2020

View reviewed changes

test/integration/fixtures/uncaught/listeners.fixture.js Show resolved Hide resolved

boneskull commented Apr 24, 2020

View reviewed changes

test/unit/throw.spec.js Show resolved Hide resolved

boneskull added a commit that referenced this pull request Apr 24, 2020

increase timeout in watch test for CI

a3094b7

the same code should be in PR #4240

boneskull added a commit that referenced this pull request Apr 24, 2020

increase timeout in watch test for CI

7ddcf5f

the same code should be in PR #4240

boneskull added a commit that referenced this pull request Apr 24, 2020

increase timeout in watch test for CI

fe2c342

the same code should be in PR #4240

juergba reviewed Apr 28, 2020

View reviewed changes

boneskull force-pushed the boneskull/issue/2839-test-fixes branch 3 times, most recently from 922b066 to 31ca866 Compare April 28, 2020 20:54

boneskull requested a review from juergba April 28, 2020 20:55

juergba approved these changes Apr 30, 2020

View reviewed changes

boneskull force-pushed the boneskull/issue/2839-test-fixes branch from 31ca866 to 798155b Compare April 30, 2020 21:03

boneskull requested a review from juergba April 30, 2020 21:19

juergba approved these changes May 1, 2020

View reviewed changes

boneskull force-pushed the boneskull/issue/2839-test-fixes branch from 798155b to d5d0c98 Compare May 1, 2020 21:56

boneskull merged commit 2509ab5 into master May 4, 2020

boneskull deleted the boneskull/issue/2839-test-fixes branch May 4, 2020 19:56

craigtaub added this to the next milestone May 21, 2020

Uh oh!

assorted test fixes & refactors #4240

assorted test fixes & refactors #4240

Uh oh!

Conversation

boneskull commented Apr 21, 2020

Uh oh!

coveralls commented Apr 21, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

boneskull Apr 24, 2020

Choose a reason for hiding this comment

Uh oh!

boneskull Apr 24, 2020

Choose a reason for hiding this comment

Uh oh!

Uh oh!

boneskull commented Apr 24, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

juergba Apr 30, 2020

Choose a reason for hiding this comment

Uh oh!

boneskull Apr 30, 2020

Choose a reason for hiding this comment

Uh oh!

juergba May 1, 2020

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

boneskull commented Apr 30, 2020

Uh oh!

juergba left a comment

Choose a reason for hiding this comment

Uh oh!

juergba May 1, 2020

Choose a reason for hiding this comment

Uh oh!

boneskull commented May 1, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

coveralls commented Apr 21, 2020 •

edited

Loading