Jest not showing skipped tests and throws if testNamePattern does not match any test #6584
Description
💥 Regression Report
I need to discover the test case names before executing them. I use the configuration option testNamePattern with value ^$a which will not match any string and hence cause jest to skip all tests. I use a custom reporter to check the test case names. After updating jest to latest version, using this test pattern and running jest I get this error.
FAIL __test__/test.js
● Test suite failed to run
Your test suite must contain at least one test.
at node_modules/jest-cli/build/test_scheduler.js:246:22
Test Suites: 1 failed, 1 total
Tests: 0 total
Snapshots: 0 total
Time: 2.68s
Ran all test suites with tests matching "^$a".Another regression, if a testNamePattern matches some tests in one suite and no tests in another suite, the unmatched tests are not reported as skipped. Consider 4 test cases in the same file suite a > test 1, suite a > test 2, suite b > test 1 and suite b > test 2
FAIL __test__/test.js
suite a
√ test 1 (1ms)
× test 2 (9ms)
● suite a › test 2
assert.(received, expected)
Expected value undefined
Received:
""
Difference:
Comparing two different types of values. Expected undefined but received string.
7 |
8 | it('test 2', () => {
> 9 | assert.fail('');
| ^
10 | });
11 | });
12 |
at Object.it (__test__/test.js:9:16)
Test Suites: 1 failed, 1 total
Tests: 1 failed, 1 passed, 2 total
Snapshots: 0 total
Time: 2.981s
Ran all test suites with tests matching "suite a".The two test cases in suite a match and are executed. Test cases for suite b don't match and are not shown as skipped.
Last working version
Worked up to version: 22.4.3
Stopped working in version: 23.2.0
To Reproduce
const assert = require('assert')
describe('suite a', () => {
it('test 1', () => {
});
it('test 2', () => {
assert.fail('');
});
});
describe('suite b', () => {
it('test 1', () => {
});
it('test 2', () => {
assert.fail('');
});
});Run jest with config
{
"testNamePattern": "^$a"
}and
{
"testNamePattern": "suite a"
}Expected behavior
For "testNamePattern": "^$a"
Jest v22.1.2 node v7.4.0 linux/amd64
Test Suites: 1 skipped, 0 of 1 total
Tests: 4 skipped, 4 total
Snapshots: 0 total
Time: 0.915s, estimated 1s
Ran all test suites with tests matching "^$a".For "testNamePattern": "suite a"
Jest v22.1.2 node v7.4.0 linux/amd64
FAIL ./add-test.js
suite a
✓ test 1 (3ms)
✕ test 2 (9ms)
suite b
○ skipped 2 tests
● suite a › test 2
assert.(received, expected)
Expected value undefined
Received:
""
Difference:
Comparing two different types of values. Expected undefined but received string.
7 |
8 | it('test 2', () => {
> 9 | assert.fail('');
10 | });
11 | });
12 |
at Object.it (add-test.js:9:16)
Test Suites: 1 failed, 1 total
Tests: 1 failed, 2 skipped, 1 passed, 4 total
Snapshots: 0 total
Time: 1.002s
Ran all test suites with tests matching "suite a".
exit status 1Link to repl or repo (highly encouraged)
I see that repl.it is running jest v22.1.2 and the output is as expected, but here it is anyway for the sake of this issue not being stalled: https://repl.it/repls/WideeyedFrayedTerabyte
Run npx envinfo --preset jest
npx: installed 1 in 2.58s
Path must be a string. Received undefined
npx: installed 1 in 2.651s
C:\Users\karsin\AppData\Roaming\npm-cache\_npx\8504\node_modules\envinfo\dist\cli.js
System:
OS: Windows 10
CPU: x64 Intel(R) Core(TM) i7-6600U CPU @ 2.60GHz
Binaries:
npm: 5.6.0 - C:\Program Files\nodejs\npm.CMD