Hi,
Starting with daa4431 a tap test that returns a non-0 exit code is treated as having passed. Which is bad, because a crashing test will typically return some non-zero errorcode, but not report anything in the tap output.
Repro:
project('repro')
test(
'foo',
find_program('bash'),
args: ['-c', 'echo ok 1; exit 139'],
protocol: 'tap',
)With daa4431:
andres@awork3:/tmp/meson-repro$ M=/home/andres/src/meson-bisect/meson.py && rm -rf build/ && $M setup build && $M test -C build/
The Meson build system
Version: 1.11.99
Source dir: /tmp/meson-repro
Build dir: /tmp/meson-repro/build
Build type: native build
Project name: repro
Project version: undefined
Host machine cpu family: x86_64
Host machine cpu: x86_64
Program bash found: YES (/usr/bin/bash)
Build targets in project: 0
Found ninja-1.14.0.git at /home/andres/bin/bin/ninja
ninja: Entering directory `/tmp/meson-repro/build'
ninja: no work to do.
1/1 repro:foo OK 0.01s (exit status 139 or signal 11 SIGSEGV)
Ok: 1
Fail: 0
Full log written to /tmp/meson-repro/build/meson-logs/testlog.txt
(the expected failure occurs)
With daa4431^:
andres@awork3:/tmp/meson-repro$ M=/home/andres/src/meson-bisect/meson.py && rm -rf build/ && $M setup build && $M test -C build/
The Meson build system
Version: 1.11.99
Source dir: /tmp/meson-repro
Build dir: /tmp/meson-repro/build
Build type: native build
Project name: repro
Project version: undefined
Host machine cpu family: x86_64
Host machine cpu: x86_64
Program bash found: YES (/usr/bin/bash)
Build targets in project: 0
Found ninja-1.14.0.git at /home/andres/bin/bin/ninja
ninja: Entering directory `/tmp/meson-repro/build'
ninja: no work to do.
1/1 repro:foo ERROR 0.01s (exit status 139 or signal 11 SIGSEGV)
>>> UBSAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1:print_stacktrace=1 MSAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1:print_stacktrace=1 MESON_TEST_ITERATION=1 MALLOC_PERTURB_=252 ASAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1 /usr/bin/bash -c 'echo ok 1; exit 139'
Ok: 0
Fail: 1
Full log written to /tmp/meson-repro/build/meson-logs/testlog.txt
Similarly, if a test just crashes, without any tap output beforehand, it's now reported as a SKIP, which also seems quite wrong.
CC: @dcbaker
Greetings,
Andres
Hi,
Starting with daa4431 a tap test that returns a non-0 exit code is treated as having passed. Which is bad, because a crashing test will typically return some non-zero errorcode, but not report anything in the tap output.
Repro:
With daa4431:
(the expected failure occurs)
With daa4431^:
Similarly, if a test just crashes, without any tap output beforehand, it's now reported as a SKIP, which also seems quite wrong.
CC: @dcbaker
Greetings,
Andres