-
Stefan Raspl authored
The current tap13 support summarizes testcases on a rather coarse granularity. Which sort of defeats the purpose, especially in CI environments, where having results for individual tests to pinpoint problems helps a lot. To improve things, this patch makes TAP mode bypass print_result and extract results for individual testcases as they are sent to the log file. It then introduces test case numbers and writes the plan at the end. One could argue that post-processing the logs is a rather fragile approach. Which is true - but apparently already the case, see e.g. extract_summary in scripts/runtime.bash. Plus this is quite cheap, while a proper solution would require to modify the kernels, so we can e.g. pass in a counter for the test case number. And we would probably have to come up with reasonable test case names, while this approach simply derives them from each test output. Before: $ ./run_tests.sh -t TAP version 13 ok selftest-setup ok intercept ok emulator ok sieve ok sthyi ok skey ok diag10 ok pfmf ok cmm ok vector ok gs # SKIP ok iep # SKIP 1..12 After: $ ./run_tests.sh -t TAP version 13 ok 1 - cmm: privileged: Program interrupt: expected(2) ok 2 - cmm: invalid ORC 8: Program interrupt: expected(6) ok 3 - diag10: lowcore freeing: 0x0000/0x0000: Program interrupt: expected(6) ok 4 - diag10: lowcore freeing: 0x1000/0x1000: Program interrupt: expected(6) ok 5 - diag10: lowcore freeing: 0x0000/0x1000: Program interrupt: expected(6) ok 6 - diag10: start/end: end < start: Program interrupt: expected(6) ok 7 - diag10: start/end: unaligned start: Program interrupt: expected(6) ok 8 - diag10: start/end: unaligned end: Program interrupt: expected(6) ok 9 - diag10: privileged: Program interrupt: expected(2) ok 10 - emulator: spm/ipm: cc=0,key=f: bit 32 and 33 set to zero ok 11 - emulator: spm/ipm: cc=0,key=f: bit 0-31, 40-63 unchanged ok 12 - emulator: spm/ipm: cc=0,key=f: cc and key applied ok 13 - emulator: spm/ipm: cc=1,key=9: bit 32 and 33 set to zero 1..180 [...] Signed-off-by: Stefan Raspl <raspl@linux.ibm.com> [Almost rewritten, but still giving a lot of credit to Stefan. - Paolo] Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
6e1d3752