mirror of
https://gitlab.isc.org/isc-projects/bind9
synced 2025-08-31 14:35:26 +00:00
Detect missing system test results
At the end of each system test suite run, the system test framework collects all existing test.output files from system test subdirectories and produces bin/tests/system/systests.output from those files. However, it does not check whether a test.output file was found for every executed test. Thus, if the test.output file is accidentally deleted by the system test itself (e.g. due to an overly broad file removal wildcard present in clean.sh), its output will not be included in bin/tests/system/systests.output. Since the result of each system test suite run is determined by bin/tests/system/testsummary.sh, which only operates on the contents of bin/tests/system/systests.output, this can lead to test failures being ignored. Fix by ensuring the number of test results found in bin/tests/system/systests.output is equal to the number of tests run and triggering a system test suite failure in case of a discrepancy between these two values.
This commit is contained in:
@@ -56,4 +56,11 @@ if [ -n "${FAILED_TESTS}" ]; then
|
||||
status=1
|
||||
fi
|
||||
|
||||
RESULTS_FOUND=`grep -c 'R:[a-z0-9_-][a-z0-9_-]*:[A-Z][A-Z]*' systests.output`
|
||||
TESTS_RUN=`echo "${SUBDIRS}" | wc -w`
|
||||
if [ "${RESULTS_FOUND}" -ne "${TESTS_RUN}" ]; then
|
||||
echofail "I:Found ${RESULTS_FOUND} test results, but ${TESTS_RUN} tests were run"
|
||||
status=1
|
||||
fi
|
||||
|
||||
exit $status
|
||||
|
Reference in New Issue
Block a user