|Age||Commit message (Collapse)||Author||Files||Lines|
The license of the build or execution scripts doesn't affect the license
of the actual program, and we want to explicitly state that these scripts may
be used in any kind of context.
We don't currently fail test runs from mismatching results, it's just for human
convenience. So don't say things like "Failing due to skipped tests", it's
Summarize once the need to update results instead.
The script takes quite long to parse each and every line O(n**2). Rather
directly pick only those lines that contain the test name in the grep that
feeds into the loop.
Compare current test results to the expected results, and exit in error on
Add compare-result.sh: (trivially) grep junit xml output to determine which
tests passed and which didn't, and compare against an expected-result.log,
another junit file from a previous run. Summarize and determine success.
Include an "xfail" feature: tests that are expected to fail are marked as
"xfail", unexpected failures as "FAIL".
In various subdirs, copy the current jenkins jobs' junit xml outputs as
expected-results.log, so that we will start getting useful output in both
jenkins runs and manual local runs.
In start-testsuite.sh, after running the tests, invoke the results comparison.
Due to the single-line parsing nature, the script so far does not distinguish
between error and failure. I doubt that we actually need to do that though.