summaryrefslogtreecommitdiffstats
path: root/compare-results.sh
AgeCommit message (Collapse)AuthorFilesLines
2019-06-21Put build / run scripts under Apache 2.0 licenseHarald Welte1-0/+14
The license of the build or execution scripts doesn't affect the license of the actual program, and we want to explicitly state that these scripts may be used in any kind of context. Change-Id: I553925cd88b05903aab52ae1afb093aa9ab9d035
2018-09-06compare_results.sh: less drastic wordingNeels Hofmeyr1-8/+9
We don't currently fail test runs from mismatching results, it's just for human convenience. So don't say things like "Failing due to skipped tests", it's confusing. Summarize once the need to update results instead. Change-Id: Ibafa75bd54df07253780c6af3a68a0a4d148e64b
2018-09-06compare_results.sh: speed up comparison loopNeels Hofmeyr1-2/+2
The script takes quite long to parse each and every line O(n**2). Rather directly pick only those lines that contain the test name in the grep that feeds into the loop. Change-Id: I0d4987b11c3feaef3a7fc56d764c43386865004f
2018-04-06add compare-results.sh, call from start-testsuite.shNeels Hofmeyr1-0/+198
Compare current test results to the expected results, and exit in error on discrepancies. Add compare-result.sh: (trivially) grep junit xml output to determine which tests passed and which didn't, and compare against an expected-result.log, another junit file from a previous run. Summarize and determine success. Include an "xfail" feature: tests that are expected to fail are marked as "xfail", unexpected failures as "FAIL". In various subdirs, copy the current jenkins jobs' junit xml outputs as expected-results.log, so that we will start getting useful output in both jenkins runs and manual local runs. In start-testsuite.sh, after running the tests, invoke the results comparison. Due to the single-line parsing nature, the script so far does not distinguish between error and failure. I doubt that we actually need to do that though. Related: OS#3136 Change-Id: I87d62a8be73d73a5eeff61a842e7c27a0066079d