1. 15 Mar, 2021 3 commits
    • Vincent Stehlé's avatar
      Add configuration file · 9ae846ea
      Vincent Stehlé authored
      
      
      Add a --config option to Parser.py, to use a configuration file.
      
      This configuration file can contain a list of transformations to apply to
      the test results. This will allow to mark tests as false positives or to
      waive failures.
      
      We describe the YAML configuration file format in details in the README.md.
      We also add a sample configuration, to demonstrate usage of configuration
      files with the already provided sample log and sequence files.
      Signed-off-by: Vincent Stehlé's avatarVincent Stehlé <vincent.stehle@arm.com>
      9ae846ea
    • Vincent Stehlé's avatar
      Better handling of skipped tests sets · d144d546
      Vincent Stehlé authored
      
      
      A skip test set is a test set, which was considered but had zero of its
      tests run. This is not the same as a dropped test set, which we do not find
      in the log at all.
      Generate artificial test entries with the result set to SKIPPED to keep
      track of the former.
      Signed-off-by: Vincent Stehlé's avatarVincent Stehlé <vincent.stehle@arm.com>
      d144d546
    • Vincent Stehlé's avatar
      parser.py: rework ekl parsing · f88b3ceb
      Vincent Stehlé authored
      
      
      Currently we are dropping the end of the device path in the ekl when it
      contains `|' characters, as in sample.elk for example. Rework the ekl
      parsing to fix this.
      
        Before: `Acpi(PNP0A08,0)/Pci(0'
         After: `Acpi(PNP0A08,0)/Pci(0|0)/Pci(0|0)'
      Signed-off-by: Vincent Stehlé's avatarVincent Stehlé <vincent.stehle@arm.com>
      f88b3ceb
  2. 11 Mar, 2021 5 commits
    • Vincent Stehlé's avatar
      Combine our databases into a single one · 62552997
      Vincent Stehlé authored
      
      
      Currently we read db1 and db2 from the log and sequence files respectively.
      Then we compute a cross_check database with all the tests, which have run,
      as well as a would_not_run database with all the tests sets, which have not
      run.
      
      Combine those databases into a unified cross_check database and get rid of
      would_not_run. We create artificial test entries for the dropped tests
      sets.
      While at it, we keep the spurious run tests, if any, and mark them as such.
      
      The benefits are that we do not need special handling for dropped tests
      sets down the pipe anymore, and that our future configuration file
      filtering rules will also apply to spurious and dropped tests.
      Signed-off-by: Vincent Stehlé's avatarVincent Stehlé <vincent.stehle@arm.com>
      62552997
    • Vincent Stehlé's avatar
      Parser.py: auto-detect result values and summary · 316c6421
      Vincent Stehlé authored
      
      
      In the future, we will be able to override tests result values with a
      config file and it will become more convenient to auto-detect the test
      result values rather than hardcoding.
      
      This changes a bit the summary table and group sections order and fields,
      but is still arguably acceptable.
      
      While at it, print a one line summary during execution.
      Signed-off-by: Vincent Stehlé's avatarVincent Stehlé <vincent.stehle@arm.com>
      316c6421
    • Vincent Stehlé's avatar
      Add a sort option · 6e9c8643
      Vincent Stehlé authored
      
      
      Add a --sort command line option to Parser.py, to allow sorting the tests
      data in a configurable manner before producing the outputs.
      
      Also, document usage in the README.md and fix parser.py case.
      Signed-off-by: Vincent Stehlé's avatarVincent Stehlé <vincent.stehle@arm.com>
      6e9c8643
    • Vincent Stehlé's avatar
      Parser.py: add json output · cf9c8c6c
      Vincent Stehlé authored
      
      
      Add a --json command line option to dump our data in json format.
      Signed-off-by: Vincent Stehlé's avatarVincent Stehlé <vincent.stehle@arm.com>
      cf9c8c6c
    • Grant Likely's avatar
      Make parser.py lowercase · c2ba6f1e
      Grant Likely authored and Vincent Stehlé's avatar Vincent Stehlé committed
      
      
      Mostly an aesthetic change, but having the filename in lowercase is more
      friendly when running from case sensitive filesystems, like on Linux.
      Signed-off-by: Grant Likely's avatarGrant Likely <grant.likely@secretlab.ca>
      c2ba6f1e
  3. 03 Mar, 2021 5 commits
  4. 01 Mar, 2021 7 commits
  5. 19 Feb, 2021 2 commits
  6. 01 Oct, 2020 1 commit
  7. 29 Sep, 2020 2 commits
  8. 23 Sep, 2020 2 commits
  9. 17 Sep, 2020 1 commit
  10. 16 Sep, 2020 1 commit
  11. 01 Sep, 2020 1 commit
    • Jeff Booher-Kaeding's avatar
      V1.0? · 3afe3a32
      Jeff Booher-Kaeding authored
      Now outputting CSV, re-factored Command line arguments, bug fixes, spelling fixes, and more.
      3afe3a32
  12. 27 Aug, 2020 1 commit
  13. 18 Aug, 2020 1 commit
  14. 17 Aug, 2020 1 commit
  15. 13 Aug, 2020 2 commits
  16. 12 Aug, 2020 1 commit