1. 09 Aug, 2021 9 commits
  2. 13 Jul, 2021 4 commits
  3. 06 Jul, 2021 1 commit
  4. 14 Jun, 2021 4 commits
  5. 10 Jun, 2021 1 commit
  6. 09 Jun, 2021 5 commits
  7. 07 Jun, 2021 1 commit
  8. 31 May, 2021 2 commits
  9. 18 May, 2021 1 commit
  10. 11 May, 2021 1 commit
  11. 24 Mar, 2021 1 commit
  12. 18 Mar, 2021 1 commit
    • Vincent Stehlé's avatar
      EBBR.yaml: add configuration with ignored and waived · 0e87dd09
      Vincent Stehlé authored
      
      
      We add a configuration file with rules for testing the EBBR specification.
      
      Some tests correspond to cases not mandated by the EBBR specification and
      we want to ignore them. Some other tests are either already fixed with a
      recent U-Boot or still present with the sandbox or qemu, and we want to
      issue waivers for those.
      
                        Original result          Forced result
        False-positive  FAILURE/WARNING/DROPPED  IGNORED
                Waived  FAILURE/WARNING/SKIPPED  WAIVED
      
      The list has been established in part by comparing SCT runs done with the
      EBBR.seq sequence file, running on arm64 U-Boot sandbox and qemu, from
      commit ddaa94978583 and version v2021.04-rc2.
      Signed-off-by: Vincent Stehlé's avatarVincent Stehlé <vincent.stehle@arm.com>
      0e87dd09
  13. 15 Mar, 2021 5 commits
  14. 11 Mar, 2021 4 commits
    • Jeffrey Booher-Kaeding's avatar
      Merge pull request #7 from vstehle/for-jeffrey · 886921c5
      Jeffrey Booher-Kaeding authored
      From Vincent:
      Lowercase change from
      JSON output
      Sorting command line option
      Internal changes, to prepare for configuration/filtering: detect values & combined databases
      886921c5
    • Vincent Stehlé's avatar
      Combine our databases into a single one · 62552997
      Vincent Stehlé authored
      
      
      Currently we read db1 and db2 from the log and sequence files respectively.
      Then we compute a cross_check database with all the tests, which have run,
      as well as a would_not_run database with all the tests sets, which have not
      run.
      
      Combine those databases into a unified cross_check database and get rid of
      would_not_run. We create artificial test entries for the dropped tests
      sets.
      While at it, we keep the spurious run tests, if any, and mark them as such.
      
      The benefits are that we do not need special handling for dropped tests
      sets down the pipe anymore, and that our future configuration file
      filtering rules will also apply to spurious and dropped tests.
      Signed-off-by: Vincent Stehlé's avatarVincent Stehlé <vincent.stehle@arm.com>
      62552997
    • Vincent Stehlé's avatar
      Parser.py: auto-detect result values and summary · 316c6421
      Vincent Stehlé authored
      
      
      In the future, we will be able to override tests result values with a
      config file and it will become more convenient to auto-detect the test
      result values rather than hardcoding.
      
      This changes a bit the summary table and group sections order and fields,
      but is still arguably acceptable.
      
      While at it, print a one line summary during execution.
      Signed-off-by: Vincent Stehlé's avatarVincent Stehlé <vincent.stehle@arm.com>
      316c6421
    • Vincent Stehlé's avatar
      Add a sort option · 6e9c8643
      Vincent Stehlé authored
      
      
      Add a --sort command line option to Parser.py, to allow sorting the tests
      data in a configurable manner before producing the outputs.
      
      Also, document usage in the README.md and fix parser.py case.
      Signed-off-by: Vincent Stehlé's avatarVincent Stehlé <vincent.stehle@arm.com>
      6e9c8643