Unverified Commit 4730e9a9 authored by Jeffrey Booher-Kaeding's avatar Jeffrey Booher-Kaeding Committed by GitHub
Browse files

Merge pull request #11 from vstehle/for-jeffrey

Handling missing Package, Updating Doc, Fixing Typos
parents 0e8f32de df27218a
...@@ -16,7 +16,7 @@ ...@@ -16,7 +16,7 @@
# `EBBR.seq'. # `EBBR.seq'.
- rule: Force false-positive simple network protocol ReceiveFilters failure as - rule: Force false-positive simple network protocol ReceiveFilters failure as
pass (EBBR Table 2.4 Notable Deviations from UEFI § 2.6.2) ignored (EBBR Table 2.4 Notable Deviations from UEFI § 2.6.2)
criteria: criteria:
descr: Simple Network Protocol Test descr: Simple Network Protocol Test
group: NetworkSupportTest group: NetworkSupportTest
...@@ -264,7 +264,7 @@ ...@@ -264,7 +264,7 @@
guid: 5CB0CDB5-AC80-4983-B710-4B0BF0191563 guid: 5CB0CDB5-AC80-4983-B710-4B0BF0191563
log: EfiCompliantBBTestPlatform_uefi.c log: EfiCompliantBBTestPlatform_uefi.c
name: UEFI Compliant - Boot from block-oriented NVMe peripheral name: UEFI Compliant - Boot from block-oriented NVMe peripheral
result: IGNORED result: FAILURE
revision: '0x00010001' revision: '0x00010001'
set guid: A0A8BED3-3D6F-4AD8-907A-84D52EE1543B set guid: A0A8BED3-3D6F-4AD8-907A-84D52EE1543B
sub set: PlatformSpecificElements sub set: PlatformSpecificElements
...@@ -306,7 +306,7 @@ ...@@ -306,7 +306,7 @@
update: update:
result: IGNORED result: IGNORED
- rule: Force false-positive uefi compliant failure (17) as ignored (EBBR Table 2.4 - rule: Force false-positive uefi compliant warning (17) as ignored (EBBR Table 2.4
Notable Deviations from UEFI § 2.6.2) Notable Deviations from UEFI § 2.6.2)
criteria: criteria:
descr: UEFI Compliant Test descr: UEFI Compliant Test
...@@ -546,7 +546,7 @@ ...@@ -546,7 +546,7 @@
set guid: 39FF9C71-4B41-4E5B-AED7-87C794187D66 set guid: 39FF9C71-4B41-4E5B-AED7-87C794187D66
sub set: GetInfo_Func sub set: GetInfo_Func
update: update:
result: WAIVED result: IGNORED
- rule: Force dropped RNG (2) as ignored (Table 2.3 UEFI Platform-Specific - rule: Force dropped RNG (2) as ignored (Table 2.3 UEFI Platform-Specific
Required Elements) Required Elements)
...@@ -570,7 +570,7 @@ ...@@ -570,7 +570,7 @@
set guid: DC5B2884-EDFB-4078-A288-4DFDA849A08D set guid: DC5B2884-EDFB-4078-A288-4DFDA849A08D
sub set: GetRNG_Func sub set: GetRNG_Func
update: update:
result: WAIVED result: IGNORED
- rule: Force dropped RNG (4) as ignored (Table 2.3 UEFI Platform-Specific - rule: Force dropped RNG (4) as ignored (Table 2.3 UEFI Platform-Specific
Required Elements) Required Elements)
...@@ -582,7 +582,7 @@ ...@@ -582,7 +582,7 @@
set guid: DDBBE5AB-206E-4F35-9556-186DA87C2A86 set guid: DDBBE5AB-206E-4F35-9556-186DA87C2A86
sub set: GetRNG_Conf sub set: GetRNG_Conf
update: update:
result: WAIVED result: IGNORED
############################################################################### ###############################################################################
# Waived # # Waived #
......
...@@ -47,6 +47,24 @@ $ ./parser.py --filter "x['result'] == 'FAILURE'" ... ...@@ -47,6 +47,24 @@ $ ./parser.py --filter "x['result'] == 'FAILURE'" ...
Filtering takes place after the configuration rules, which are described below. Filtering takes place after the configuration rules, which are described below.
This filtering mechanism can also be (ab)used to transform tests results.
Example command, which adds a "comment" field (and keeps all the tests):
``` {.sh}
$ ./parser.py \
--filter "x.update({'comment': 'Transformed'}) or True" ...
```
Example command, which removes filenames prefixes in the "log" field (and keeps
all the tests):
``` {.sh}
$ ./parser.py \
--filter "x.update({'log': re.sub(r'/.*/', '', x['log'])}) \
or True" ...
```
## Configuration file ## Configuration file
It is possible to use a configuration file with command line option `--config It is possible to use a configuration file with command line option `--config
...@@ -141,12 +159,8 @@ This generated configuration can then be further edited manually. ...@@ -141,12 +159,8 @@ This generated configuration can then be further edited manually.
It is possible to convert this `README.md` into `README.pdf` with pandoc using It is possible to convert this `README.md` into `README.pdf` with pandoc using
`make doc`. See `make help`. `make doc`. See `make help`.
### TODO:
* double check concatenation of all `.ekl` logs, preliminary tests show small Divergence between them and `summary.ekl` found in `Overall` folder. Cat.sh will generate this file.
* look into large number of dropped tests.
### db structure: ### db structure:
``` {.python} ``` {.python}
tests = [ tests = [
test_dict, test_dict,
......
...@@ -8,7 +8,11 @@ import csv ...@@ -8,7 +8,11 @@ import csv
import logging import logging
import json import json
import re import re
from packaging import version
try:
from packaging import version
except ImportError:
print('No packaging...')
try: try:
import yaml import yaml
...@@ -22,7 +26,8 @@ if 'yaml' in sys.modules: ...@@ -22,7 +26,8 @@ if 'yaml' in sys.modules:
from yaml import Dumper from yaml import Dumper
# Not all yaml versions have a Loader argument. # Not all yaml versions have a Loader argument.
if version.parse(yaml.__version__) >= version.parse('5.1'): if 'packaging.version' in sys.modules and \
version.parse(yaml.__version__) >= version.parse('5.1'):
yaml_load_args = {'Loader': yaml.FullLoader} yaml_load_args = {'Loader': yaml.FullLoader}
else: else:
yaml_load_args = {} yaml_load_args = {}
...@@ -322,12 +327,16 @@ def use_config(cross_check, filename): ...@@ -322,12 +327,16 @@ def use_config(cross_check, filename):
# Otherwise it is dropped # Otherwise it is dropped
def filter_data(cross_check, Filter): def filter_data(cross_check, Filter):
logging.debug(f"Filtering with `{Filter}'") logging.debug(f"Filtering with `{Filter}'")
before = len(cross_check)
# This function "wraps" the filter and is called for each test # This function "wraps" the filter and is called for each test
def function(x): def function(x):
return eval(Filter) return eval(Filter)
return list(filter(function, cross_check)) r = list(filter(function, cross_check))
after = len(r)
logging.info(f"Filtered out {before - after} test(s), kept {after}")
return r
# Sort tests data in-place # Sort tests data in-place
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment