- 17 Sep, 2021 2 commits
-
-
Vincent Stehlé authored
We add a database of the sequence files we know about with their sha256, and we try to identify the input sequence file among those. A `--seq-db' option is added to allow specifying the database. Update documentation accordingly. While at it, fix spacing. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
Vincent Stehlé authored
Make printing a bit closer to markdown, with header line, wider separations between columns and capitalized header fields. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
- 26 Aug, 2021 1 commit
-
-
Vincent Stehlé authored
Handle more cases gracefully during printing, where not all the fields are present for some of the records. This makes the original commit 29d85950 ("parser.py: add an option for printing") more robust and fixes a bug in the first hardening attempt of commit f153e735 ("parser.py: more robust printing"). Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
- 19 Aug, 2021 8 commits
-
-
Vincent Stehlé authored
We can stress the markdown generation code a bit more now that we can re-read previously generated markdown result. Make markdown generation more robust to missing keys to avoid exceptions. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
Vincent Stehlé authored
This is useful to perform further processing. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
Vincent Stehlé authored
This simplifies the main path and prepares for future conditional .md generation. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
Vincent Stehlé authored
dict_2_md implicitly relies on the tests all having the same order for their dict keys during printing. This works well with all the tests read from the .ekl log file but does not work with the "artificial" SPURIOUS test entries, which end up having their keys in a different order. Fix the keys order to fix .md printing. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
Vincent Stehlé authored
Simplify the printing by group, to help debugging. As a bonus, this is more efficient. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
Vincent Stehlé authored
Add a checker for python scripts and a checker for shell scripts to the `make check' target. Fix all issues reported by the checker in parser.py. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
Vincent Stehlé authored
This makes the main path more concise and prepares for future re-reading of the .md summary. While at it, reformat a bit. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
Vincent Stehlé authored
This reduces the use of db1 in the code for clarity. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
- 18 Aug, 2021 2 commits
-
-
Grant Likely authored
The sample files are just for testing. Using them by default could hide from the user that they are calling the script wrong. Make it required to supply the .ekl and .seq files. Signed-off-by:
Grant Likely <grant.likely@arm.com>
-
Grant Likely authored
Use the typical __init__ == '__main__' pattern to determine if the code is being run as a script, and only execute the main code if true. This makes the code more portable to other projects. Signed-off-by:
Grant Likely <grant.likely@arm.com>
-
- 13 Jul, 2021 1 commit
-
-
Vincent Stehlé authored
Deal with the case where not all records have a given field more gracefully during printing. This happens for those fields, which are generated on-the-fly by configuration rules for example. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
- 14 Jun, 2021 4 commits
-
-
Vincent Stehlé authored
This is useful to print a quick summary, after filtering. We document some usage in the README.md. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
Vincent Stehlé authored
This is similar in its principle to the UNIX `uniq -c' and is handy for triaging results, in conjunction with the `--fields' option. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
Vincent Stehlé authored
Keeping only some fields is handy to generate summaries for triaging. This does not apply to the markdown and config template formats, which depend on certain fields being present. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
Vincent Stehlé authored
This is helpful to debug, especially with large SCT logs. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
- 09 Jun, 2021 2 commits
-
-
Vincent Stehlé authored
It is always convenient to see the effect of a filter in a quantitative manner. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
Vincent Stehlé authored
The packaging library is not always installed even though it is standard. Try to handle its absence gracefully. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
- 31 May, 2021 2 commits
-
-
Vincent Stehlé authored
Allow to filter tests data using a python expression. This complements the existing key/value filtering mechanism. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
Vincent Stehlé authored
Try to semi-automate the writing of yaml configuration files by generating a template yaml configuration from tests data, which can be further edited manually. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
- 11 May, 2021 1 commit
-
-
Vincent Stehlé authored
The yaml module has the FullLoader class since version 5.1 only; handle this gracefully at runtime. This fixes the following error: AttributeError: module 'yaml' has no attribute 'FullLoader' Reported-by:
Zhifei Yang <zhifei.yang@arm.com> Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
- 15 Mar, 2021 5 commits
-
-
The libyaml implementation is significantly faster than the pure python YAML parser. If it is present, then use CDumper when dumping the results. Signed-off-by:
Grant Likely <grant.likely@arm.com> [Rebased, added conditional.] Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
Vincent Stehlé authored
Now that we are using yaml to load configurations, we might as well dump our test data in yaml, too. This helps writing the configuration rules. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
Vincent Stehlé authored
Add a --config option to Parser.py, to use a configuration file. This configuration file can contain a list of transformations to apply to the test results. This will allow to mark tests as false positives or to waive failures. We describe the YAML configuration file format in details in the README.md. We also add a sample configuration, to demonstrate usage of configuration files with the already provided sample log and sequence files. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
Vincent Stehlé authored
A skip test set is a test set, which was considered but had zero of its tests run. This is not the same as a dropped test set, which we do not find in the log at all. Generate artificial test entries with the result set to SKIPPED to keep track of the former. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
Vincent Stehlé authored
Currently we are dropping the end of the device path in the ekl when it contains `|' characters, as in sample.elk for example. Rework the ekl parsing to fix this. Before: `Acpi(PNP0A08,0)/Pci(0' After: `Acpi(PNP0A08,0)/Pci(0|0)/Pci(0|0)' Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
- 11 Mar, 2021 5 commits
-
-
Vincent Stehlé authored
Currently we read db1 and db2 from the log and sequence files respectively. Then we compute a cross_check database with all the tests, which have run, as well as a would_not_run database with all the tests sets, which have not run. Combine those databases into a unified cross_check database and get rid of would_not_run. We create artificial test entries for the dropped tests sets. While at it, we keep the spurious run tests, if any, and mark them as such. The benefits are that we do not need special handling for dropped tests sets down the pipe anymore, and that our future configuration file filtering rules will also apply to spurious and dropped tests. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
Vincent Stehlé authored
In the future, we will be able to override tests result values with a config file and it will become more convenient to auto-detect the test result values rather than hardcoding. This changes a bit the summary table and group sections order and fields, but is still arguably acceptable. While at it, print a one line summary during execution. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
Vincent Stehlé authored
Add a --sort command line option to Parser.py, to allow sorting the tests data in a configurable manner before producing the outputs. Also, document usage in the README.md and fix parser.py case. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
Vincent Stehlé authored
Add a --json command line option to dump our data in json format. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
Mostly an aesthetic change, but having the filename in lowercase is more friendly when running from case sensitive filesystems, like on Linux. Signed-off-by:
Grant Likely <grant.likely@secretlab.ca>
-
- 03 Mar, 2021 5 commits
-
-
Vincent Stehlé authored
Use logging to output debug messages and add a --debug command line option to change the log level. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
Vincent Stehlé authored
Add the --csv command line option to generate a .csv file. This is handy to manually triage issues in a spreadsheet. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
Vincent Stehlé authored
Use argparse for command line arguments. This gives us the online help and will give use more flexibility to add command line options. We preserve the current behaviour with up to two positional arguments for the ekl and seq input files, and up to two arguments for the key:value search. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
Vincent Stehlé authored
We capture the group iteration, start date and time, revision, description and device path. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
Vincent Stehlé authored
This is easier for argument passing and more flexible for adding fields in the future. We take care to preserve current fields ordering in output result.md. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
- 01 Mar, 2021 2 commits
-
-
Vincent Stehlé authored
We generate a table for the summary rather than sections, as this is arguably more readable. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-
Vincent Stehlé authored
Avoid duplicating the GUID, result and name in the log. Signed-off-by:
Vincent Stehlé <vincent.stehle@arm.com>
-