exekall rework (part 2)
-
Mostly rework/cleanup the
ValueDBwhere exekall stores the objects that were computed for each "stages" of the test (what is displayed inSTRUCTUREfile in tests' artifact folder). That DB now has a simple API to allow reloading object instances in a script or notebook. Documentation/examples coming soon. -
bisectoris now integrated with exekall'sValueDB, which can be stored directly inside bisector's report. That allows to remove the xUnit ad-hoc file sincebisectorcan now deal withlisaobjects directly (mostlyResultBundleandResult). As a consequence, opening a report usingbisector reportwill likely require thelisapackage to be importable, which meanslisashould installed as well. -
exekalllearns aexekall comparesubcommand that allows comparing twoVALUE_DB.pickle.xzfiles. It will expect multiple values for each test, and look for significant regressions. In order to have more than one value for each test in one DB, useexekall mergeorbisector report myreport.yml.gz -oexport-db=merged_db.pickle.xz -
exekall compareis implemented in term of functions/classes found in lisa/regression.py, which can be readily used for other purposes if needed.
breaking changes:
- Since
xUnitis replaced byValueDBinbisector's reports, that version won't be able to open previous lisa-next reports. lisa legacy reports should still work fine. -
ValueDBstructure has changed, so reloading previously-generated DB won't work.
note:
Since a lot more objects are now stored in bisector's report, the report loading time has increased. Use bisector report --cache to use a pickle cache of the YAML report, which is around ~10 times faster to open.