Skip to content

lisa.regression: Add Regression.fix_validation_min_iter_nr

Douglas Raillard requested to merge github/fork/douglas-raillard-arm/_pr140 into master

Compute the number of iterations required to observe the effects of a fix to a test case, compared to the new failing rate. This gives an indication on how many iterations are needed to have exekall compare answer the question "is my fix fixing this regression ?", assuming that you actually fixed it. If exekall compare says that there is no significant change after that number of iterations, chances are that the fix is only partial (or not fixing anything at all).

Merge request reports