Skip to content

ftrace: Ensure dataframe indexes are monotonic

Darryl Green requested to merge github/fork/valschneider/fix/non-monotonic into master

Created by: valschneider

In the unlikely event that a trace event of a given class ends up appearing AFTER it happened (IOW, its timestamp is before the previous event in the report/dataframe), we'll hit this error:

  File "/home/valsch01/Work/lisa/libs/trappy/trappy/ftrace.py", line 732, in __init__
    window, abs_window)
  File "/home/valsch01/Work/lisa/libs/trappy/trappy/ftrace.py", line 100, in __init__
    self._do_parse()
  File "/home/valsch01/Work/lisa/libs/trappy/trappy/ftrace.py", line 309, in _do_parse
    self._apply_user_parameters()
  File "/home/valsch01/Work/lisa/libs/trappy/trappy/ftrace.py", line 286, in _apply_user_parameters
    self._windowify_class(trace_class, self.max_window)
  File "/home/valsch01/Work/lisa/libs/trappy/trappy/ftrace.py", line 166, in _windowify_class
    window[0]:]
  File "/usr/local/lib/python2.7/dist-packages/pandas/core/frame.py", line 2127, in __getitem__
    indexer = convert_to_index_sliceable(self, key)
  File "/usr/local/lib/python2.7/dist-packages/pandas/core/indexing.py", line 1980, in convert_to_index_sliceable
    return idx._convert_slice_indexer(key, kind='getitem')
  File "/usr/local/lib/python2.7/dist-packages/pandas/core/indexes/numeric.py", line 328, in _convert_slice_indexer
    return self.slice_indexer(key.start, key.stop, key.step, kind=kind)
  File "/usr/local/lib/python2.7/dist-packages/pandas/core/indexes/base.py", line 3457, in slice_indexer
    kind=kind)
  File "/usr/local/lib/python2.7/dist-packages/pandas/core/indexes/base.py", line 3658, in slice_locs
    start_slice = self.get_slice_bound(start, 'left', kind)
  File "/usr/local/lib/python2.7/dist-packages/pandas/core/indexes/base.py", line 3594, in get_slice_bound
    raise err
KeyError: 7115.449586

Although terribly obscure, this error is due to unsorted indexes in the dataframe. This is fixed by ensuring the indexes are always sorted before doing any timestamp-based slicing.

It's still unclear how this can happen - is it bug in the trace collection, the trace reporting...?

A test has also been added for good measure.

Merge request reports