Skip to content

lisa.analysis.functions: Add profiler

Douglas Raillard requested to merge github/fork/douglas-raillard-arm/_home_pr1200 into master

WIP

TODO:

  • add funcgraph_entry and exit event parsers to speedup parsing
  • replace the custom stats by lisa.stats.Stats instead
  • allow getting the dataframe passed to lisa.stats.Stats, so multiple results can be aggregated (for comparison or just to add the results of multiple sessions)
  • Check difference between ftrace timestamps and calltime/rettime (and use the later if they are more precise)
  • handle recursive calls:
    • maybe flatten the calls and reparent the nested calls to the caller of the recursive function. This would allow regular accounting to be done properly. Only the exit_time needs to be fixed up to remove the recursive calls contribution (otherwise self_time will be wrong, as we will hide some children of the function)
  • Detect IRQ entry points and remove the time spent interrupted by the IRQ routine from the self_time of each node.
  • Handle indirectly recursive
  • Allow comparing a function against another one (for tagged variants)
  • Try to normalize times with CPU capacity using cpu_capacity
  • Allow displaying stats per-CPU, so that people fixing frequency can see the variations per CPU
  • Check on empty dataframes
  • Use NaN for invalid metrics
  • Make "autoclosed" calls invalid (the ones that did not have a closing event) to avoid counting a broken sample
  • Add option to ignore calls for a given amount of time after a change in CPU capacity (when we don't know what the frequency is yet)
  • (optional) when no tag dataframe is used, we could simply use df_delta introduced by https://github.com/ARM-software/lisa/pull/1543 by grouping on the ['dept', 'func'] columns of the funcgraph events. This would be much faster than building a pure Python graph.

Allow using the function grapher events alongside custom tag events to generate profiling statistics.

The tag event can be used to tag whole paths in the callgraph and each (function, tags) pair will be considered independently when computing the statistics.

Should be testable with something like:

from lisa.trace import FtraceCollector
from lisa.target import Target
from lisa.utils import setup_logging
setup_logging()

target = Target.from_default_conf()

# List of functions to profile
functions = [...]

# To be replaced with FtraceCollector/Trace if persistent trace file is wanted
with Trace.from_target(target, events=['funcgraph_entry', 'funcgraph_exit''], functions=functions) as trace:
    target.execute('echo hello world')

# An event can be used to tag paths in the callgraph. Function calls along each path will be treated independently for the statistics.
# E.g. using
# trace_printk("pathtag: path=%s", "path1")
# tag_df = trace.df_event('trace_printk@pathtag')
tag_df = None
stats = trace.analysis.functions.profile_stats(functions=functions, tag_df=tag_df)
stats.plot_stats()

Merge request reports