1. 19 Aug, 2019 1 commit
  2. 25 Apr, 2019 2 commits
    • Valentin Schneider's avatar
      Merge pull request #290 from derkling/optimize-df-memory · a8abc5d8
      Valentin Schneider authored
      trace/base: Optimizing DataFrame memory footprint
      a8abc5d8
    • Patrick Bellasi's avatar
      trace/base: Optimizing DataFrame memory footprint · 9b3dc0c6
      Patrick Bellasi authored
      Under the hood pandas represents numeric values as NumPy ndarrays and
      stores them in a continuous block of memory. Values of the same column
      are represented using the same type and thus number of bytes.
      
      Many types in pandas have multiple subtypes that can use fewer bytes to
      represent each value. For example, the float type has the float16,
      float32, and float64 subtypes.
      
      Use the function pd.to_numeric() to downcast numeric types to use for
      each value the minumum number of bytes which is still enough to
      represent the maximum value for a given column.
      
      Use also "Categoricals" introduced in Pandas since version 0.15.
      The category type uses integer values under the hood to represent the
      values in a column, rather than the raw values. Use category to
      efficiently compress the representation of string values by replacing
      64bit string pointers with an index using less bits.
      
      Credits goes to:
      
         Using pandas with Large Data Sets
         https://www.dataquest.io/blog/pandas-big-data/
      
      
      
      where these changes are proposed and discussed in details.
      
      The proposed change applied to a 473M example trace gives the following
      results:
      
                         | Events |  Memory (MB)  |  Compression
                         |  count | Before  After |      percent
        -----------------+--------+ --------------+-------------
         clock_disable   |  18368 |   3.34   0.88 |    73.652695
         clock_enable    |  19149 |   3.43   0.92 |    73.177843
         clock_set_rate  |  42099 |   7.63   2.01 |    73.656619
         cpu_idle        | 272726 |  28.87  12.74 |    55.871147
         sched_switch    | 315951 |  82.86  23.26 |    71.928554
      
      Signed-off-by: default avatarPatrick Bellasi <patrick.bellasi@arm.com>
      9b3dc0c6
  3. 24 Apr, 2019 4 commits
  4. 15 Apr, 2019 1 commit
  5. 12 Apr, 2019 3 commits
  6. 06 Feb, 2019 1 commit
  7. 03 Jan, 2019 1 commit
    • Valentin Schneider's avatar
      tests/test_ftrace: Fix test_get_all_freqs_data() · a8d752eb
      Valentin Schneider authored
      In Python2, iterating over a dict's keys will always give the same
      sequence (although the ordering is arbitrary). In Python3 that
      ordering is still abitrary, but to make things even better it can
      change from one execution to another - see [1].
      
      GenericFTrace.get_all_freqs_data() returns a list which's ordering
      depends on the keys of the input dictionnary. That in itself would
      need to be changed to a better design, but for the time being we can
      fix the test and make Trappy pass reliably by converting the output
      of get_all_freqs_data() to a dict().
      
      [1]: https://docs.python.org/2/using/cmdline.html#cmdoption-r
      a8d752eb
  8. 19 Dec, 2018 2 commits
    • Javi Merino's avatar
      Merge pull request #288 from valschneider/trappy-hex-cast · aa6f8e2a
      Javi Merino authored
      base: Fix string to int conversion
      aa6f8e2a
    • Valentin Schneider's avatar
      base: Fix string to int conversion · 9eca88cf
      Valentin Schneider authored
      Because it relied on str.isdigit(), Base.string_cast() would
      erroneously assume a hex string (e.g. "0x42") wouldn't be castable to
      int.
      
      Use a more robust pattern using try/except blocks. This will happily
      cope with negative and hex values.
      
      While at it, rename string_cast() to string_cast_int(), since the
      former only handles ints and does an ugly empty return when the
      specified type isn't int.
      9eca88cf
  9. 28 Sep, 2018 3 commits
  10. 27 Sep, 2018 3 commits
  11. 20 Sep, 2018 1 commit
    • Douglas Raillard's avatar
      ftrace: fix __getattr__ implementation · 836d6438
      Douglas Raillard authored
      Since __getattr__ can be called on uninitialized instances (i.e. with no
      attributes), it can enter an infinite recursion while trying to access
      some instance attributes.
      
      Fix that by ignoring requests for dunder names (__XXX__), since they are
      resolved without __getattr__ when needed by a builtin, and are mostly
      supposed to be class attributes in other cases. That means __getattr__
      is not supposed to receive requests to such names often, and these will
      (almost) surely not be anything the API can offer since usage of such
      names is reserved for future expansion of the language.
      836d6438
  12. 14 Sep, 2018 1 commit
  13. 07 Sep, 2018 3 commits
  14. 06 Sep, 2018 1 commit
  15. 05 Sep, 2018 13 commits