Skip to content
  • Joonsoo Kim's avatar
    mm/slab: re-implement pfmemalloc support · f68f8ddd
    Joonsoo Kim authored
    Current implementation of pfmemalloc handling in SLAB has some problems.
    1) pfmemalloc_active is set to true when there is just one or more
       pfmemalloc slabs in the system, but it is cleared when there is no
       pfmemalloc slab in one arbitrary kmem_cache.  So, pfmemalloc_active
       could be wrongly cleared.
    2) Search to partial and free list doesn't happen when non-pfmemalloc
       object are not found in cpu cache.  Instead, allocating new slab
       happens and it is not optimal.
    3) Even after sk_memalloc_socks() is disabled, cpu cache would keep
       pfmemalloc objects tagged with SLAB_OBJ_PFMEMALLOC.  It isn't cleared
       if sk_memalloc_socks() is disabled so it could cause problem.
    4) If cpu cache is filled with pfmemalloc objects, it would cause slow
       down non-pfmemalloc allocation.
    To me, current pointer tagging approach looks complex and fragile so this
    patch re-implement whole thing instead of fixing problems one by one.
    Design principle for new implementation is that
    1) Don't disrupt non-pfmemalloc allocation in fast path even if
       sk_memalloc_socks() is enabled.  It's more likely case than pfmemalloc
    2) Ensure that pfmemalloc slab is used only for pfmemalloc allocation.
    3) Don't consider performance of pfmemalloc allocation in memory
       deficiency state.
    As a result, all pfmemalloc alloc/free in memory tight state will be
    handled in slow-path.  If there is non-pfmemalloc free object, it will be
    returned first even for pfmemalloc user in fast-path so that performance
    of pfmemalloc user isn't affected in normal case and pfmemalloc objects
    will be kept as long as possible.
    Signed-off-by: default avatarJoonsoo Kim <>
    Tested-by: default avatarMel Gorman <>
    Cc: Christoph Lameter <>
    Cc: Pekka Enberg <>
    Cc: David Rientjes <>
    Signed-off-by: default avatarAndrew Morton <>
    Signed-off-by: default avatarLinus Torvalds <>