• Greg Thelen's avatar
    slab: avoid IPIs when creating kmem caches · a87c75fb
    Greg Thelen authored
    Each slab kmem cache has per cpu array caches.  The array caches are
    created when the kmem_cache is created, either via kmem_cache_create()
    or lazily when the first object is allocated in context of a kmem
    enabled memcg.  Array caches are replaced by writing to /proc/slabinfo.
    
    Array caches are protected by holding slab_mutex or disabling
    interrupts.  Array cache allocation and replacement is done by
    __do_tune_cpucache() which holds slab_mutex and calls
    kick_all_cpus_sync() to interrupt all remote processors which confirms
    there are no references to the old array caches.
    
    IPIs are needed when replacing array caches.  But when creating a new
    array cache, there's no need to send IPIs because there cannot be any
    references to the new cache.  Outside of memcg kmem accounting these
    IPIs occur at boot time, so they're not a problem.  But with memcg kmem
    accounting each container can create kmem caches, so the IPIs are
    wasteful.
    
    Avoid unnecessary IPIs when creating array caches.
    
    Test which reports the IPI count of allocating slab in 10000 memcg:
    
    	import os
    
    	def ipi_count():
    		with open("/proc/interrupts") as f:
    			for l in f:
    				if 'Function call interrupts' in l:
    					return int(l.split()[1])
    
    	def echo(val, path):
    		with open(path, "w") as f:
    			f.write(val)
    
    	n = 10000
    	os.chdir("/mnt/cgroup/memory")
    	pid = str(os.getpid())
    	a = ipi_count()
    	for i in range(n):
    		os.mkdir(str(i))
    		echo("1G\n", "%d/memory.limit_in_bytes" % i)
    		echo("1G\n", "%d/memory.kmem.limit_in_bytes" % i)
    		echo(pid, "%d/cgroup.procs" % i)
    		open("/tmp/x", "w").close()
    		os.unlink("/tmp/x")
    	b = ipi_count()
    	print "%d loops: %d => %d (+%d ipis)" % (n, a, b, b-a)
    	echo(pid, "cgroup.procs")
    	for i in range(n):
    		os.rmdir(str(i))
    
    patched:   10000 loops: 1069 => 1170 (+101 ipis)
    unpatched: 10000 loops: 1192 => 48933 (+47741 ipis)
    
    Link: http://lkml.kernel.org/r/20170416214544.109476-1-gthelen@google.com
    
    Signed-off-by: default avatarGreg Thelen <gthelen@google.com>
    Acked-by: default avatarJoonsoo Kim <iamjoonsoo.kim@lge.com>
    Acked-by: default avatarDavid Rientjes <rientjes@google.com>
    Cc: Christoph Lameter <cl@linux.com>
    Cc: Pekka Enberg <penberg@kernel.org>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    a87c75fb