Skip to content
  • Eric Dumazet's avatar
    [INET]: speedup inet (tcp/dccp) lookups · 81c3d547
    Eric Dumazet authored
    Arnaldo and I agreed it could be applied now, because I have other
    pending patches depending on this one (Thank you Arnaldo)
    
    (The other important patch moves skc_refcnt in a separate cache line,
    so that the SMP/NUMA performance doesnt suffer from cache line ping pongs)
    
    1) First some performance data :
    --------------------------------
    
    tcp_v4_rcv() wastes a *lot* of time in __inet_lookup_established()
    
    The most time critical code is :
    
    sk_for_each(sk, node, &head->chain) {
         if (INET_MATCH(sk, acookie, saddr, daddr, ports, dif))
             goto hit; /* You sunk my battleship! */
    }
    
    The sk_for_each() does use prefetch() hints but only the begining of
    "struct sock" is prefetched.
    
    As INET_MATCH first comparison uses inet_sk(__sk)->daddr, wich is far
    away from the begining of "struct sock", it has to bring into CPU
    cache cold cache line. Each iteration has to use at least 2 cache
    lines.
    
    This can be problematic if some chains are very long.
    
    2) The goal
    -----------
    
    ...
    81c3d547