• Eric Dumazet's avatar
    net: reorder struct sock fields · b178bb3d
    Eric Dumazet authored
    
    
    Right now, fields in struct sock are not optimally ordered, because each
    path (RX softirq, TX completion, RX user,  TX user) has to touch fields
    that are contained in many different cache lines.
    
    The really critical thing is to shrink number of cache lines that are
    used at RX softirq time : CPU handling softirqs for a device can receive
    many frames per second for many sockets. If load is too big, we can drop
    frames at NIC level. RPS or multiqueue cards can help, but better reduce
    latency if possible.
    
    This patch starts with UDP protocol, then additional patches will try to
    reduce latencies of other ones as well.
    
    At RX softirq time, fields of interest for UDP protocol are :
    (not counting ones in inet struct for the lookup)
    
    Read/Written:
    sk_refcnt   (atomic increment/decrement)
    sk_rmem_alloc & sk_backlog.len (to check if there is room in queues)
    sk_receive_queue
    sk_backlog (if socket locked by user program)
    sk_rxhash
    sk_forward_alloc
    sk_drops
    
    Read only:
    sk_rcvbuf (sk_rcvqueues_full())
    sk_filter
    sk_wq
    sk_policy[0]
    sk_flags
    
    Additional notes :
    
    - sk_backlog has one hole on 64bit arches. We can fill it to save 8
    bytes.
    - sk_backlog is used only if RX sofirq handler finds the socket while
    locked by user.
    - sk_rxhash is written only once per flow.
    - sk_drops is written only if queues are full
    
    Final layout :
    
    [1] One section grouping all read/write fields, but placing rxhash and
    sk_backlog at the end of this section.
    
    [2] One section grouping all read fields in RX handler
       (sk_filter, sk_rcv_buf, sk_wq)
    
    [3] Section used by other paths
    
    I'll post a patch on its own to put sk_refcnt at the end of struct
    sock_common so that it shares same cache line than section [1]
    
    New offsets on 64bit arch :
    
    sizeof(struct sock)=0x268
    offsetof(struct sock, sk_refcnt)  =0x10
    offsetof(struct sock, sk_lock)    =0x48
    offsetof(struct sock, sk_receive_queue)=0x68
    offsetof(struct sock, sk_backlog)=0x80
    offsetof(struct sock, sk_rmem_alloc)=0x80
    offsetof(struct sock, sk_forward_alloc)=0x98
    offsetof(struct sock, sk_rxhash)=0x9c
    offsetof(struct sock, sk_rcvbuf)=0xa4
    offsetof(struct sock, sk_drops) =0xa0
    offsetof(struct sock, sk_filter)=0xa8
    offsetof(struct sock, sk_wq)=0xb0
    offsetof(struct sock, sk_policy)=0xd0
    offsetof(struct sock, sk_flags) =0xe0
    
    Instead of :
    
    sizeof(struct sock)=0x270
    offsetof(struct sock, sk_refcnt)  =0x10
    offsetof(struct sock, sk_lock)    =0x50
    offsetof(struct sock, sk_receive_queue)=0xc0
    offsetof(struct sock, sk_backlog)=0x70
    offsetof(struct sock, sk_rmem_alloc)=0xac
    offsetof(struct sock, sk_forward_alloc)=0x10c
    offsetof(struct sock, sk_rxhash)=0x128
    offsetof(struct sock, sk_rcvbuf)=0x4c
    offsetof(struct sock, sk_drops) =0x16c
    offsetof(struct sock, sk_filter)=0x198
    offsetof(struct sock, sk_wq)=0x88
    offsetof(struct sock, sk_policy)=0x98
    offsetof(struct sock, sk_flags) =0x130
    
    Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
    Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    b178bb3d