Skip to content
  • Hagen Paul Pfeifer's avatar
    net: optimize Berkeley Packet Filter (BPF) processing · 01f2f3f6
    Hagen Paul Pfeifer authored
    
    
    Gcc is currenlty not in the ability to optimize the switch statement in
    sk_run_filter() because of dense case labels. This patch replace the
    OR'd labels with ordered sequenced case labels. The sk_chk_filter()
    function is modified to patch/replace the original OPCODES in a
    ordered but equivalent form. gcc is now in the ability to transform the
    switch statement in sk_run_filter into a jump table of complexity O(1).
    
    Until this patch gcc generates a sequence of conditional branches (O(n) of 567
    byte .text segment size (arch x86_64):
    
    7ff: 8b 06                 mov    (%rsi),%eax
    801: 66 83 f8 35           cmp    $0x35,%ax
    805: 0f 84 d0 02 00 00     je     adb <sk_run_filter+0x31d>
    80b: 0f 87 07 01 00 00     ja     918 <sk_run_filter+0x15a>
    811: 66 83 f8 15           cmp    $0x15,%ax
    815: 0f 84 c5 02 00 00     je     ae0 <sk_run_filter+0x322>
    81b: 77 73                 ja     890 <sk_run_filter+0xd2>
    81d: 66 83 f8 04           cmp    $0x4,%ax
    821: 0f 84 17 02 00 00     je     a3e <sk_run_filter+0x280>
    827: 77 29                 ja     852 <sk_run_filter+0x94>
    829: 66 83 f8 01           cmp    $0x1,%ax
    [...]
    
    With the modification the compiler translate the switch statement into
    the following jump table fragment:
    
    7ff: 66 83 3e 2c           cmpw   $0x2c,(%rsi)
    803: 0f 87 1f 02 00 00     ja     a28 <sk_run_filter+0x26a>
    809: 0f b7 06              movzwl (%rsi),%eax
    80c: ff 24 c5 00 00 00 00  jmpq   *0x0(,%rax,8)
    813: 44 89 e3              mov    %r12d,%ebx
    816: e9 43 03 00 00        jmpq   b5e <sk_run_filter+0x3a0>
    81b: 41 89 dc              mov    %ebx,%r12d
    81e: e9 3b 03 00 00        jmpq   b5e <sk_run_filter+0x3a0>
    
    Furthermore, I reordered the instructions to reduce cache line misses by
    order the most common instruction to the start.
    
    Signed-off-by: default avatarHagen Paul Pfeifer <hagen@jauu.net>
    Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    01f2f3f6