• Sean Christopherson's avatar
    x86/retpoline: Don't clobber RFLAGS during CALL_NOSPEC on i386 · b63f20a7
    Sean Christopherson authored
    Use 'lea' instead of 'add' when adjusting %rsp in CALL_NOSPEC so as to
    avoid clobbering flags.
    KVM's emulator makes indirect calls into a jump table of sorts, where
    the destination of the CALL_NOSPEC is a small blob of code that performs
    fast emulation by executing the target instruction with fixed operands.
         0x000339f8 <+0>:   adc    %dl,%al
         0x000339fa <+2>:   ret
    A major motiviation for doing fast emulation is to leverage the CPU to
    handle consumption and manipulation of arithmetic flags, i.e. RFLAGS is
    both an input and output to the target of CALL_NOSPEC.  Clobbering flags
    results in all sorts of incorrect emulation, e.g. Jcc instructions often
    take the wrong path.  Sans the nops...
      asm("push %[flags]; popf; " CALL_NOSPEC " ; pushf; pop %[flags]\n"
         0x0003595a <+58>:  mov    0xc0(%ebx),%eax
         0x00035960 <+64>:  mov    0x60(%ebx),%edx
         0x00035963 <+67>:  mov    0x90(%ebx),%ecx
         0x00035969 <+73>:  push   %edi
         0x0003596a <+74>:  popf
         0x0003596b <+75>:  call   *%esi
         0x000359a0 <+128>: pushf
         0x000359a1 <+129>: pop    %edi
         0x000359a2 <+130>: mov    %eax,0xc0(%ebx)
         0x000359b1 <+145>: mov    %edx,0x60(%ebx)
      ctxt->eflags = (ctxt->eflags & ~EFLAGS_MASK) | (flags & EFLAGS_MASK);
         0x000359a8 <+136>: mov    -0x10(%ebp),%eax
         0x000359ab <+139>: and    $0x8d5,%edi
         0x000359b4 <+148>: and    $0xfffff72a,%eax
         0x000359b9 <+153>: or     %eax,%edi
         0x000359bd <+157>: mov    %edi,0x4(%ebx)
    For the most part this has gone unnoticed as emulation of guest code
    that can trigger fast emulation is effectively limited to MMIO when
    running on modern hardware, and MMIO is rarely, if ever, accessed by
    instructions that affect or consume flags.
    Breakage is almost instantaneous when running with unrestricted guest
    disabled, in which case KVM must emulate all instructions when the guest
    has invalid state, e.g. when the guest is in Big Real Mode during early
    Fixes: 776b043848fd2 ("x86/retpoline: Add initial retpoline support")
    Fixes: 1a29b5b7
     ("KVM: x86: Make indirect calls in emulator speculation safe")
    Signed-off-by: default avatarSean Christopherson <sean.j.christopherson@intel.com>
    Signed-off-by: default avatarThomas Gleixner <tglx@linutronix.de>
    Acked-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
    Cc: stable@vger.kernel.org
    Link: https://lkml.kernel.org/r/20190822211122.27579-1-sean.j.christopherson@intel.com