1. 12 Mar, 2021 6 commits
    • Vincenzo Frascino's avatar
      kselftest/arm64: Verify that TCO is enabled in load_unaligned_zeropad() · 15bfd167
      Vincenzo Frascino authored
      load_unaligned_zeropad() and __get/put_kernel_nofault() functions can
      read passed some buffer limits which may include some MTE granule with a
      different tag.
      
      When MTE async mode is enable, the load operation crosses the boundaries
      and the next granule has a different tag the PE sets the TFSR_EL1.TF1
      bit as if an asynchronous tag fault is happened:
      
       ==================================================================
       BUG: KASAN: invalid-access
       Asynchronous mode enabled: no access details available
      
       CPU: 0 PID: 1 Comm: init Not tainted 5.12.0-rc1-ge1045c86
      
      -dirty #8
       Hardware name: FVP Base RevC (DT)
       Call trace:
         dump_backtrace+0x0/0x1c0
         show_stack+0x18/0x24
         dump_stack+0xcc/0x14c
         kasan_report_async+0x54/0x70
         mte_check_tfsr_el1+0x48/0x4c
         exit_to_user_mode+0x18/0x38
         finish_ret_to_user+0x4/0x15c
       ==================================================================
      
      Verify that Tag Check Override (TCO) is enabled in these functions before
      the load and disable it afterwards to prevent this to happen.
      
      Note: The issue has been observed only with an MTE enabled userspace.
      
      Cc: Catalin Marinas <catalin.marinas@arm.com>
      Cc: Will Deacon <will@kernel.org>
      Reported-by: default avatarBranislav Rankov <Branislav.Rankov@arm.com>
      Signed-off-by: Vincenzo Frascino's avatarVincenzo Frascino <vincenzo.frascino@arm.com>
      15bfd167
    • Vincenzo Frascino's avatar
      arm64: mte: Report async tag faults before suspend · 1b8aed65
      Vincenzo Frascino authored
      
      
      When MTE async mode is enabled TFSR_EL1 contains the accumulative
      asynchronous tag check faults for EL1 and EL0.
      
      During the suspend/resume operations the firmware might perform some
      operations that could change the state of the register resulting in
      a spurious tag check fault report.
      
      Report asynchronous tag faults before suspend and clear the TFSR_EL1
      register after resume to prevent this to happen.
      
      Cc: Catalin Marinas <catalin.marinas@arm.com>
      Cc: Will Deacon <will@kernel.org>
      Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
      Reviewed-by: Catalin Marinas's avatarCatalin Marinas <catalin.marinas@arm.com>
      Reviewed-by: Lorenzo Pieralisi's avatarLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
      Signed-off-by: Vincenzo Frascino's avatarVincenzo Frascino <vincenzo.frascino@arm.com>
      1b8aed65
    • Vincenzo Frascino's avatar
      arm64: mte: Enable async tag check fault · f88f1d1e
      Vincenzo Frascino authored
      
      
      MTE provides a mode that asynchronously updates the TFSR_EL1 register
      when a tag check exception is detected.
      
      To take advantage of this mode the kernel has to verify the status of
      the register at:
        1. Context switching
        2. Return to user/EL0 (Not required in entry from EL0 since the kernel
        did not run)
        3. Kernel entry from EL1
        4. Kernel exit to EL1
      
      If the register is non-zero a trace is reported.
      
      Add the required features for EL1 detection and reporting.
      
      Note: ITFSB bit is set in the SCTLR_EL1 register hence it guaranties that
      the indirect writes to TFSR_EL1 are synchronized at exception entry to
      EL1. On the context switch path the synchronization is guarantied by the
      dsb() in __switch_to().
      The dsb(nsh) in mte_check_tfsr_exit() is provisional pending
      confirmation by the architects.
      
      Cc: Catalin Marinas <catalin.marinas@arm.com>
      Cc: Will Deacon <will@kernel.org>
      Reviewed-by: Catalin Marinas's avatarCatalin Marinas <catalin.marinas@arm.com>
      Acked-by: default avatarAndrey Konovalov <andreyknvl@google.com>
      Signed-off-by: Vincenzo Frascino's avatarVincenzo Frascino <vincenzo.frascino@arm.com>
      f88f1d1e
    • Vincenzo Frascino's avatar
      arm64: mte: Enable TCO in functions that can read beyond buffer limits · b49aecf4
      Vincenzo Frascino authored
      
      
      load_unaligned_zeropad() and __get/put_kernel_nofault() functions can
      read past some buffer limits which may include some MTE granule with a
      different tag.
      
      When MTE async mode is enabled, the load operation crosses the boundaries
      and the next granule has a different tag the PE sets the TFSR_EL1.TF1 bit
      as if an asynchronous tag fault is happened.
      
      Enable Tag Check Override (TCO) in these functions  before the load and
      disable it afterwards to prevent this to happen.
      
      Note: The same condition can be hit in MTE sync mode but we deal with it
      through the exception handling.
      In the current implementation, mte_async_mode flag is set only at boot
      time but in future kasan might acquire some runtime features that
      that change the mode dynamically, hence we disable it when sync mode is
      selected for future proof.
      
      Cc: Catalin Marinas <catalin.marinas@arm.com>
      Cc: Will Deacon <will@kernel.org>
      Reported-by: default avatarBranislav Rankov <Branislav.Rankov@arm.com>
      Tested-by: default avatarBranislav Rankov <Branislav.Rankov@arm.com>
      Signed-off-by: Vincenzo Frascino's avatarVincenzo Frascino <vincenzo.frascino@arm.com>
      b49aecf4
    • Vincenzo Frascino's avatar
      kasan: Add report for async mode · e889f587
      Vincenzo Frascino authored
      
      
      KASAN provides an asynchronous mode of execution.
      
      Add reporting functionality for this mode.
      
      Cc: Dmitry Vyukov <dvyukov@google.com>
      Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
      Cc: Alexander Potapenko <glider@google.com>
      Cc: Andrey Konovalov <andreyknvl@google.com>
      Reviewed-by: default avatarAndrey Konovalov <andreyknvl@google.com>
      Signed-off-by: Vincenzo Frascino's avatarVincenzo Frascino <vincenzo.frascino@arm.com>
      Signed-off-by: default avatarAndrey Konovalov <andreyknvl@google.com>
      e889f587
    • Vincenzo Frascino's avatar
      arm64: mte: Drop arch_enable_tagging() · 1121ebd8
      Vincenzo Frascino authored
      
      
      arch_enable_tagging() was left in memory.h after the introduction of
      async mode to not break the bysectability of the KASAN KUNIT tests.
      
      Remove the function now that KASAN has been fully converted.
      
      Cc: Catalin Marinas <catalin.marinas@arm.com>
      Cc: Will Deacon <will@kernel.org>
      Acked-by: Catalin Marinas's avatarCatalin Marinas <catalin.marinas@arm.com>
      Signed-off-by: Vincenzo Frascino's avatarVincenzo Frascino <vincenzo.frascino@arm.com>
      1121ebd8
  2. 08 Mar, 2021 2 commits
    • Vincenzo Frascino's avatar
      kasan: Add KASAN mode kernel parameter · 75107546
      Vincenzo Frascino authored
      
      
      Architectures supported by KASAN_HW_TAGS can provide a sync or async mode
      of execution. On an MTE enabled arm64 hw for example this can be identified
      with the synchronous or asynchronous tagging mode of execution.
      In synchronous mode, an exception is triggered if a tag check fault occurs.
      In asynchronous mode, if a tag check fault occurs, the TFSR_EL1 register is
      updated asynchronously. The kernel checks the corresponding bits
      periodically.
      
      KASAN requires a specific kernel command line parameter to make use of this
      hw features.
      
      Add KASAN HW execution mode kernel command line parameter.
      
      Note: This patch adds the kasan.mode kernel parameter and the
      sync/async kernel command line options to enable the described features.
      
      Cc: Dmitry Vyukov <dvyukov@google.com>
      Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
      Cc: Alexander Potapenko <glider@google.com>
      Cc: Andrey Konovalov <andreyknvl@google.com>
      Reviewed-by: default avatarAndrey Konovalov <andreyknvl@google.com>
      Signed-off-by: Vincenzo Frascino's avatarVincenzo Frascino <vincenzo.frascino@arm.com>
      [ Add a new var instead of exposing kasan_arg_mode to be consistent with
        flags for other command line arguments. ]
      Signed-off-by: default avatarAndrey Konovalov <andreyknvl@google.com>
      75107546
    • Vincenzo Frascino's avatar
      arm64: mte: Add asynchronous mode support · 788ec4f3
      Vincenzo Frascino authored
      
      
      MTE provides an asynchronous mode for detecting tag exceptions. In
      particular instead of triggering a fault the arm64 core updates a
      register which is checked by the kernel after the asynchronous tag
      check fault has occurred.
      
      Add support for MTE asynchronous mode.
      
      The exception handling mechanism will be added with a future patch.
      
      Note: KASAN HW activates async mode via kasan.mode kernel parameter.
      The default mode is set to synchronous.
      The code that verifies the status of TFSR_EL1 will be added with a
      future patch.
      
      Cc: Catalin Marinas <catalin.marinas@arm.com>
      Cc: Will Deacon <will@kernel.org>
      Reviewed-by: Catalin Marinas's avatarCatalin Marinas <catalin.marinas@arm.com>
      Reviewed-by: default avatarAndrey Konovalov <andreyknvl@google.com>
      Signed-off-by: Vincenzo Frascino's avatarVincenzo Frascino <vincenzo.frascino@arm.com>
      788ec4f3
  3. 05 Mar, 2021 32 commits