1. 25 Mar, 2017 1 commit
  2. 10 Jan, 2017 4 commits
  3. 12 Dec, 2016 2 commits
    • Andrew Boyer's avatar
      IB/rxe: Hold refs when running tasklets · 37f69f43
      Andrew Boyer authored
      
      
      It might be possible for all of a QP's references to be dropped
      while one of that QP's tasklets is running.
      
      For example, the completer might run during QP destroy.
      If qp->valid is false, it will drop all of the packets on
      the resp_pkts list, potentially removing the last reference.
      Then it tries to advance the SQ consumer pointer. If the
      SQ's buffer has already been destroyed, the system will
      panic.
      
      To be safe, hold a reference on the QP for the duration
      of each tasklet.
      
      Signed-off-by: default avatarAndrew Boyer <andrew.boyer@dell.com>
      Signed-off-by: default avatarDoug Ledford <dledford@redhat.com>
      37f69f43
    • Arnd Bergmann's avatar
      IB/rxe: avoid putting a large struct rxe_qp on stack · a0fa7268
      Arnd Bergmann authored
      A race condition fix added an rxe_qp structure to the stack in order
      to be able to perform rollback in rxe_requester(), but the structure
      is large enough to trigger the warning for possible stack overflow:
      
      drivers/infiniband/sw/rxe/rxe_req.c: In function 'rxe_requester':
      drivers/infiniband/sw/rxe/rxe_req.c:757:1: error: the frame size of 2064 bytes is larger than 1024 bytes [-Werror=frame-larger-than=]
      
      This changes the rollback function to only save the psn inside
      the qp, which is the only field we access in the rollback_qp
      anyway.
      
      Fixes: 3050b998
      
       ("IB/rxe: Fix race condition between requester and completer")
      Signed-off-by: default avatarArnd Bergmann <arnd@arndb.de>
      Reviewed-by: default avatarLeon Romanovsky <leonro@mellanox.com>
      Signed-off-by: default avatarDoug Ledford <dledford@redhat.com>
      a0fa7268
  4. 17 Nov, 2016 1 commit
    • Yonatan Cohen's avatar
      IB/rxe: Fix handling of erroneous WR · 002e062e
      Yonatan Cohen authored
      To correctly handle a erroneous WR this fix does the following
      1. Make sure the bad WQE causes a user completion event.
      2. Call rxe_completer to handle the erred WQE.
      
      Before the fix, when rxe_requester found a bad WQE, it changed its
      status to IB_WC_LOC_PROT_ERR and exit with 0 for non RC QPs.
      
      If this was the 1st WQE then there would be no ACK to invoke the
      completer and this bad WQE would be stuck in the QP's send-q.
      
      On top of that the requester exiting with 0 caused rxe_do_task to
      endlessly invoke rxe_requester, resulting in a soft-lockup attached
      below.
      
      In case the WQE was not the 1st and rxe_completer did get a chance to
      handle the bad WQE, it did not cause a complete event since the WQE's
      IB_SEND_SIGNALED flag was not set.
      
      Setting WQE status to IB_SEND_SIGNALED is subject to IBA spec
      version 1.2.1, section 10.7.3.1 Signaled Completions.
      
      NMI watchdog: BUG: soft lockup - CPU#7 stuck for 22s!
      [<ffffffffa0590145>] ? rxe_pool_get_index+0x35/0xb0 [rdma_rxe]
      [<ffffffffa05952ec>] lookup_mem+0x3c/0xc0 [rdma_rxe]
      [<ffffffffa0595534>] copy_data+0x1c4/0x230 [rdma_rxe]
      [<ffffffffa058c180>] rxe_requester+0x9d0/0x1100 [rdma_rxe]
      [<ffffffff8158e98a>] ? kfree_skbmem+0x5a/0x60
      [<ffffffffa05962c9>] rxe_do_task+0x89/0xf0 [rdma_rxe]
      [<ffffffffa05963e2>] rxe_run_task+0x12/0x30 [rdma_rxe]
      [<ffffffffa059110a>] rxe_post_send+0x41a/0x550 [rdma_rxe]
      [<ffffffff811ef922>] ? __kmalloc+0x182/0x200
      [<ffffffff816ba512>] ? down_read+0x12/0x40
      [<ffffffffa054bd32>] ib_uverbs_post_send+0x532/0x540 [ib_uverbs]
      [<ffffffff815f8722>] ? tcp_sendmsg+0x402/0xb80
      [<ffffffffa05453dc>] ib_uverbs_write+0x18c/0x3f0 [ib_uverbs]
      [<ffffffff81623c2e>] ? inet_recvmsg+0x7e/0xb0
      [<ffffffff8158764d>] ? sock_recvmsg+0x3d/0x50
      [<ffffffff81215b87>] __vfs_write+0x37/0x140
      [<ffffffff81216892>] vfs_write+0xb2/0x1b0
      [<ffffffff81217ce5>] SyS_write+0x55/0xc0
      [<ffffffff816bc672>] entry_SYSCALL_64_fastpath+0x1a/0xa
      
      Fixes: 8700e3e7
      
       ("Soft RoCE driver")
      Signed-off-by: default avatarYonatan Cohen <yonatanc@mellanox.com>
      Reviewed-by: default avatarMoni Shoua <monis@mellanox.com>
      Signed-off-by: default avatarLeon Romanovsky <leon@kernel.org>
      Signed-off-by: default avatarDoug Ledford <dledford@redhat.com>
      002e062e
  5. 06 Oct, 2016 1 commit
    • Parav Pandit's avatar
      IB/rxe: improved debug prints & code cleanup · e404f945
      Parav Pandit authored
      
      
      1. Debugging qp state transitions and qp errors in loopback and
      multiple QP tests is difficult without qp numbers in debug logs.
      This patch adds qp number to important debug logs.
      
      2. Instead of having rxe: prefix in few logs and not having in
      few logs, using uniform module name prefix using pr_fmt macro.
      
      3. Code cleanup for various warnings reported by checkpatch for
      incomplete unsigned data type, line over 80 characters, return
      statements.
      
      Signed-off-by: default avatarParav Pandit <pandit.parav@gmail.com>
      Signed-off-by: default avatarDoug Ledford <dledford@redhat.com>
      e404f945
  6. 16 Sep, 2016 1 commit
  7. 04 Aug, 2016 1 commit
    • Moni Shoua's avatar
      Soft RoCE driver · 8700e3e7
      Moni Shoua authored
      Soft RoCE (RXE) - The software RoCE driver
      
      ib_rxe implements the RDMA transport and registers to the RDMA core
      device as a kernel verbs provider. It also implements the packet IO
      layer. On the other hand ib_rxe registers to the Linux netdev stack
      as a udp encapsulating protocol, in that case RDMA, for sending and
      receiving packets over any Ethernet device.  This yields a RDMA
      transport over the UDP/Ethernet network layer forming a RoCEv2
      compatible device.
      
      The configuration procedure of the Soft RoCE drivers requires
      binding to any existing Ethernet network device. This is done with
      /sys interface.
      
      A userspace Soft RoCE library (librxe) provides user applications
      the ability to run with Soft RoCE devices.  The use of rxe verbs ins
      user space requires the inclusion of librxe as a device specifics
      plug-in to libibverbs. librxe is packaged separately.
      
      Architecture:
      
           +-----------------------------------------------------------+
           |                          Application                      |
           +-----------------------------------------------------------+
                                  +-----------------------------------+
                                  |             libibverbs            |
      User                        +-----------------------------------+
                                  +----------------+ +----------------+
                                  | librxe         | | HW RoCE lib    |
                                  +----------------+ +----------------+
      +---------------------------------------------------------------+
           +--------------+                           +------------+
           | Sockets      |                           | RDMA ULP   |
           +--------------+                           +------------+
           +--------------+                  +---------------------+
           | TCP/IP       |                  | ib_core             |
           +--------------+                  +---------------------+
                                   +------------+ +----------------+
      Kernel                       | ib_rxe     | | HW RoCE driver |
                                   +------------+ +----------------+
           +------------------------------------+
           | NIC driver                         |
           +------------------------------------+
      
      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           +-----------------------------------------------------------+
           |                          Application                      |
           +-----------------------------------------------------------+
                                  +-----------------------------------+
                                  |             libibverbs            |
      User                        +-----------------------------------+
                                  +----------------+ +----------------+
                                  | librxe         | | HW RoCE lib    |
                                  +----------------+ +----------------+
      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           +--------------+                           +------------+
           | Sockets      |                           | RDMA ULP   |
           +--------------+                           +------------+
           +--------------+                  +---------------------+
           | TCP/IP       |                  | ib_core             |
           +--------------+                  +---------------------+
                                   +------------+ +----------------+
      Kernel                       | ib_rxe     | | HW RoCE driver |
                                   +------------+ +----------------+
           +------------------------------------+
           | NIC driver                         |
           +------------------------------------+
      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      
      Soft RoCE resources:
      
      [1[ https://github.com/SoftRoCE/librxe-dev librxe - source code in
      Github
      [2] https://github.com/SoftRoCE/rxe-dev/wiki/rxe-dev:-Home - Soft RoCE
      Wiki page
      [3] https://github.com/SoftRoCE/librxe-dev
      
       - Soft RoCE userspace library
      
      Signed-off-by: default avatarKamal Heib <kamalh@mellanox.com>
      Signed-off-by: default avatarAmir Vadai <amirv@mellanox.com>
      Signed-off-by: default avatarMoni Shoua <monis@mellanox.com>
      Reviewed-by: default avatarHaggai Eran <haggaie@mellanox.com>
      Signed-off-by: default avatarDoug Ledford <dledford@redhat.com>
      8700e3e7