Commit dc35a10f authored by Marcin Wojtas's avatar Marcin Wojtas Committed by David S. Miller
Browse files

net: mvneta: bm: add support for hardware buffer management



Buffer manager (BM) is a dedicated hardware unit that can be used by all
ethernet ports of Armada XP and 38x SoC's. It allows to offload CPU on RX
path by sparing DRAM access on refilling buffer pool, hardware-based
filling of descriptor ring data and better memory utilization due to HW
arbitration for using 'short' pools for small packets.

Tests performed with A388 SoC working as a network bridge between two
packet generators showed increase of maximum processed 64B packets by
~20k (~555k packets with BM enabled vs ~535 packets without BM). Also
when pushing 1500B-packets with a line rate achieved, CPU load decreased
from around 25% without BM to 20% with BM.

BM comprise up to 4 buffer pointers' (BP) rings kept in DRAM, which
are called external BP pools - BPPE. Allocating and releasing buffer
pointers (BP) to/from BPPE is performed indirectly by write/read access
to a dedicated internal SRAM, where internal BP pools (BPPI) are placed.
BM hardware controls status of BPPE automatically, as well as assigning
proper buffers to RX descriptors. For more details please refer to
Functional Specification of Armada XP or 38x SoC.

In order to enable support for a separate hardware block, common for all
ports, a new driver has to be implemented ('mvneta_bm'). It provides
initialization sequence of address space, clocks, registers, SRAM,
empty pools' structures and also obtaining optional configuration
from DT (please refer to device tree binding documentation). mvneta_bm
exposes also a necessary API to mvneta driver, as well as a dedicated
structure with BM information (bm_priv), whose presence is used as a
flag notifying of BM usage by port. It has to be ensured that mvneta_bm
probe is executed prior to the ones in ports' driver. In case BM is not
used or its probe fails, mvneta falls back to use software buffer
management.

A sequence executed in mvneta_probe function is modified in order to have
an access to needed resources before possible port's BM initialization is
done. According to port-pools mapping provided by DT appropriate registers
are configured and the buffer pools are filled. RX path is modified
accordingly. Becaues the hardware allows a wide variety of configuration
options, following assumptions are made:
* using BM mechanisms can be selectively disabled/enabled basing
  on DT configuration among the ports
* 'long' pool's single buffer size is tied to port's MTU
* using 'long' pool by port is obligatory and it cannot be shared
* using 'short' pool for smaller packets is optional
* one 'short' pool can be shared among all ports

This commit enables hardware buffer management operation cooperating with
existing mvneta driver. New device tree binding documentation is added and
the one of mvneta is updated accordingly.

[gregory.clement@free-electrons.com: removed the suspend/resume part]

Signed-off-by: default avatarMarcin Wojtas <mw@semihalf.com>
Signed-off-by: default avatarGregory CLEMENT <gregory.clement@free-electrons.com>
Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
parent f2900ace
......@@ -18,15 +18,30 @@ Optional properties:
"core" for core clock and "bus" for the optional bus clock.
Optional properties (valid only for Armada XP/38x):
- buffer-manager: a phandle to a buffer manager node. Please refer to
Documentation/devicetree/bindings/net/marvell-neta-bm.txt
- bm,pool-long: ID of a pool, that will accept all packets of a size
higher than 'short' pool's threshold (if set) and up to MTU value.
Obligatory, when the port is supposed to use hardware
buffer management.
- bm,pool-short: ID of a pool, that will be used for accepting
packets of a size lower than given threshold. If not set, the port
will use a single 'long' pool for all packets, as defined above.
Example:
ethernet@d0070000 {
ethernet@70000 {
compatible = "marvell,armada-370-neta";
reg = <0xd0070000 0x2500>;
reg = <0x70000 0x2500>;
interrupts = <8>;
clocks = <&gate_clk 4>;
tx-csum-limit = <9800>
status = "okay";
phy = <&phy0>;
phy-mode = "rgmii-id";
buffer-manager = <&bm>;
bm,pool-long = <0>;
bm,pool-short = <1>;
};
* Marvell Armada 380/XP Buffer Manager driver (BM)
Required properties:
- compatible: should be "marvell,armada-380-neta-bm".
- reg: address and length of the register set for the device.
- clocks: a pointer to the reference clock for this device.
- internal-mem: a phandle to BM internal SRAM definition.
Optional properties (port):
- pool<0 : 3>,capacity: size of external buffer pointers' ring maintained
in DRAM. Can be set for each pool (id 0 : 3) separately. The value has
to be chosen between 128 and 16352 and it also has to be aligned to 32.
Otherwise the driver would adjust a given number or choose default if
not set.
- pool<0 : 3>,pkt-size: maximum size of a packet accepted by a given buffer
pointers' pool (id 0 : 3). It will be taken into consideration only when pool
type is 'short'. For 'long' ones it would be overridden by port's MTU.
If not set a driver will choose a default value.
In order to see how to hook the BM to a given ethernet port, please
refer to Documentation/devicetree/bindings/net/marvell-armada-370-neta.txt.
Example:
- main node:
bm: bm@c8000 {
compatible = "marvell,armada-380-neta-bm";
reg = <0xc8000 0xac>;
clocks = <&gateclk 13>;
internal-mem = <&bm_bppi>;
status = "okay";
pool2,capacity = <4096>;
pool1,pkt-size = <512>;
};
- internal SRAM node:
bm_bppi: bm-bppi {
compatible = "mmio-sram";
reg = <MBUS_ID(0x0c, 0x04) 0 0x100000>;
ranges = <0 MBUS_ID(0x0c, 0x04) 0 0x100000>;
#address-cells = <1>;
#size-cells = <1>;
clocks = <&gateclk 13>;
status = "okay";
};
......@@ -40,6 +40,19 @@ config MVMDIO
This driver is used by the MV643XX_ETH and MVNETA drivers.
config MVNETA_BM
tristate "Marvell Armada 38x/XP network interface BM support"
depends on MVNETA
---help---
This driver supports auxiliary block of the network
interface units in the Marvell ARMADA XP and ARMADA 38x SoC
family, which is called buffer manager.
This driver, when enabled, strictly cooperates with mvneta
driver and is common for all network ports of the devices,
even for Armada 370 SoC, which doesn't support hardware
buffer management.
config MVNETA
tristate "Marvell Armada 370/38x/XP network interface support"
depends on PLAT_ORION
......
......@@ -4,6 +4,7 @@
obj-$(CONFIG_MVMDIO) += mvmdio.o
obj-$(CONFIG_MV643XX_ETH) += mv643xx_eth.o
obj-$(CONFIG_MVNETA_BM) += mvneta_bm.o
obj-$(CONFIG_MVNETA) += mvneta.o
obj-$(CONFIG_MVPP2) += mvpp2.o
obj-$(CONFIG_PXA168_ETH) += pxa168_eth.o
......
This diff is collapsed.
/*
* Driver for Marvell NETA network controller Buffer Manager.
*
* Copyright (C) 2015 Marvell
*
* Marcin Wojtas <mw@semihalf.com>
*
* This file is licensed under the terms of the GNU General Public
* License version 2. This program is licensed "as is" without any
* warranty of any kind, whether express or implied.
*/
#include <linux/kernel.h>
#include <linux/genalloc.h>
#include <linux/platform_device.h>
#include <linux/netdevice.h>
#include <linux/skbuff.h>
#include <linux/mbus.h>
#include <linux/module.h>
#include <linux/io.h>
#include <linux/of.h>
#include <linux/clk.h>
#include "mvneta_bm.h"
#define MVNETA_BM_DRIVER_NAME "mvneta_bm"
#define MVNETA_BM_DRIVER_VERSION "1.0"
static void mvneta_bm_write(struct mvneta_bm *priv, u32 offset, u32 data)
{
writel(data, priv->reg_base + offset);
}
static u32 mvneta_bm_read(struct mvneta_bm *priv, u32 offset)
{
return readl(priv->reg_base + offset);
}
static void mvneta_bm_pool_enable(struct mvneta_bm *priv, int pool_id)
{
u32 val;
val = mvneta_bm_read(priv, MVNETA_BM_POOL_BASE_REG(pool_id));
val |= MVNETA_BM_POOL_ENABLE_MASK;
mvneta_bm_write(priv, MVNETA_BM_POOL_BASE_REG(pool_id), val);
/* Clear BM cause register */
mvneta_bm_write(priv, MVNETA_BM_INTR_CAUSE_REG, 0);
}
static void mvneta_bm_pool_disable(struct mvneta_bm *priv, int pool_id)
{
u32 val;
val = mvneta_bm_read(priv, MVNETA_BM_POOL_BASE_REG(pool_id));
val &= ~MVNETA_BM_POOL_ENABLE_MASK;
mvneta_bm_write(priv, MVNETA_BM_POOL_BASE_REG(pool_id), val);
}
static inline void mvneta_bm_config_set(struct mvneta_bm *priv, u32 mask)
{
u32 val;
val = mvneta_bm_read(priv, MVNETA_BM_CONFIG_REG);
val |= mask;
mvneta_bm_write(priv, MVNETA_BM_CONFIG_REG, val);
}
static inline void mvneta_bm_config_clear(struct mvneta_bm *priv, u32 mask)
{
u32 val;
val = mvneta_bm_read(priv, MVNETA_BM_CONFIG_REG);
val &= ~mask;
mvneta_bm_write(priv, MVNETA_BM_CONFIG_REG, val);
}
static void mvneta_bm_pool_target_set(struct mvneta_bm *priv, int pool_id,
u8 target_id, u8 attr)
{
u32 val;
val = mvneta_bm_read(priv, MVNETA_BM_XBAR_POOL_REG(pool_id));
val &= ~MVNETA_BM_TARGET_ID_MASK(pool_id);
val &= ~MVNETA_BM_XBAR_ATTR_MASK(pool_id);
val |= MVNETA_BM_TARGET_ID_VAL(pool_id, target_id);
val |= MVNETA_BM_XBAR_ATTR_VAL(pool_id, attr);
mvneta_bm_write(priv, MVNETA_BM_XBAR_POOL_REG(pool_id), val);
}
/* Allocate skb for BM pool */
void *mvneta_buf_alloc(struct mvneta_bm *priv, struct mvneta_bm_pool *bm_pool,
dma_addr_t *buf_phys_addr)
{
void *buf;
dma_addr_t phys_addr;
buf = mvneta_frag_alloc(bm_pool->frag_size);
if (!buf)
return NULL;
/* In order to update buf_cookie field of RX descriptor properly,
* BM hardware expects buf virtual address to be placed in the
* first four bytes of mapped buffer.
*/
*(u32 *)buf = (u32)buf;
phys_addr = dma_map_single(&priv->pdev->dev, buf, bm_pool->buf_size,
DMA_FROM_DEVICE);
if (unlikely(dma_mapping_error(&priv->pdev->dev, phys_addr))) {
mvneta_frag_free(bm_pool->frag_size, buf);
return NULL;
}
*buf_phys_addr = phys_addr;
return buf;
}
/* Refill processing for HW buffer management */
int mvneta_bm_pool_refill(struct mvneta_bm *priv,
struct mvneta_bm_pool *bm_pool)
{
dma_addr_t buf_phys_addr;
void *buf;
buf = mvneta_buf_alloc(priv, bm_pool, &buf_phys_addr);
if (!buf)
return -ENOMEM;
mvneta_bm_pool_put_bp(priv, bm_pool, buf_phys_addr);
return 0;
}
EXPORT_SYMBOL_GPL(mvneta_bm_pool_refill);
/* Allocate buffers for the pool */
int mvneta_bm_bufs_add(struct mvneta_bm *priv, struct mvneta_bm_pool *bm_pool,
int buf_num)
{
int err, i;
if (bm_pool->buf_num == bm_pool->size) {
dev_dbg(&priv->pdev->dev, "pool %d already filled\n",
bm_pool->id);
return bm_pool->buf_num;
}
if (buf_num < 0 ||
(buf_num + bm_pool->buf_num > bm_pool->size)) {
dev_err(&priv->pdev->dev,
"cannot allocate %d buffers for pool %d\n",
buf_num, bm_pool->id);
return 0;
}
for (i = 0; i < buf_num; i++) {
err = mvneta_bm_pool_refill(priv, bm_pool);
if (err < 0)
break;
}
/* Update BM driver with number of buffers added to pool */
bm_pool->buf_num += i;
dev_dbg(&priv->pdev->dev,
"%s pool %d: pkt_size=%4d, buf_size=%4d, frag_size=%4d\n",
bm_pool->type == MVNETA_BM_SHORT ? "short" : "long",
bm_pool->id, bm_pool->pkt_size, bm_pool->buf_size,
bm_pool->frag_size);
dev_dbg(&priv->pdev->dev,
"%s pool %d: %d of %d buffers added\n",
bm_pool->type == MVNETA_BM_SHORT ? "short" : "long",
bm_pool->id, i, buf_num);
return i;
}
EXPORT_SYMBOL_GPL(mvneta_bm_bufs_add);
/* Create pool */
static int mvneta_bm_pool_create(struct mvneta_bm *priv,
struct mvneta_bm_pool *bm_pool)
{
struct platform_device *pdev = priv->pdev;
u8 target_id, attr;
int size_bytes, err;
size_bytes = sizeof(u32) * bm_pool->size;
bm_pool->virt_addr = dma_alloc_coherent(&pdev->dev, size_bytes,
&bm_pool->phys_addr,
GFP_KERNEL);
if (!bm_pool->virt_addr)
return -ENOMEM;
if (!IS_ALIGNED((u32)bm_pool->virt_addr, MVNETA_BM_POOL_PTR_ALIGN)) {
dma_free_coherent(&pdev->dev, size_bytes, bm_pool->virt_addr,
bm_pool->phys_addr);
dev_err(&pdev->dev, "BM pool %d is not %d bytes aligned\n",
bm_pool->id, MVNETA_BM_POOL_PTR_ALIGN);
return -ENOMEM;
}
err = mvebu_mbus_get_dram_win_info(bm_pool->phys_addr, &target_id,
&attr);
if (err < 0) {
dma_free_coherent(&pdev->dev, size_bytes, bm_pool->virt_addr,
bm_pool->phys_addr);
return err;
}
/* Set pool address */
mvneta_bm_write(priv, MVNETA_BM_POOL_BASE_REG(bm_pool->id),
bm_pool->phys_addr);
mvneta_bm_pool_target_set(priv, bm_pool->id, target_id, attr);
mvneta_bm_pool_enable(priv, bm_pool->id);
return 0;
}
/* Notify the driver that BM pool is being used as specific type and return the
* pool pointer on success
*/
struct mvneta_bm_pool *mvneta_bm_pool_use(struct mvneta_bm *priv, u8 pool_id,
enum mvneta_bm_type type, u8 port_id,
int pkt_size)
{
struct mvneta_bm_pool *new_pool = &priv->bm_pools[pool_id];
int num, err;
if (new_pool->type == MVNETA_BM_LONG &&
new_pool->port_map != 1 << port_id) {
dev_err(&priv->pdev->dev,
"long pool cannot be shared by the ports\n");
return NULL;
}
if (new_pool->type == MVNETA_BM_SHORT && new_pool->type != type) {
dev_err(&priv->pdev->dev,
"mixing pools' types between the ports is forbidden\n");
return NULL;
}
if (new_pool->pkt_size == 0 || type != MVNETA_BM_SHORT)
new_pool->pkt_size = pkt_size;
/* Allocate buffers in case BM pool hasn't been used yet */
if (new_pool->type == MVNETA_BM_FREE) {
new_pool->type = type;
new_pool->buf_size = MVNETA_RX_BUF_SIZE(new_pool->pkt_size);
new_pool->frag_size =
SKB_DATA_ALIGN(MVNETA_RX_BUF_SIZE(new_pool->pkt_size)) +
SKB_DATA_ALIGN(sizeof(struct skb_shared_info));
/* Create new pool */
err = mvneta_bm_pool_create(priv, new_pool);
if (err) {
dev_err(&priv->pdev->dev, "fail to create pool %d\n",
new_pool->id);
return NULL;
}
/* Allocate buffers for this pool */
num = mvneta_bm_bufs_add(priv, new_pool, new_pool->size);
if (num != new_pool->size) {
WARN(1, "pool %d: %d of %d allocated\n",
new_pool->id, num, new_pool->size);
return NULL;
}
}
return new_pool;
}
EXPORT_SYMBOL_GPL(mvneta_bm_pool_use);
/* Free all buffers from the pool */
void mvneta_bm_bufs_free(struct mvneta_bm *priv, struct mvneta_bm_pool *bm_pool,
u8 port_map)
{
int i;
bm_pool->port_map &= ~port_map;
if (bm_pool->port_map)
return;
mvneta_bm_config_set(priv, MVNETA_BM_EMPTY_LIMIT_MASK);
for (i = 0; i < bm_pool->buf_num; i++) {
dma_addr_t buf_phys_addr;
u32 *vaddr;
/* Get buffer physical address (indirect access) */
buf_phys_addr = mvneta_bm_pool_get_bp(priv, bm_pool);
/* Work-around to the problems when destroying the pool,
* when it occurs that a read access to BPPI returns 0.
*/
if (buf_phys_addr == 0)
continue;
vaddr = phys_to_virt(buf_phys_addr);
if (!vaddr)
break;
dma_unmap_single(&priv->pdev->dev, buf_phys_addr,
bm_pool->buf_size, DMA_FROM_DEVICE);
mvneta_frag_free(bm_pool->frag_size, vaddr);
}
mvneta_bm_config_clear(priv, MVNETA_BM_EMPTY_LIMIT_MASK);
/* Update BM driver with number of buffers removed from pool */
bm_pool->buf_num -= i;
}
EXPORT_SYMBOL_GPL(mvneta_bm_bufs_free);
/* Cleanup pool */
void mvneta_bm_pool_destroy(struct mvneta_bm *priv,
struct mvneta_bm_pool *bm_pool, u8 port_map)
{
bm_pool->port_map &= ~port_map;
if (bm_pool->port_map)
return;
bm_pool->type = MVNETA_BM_FREE;
mvneta_bm_bufs_free(priv, bm_pool, port_map);
if (bm_pool->buf_num)
WARN(1, "cannot free all buffers in pool %d\n", bm_pool->id);
if (bm_pool->virt_addr) {
dma_free_coherent(&priv->pdev->dev, sizeof(u32) * bm_pool->size,
bm_pool->virt_addr, bm_pool->phys_addr);
bm_pool->virt_addr = NULL;
}
mvneta_bm_pool_disable(priv, bm_pool->id);
}
EXPORT_SYMBOL_GPL(mvneta_bm_pool_destroy);
static void mvneta_bm_pools_init(struct mvneta_bm *priv)
{
struct device_node *dn = priv->pdev->dev.of_node;
struct mvneta_bm_pool *bm_pool;
char prop[15];
u32 size;
int i;
/* Activate BM unit */
mvneta_bm_write(priv, MVNETA_BM_COMMAND_REG, MVNETA_BM_START_MASK);
/* Create all pools with maximum size */
for (i = 0; i < MVNETA_BM_POOLS_NUM; i++) {
bm_pool = &priv->bm_pools[i];
bm_pool->id = i;
bm_pool->type = MVNETA_BM_FREE;
/* Reset read pointer */
mvneta_bm_write(priv, MVNETA_BM_POOL_READ_PTR_REG(i), 0);
/* Reset write pointer */
mvneta_bm_write(priv, MVNETA_BM_POOL_WRITE_PTR_REG(i), 0);
/* Configure pool size according to DT or use default value */
sprintf(prop, "pool%d,capacity", i);
if (of_property_read_u32(dn, prop, &size)) {
size = MVNETA_BM_POOL_CAP_DEF;
} else if (size > MVNETA_BM_POOL_CAP_MAX) {
dev_warn(&priv->pdev->dev,
"Illegal pool %d capacity %d, set to %d\n",
i, size, MVNETA_BM_POOL_CAP_MAX);
size = MVNETA_BM_POOL_CAP_MAX;
} else if (size < MVNETA_BM_POOL_CAP_MIN) {
dev_warn(&priv->pdev->dev,
"Illegal pool %d capacity %d, set to %d\n",
i, size, MVNETA_BM_POOL_CAP_MIN);
size = MVNETA_BM_POOL_CAP_MIN;
} else if (!IS_ALIGNED(size, MVNETA_BM_POOL_CAP_ALIGN)) {
dev_warn(&priv->pdev->dev,
"Illegal pool %d capacity %d, round to %d\n",
i, size, ALIGN(size,
MVNETA_BM_POOL_CAP_ALIGN));
size = ALIGN(size, MVNETA_BM_POOL_CAP_ALIGN);
}
bm_pool->size = size;
mvneta_bm_write(priv, MVNETA_BM_POOL_SIZE_REG(i),
bm_pool->size);
/* Obtain custom pkt_size from DT */
sprintf(prop, "pool%d,pkt-size", i);
if (of_property_read_u32(dn, prop, &bm_pool->pkt_size))
bm_pool->pkt_size = 0;
}
}
static void mvneta_bm_default_set(struct mvneta_bm *priv)
{
u32 val;
/* Mask BM all interrupts */
mvneta_bm_write(priv, MVNETA_BM_INTR_MASK_REG, 0);
/* Clear BM cause register */
mvneta_bm_write(priv, MVNETA_BM_INTR_CAUSE_REG, 0);
/* Set BM configuration register */
val = mvneta_bm_read(priv, MVNETA_BM_CONFIG_REG);
/* Reduce MaxInBurstSize from 32 BPs to 16 BPs */
val &= ~MVNETA_BM_MAX_IN_BURST_SIZE_MASK;
val |= MVNETA_BM_MAX_IN_BURST_SIZE_16BP;
mvneta_bm_write(priv, MVNETA_BM_CONFIG_REG, val);
}
static int mvneta_bm_init(struct mvneta_bm *priv)
{
mvneta_bm_default_set(priv);
/* Allocate and initialize BM pools structures */
priv->bm_pools = devm_kcalloc(&priv->pdev->dev, MVNETA_BM_POOLS_NUM,
sizeof(struct mvneta_bm_pool),
GFP_KERNEL);
if (!priv->bm_pools)
return -ENOMEM;
mvneta_bm_pools_init(priv);
return 0;
}
static int mvneta_bm_get_sram(struct device_node *dn,
struct mvneta_bm *priv)
{
priv->bppi_pool = of_gen_pool_get(dn, "internal-mem", 0);
if (!priv->bppi_pool)
return -ENOMEM;
priv->bppi_virt_addr = gen_pool_dma_alloc(priv->bppi_pool,
MVNETA_BM_BPPI_SIZE,
&priv->bppi_phys_addr);
if (!priv->bppi_virt_addr)
return -ENOMEM;
return 0;
}
static void mvneta_bm_put_sram(struct mvneta_bm *priv)
{
gen_pool_free(priv->bppi_pool, priv->bppi_phys_addr,
MVNETA_BM_BPPI_SIZE);
}
static int mvneta_bm_probe(struct platform_device *pdev)
{
struct device_node *dn = pdev->dev.of_node;
struct mvneta_bm *priv;
struct resource *res;
int err;
priv = devm_kzalloc(&pdev->dev, sizeof(struct mvneta_bm), GFP_KERNEL);
if (!priv)
return -ENOMEM;
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
priv->reg_base = devm_ioremap_resource(&pdev->dev, res);
if (IS_ERR(priv->reg_base))
return PTR_ERR(priv->reg_base);
priv->clk = devm_clk_get(&pdev->dev, NULL);
if (IS_ERR(priv->clk))
return PTR_ERR(priv->clk);
err = clk_prepare_enable(priv->clk);
if (err < 0)
return err;
err = mvneta_bm_get_sram(dn, priv);
if (err < 0) {
dev_err(&pdev->dev, "failed to allocate internal memory\n");
goto err_clk;
}
priv->pdev = pdev;
/* Initialize buffer manager internals */
err = mvneta_bm_init(priv);
if (err < 0) {
dev_err(&pdev->dev, "failed to initialize controller\n");
goto err_sram;