Home / Alt manpages / rdma-link(8)

  • rdma-link(8)
  • Admin command
  • linux

Build and Tear Down a Test RDMA Link with rdma-link

No InfiniBand card in the box but you need a real RDMA link to throw traffic at? rdma-link fakes one, a Soft RoCE or Soft iWARP link bolted onto an ordinary network device, gone the moment you delete it. Budget about fifteen minutes if you already know the interface name. The examples use the installed rdma utility from iproute2 6.1.0-1ubuntu6.4 and the local rdma-link(8) manual page.

The route

Jump straight to the step you need, or tick off Done means at the end.

This covers the two link types documented by that version: rxe (Soft RoCE) and siw (Soft iWARP). It does not touch an RDMA application, IP addressing or a physical adapter.

1. Check the tool and pick a network device

Start with read-only checks. None of these need elevated privileges:

$ command -v rdma
/usr/bin/rdma
$ rdma -V
rdma utility, iproute2-6.1.0
$ ip -brief link
  • Use a real interface. Replace eth0 below with the exact name from your own output.
  • Do not mix up the names. An RDMA link name is not a network-device name: the device sits after netdev.

Checkpoint

Write down the network-device name and pick a link type. If you do not actually need a software RDMA link, stop here: the rest of this guide changes kernel networking state.

Check what exists before you add anything:

$ rdma link show

On the reference host this printed nothing and exited 0. That is normal: a configured machine lists its links, a machine with none may print nothing and still succeed. A link is addressed as DEV/PORT_INDEX, for example mlx5_0/1. Inspect one known link directly:

$ rdma link show mlx5_0/1

Use the device and port index from your own listing, not this example, unless your host genuinely has that link. The exact attributes depend on the driver, so treat the listing as the source of truth for names already in use. Do not invent a new name by guessing from a physical port.

Warning

Adding a link changes live kernel state and can affect any software that discovers RDMA devices. Pick an interface meant for this test, check the name twice, and use sudo only for this state-changing command.

For Soft RoCE, choose a new RDMA link name and bind it to the existing network device:

$ sudo rdma link add rxe_eth0 type rxe netdev eth0

rxe_eth0 is only an example name: it is what you will use to inspect and remove the link later. Need Soft iWARP instead? Use the documented siw type:

$ sudo rdma link add siw_eth0 type siw netdev eth0
  • Do not run both examples unless you deliberately want two software links on the same network device.
  • Name already exists? Pick a different one, or inspect the existing link first.
  • Type or device unavailable? Check the spelling and whether the relevant driver is actually loaded.

Checkpoint

A successful add normally prints nothing. That silence is not proof it worked, so verify it in the next step.

List every link again and look for the name you chose:

$ rdma link show

For a software link, the port index and state come from the driver. Check that your chosen name appears, that the reported netdev is the interface you intended, and that the command exits successfully. Once you know the device and port index, you can also ask for the link directly:

$ rdma link show rxe_eth0/1

Replace rxe_eth0/1 with the exact device and port index from the listing above. If the link is missing, note the error from the add command and check the list again. Do not keep retrying a failure you do not understand: a partial result or a name clash is easier to untangle from the current listing than by guessing.

This is the undo for everything above. Confirm the exact name first, then delete it with elevated privileges:

$ sudo rdma link delete rxe_eth0
$ rdma link show

The final listing should no longer show rxe_eth0. Deleting the link does not delete or rename eth0 itself, though any application using the RDMA link can lose access while it is gone. Created siw_eth0 instead? Substitute that exact name.

There is no safe generic delete for an unknown name. If deletion fails with a name or device error, do not widen the command: re-run the listing, copy the exact name, and retry that one deletion.

Common traps

  • rdma link show is not ip link show. The first lists RDMA links, the second lists network devices.
  • netdev eth0 is the underlying device, not the new RDMA link's name.
  • rxe means Soft RoCE, siw means Soft iWARP. Pick the one your test or application actually expects.
  • DEV/PORT_INDEX is a lookup form, not something to guess: get the port index from rdma link show.
  • rdma link add and rdma link delete change live state. Keep them out of unattended scripts until failure handling and cleanup are explicit.

For the exact syntax on your install, run rdma link help. It lists show, add, delete and their arguments without changing anything.

Done means

  • Version checked: you confirmed the installed iproute2 version and a real network-device name.
  • Existing links reviewed: you listed what was already there before adding a new name.
  • Type chosen deliberately: you used rxe or siw on purpose and verified the resulting netdev.
  • Name recorded: you wrote down the exact RDMA link name so it can be removed without guessing.
  • Cleaned up or kept on purpose: you deleted the test link, or left it because something still needs it.