Audit RDMA Resources with rdma resource show

Something is holding RDMA resources, and rdma resource show is how you look without risking a restart. It gives a read-only view of devices, ports and applications: a whole-system summary, a queue-pair query for one link or port, detailed output, and filters for common resource types. Allow about ten minutes. You need the rdma utility from iproute2; the version used here is iproute2-6.1.0-1ubuntu6.4, which reports itself as iproute2-6.1.0.

Everything below is read-only. Run it as your normal user first. Reach for sudo only if the system says its RDMA resource interface cannot be read, not as a default habit.

1. Confirm the tool and list every link

Check which executable runs, then ask for the summary view:

$ command -v rdma
/usr/bin/rdma
$ rdma resource show

Omit the device and port selector and you get a summary for every RDMA link. On a machine with none, empty output and exit status 0 is a perfectly valid result. If a script needs to tell a successful empty query apart from a real error, check the status straight away:

$ status=$?
$ printf 'rdma exited with status %s\n' "$status"
rdma exited with status 0

Checkpoint: you now know whether this host exposes any links at all. Do not assume RDMA hardware exists just because the command is installed.

2. Narrow the view to one device

Pass a device name after show to request only that link:

$ rdma resource show mlx5_2

• mlx5_2 is a manual-page example, not a name guaranteed to exist on your host. Swap in a real one from your own tooling.
• Wrong name, wrong result. The command reports Wrong device name and a non-zero status. That is a lookup failure, not proof the device has no resources.

A device and port can be written as DEVICE/PORT_INDEX for queue-pair queries. Port indexes are normally positive; the manual uses mlx5_4/0 as its example of an illegal port, so do not reach for zero as a discovery value.

3. Inspect queue pairs by link or port

Queue pairs use the qp link form. Request every QP on a device:

$ rdma resource show qp link mlx5_4

Add a port index to narrow it further:

$ rdma resource show qp link mlx5_4/1

Either command may return no rows on a device with no matching QPs, or fail with Wrong device name on a host without mlx5_4. Substitute a real device and a valid port before you treat either result as inventory.

Looking for QPs with no assigned port yet? Use a hyphen as the port selector, kept attached to the device name:

$ rdma resource show qp link mlx5_4/-

That hyphen is not a shell option and does not mean all ports. It is a specific selector documented for exactly this case.

4. Pull detail or machine-readable output

The -d option asks for a detailed view; the long spelling -details works too:

$ rdma -d resource show qp link mlx5_4/-
$ rdma resource show qp link mlx5_4/- -dd

The second form adds driver-specific detail. Those fields are not portable: their presence and names depend on the RDMA driver and the running kernel, so treat them as diagnostic data rather than a stable schema for a cross-host parser.

For anything you plan to parse, use JSON instead:

$ rdma -j resource show
[]

The empty array here just means this machine has no reported RDMA resources. On a host that does, look at the actual object shape before you write a parser against it. The long form of the flag is -json.

Checkpoint: reach for JSON when automating, detail mode when investigating by hand. Do not parse driver-specific detail text as though it were a stable interface.

5. Query other resource types

The resource selector accepts cm_id, cq, mr, pd, qp, ctx and srq. Start broad, then add the filter documented for the type you care about:

$ rdma resource show cm_id dst-port 7174
$ rdma resource show cm_id src-addr 172.16.0.100
$ rdma resource show cq pid 30489
$ rdma resource show ctx ctxn 1
$ rdma resource show srq lqpn 5-7

These are exact selectors, not text-matching guesses. dst-port 7174 asks for CM_IDs whose destination IP port is 7174; pid 30489 asks for completion queues owned by that process. Swap in the real address, port, PID and QPN range from the incident you are chasing. A process ID can vanish between observation and query, so an empty result does not prove it never held a resource.

6. Handle errors without touching the system

rdma resource show is inspection only, it has no delete or reset operation, so rerunning the query is the normal recovery step. If a query fails, check spelling and device names first:

$ rdma resource help
$ rdma -V
rdma utility, iproute2-6.1.0

• Command missing? Install the distribution package through your normal process.
• Device present but unreadable? Capture the exact error, kernel version and link name before you touch permissions or restart RDMA services.
• Tempted by a privileged retry? It can help diagnose access restrictions, but hides a deployment problem if it ends up in an unattended script.

There is no undo command because nothing here changes state. Do not redirect output to a file that matters without an intentional destination, and do not turn an empty response into an automatic device restart. Resource visibility and resource ownership are separate questions.

Done means

• Version confirmed: rdma -V identifies the installed iproute2 version.
• Baseline captured: rdma resource show completed, and you know what an empty result means on this host.
• Selectors are real: device and port values are genuine, and any port index is a valid positive number.
• Detail treated as detail: driver-specific output stays diagnostic; JSON gets inspected before automation relies on it.
• Filters are current: any port, address, PID or QPN range came from the actual investigation.
• Nothing changed: no device, RDMA service or application state was touched.