Inspect and Configure RDMA QP Counters with rdma statistic

You will inspect RDMA counters, identify the QP counter mode available from the installed utility, and make a controlled automatic or manual binding change when you actually need one. The examples use rdma from iproute2 6.1.0-1ubuntu6.4. Allow about fifteen minutes for inspection, or longer if you need to coordinate a live workload.

You need the iproute2 package and an RDMA device with a known device name and port. Read-only queries normally run as an ordinary user. The set, bind and unbind commands change kernel counter state and commonly need root or the relevant network-management capability. Do not test them against a production port until you know how the application will be affected.

1. Check the installed command

Start with the binary and its version. This is read-only and does not need sudo:

$ command -v rdma
/usr/bin/rdma
$ rdma -V
rdma utility, iproute2-6.1.0
$ rdma statistic help
Usage: rdma [ OPTIONS ] statistic { COMMAND | help }

The installed help is the contract for this host. In this iproute2 build it advertises qp as the supported object, type as the automatic-mode criterion, and cntn, lqpn and pid as filters. The installed manpage is broader and includes examples for MR counters, qp-type, and optional link counters. Do not assume those additional forms work just because they appear in that older documentation; check rdma statistic help on the target machine first.

2. Establish the device name and port

Use the RDMA device name, not an Ethernet interface name. List devices without changing anything:

$ rdma dev show
$ rdma link show

Output is host-specific. A link may be written as DEVICE/PORT, for example mlx5_2/1. Replace DEVICE/PORT in the examples with an exact value from your own output. If both commands are empty, stop here: there is no local port on which to inspect a counter. Installing a package or adding a guessed device name will not create an RDMA port.

Checkpoint: save the value you intend to use, but keep it as a shell variable so a later command is easy to review:

$ RDMA_PORT='DEVICE/PORT'
$ printf '%s\n' "$RDMA_PORT"
DEVICE/PORT

3. Read the default and QP counters

Query the default counters first. These commands are ordinary read-only queries:

$ rdma statistic show
$ rdma statistic show link "$RDMA_PORT"
$ rdma statistic qp show
$ rdma statistic qp show link "$RDMA_PORT"

The utility prints counter records when the kernel and driver expose them. On a machine with no RDMA device, the installed command on this test host returned no text and status 0, so an empty result is not proof that a counter exists. On a real host, compare the device and port in the output before acting on a record.

Use a filter only when you know the field's value. The installed help supports counter number cntn, local QP number lqpn, and process ID pid:

$ rdma statistic qp show link "$RDMA_PORT" pid 30489
$ rdma statistic qp show link "$RDMA_PORT" lqpn 178

These values are examples, not defaults. A filter that returns nothing means there is no matching record at that instant, or that the field is not available for the selected port. Do not turn a filter into a state change by adding set, bind or unbind until you have identified the target.

4. Check the current QP mode

Read the current automatic-binding mode before changing it:

$ rdma statistic qp mode
$ rdma statistic qp mode link "$RDMA_PORT"

The first command reports modes across devices; the second narrows the query to one link. Record the result before making a change. A mode query that prints nothing may simply mean that the driver has no report for the selected link, so check the device name and port again rather than assuming a default.

5. Enable automatic QP counters only for a planned workload

Automatic mode binds new user QPs into counter groups according to a criterion. In the installed utility the supported criterion is type. This is a state-changing operation and may allocate counters or alter how a busy workload is accounted for:

$ sudo rdma statistic qp set link "$RDMA_PORT" auto type on

For the example port, the intended result is one counter grouping per QP type as new QPs are created. The command does not retroactively make every existing object conform to a new reporting policy. Re-read the mode and then create or observe a test workload:

$ sudo rdma statistic qp mode link "$RDMA_PORT"
$ rdma statistic qp show link "$RDMA_PORT"

There is no persistent configuration file in this command. The setting belongs to the live kernel state and driver. To stop automatic mode for this scope, use the documented off form:

$ sudo rdma statistic qp set link "$RDMA_PORT" auto off
$ rdma statistic qp mode link "$RDMA_PORT"

That is the recovery action for the example. The counters that remain may still be manually accessible, so inspect the next listing rather than assuming that disabling automatic mode deletes every counter immediately.

6. Bind one QP manually when automatic mode is off

Manual binding is useful when you need a specific QP's statistics and do not want automatic grouping. First obtain the QP's local number from the workload or a filtered listing. Then let the kernel allocate a counter by omitting the counter number:

$ sudo rdma statistic qp bind link "$RDMA_PORT" lqpn 178
$ rdma statistic qp show link "$RDMA_PORT" lqpn 178

The manpage documents the omitted counter ID as a request for a new counter. If you already have a counter number, the installed help also shows the explicit form:

$ sudo rdma statistic qp bind link "$RDMA_PORT" lqpn 178 cntn 4

Use an explicit counter only after confirming that cntn 4 belongs to the same device and port. A successful command is not a substitute for checking the resulting listing.

7. Unbind and release a manual counter

Unbind the specific QP when its observation window is over. This changes state and can make its statistics unavailable:

$ sudo rdma statistic qp unbind link "$RDMA_PORT" cntn 4 lqpn 178
$ rdma statistic qp show link "$RDMA_PORT" lqpn 178

If you omit the QP identifier, the documented operation unbinds all QPs from that counter:

$ sudo rdma statistic qp unbind link "$RDMA_PORT" cntn 4

Do not use the second form as a cleanup shortcut on a shared port. The manpage says an empty counter is released automatically by the kernel, but an unbind can remove useful live measurements for every QP using it. If you unbound the wrong QP or counter, stop further changes and restore the binding with the same bind form, using the verified QP and counter IDs.

8. Diagnose an empty or rejected result

Keep three questions separate: does the device exist, does the driver expose the requested statistic, and is the requested object present now? Run the read-only checks again:

$ rdma dev show
$ rdma link show
$ rdma statistic help
$ rdma statistic qp show link "$RDMA_PORT"
$ printf 'status: %s\n' "$?"

An unknown object or filter usually means the command syntax is ahead of the installed utility. A valid syntax with no records can mean that no matching QP or counter is allocated. A permission error on a change command means you need the appropriate privilege, not that a guessed counter ID is safe. Do not repeatedly retry a state-changing command against a live workload while changing several arguments at once; return to the read-only listing and verify one value at a time.

If the older manpage examples for MR or optional counters matter to your deployment, treat them as a compatibility question. Check the target machine's iproute2 version and its own help, then consult the matching package documentation before using them. This guide does not claim those commands are available in iproute2 6.1.0.