Set RDMA Network Namespace Mode Safely with rdma system

You will finish with a checked RDMA network namespace mode and a clear choice between sharing RDMA devices and isolating them per namespace. The examples use the installed rdma utility from iproute2 6.1.0-1ubuntu6.4, where the current mode is reported as netns shared copy-on-fork on.

Allow about fifteen minutes for an inspection or a planned change. You need an RDMA-capable Linux host, the rdma command from iproute2, and enough privilege to change kernel RDMA state. Reading the current mode is normally unprivileged. Changing it returned Operation not permitted without elevated privilege on the test host.

1. Confirm the installed command

Start with read-only checks. This avoids assuming that a different distribution's iproute2 release has the same interface:

$ command -v rdma
/usr/bin/rdma
$ dpkg-query -W -f='${Package} ${Version}\n' iproute2
iproute2 6.1.0-1ubuntu6.4
$ rdma -V
rdma utility, iproute2-6.1.0
$ rdma system help
Usage: rdma system show [ netns ]
       rdma system set netns { shared | exclusive }

The command uses rdma system. The shorter rdma sys spelling is also accepted by this installed utility, but using the full form in scripts makes the operation easier to recognise.

2. Check the current mode

Ask for the current setting before changing anything:

$ rdma system show
netns shared copy-on-fork on

The useful part for this guide is shared. In shared mode, an RDMA device is accessible in all network namespaces. The extra copy-on-fork on text is status reported by this version of the utility; it is not a mode value that you pass to rdma system set. The set command accepts only shared or exclusive.

Checkpoint: record the word after netns. If it already matches the isolation model you need, stop here and use the verification in step 5. A successful read does not mean that a later change will be permitted.

3. Choose shared or exclusive deliberately

Use shared when RDMA devices should remain visible across network namespaces and isolation between those namespaces is not required. This is the mode shown by the tested host.

Use exclusive when a dedicated RDMA device must be assigned to one particular network namespace. Set that mode before creating any network namespace. The manpage warns that changing from shared to exclusive fails with EBUSY when active network namespaces and one or more RDMA devices exist.

This is a system-wide choice, not a per-device switch. Do not change it while RDMA traffic is active unless you have deliberately accepted the operational risk. A mode change can affect how applications find devices, so schedule it with the same care as other networking changes.

4. Apply the selected mode

Changing the setting is state-changing and normally needs elevated privilege. Stop or drain dependent RDMA workloads first, then run exactly one of these commands:

# Keep RDMA devices visible in every network namespace
$ sudo rdma system set netns shared

# Or isolate an RDMA device to one network namespace
$ sudo rdma system set netns exclusive

There is no need to add copy-on-fork, a device name or a namespace name. Those are not arguments accepted by this command. Keep shared or exclusive as a literal value rather than interpolating an unchecked shell variable.

If you run the command without sufficient privilege, the tested utility reports:

$ rdma system set netns shared
error: Operation not permitted

That error means the attempted change was not applied. Re-run the same operation with the privilege required by your host's policy, then verify it in the next step. Do not treat sudo as a repair for an EBUSY result: an active namespace or RDMA device still has to be dealt with safely.

5. Verify the result

Always read the mode back after a successful set:

$ rdma system show
netns exclusive copy-on-fork on

For shared mode, expect netns shared; for exclusive mode, expect netns exclusive. The rest of the line can contain version-specific status text, so verify the selected mode rather than comparing an entire line byte for byte.

A small shell check is useful in a deployment or maintenance script:

mode=$(rdma system show | awk '$1 == "netns" { print $2; exit }')
case "$mode" in
    shared|exclusive) printf 'RDMA netns mode: %s\n' "$mode" ;;
    *) printf 'Unexpected rdma system output\n' >&2; exit 1 ;;
esac

This check validates the mode field, not whether an application has opened an RDMA device successfully. Test the actual workload separately, inside the namespace where it is meant to run.

6. Recover from a wrong choice

If you selected the wrong mode, return to the intended value during a maintenance window:

$ sudo rdma system set netns shared
$ rdma system show
netns shared copy-on-fork on

Replace shared with exclusive only when exclusive isolation is the deliberate target and its prerequisites are satisfied. The undo operation is not a substitute for stopping traffic. If the mode change fails, keep the last verified mode, inspect active network namespaces and RDMA users, and plan the retry rather than repeatedly issuing the command.

For a failed exclusive-mode attempt, the important boundary is the documented EBUSY case: shared mode remains the effective mode when the change is rejected. Confirm that with rdma system show before changing anything else.