Inspect, dump and safely pin eBPF programs with bpftool

You will use bpftool prog to list programs already loaded into the kernel, inspect their identifiers and instructions, then load and pin one program for later use. Allow 15 minutes for inspection, or longer if you are loading an object supplied by another project. The examples are deliberately split between read-only checks and commands that change kernel state.

This guide follows the bpftool-prog(8) page installed with Ubuntu's linux-tools-common package, version 6.8.0-139.139. The executable is normally supplied by the matching kernel tools package as well, so check both the package and the command before relying on a feature.

1. Check the tool and your privileges

Start with read-only checks. The command may be under a kernel-specific directory, depending on the distribution:

$ command -v bpftool
$ bpftool version
$ dpkg-query -W -f='${Package} ${Version}\n' linux-tools-common

Expect bpftool version to print a bpftool version, the libbpf version and compiled-in features. If the command is missing, install the tools package matching the running kernel through your normal package-management process. Do not treat the manpage package alone as proof that the executable is available.

Listing may work as an ordinary user on some systems, but access to BPF objects and bpffs varies with kernel policy and permissions. Use sudo only when the command reports a permission failure, and keep the first investigation read-only.

2. List loaded programs and select one precisely

List every program visible to the command:

$ sudo bpftool prog show
10: xdp  name some_prog  tag 005a3d2123620c8b  gpl
        loaded_at 2017-09-29T20:11:00+0000  uid 0
        xlated 528B  jited 370B  memlock 4096B  map_ids 10

The numbers are host-specific. The leading number is the program ID. The type, name and tag help you identify the right object; do not copy an ID from this example. A program selector can be id, pinned, tag or name. A name or tag can match more than one program, so use an ID or a pinned path when a later command requires exactly one.

For scripts, request JSON rather than scraping aligned text:

$ sudo bpftool --json --pretty prog show
[{"id":10,"type":"xdp","tag":"005a3d2123620c8b","jited":true}]
$ sudo bpftool -j prog show > /tmp/bpftool-progs.json

The fields shown depend on the kernel and the program. Since Linux 5.1, run-time statistics can be reported when the kernel statistics switch is enabled, but collection is not enabled by default because it has a performance cost. Since Linux 5.8, the listing can also show processes holding BPF program file descriptors.

3. Dump the verified program before changing anything

Replace PROGRAM_ID with an ID you just checked. The translated eBPF instructions are the useful first view:

$ sudo bpftool prog dump xlated id PROGRAM_ID linum
0: (b7) r0 = 0
1: (95) exit

Exact instructions and source locations vary. Add opcodes for raw opcode bytes, or visual for a control-flow graph in DOT format. The latter needs a single match, so do not combine it with a broad name or tag that selects several programs.

JIT output is host machine code, not portable eBPF:

$ sudo bpftool prog dump jited id PROGRAM_ID opcodes
0:   push   %rbp
     55

Save a binary dump when you need to compare it later. The destination is overwritten by ordinary shell redirection, so use a new temporary name if the old evidence matters:

$ sudo bpftool prog dump xlated id PROGRAM_ID file /tmp/program.xlated
$ test -s /tmp/program.xlated && echo "translated dump written"
translated dump written

4. Check bpffs before pinning

Pinning gives a BPF program a filesystem reference that can be reopened after the creating process exits. The destination must be inside a mounted BPF filesystem, usually /sys/fs/bpf, and its final name must not contain a dot:

$ findmnt /sys/fs/bpf
TARGET    SOURCE FSTYPE OPTIONS
/sys/fs/bpf bpf   bpf    rw,nosuid,nodev,noexec,relatime
$ sudo bpftool prog show id PROGRAM_ID

If the mount is absent, ask the system administrator to provide the expected bpffs mount rather than mounting a new filesystem in a production namespace without checking its ownership and boot configuration. Some bpftool commands try to mount virtual filesystems automatically; use --nomount when you want a command to fail instead of attempting that convenience.

5. Pin one program, then verify the reference

This is the first state-changing command. Confirm the ID and destination twice before running it:

$ sudo bpftool prog pin id PROGRAM_ID /sys/fs/bpf/example_prog
$ sudo bpftool -f prog show id PROGRAM_ID
10: xdp  name some_prog  ...
        pinned /sys/fs/bpf/example_prog

The -f or --bpffs option asks the listing to show pinned filenames. Verify the path is present with stat. To undo this particular pin, remove the bpffs entry after confirming that no service still needs it:

$ sudo stat /sys/fs/bpf/example_prog
$ sudo rm -- /sys/fs/bpf/example_prog

Removing the pin removes that filesystem reference; it does not necessarily unload a program still referenced by a process, link or another pin. The rm command is irreversible for that path, so do not use it as a cleanup step until you have checked consumers.

6. Load an object only after reviewing its scope

Loading an ELF object can create maps, consume kernel resources and, with autoattach, attach a program before pinning it. Treat an object from outside your build as executable kernel-facing input. Review its provenance, program sections, map definitions and intended hook before using elevated privileges.

load pins only the first program. loadall pins every program under a directory. The type can be inferred from section names, but specifying it makes the operation easier to review:

$ sudo bpftool prog load ./PROGRAM.o /sys/fs/bpf/example_prog type xdp
$ sudo bpftool -f prog show pinned /sys/fs/bpf/example_prog

By default, maps declared in the object are created afresh. The map name MAP_NAME id MAP_ID or map name MAP_NAME pinned FILE form lets you reuse an existing map; check the map's definition and owner before doing so. pinmaps MAP_DIR pins maps under a directory. If you use autoattach, verify the corresponding link with bpftool link show -f, because the pinned path represents the link rather than the program itself.

7. Diagnose without widening the change

Use -d or --debug when a load fails. It includes libbpf and verifier messages, which often identify the instruction, map or permission that needs attention:

$ sudo bpftool -d prog load ./PROGRAM.o /sys/fs/bpf/example_prog type xdp
libbpf: loading object './PROGRAM.o'
libbpf: prog 'example' rejected: Permission denied

The wording and verifier details are kernel-specific. A non-zero exit status is the useful signal; do not retry a failing load repeatedly against a live hook. For loader-program diagnostics, -L attempts the load without pinning, and bpftool prog tracelog prints the kernel trace pipe until you press Ctrl+C. Tracelog is for debugging only and can expose data written by other BPF programs.