Turn MariaDB Numeric Stack Addresses into Function Names

You will turn a numeric stack trace from mysqld into lines containing function names and offsets. The key requirement is a symbols file made from the matching server binary with nm --numeric-sort. Allow 10 to 15 minutes if both files are ready, or longer if you need to locate the exact MariaDB build.

1. Check the installed tool

This guide follows the MariaDB 10.11 command installed with the Ubuntu package on this machine. The local executable reports version 10.11.14, while the installed package is mariadb-client 1:10.11.14-0ubuntu0.24.04.1. Your package version may differ, so record it before comparing a later investigation with this one.

$ command -v resolve_stack_dump
/usr/bin/resolve_stack_dump
$ resolve_stack_dump --version
resolve_stack_dump  Ver 1.4 Distrib 10.11.14-MariaDB, for debian-linux-gnu (x86_64)
$ dpkg-query -W -f='${Package} ${Version}\n' mariadb-client
mariadb-client 1:10.11.14-0ubuntu0.24.04.1

There is no need for sudo here. Reading the command, symbol file and dump normally requires only your account's permissions.

2. Produce the symbols file from the matching server

The symbols file is the output of nm --numeric-sort mysqld. Run it against the exact server executable that produced the trace, not an arbitrary mysqld found on the current PATH. A different package revision, build or binary can put functions at different addresses and produce plausible-looking but wrong names.

$ command -v mysqld
/usr/sbin/mysqld
$ nm --numeric-sort /path/to/matching/mysqld > /path/to/mysqld.symbols

The command writes the symbol listing to standard output, so the redirection creates or truncates the destination. Choose a new destination or make a backup before running it if that file already contains useful evidence. Keep the original server binary and symbols file together while you investigate.

Some distribution binaries are stripped and produce no usable symbols. That is a property of the binary, not a reason to substitute a different release. Obtain the unstripped or debug build that corresponds to the server which emitted the trace, then generate the listing from that binary.

$ test -s /path/to/mysqld.symbols && echo 'symbols file is non-empty'
symbols file is non-empty
$ head -n 3 /path/to/mysqld.symbols
0000000000401000 T first_symbol
0000000000401050 T another_symbol
0000000000401100 T later_symbol

The names and addresses in that sample are illustrative. Your file should contain the real output from nm, not hand-written replacements.

3. Resolve a saved numeric dump

Pass the symbols file first and the numeric dump second. The dump should be the numeric stack trace captured from mysqld. The installed utility accepts addresses such as 0x401050 and emits the original address, the nearest symbol and an offset.

$ resolve_stack_dump /path/to/mysqld.symbols /path/to/numeric-dump.txt
0x401050 another_symbol + 0
0x4010f0 another_symbol + 160
0x401100 later_symbol + 0

The displayed names and offsets will depend on your files. An address exactly at a symbol starts with offset zero. An address between two symbols can be reported as an offset from the preceding symbol. That offset is useful for finding the call site, but it is not a source line number.

Do not treat a successful exit status as proof that the result is meaningful. Compare the output with the dump, check that the addresses have the expected format, and confirm that the symbols came from the matching binary.

4. Use standard input when the dump is a pipe

If you omit the numeric dump filename, resolve_stack_dump reads the trace from standard input. This is useful when another diagnostic command or a stored file is already being streamed, but the symbols file remains the required positional argument.

$ cat /path/to/numeric-dump.txt | resolve_stack_dump /path/to/mysqld.symbols
0x401050 another_symbol + 0
0x401100 later_symbol + 0

For scripts, the long options make the two inputs harder to swap accidentally:

$ resolve_stack_dump \
    --symbols-file=/path/to/mysqld.symbols \
    --numeric-dump-file=/path/to/numeric-dump.txt

The short forms are -s and -n. They select the same files. The utility does not offer an option to rewrite the trace, change symbol names or produce source file and line information.

5. Diagnose an unhelpful result

If addresses pass through unchanged, first inspect the input. The installed tool resolved a test address written as 0x401050, whereas the same address without the 0x prefix was left unchanged. Preserve the format produced by the MariaDB diagnostic that created the trace rather than normalising it by guesswork.

Next, verify the symbol file and binary relationship. A non-empty file is not necessarily a matching file. Check the executable path, package version and file metadata, then regenerate the symbols listing from the exact executable if you can access it. If the original server was upgraded or replaced, the current mysqld is not a safe substitute.

$ file /path/to/matching/mysqld /path/to/mysqld.symbols /path/to/numeric-dump.txt
$ wc -l /path/to/mysqld.symbols /path/to/numeric-dump.txt
$ resolve_stack_dump --help

If the command cannot open a file, fix the path or read permission. Do not make a database service world-readable merely to run this analysis. Copy the evidence to a controlled directory with suitable ownership and permissions instead. The resolver does not alter MariaDB data, service configuration or the input files, so there is no service rollback step.