Make a gprofng Experiment Portable with gp-archive

An experiment recorded on one machine is close to useless on another until gp-archive copies in the source files and objects it needs to travel. The workflow uses the installed GNU Binutils 2.42 tools, takes about fifteen minutes for an existing experiment, and does not need root privileges when you can already write the experiment directory.

gp-archive is also available under the target-specific names aarch64-linux-gnu-gp-archive and x86_64-linux-gnu-gp-archive. They are the same gprofng archive operation for their respective toolchain packages. The examples use the unprefixed command.

1. Check the installed command and experiment

Start with read-only checks. Replace /path/to/experiment.er with the directory produced by gprofng collect app:

$ command -v gp-archive
/usr/bin/gp-archive
$ gp-archive --version
GNU x86_64-linux-gnu-gp-archive binutils version 2.42
$ test -d /path/to/experiment.er && echo 'experiment directory found'
experiment directory found

The archive command must run on the same system that recorded the experiment, because it resolves the executable, library, object and source paths recorded there. An experiment from an older gprofng release may produce a warning or fail to archive correctly. Match the archive tool to the release that recorded the data where possible.

Checkpoint: stop here if the path is wrong or the command is from an unexpected release. Do not try to repair an invalid experiment by pointing gp-archive at an arbitrary directory.

2. Understand what is already present

Normal collection archives application binaries as part of creating the experiment, but source files are not included by default. Automatic binary archiving can also be absent if the profiled program ended prematurely, was still running, or collection explicitly disabled archiving with -a off.

Inspect the experiment before changing it:

$ find /path/to/experiment.er -maxdepth 2 -type f -printf '%P\n' | sort
archives/...
overview
profile
...

The exact file list varies. An archives directory containing hashed copies of libraries and executables is normal. Its presence does not prove that source files are present, so check for the particular source or object you need rather than trusting the directory alone.

3. Add source files with the installed option syntax

On this installed 2.42 command, -a src archives source files and related objects that can be found. It also keeps the normal binary archiving behaviour:

$ gp-archive -a src /path/to/experiment.er
Copying `/build/project/main.c' to `/path/to/experiment.er/archives/main.c_...'

The suffixes and paths are host-specific. A successful run can print one line for each copied file, or stay quiet if there was nothing new to add. Verify the result by listing the archive:

$ find /path/to/experiment.er/archives -maxdepth 1 -type f -printf '%f\n' | sort
main.c_...
libc.so.6_...
project-binary_...

The manual page also describes a -s all form for source archiving, but the executable installed on this machine rejects -s as an unrecognised option. Prefer the syntax accepted by your command's own --help output. For this 2.42 installation, use -a src or the narrower -a usedsrc selection when you only want sources associated with recorded program-counter data.

4. Limit the files when the experiment is large

Archiving every source, object and debug-information file can consume substantial disk space. Use -m with a POSIX regular expression matching the full path when you need a narrower archive:

$ gp-archive -a src -m '/srv/project/.*' /path/to/experiment.er
Copying `/srv/project/main.c' to `/path/to/experiment.er/archives/main.c_...'

Quote the expression so the shell does not reinterpret its characters. Check the copied names afterwards. A regular expression that matches nothing is not a useful portability result, even if the command exits successfully.

To process only the named experiment and not descendants, add -n:

$ gp-archive -n -a src /path/to/experiment.er

This does not remove descendants. It only limits the operation to the experiment named on the command line.

5. Choose a common archive deliberately

Repeated experiments can share archived files. -d takes an absolute path to a common archive directory; -r takes a relative path. The tool creates a missing directory and places a symbolic link in the experiment archive. These options change state outside the experiment, so check the destination first and make sure its ownership and backup policy are suitable:

$ test -d /srv/gprofng-common || echo 'destination does not exist yet'
$ gp-archive -a src -d /srv/gprofng-common /path/to/experiment.er
$ ls -ld /srv/gprofng-common /path/to/experiment.er/archives

Warning: -d and -r are mutually exclusive. Do not use a shared directory merely to save a few megabytes: it becomes part of the experiment's portability boundary and must travel with it. If you need to undo this choice, stop using the experiment, preserve any required files, then remove the common archive and its links only after checking every experiment that refers to it. That removal is destructive and is not included in the example.

6. Handle missing files and warnings

If a source or executable cannot be found, first identify where the original build stored it. The gprofng configuration file .er.rc can contain addpath and pathmap commands to tell the tools where missing files now live. Add only paths you trust, then rerun the archive operation. For Java applications, shared objects inside a JAR may also need an addpath entry naming the JAR itself.

Do not hide warnings on the first run. The -q option suppresses warnings on standard error and records them in the experiment's archive information for later display with gprofng display text. Use it only after you have a reason not to show warnings to the calling process.

Warning: if you need to replace an existing archive, treat -F as destructive: it removes and recreates archived files, except when -n or -m is used and for subexperiments. Take a copy of the experiment or ensure the original profiling data can be regenerated before using it. There is no general undo operation for files removed by a forced rewrite.

7. Verify before moving the experiment

Use the display command to expose warnings recorded during archiving, then check the archive directory and its links:

$ gprofng display text /path/to/experiment.er | sed -n '1,120p'
$ find /path/to/experiment.er/archives -maxdepth 1 -type f -printf '%f\n' | sort
$ find /path/to/experiment.er -type l -ls

There is no single success message that proves every source was found. A portable result is one where the required executable, libraries, source files and debug information are present, warnings have been reviewed, and any common archive directory is included in the transfer plan. Keep the original experiment until the copy has been opened and inspected on the destination.

Done means