Find Perl Hash-Order Bugs with Repeatable Runs

You will test Perl code for accidental dependence on hash key order, capture a repeatable failing case, and return to Perl's normal randomised behaviour afterwards. This is useful when a test, report, serialiser or command-line tool sometimes changes output without any input change.

Allow about fifteen minutes for a small program, or longer if you need to isolate a failing test. You need a shell and Perl. The examples use the installed Perl v5.38.2 and the local perl5180delta documentation, which describes the hash changes introduced in Perl v5.18.0. The controls are process-local environment variables; they do not require sudo and do not change system configuration.

1. Confirm the interpreter and make a small order check

Start by recording which Perl will run the test:

$ command -v perl
/usr/bin/perl
$ perl -e 'printf "%vd\n", $^V'
v5.38.2

Now print the keys of a hash twice. The order is deliberately not part of the data model:

$ perl -e '%h = (alpha => 1, beta => 2, gamma => 3, delta => 4); print join(" ", keys %h), "\n"'
alpha gamma beta delta
$ perl -e '%h = (alpha => 1, beta => 2, gamma => 3, delta => 4); print join(" ", keys %h), "\n"'
delta alpha gamma beta

Your two lines will probably differ from these examples. They can also happen to match; a match is not evidence that an order is stable. Perl v5.18 changed the hash implementation so that the seed is random by default and functions such as keys, values and each can expose different orders between runs.

2. Sort data when order is not part of the result

The normal fix is in the program, not in the environment. If a report is meant to be alphabetical, say so explicitly:

my %totals = (
    alpha => 12,
    beta  => 7,
    gamma => 19,
);

for my $name (sort keys %totals) {
    printf "%s %d\n", $name, $totals{$name};
}

If the order is defined by another rule, use that rule instead: for example, sort by a timestamp or use an array for an inherently ordered list. Do not make tests pass by hard-coding the order returned by keys %hash. A hash remains an unordered collection even when one process happens to print a pleasing sequence.

Checkpoint: run the real test or report with its ordinary environment after making the smallest code change. It should pass without setting PERL_HASH_SEED or PERL_PERTURB_KEYS.

3. Repeat a test with the secure default

Perl v5.18 defines PERL_PERTURB_KEYS. Its default value, 1, applies non-repeatable key randomisation and is the most secure mode described by the manual. Run a test several times in separate interpreter processes:

$ for run in 1 2 3 4 5; do
>     PERL_PERTURB_KEYS=1 perl /path/to/test.pl || break
> done

Replace /path/to/test.pl with a real test file. The loop stops at the first non-zero status, which keeps the failure visible. If the program emits output that is supposed to be order-independent, compare a normalised form or inspect the code that consumes the hash. If the output is a protocol, signed document or public API, treating an accidental key order as harmless can become a compatibility or security problem.

Do not use PERL_PERTURB_KEYS=0 as the fix. It disables key randomisation for that process and can hide the bug. It is a diagnostic comparison only.

4. Freeze one failure for debugging

Once you have a failing case, choose a non-zero hexadecimal hash seed. A non-zero PERL_HASH_SEED makes key randomisation repeatable, and the v5.18 interface expects a hexadecimal value rather than an old-style integer:

$ PERL_HASH_SEED=0x1234 PERL_PERTURB_KEYS=2 perl /path/to/test.pl

The explicit perturbation setting makes the intention clear. With PERL_PERTURB_KEYS=2, repeated runs of the same program should use the same repeatable key randomisation:

$ for run in 1 2 3; do
>     PERL_HASH_SEED=0x1234 PERL_PERTURB_KEYS=2 perl /path/to/test.pl
> done

Use this only to reproduce and diagnose the failure. It is not a production setting and it does not make hash order a supported interface. Do not put a seed into a shared log or bug report if it could reveal information about a process whose environment is sensitive.

To inspect the seed and compiled hash function for one process, use the debug variable:

$ PERL_HASH_SEED_DEBUG=1 perl -e '1'
HASH_FUNCTION = SBOX32_WITH_SIPHASH_1_3 HASH_SEED = 0x... PERTURB_KEYS = 1 (RANDOM)

The exact function name and seed vary by build and process. Treat the seed as diagnostic data, not as stable output for a parser. The manual specifically says that PERL_HASH_SEED_DEBUG changes the output format to include both the function and the hexadecimal seed.

5. Inspect the seed without assuming it is an integer

Perl v5.18 changed Hash::Util::hash_seed() to return a string, because seeds can be wider than an integer:

$ perl -MHash::Util=hash_seed -e 'my $seed = hash_seed(); print ref($seed) || "scalar", " ", length($seed), " bytes\n"'
scalar 32 bytes

The length depends on the hash function and build. The useful point is that this is a string of seed bytes, not a portable numeric value. Do not convert it to a signed integer or use its numeric value as an identifier.

6. Restore the normal environment

Variables written before a command affect that command only. Variables exported in your shell last longer, so check and remove them before running production code:

$ env | grep '^PERL_\(HASH_SEED\|PERTURB_KEYS\|HASH_SEED_DEBUG\)=' || true
$ unset PERL_HASH_SEED PERL_PERTURB_KEYS PERL_HASH_SEED_DEBUG
$ perl -e 'print "normal environment\n"'
normal environment

If you changed a shell profile, CI job or service environment, remove the diagnostic assignment there as well and restart only the affected process. No Perl data is modified by these commands. If you used PERL_PERTURB_KEYS=0 for comparison, unset it rather than carrying the workaround into later tests.

7. Keep the security boundary in view

The hash overhaul was partly intended to make Perl more resistant to algorithmic-complexity attacks and to expose code that relies on iteration order. Keep randomisation enabled for ordinary execution, and never expose hash order as an unexamined public format.

The same local documentation warns that deserialising untrusted Storable data can load modules and execute code. A fixed hash seed does not make Storable safe. Do not accept a seed-setting workaround as permission to deserialize data from an untrusted source.