Load Text and Berkeley DB Dumps Safely with db5.3_load

Load a text file or a Berkeley DB dump into a fresh database with db5.3_load, then prove the result is structurally sound. It takes about 10 minutes, verification pass included, and you will build a small hash database from paired text along the way.

The route

Jump straight to the step you need, or tick off Done means at the end.

Before you start
1. Choose an empty destination
2. Load paired text
3. Verify the database
4. Reload a Berkeley DB dump
5. Respect the environment

Before you start

Which tools. This targets the Debian or Ubuntu Berkeley DB 5.3 utilities. On this machine db5.3_load -V reports Berkeley DB 5.3.28, with db5.3-util version 5.3.28+dfsg2-7 and db-util version 1:5.3.21ubuntu2 installed.
Alias. The db_load alias uses the same local 5.3 utility behaviour.
Access. You need a writable destination directory, plus a backup if the destination database already holds anything useful. Normal loading needs no elevated privileges.
sudo. Use sudo only when the database directory is intentionally owned by a service account. Do not make the database world-writable to dodge that decision.

1. Choose an empty destination

The final argument is the database file, and it is created if it does not exist. Check the path before you run anything that changes state.

db_file=/path/to/data/users.db
test ! -e "$db_file" && echo "destination is unused" || echo "STOP: destination already exists"

If the file already exists, make a recoverable copy before loading. The utility can update an existing database, and by default loaded keys replace existing keys. To keep existing keys, use -n: duplicate key/data pairs are skipped and produce warnings on standard error.

cp -- "$db_file" "$db_file.backup"
printf '%s\n' 'alice' 'uid=1001' 'bob' 'uid=1002' | \
  db5.3_load -n -T -t hash "$db_file"
printf 'exit status: %s\n' "$?"

Warning

Do not use the example destination literally. Swap in a path suitable for your application. The backup is your undo: stop the application that uses the database, then restore the copy only after confirming that replacing the current file is safe.

2. Load paired text

With -T, a Btree or Hash database expects alternating key and data lines. The -t hash option supplies the access method, because text input carries no such metadata. This creates two records:

mkdir -p /tmp/db5-load-example
printf '%s\n' 'alice' 'uid=1001' 'bob' 'uid=1002' | \
  db5.3_load -T -t hash /tmp/db5-load-example/users.db
echo "load exit status: $?"

Checkpoint

You should see load exit status: 0. Status 0 means the load completed. Status 1 means one or more key/data pairs were not loaded because the key already existed. Anything above 1 is an error, so do not treat every non-zero status as harmless.

Backslashes are special in -T mode, so handle these with care:

Doubled backslash. A pair of backslashes is one literal backslash.
Backslash plus two characters. That is read as a hexadecimal byte.
Natural backslashes. Escape them before loading. For example, this supplies a literal backslash in the data value:

printf '%s\n' 'alice' 'uid=1001\\home' | \
  db5.3_load -T -t hash /tmp/db5-load-example/paths.db

Warning

A newline separates records, so a value containing a real newline needs a deliberate encoding scheme before it reaches this interface. Do not pipe arbitrary multiline fields into paired text mode and assume each stays one value.

3. Verify the database

Run the matching verification utility against the file you just made.

db5.3_verify /tmp/db5-load-example/users.db

Expected output includes:

BDB5105 Verification of /tmp/db5-load-example/users.db succeeded.

This checks the Berkeley DB structure, not whether your application reads every value as intended.

Status 1. Inspect the warnings and confirm the skipped keys were expected.
Status above 1. Keep the source input and the destination backup, fix the cause, and retry into a new file.

4. Reload a Berkeley DB dump

For a dump made by db5.3_dump, leave out -T and the access method. The dump carries the database type and other metadata the loader can use.

db5.3_dump /path/to/source.db > /tmp/users.dump
db5.3_load -f /tmp/users.dump /path/to/restored.db
db5.3_verify /path/to/restored.db

The -f option reads the named input file instead of standard input. If you use a pipeline instead, keep the destination explicit and check the loader's exit status:

db5.3_dump /path/to/source.db | \
  db5.3_load /path/to/restored.db
echo "load exit status: $?"

Warning

A dump does not necessarily preserve application-specific hash, prefix or comparison functions. The Berkeley DB documentation warns that a reload can fall back to defaults, which may break an application that depends on custom functions. Test the restored file with the real application before you switch over.

5. Respect the environment

Use -h /path/to/environment when the database belongs to a Berkeley DB environment. Without -h, DB_HOME may supply the environment home.

The local manual says the utility tries to open the environment with the locking, logging, memory-pool and transaction subsystems. If that open fails, the database may still be updated without environment protection.

Warning

Treat that fallback as a failure for a live service. Fix the environment path or arrange downtime instead of pressing on.

When an environment is in use, give the utility time to detach cleanly. The local manual specifically advises sending SIGINT so it can release environment resources.

Warning

Do not use -r lsn or -r fileid as a routine repair command. Each resets metadata in place, so the physical database file is modified. Take a tested backup and follow your environment's recovery procedure before considering either reset.

Done means

Destination checked. You looked at the path first, and a backup exists if data was already there.
Paired input. Text input used alternating key/data lines with the right -t access method.
Exit status understood. The loader returned 0, or a status 1 was investigated and accepted deliberately.
Verified. db5.3_verify reported that verification succeeded.
Environment handled. A live environment was loaded with the correct home and coordination, not an accidental unlocked fallback.