Encode and Decode Files Reversibly with basenc

When binary bytes have to survive a text-only pipe, JSON field or shell variable, basenc converts them into a safe alphabet and back again. This guide encodes or decodes a small text or binary file with GNU basenc, picks an alphabet that fits the destination, and verifies the result without overwriting useful data. Allow about ten minutes for a first run. The examples use ordinary user permissions and do not need sudo.

1. Check the installed command

This guide describes GNU coreutils 9.4, the version installed on this machine. The exact version matters because command-line tools can gain formats or change diagnostics between releases. Check the executable and package before putting an example into a script:

$ command -v basenc
/usr/bin/basenc
$ basenc --version | sed -n '1p'
basenc (GNU coreutils) 9.4
$ dpkg-query -W -f='${Package} ${Version}\n' coreutils
coreutils 9.4-3ubuntu6.3

basenc reads a file named on the command line, or standard input when there is no file or the file is -, and writes the converted bytes to standard output. That default is handy in pipelines, but it also means a misplaced redirection can create or truncate a file.

Checkpoint: confirm that command -v points to the executable you actually intend to run, especially on a system with more than one coreutils installation.

2. Encode text as ordinary Base64

Base64 is the usual choice when arbitrary bytes must travel through text-oriented systems. It uses the RFC 4648 section 4 alphabet and, by default, wraps encoded output after 76 characters. For a short value, feed exact bytes with printf rather than echo, whose newline behaviour varies:

$ printf %s 'hello' | basenc --base64
aGVsbG8=
$ printf %s 'hello' | basenc --base64 | basenc --decode --base64
hello

The second command proves the round trip in memory; the newline your terminal prints is not part of the decoded value. To encode a file, name it after the option:

$ basenc --base64 /path/to/input.bin > /path/to/output.b64
$ basenc --decode --base64 /path/to/output.b64 > /path/to/round-trip.bin
$ cmp -- /path/to/input.bin /path/to/round-trip.bin && echo 'round trip verified'
round trip verified

Warning: do not treat Base64 as encryption. Anyone who receives the encoded text can decode it. Keep passwords and tokens out of shell history, and never paste confidential data into an online decoder.

3. Choose a format for the destination

The option selects the alphabet; it does not identify the input automatically. Use the format the other program or protocol expects:

Base64url is a separate format, not a spelling variant you can always swap in:

$ printf '\376\117\202' | basenc --base64
/k+C
$ printf '\376\117\202' | basenc --base64url
_k-C
$ printf '\376\117\202' | basenc --base16
FE4F82

Check the receiving specification before picking an alphabet. A decoder expecting ordinary Base64 may reject Base64url punctuation, and one expecting lower-case hex may reject output that represents the same bytes.

4. Control wrapping when producing text

Wrapping affects presentation, not the bytes represented. The default is 76 columns. Use --wrap=0 when a single line is required, such as a value going into a shell variable or a line-oriented configuration field:

$ printf %s 'hello' | basenc --base64 --wrap=0
aGVsbG8=
$ basenc --base64 --wrap=0 /path/to/input.bin > /path/to/value.txt
$ awk 'NR > 1 { exit 1 }' /path/to/value.txt && echo 'one encoded line'
one encoded line

Do not assume arbitrary text is Base64 just because you stripped line breaks. Decoding accepts newlines as part of the stream, but other non-alphabet bytes are rejected unless you turn on the recovery behaviour in step 5.

5. Decode defensively and handle garbage

Normal decoding rejects bytes outside the selected alphabet, which is useful when corruption should stop a pipeline:

$ printf %s 'aGVs!bG8=' | basenc --decode --base64 > /tmp/basenc-result
basenc: invalid input
$ printf 'exit status: %s\n' "$?"
exit status: 1

The output file may already hold partial output when a decoder errors out. Treat it as untrusted and remove it only once you have confirmed it is the temporary path you created:

$ rm -- /tmp/basenc-result

--ignore-garbage tells the decoder to skip non-alphabet characters. Use it only when the extra bytes are known transport noise, such as labels or stray whitespace, and check the decoded result afterwards:

$ printf %s 'aGVs!bG8=' | basenc --decode --ignore-garbage --base64
hello

Warning: this option can hide damaged or injected input. It is not a repair algorithm, and it does not make an untrusted encoded document safe to execute.

6. Write output without losing the original

Shell redirection with > truncates an existing destination before basenc even starts. For a file you care about, write to a new name, validate it, then replace the old file deliberately:

$ basenc --decode --base64 /path/to/value.b64 > /path/to/value.bin.new
$ cmp -- /path/to/value.bin /path/to/value.bin.new
$ mv -- /path/to/value.bin.new /path/to/value.bin

Recovery: if decoding fails, leave the original untouched and inspect the diagnostic. If a temporary file exists, remove that exact path once you have checked it is yours. The mv is the state-changing step here; it needs write permission on the destination directory but normally no elevated privilege. There is no undo for an overwrite, so keep a backup when the original cannot be recreated.

7. Diagnose format and length errors

A successful command only proves the installed program accepted the input. It does not prove the other system chose the same alphabet, or that the decoded bytes mean what you think. Compare bytes, not a visual text rendering:

$ printf %s 'hello' | basenc --base16
68656C6C6F
$ printf %s '68656C6C6F' | basenc --decode --base16
hello

For Z85, check the length before encoding or decoding. Three input bytes are invalid for encoding because the length is not a multiple of four:

$ printf %s 'abc' | basenc --z85 > /tmp/basenc-z85
basenc: invalid input (length must be multiple of 4 characters)
$ printf %s 'test' | basenc --z85
By/Jn

Keep error output separate from converted data when scripting. A pipeline can otherwise look successful while a later command consumes incomplete output. Use the exit status and a byte comparison such as cmp as your checkpoint.

Done means