Build a Small ICU Common Data File with gencmn

You will package a set of local files into an ICU common data file, such as demo.dat, and confirm which inputs were included. This is useful when an ICU-based program expects related data in one memory-mappable package rather than as separate files.

Allow about 10 minutes for a first build. You need the gencmn executable from icu-devtools, a writable working directory, and input files that belong in the package. The examples use ordinary user permissions. You only need elevated privileges if the chosen destination is protected; a private build directory is easier to inspect and recover.

Checkpoint: confirm the installed tool

This guide was checked against ICU 74.2 from Ubuntu package icu-devtools 74.2-1ubuntu3.1. Check your own installation before relying on the default data name, because that name contains the ICU release number and machine endianness.

command -v gencmn
gencmn --help
gencmn --version

The local 74.2 build rejects --version and then prints its usage text; it has no separate version switch. The installed manpage identifies the tool as version 74.2 and shows a default name of icudt74l on this little-endian machine. The help output is the authoritative check when another ICU release is installed.

1. Create a dedicated build directory

Keep the list, source files and generated output together while testing. This avoids accidentally packaging an absolute path or overwriting a system ICU data file.

mkdir -p "$HOME/icu-package-demo/input" "$HOME/icu-package-demo/out"
cd "$HOME/icu-package-demo"
printf 'first payload for ICU\n' > input/alpha.bin
printf 'second payload for ICU\n' > input/beta.bin

The files in this demonstration are longer than 20 bytes, which matters for the ICU 74.2 implementation: files at 20 bytes or less are rejected when creating a binary common data file. Do not use a tiny empty fixture to test a successful build.

2. Write a relative input list

Put one relative pathname on each line. Paths are resolved from the directory in which gencmn runs. The list can also contain a comment line beginning with #.

cat > files.lst <<'EOF'
# Files to package
input/alpha.bin
input/beta.bin
EOF
sed -n '1,10p' files.lst

Expected output is the two input paths, preceded by the comment. The path restriction is an easy trap: ICU 74.2 rejects an absolute entry such as /tmp/alpha.bin with an "absolute path encountered" error. Change into the project directory and list paths below it instead.

3. Generate and inspect the package

Use 0 as maxsize when every listed file should be eligible, choose an explicit output name, and enable verbose output while learning the result.

gencmn --verbose \
  --destdir out \
  --name demo \
  --type dat \
  0 files.lst

Expected output is similar to this:

generating demo.dat (common data file with table of contents)
adding ./input/alpha.bin (22 bytes)
adding ./input/beta.bin (23 bytes)

The exact byte counts depend on your payloads. The generated file is out/demo.dat. Check that it exists and is not empty:

test -s out/demo.dat && stat -c '%n %s bytes' out/demo.dat

The resulting file is an ICU common data package, not a general archive. ICU can use its table of contents to locate the packaged entries. Do not try to unpack it with tar or treat it as a normal filesystem directory.

Checkpoint: control large inputs

A positive maxsize is a per-file limit in bytes, not a limit on the final package. Files larger than the limit are omitted. The value is inclusive, so a file exactly at the limit is accepted.

gencmn --verbose \
  --destdir out \
  --name limited \
  --type dat \
  24 files.lst

For a 32-byte alpha.bin and a 24-byte beta.bin, the useful part of the output is:

./input/alpha.bin ignored (size 32 > 24)
adding ./input/beta.bin (24 bytes)

Use this as a deliberate filter, not as silent error handling. A successful exit does not mean every requested file was included; read verbose output or compare the list with the files you expected.

4. Use standard input when another command owns the list

The list filename is optional. Without it, gencmn reads standard input, so a carefully selected file list can be piped into the same build.

printf '%s\n' input/alpha.bin input/beta.bin | \
  gencmn --verbose --destdir out --name piped --type dat 0

This produces out/piped.dat. Quote or otherwise constrain generated pathnames before passing them to a packaging command. The list format is whitespace-sensitive: the tool takes the first space-delimited item on a line, so do not use unescaped filenames containing spaces.

5. Choose the destination deliberately

--destdir selects the output directory. If it is omitted, ICU uses ICU_DATA; if that variable is unset, the 74.2 installation defaults to /usr/share/icu/74.2/. Some ICU tools expect a trailing slash when ICU_DATA is set.

ICU_DATA="$HOME/icu-package-demo/out/" \
  gencmn --verbose --name envdemo --type dat 0 files.lst
test -s "$HOME/icu-package-demo/out/envdemo.dat"

For a shared or system-wide location, stop and confirm the required ownership and permissions first. A generated package may replace an existing file with the same name, and changing the data used by running services can alter their behaviour. Recovery is simple for this guide: remove the generated file in your private out directory, or restore the previous file from your own backup. Do not remove files from /usr/share/icu merely to tidy up a test.

Common failure points

• Absolute path error: replace entries such as /home/me/data/foo with paths relative to the working directory.
• Unable to get length: check that each file exists, is readable, and is larger than 20 bytes for a binary package on this ICU release.
• No files listed: inspect blank input, comments and the list filename. A typo in the list path is not the same as an empty package.
• Unexpected output name: remember that --name supplies the base name and --type supplies the extension. The default is normally an ICU release-and-endianness name with a .dat type.
• Missing data later: rerun with --verbose and confirm that every intended file was added. A positive size limit can omit inputs without failing the whole command.