Read Git's Chunk Tables Without Guessing at Offsets

Git's commit-graph and multi-pack-index files share one low-level container: the chunk format. This guide shows you how to find its chunk table, work out where each payload ends, and verify the two files built on it, commit-graph files and multi-pack indexes. The examples match the installed Git 2.43.0 manpages.

Allow about 15 minutes. You need Git 2.43.0 or a nearby release, a repository whose metadata you can inspect, and ordinary read access to its .git directory. None of the commands below needs sudo. Work on a disposable clone if you plan to experiment with files rather than only verify them.

1. Know what the format is for

A chunk-based file is a container for several data sections sitting one after another. Git uses this shared format for commit-graph files and for the multi-pack-index, usually abbreviated MIDX. Each file starts with its own format-specific header, giving the format, its version and the chunk count, and the chunk table follows straight after.

This is an on-disk layout, not something you invoke directly. What you actually run day to day are the writers and verifiers for the individual formats:

git commit-graph write
git commit-graph verify
git multi-pack-index write
git multi-pack-index verify

Checkpoint: if all you need is to check repository metadata, stop here and run the relevant verify command. You do not need to decode raw bytes to diagnose most problems.

2. Read the table of contents

The chunk region has one 12-byte row per chunk, plus one final row. Every row is a four-byte chunk identifier followed by an eight-byte offset, and the integers are big-endian, what the spec calls network byte order.

Chunk ID (4 bytes) | Chunk Offset (8 bytes)
ID[0]              | OFFSET[0]
...                | ...
ID[C]              | OFFSET[C]
0x0000             | OFFSET[C+1]

Each identifier names the data starting at its row's offset, and that data ends right where the next row's offset begins. So chunk i has a size of OFFSET[i+1] - OFFSET[i]. The format demands payloads appear contiguously and in the same order as their rows: no gaps, no reordering.

The last identifier is four zero bytes, a terminator telling the reader the table has ended and handing over the offset just after the final chunk. A valid file also has a trailing hash after that offset, and the hash sits outside the chunk payload region: do not count it as another chunk.

• Reading offsets as little-endian hands you absurd positions. They are big-endian.
• Treating the terminator as a payload identifier loses your final boundary.
• Assuming the last chunk runs to end of file swallows the trailing hash and gives you the wrong size.

3. Verify a commit-graph

From the root of a repository, write a graph for the commits in its packfiles, then ask Git to check it against the object database:

git commit-graph write
git commit-graph verify

Success just returns you to the shell without an error. On a terminal, Git may show progress while writing. The verifier is the useful part here: it checks the graph's contents, not just that a file happens to exist.

The graph normally lives under the repository's object directory. A repository can also use a split graph chain, several graph files under an info/commit-graphs directory. To check only the tip of a chain, there is a documented shallow option:

git commit-graph verify --shallow

Reach for the full verification when you are chasing corruption or a chain problem. --shallow is a narrower check, not a faster stand-in for validating every layer.

Checkpoint: run git commit-graph verify straight after a write. If the write reports success but nothing useful shows up, check whether core.commitGraph is disabled: the Git 2.43.0 documentation says a write can warn and return success without writing anything when that setting is off.

4. Verify a multi-pack-index

A MIDX indexes objects across several packfiles so you are not stuck with one index per lookup path. Create or refresh it, then verify it:

git multi-pack-index write
git multi-pack-index verify

By default this works on the current repository's packfiles, and the MIDX sits with them under the object directory's packs area. If your repository uses an alternate object directory, point at it, but only if it is a genuine known alternate for this repository:

git multi-pack-index --object-dir /path/to/alternate-objects write
git multi-pack-index --object-dir /path/to/alternate-objects verify

Replace the placeholder with a real alternate object directory. The command expects its packs subdirectory to hold the pack and pack-index pairs. Pointing it at a random directory is not a safe test, it just fails the alternate-object check.

A MIDX can also carry a bitmap when written with --bitmap. That adds another indexed section, but the chunk-table rule does not change: every section still gets a row, offsets are still big-endian, and the zero identifier still ends the table.

5. Inspect bytes only when verification is not enough

Raw inspection earns its keep when you are writing a parser or explaining a damaged file. First pin down the exact file Git is using. For a normal repository, list the metadata without touching it:

git rev-parse --git-path objects/info/commit-graph
git rev-parse --git-path objects/info/commit-graphs
git rev-parse --git-path objects/pack/multi-pack-index

Some of these paths may not exist yet, especially before the matching writer has run. Do not read validity into a pathname alone: run the relevant verifier and only inspect the file itself if you need byte-level evidence.

For a parser, read the format-specific header first, grab the chunk count and the start of the chunk region, then read exactly C + 1 rows of 12 bytes. Decode each identifier as four bytes and each offset as an unsigned eight-byte big-endian integer. Check that offsets are ordered, sit within the mapped file, and leave room for the trailing hash. Reject a missing zero identifier, overlapping ranges, or an offset that points back into the table.

Git's C API follows the same lifecycle. A writer sets up a struct chunkfile with init_chunkfile(), registers each payload with add_chunk(), and calls write_chunkfile(), which checks every callback produced the expected amount of data. The caller then frees the chunk state, writes the trailing hash and closes the hashfile.

A reader memory-maps the whole file, sets up state with init_chunkfile(NULL), and calls read_table_of_contents(). pair_chunk() hands back a pointer to a present chunk and leaves the pointer untouched when the identifier is absent. read_chunk() instead calls a callback with the chunk's pointer and size, and skips the callback entirely for a missing chunk. Keep the mapping alive while those pointers are in use, then call free_chunkfile(); freeing the parser state does not unmap the file.

6. Recover from a failed or unwanted write

These writers change derived files inside .git, never tracked working-tree content. If a write fails, leave the existing files alone and run the matching verifier first: it may show whether the old metadata still works. Do not delete packfiles or graph layers as your first move, pack maintenance can discard objects, and deletion is never required just to understand the table format.

Warning: if you deliberately generated metadata in a disposable clone and want that clone back the way it was, remove only the exact generated file or directory after confirming its path with git rev-parse --git-path. This is an irreversible filesystem action, so take a backup or reclone instead if the repository holds valuable local state. A later git commit-graph write or git multi-pack-index write can rebuild derived indexes from the object database, provided the underlying objects are still intact.