Validate Thin-Provisioning Metadata Safely with thin_check

thin_check is the command that tells you whether device-mapper thin-provisioning metadata is trustworthy before you hand a pool back to production. The ordinary check is read-only, but the same binary can also clear a superblock flag or attempt a limited repair, and those are separate decisions with separate risks. Allow about ten minutes for a single metadata device, plus whatever maintenance window is needed to stop its users first.

This guide describes the installed thin-provisioning-tools package, version 0.9.0-2ubuntu5.1, and its thin_check 0.9.0 binary. Storage metadata is valuable, so have a current backup or snapshot plan in place before taking a pool out of service.

1. Confirm the installed command

Start as your ordinary account. Checking the executable and version changes nothing:

$ command -v thin_check
/usr/sbin/thin_check
$ dpkg-query -W -f='${Package} ${Version}\n' thin-provisioning-tools
thin-provisioning-tools 0.9.0-2ubuntu5.1
$ thin_check --version
0.9.0

The path can differ on another distribution. If command -v finds nothing, install the package through your normal operating-system process rather than copying a binary into a storage directory.

Checkpoint: you know which binary will run and have recorded its version. The examples below use /dev/vg/metadata as a placeholder. Replace it with the metadata device or file belonging to your own pool.

2. Identify the metadata target

thin_check accepts one device or file containing metadata created by the device-mapper thin-provisioning target. It does not take a thin volume data path as a substitute for the metadata target. If the pool is managed by LVM, inspect the pool configuration and your runbook to establish the exact metadata LV before running a check.

Confirm the path without opening it for modification:

$ ls -l /dev/vg/metadata
$ test -r /dev/vg/metadata && echo 'metadata path is readable'
metadata path is readable

Do not guess from a similarly named LV. A wrong readable device can still be the wrong metadata. If the path needs elevated access, use sudo only for the check itself:

$ sudo thin_check /dev/vg/metadata

3. Stop the target before an ordinary check

Warning: The ordinary check must not run against live metadata. The manpage says the device must not be actively used by the thin-provisioning target. Schedule a maintenance window, stop or deactivate the services and volumes that use the pool, and follow your storage manager's documented deactivation procedure.

Do not improvise a deactivation command here. The correct order depends on whether the pool is controlled by LVM, device-mapper commands or another service. Verify that the target is no longer active using the tooling already used to administer that pool. Only then proceed.

If you cannot stop the pool, do not run the ordinary command on its live metadata. The supported live-data route is a metadata snapshot, covered in step 6, with reduced coverage.

Checkpoint: the pool is inactive, the metadata path is correct, and you have a route to reactivate the pool after the check. No root shell is required; use a narrowly scoped sudo command if permissions demand it.

4. Run the normal validation

Run the command with no repair or suppression flags:

$ sudo thin_check /dev/vg/metadata

A successful check returns exit status 0. It may produce little or no output, so check the status explicitly when this is part of a script or incident record:

$ printf 'thin_check exit status: %s\n' "$?"
thin_check exit status: 0

Exit status 1 means an error was found. Keep the diagnostic output and the exact command in your incident notes. A non-zero result is a reason to stop and investigate, not a reason to add --ignore-non-fatal-errors and continue activation.

The --quiet or -q option suppresses messages and leaves only the exit code. It is useful for a carefully designed health check, but it is a poor first run because it removes information needed for diagnosis:

$ sudo thin_check --quiet /dev/vg/metadata
$ printf 'status: %s\n' "$?"
status: 0

5. Narrow the check only when you have a reason

--super-block-only checks only the superblock. --skip-mappings skips the block mappings, which make up most of the metadata. Both can answer a narrow diagnostic question, but neither is equivalent to a full validation. Record the reduced scope in the result:

$ sudo thin_check --super-block-only /dev/vg/metadata
$ printf 'superblock-only status: %s\n' "$?"
superblock-only status: 0

Use a full check before treating metadata as validated. If a full check is too slow for a monitoring probe, document the limited check and arrange a complete check during maintenance.

The manpage documents --metadata-snapshot, also written as -m, for checking a metadata snapshot. On this installed 0.9.0 binary, thin_check --help prints the long spelling --metadata-snap instead. Prefer the short, documented option when working on this host, and verify the local help before putting the long spelling into automation:

$ thin_check --help | grep -- '--metadata'
  {-m|--metadata-snap}

6. Use a metadata snapshot for live metadata

--metadata-snapshot or -m is the exception to the offline rule. It checks the devices tree and mappings in a metadata snapshot, so it can be used with live metadata. The snapshot does not contain space maps, which means those are not checked. Treat a successful snapshot check as useful evidence about that snapshot, not as a full offline validation of every metadata structure.

Create or expose the snapshot using the pool administration procedure appropriate to your system, then pass the resulting snapshot device or file:

$ sudo thin_check -m /dev/vg/metadata-snapshot
$ printf 'snapshot check status: %s\n' "$?"
snapshot check status: 0

Do not invent a snapshot path or remove a live snapshot while a service still depends on it. Follow the procedure that created it for cleanup. If the snapshot check fails, preserve its diagnostics and escalate to the person responsible for the pool before trying repair.

7. Treat repair and flag-clearing options as state changes

Warning: --clear-needs-check-flag changes the metadata superblock. The kernel can set this flag to require a check before the pool is activated. Only clear it after a successful check, and only when your storage procedure says the flag should be cleared:

$ sudo thin_check --clear-needs-check-flag /dev/vg/metadata
$ printf 'flag-clear check status: %s\n' "$?"
flag-clear check status: 0

The three options below all change something. None of them is a substitute for fixing the underlying problem:

Done means