Measure Pipe Throughput and Safe File Copies with pv

pv sits inside a copy or a pipeline and shows elapsed time, transfer rate and a progress bar. No more blinking cursor with no clue whether anything is happening. This guide builds a small, repeatable set of pv commands for watching a file copy or a shell pipeline, using pv 1.8.5 from package version 1.8.5-2build1, matching the installed manual dated November 2023.

Allow about fifteen minutes. You need a shell, a readable input file, and a writable working directory. The normal examples are unprivileged. Writing a device, changing another process or reading protected files may need elevated privileges and can damage data, so those operations sit outside the first workflow.

1. Check the installed command

Confirm which executable and package you are using before relying on an option in a script:

$ command -v pv
/usr/bin/pv
$ pv --version
pv 1.8.5
Copyright 2023 Andrew Wood
$ dpkg-query -W -f='${Package} ${Version}\n' pv
pv 1.8.5-2build1

Option names and output formatting can vary between releases; this guide is written for the version above. If pv is missing, install the package through your normal system-management process rather than copying an unrelated binary into a pipeline.

2. Put pv between a source and a destination

pv copies standard input to standard output and writes its status display to standard error. The simplest useful shape is:

$ pv /path/to/input.iso > /path/to/input.iso.copy

When a regular file is supplied as an argument, pv can work out its size, so the default display includes a progress bar, elapsed time, ETA, current rate and bytes transferred. It is temporary terminal output, so it may redraw on one line while the copy runs.

Checkpoint: wait for the command to return, then compare the files:

$ cmp --silent /path/to/input.iso /path/to/input.iso.copy
$ printf 'cmp exit status: %s\n' "$?"
cmp exit status: 0

Warning: do not use the same path on both sides. pv reports an input and output collision with a non-zero status, but shell redirection can truncate a destination before the program gets a chance to help you. If the output already matters, choose a new name or take a backup first.

3. Supply the size for a pipe

A pipe usually carries no total byte count, so pv can show that data is moving but cannot calculate a percentage or ETA. Give it the expected number of bytes with -s:

$ input=/path/to/input.iso
$ pv -s "$(stat -c '%s' "$input")" < "$input" > /path/to/input.iso.copy

stat measures the file before the pipeline starts. The size argument is in bytes, and pv also accepts suffixes such as K, M, G and T, interpreted as powers of 1024. The quoted variables keep spaces in a filename from breaking the command's structure.

If you already know the size, a literal is easier to audit:

$ cat /path/to/input.iso | pv --size 4G > /path/to/input.iso.copy

Only use a size that describes the data actually sent: a wrong estimate makes the percentage and ETA misleading, though on its own it does not limit the input. Add --stop-at-size only when you deliberately want pv to stop after the specified number of bytes:

$ head -c 4096 /dev/zero | pv --size 4096 --stop-at-size > /tmp/pv-four-kib

This test changes only a file under /tmp. Remove it when no longer needed with rm -- /tmp/pv-four-kib; that removal is irreversible.

4. Make output useful in scripts and logs

pv does not normally draw a visual display when its standard error is not a terminal. Use --force when a script, capture command or monitoring wrapper needs the display anyway. Status output stays separate from the data stream: pv's copied bytes remain on standard output.

$ head -c 1000000 /dev/zero | pv --size 1000000 --force > /dev/null 2> /tmp/pv-status
$ sed -n '1p' /tmp/pv-status
 1.00MiB 0:00:00 [ ... ] 100%

The spacing and rate vary with timing and terminal width. What actually matters is that the data destination is correct, the status file is separate, and pv exits successfully:

$ test -s /tmp/pv-status && echo 'status captured'
status captured
$ printf 'pv exit status: %s\n' "$?"
pv exit status: 0

Choose one meaning and document it for whatever consumes it. Do not parse the normal progress bar as if it were a stable API.

5. Throttle a transfer without changing its contents

Use --rate-limit when a copy must not consume all available bandwidth or storage throughput:

$ pv --rate-limit 10M /path/to/input.iso > /path/to/input.iso.copy

The suffix M means mebibytes per second in this option. Add --quiet if you want the limiter without a display:

$ pv --quiet --rate-limit 10M /path/to/input.iso > /path/to/input.iso.copy

A rate limit controls pv's own transfer; it does not reserve capacity for another process and does not make a remote destination reliable. If something in the pipeline fails, check every component rather than treating a completed-looking display as proof the whole job succeeded.

6. Handle lines, delays and common traps

Use --line-mode when the useful unit is newline-delimited records rather than bytes. Without --size, pv may need to read regular input once to count lines before transferring it, which can be surprising for a large input or a pipe. Give an expected line count when you have one:

$ pv --line-mode --size 1000 /path/to/records.txt > /path/to/records.copy

For a job that normally finishes too quickly to watch, --delay-start 2 waits two seconds before displaying progress. --wait waits for the first byte before showing progress or calculating ETAs, useful when an upstream program pauses for input. Both affect display timing, not the bytes copied.

If you see no progress, check whether standard error is a terminal, whether the transfer has actually started, and whether a delay option was supplied. Use --force for a deliberate non-terminal display. If the percentage will not calculate, give it an accurate size. Do not reach for sudo as a diagnostic shortcut: it changes file access and ownership context, not pv's ability to work out the length of an arbitrary stream.

7. Treat device images as a separate risk level

Warning: pv can read from or write to block devices, but a redirection such as pv image > /dev/your-device overwrites the target. A single typo can destroy a disk or a partition. Do not run a device-writing command until you have identified the device independently with tools such as lsblk, unmounted it where appropriate, confirmed the image checksum, and accepted that the operation needs elevated privileges. This guide deliberately does not give you a live device path to paste.

--skip-errors can push past read errors by filling the skipped portions with null bytes. That is a recovery trade-off, not a way to get a trustworthy copy: record the errors and validate the resulting image against what you meant to recover. For normal files, fix the underlying storage or input problem instead.

Done means