Turn Delimited Text into a Reliable Report with mawk

You will finish with a small mawk report that reads comma-separated service data, selects failed checks, calculates a total, and leaves the input file untouched. The examples use mawk 1.3.4 (package version 1.3.4.20240123-1build1) on this machine.

Allow about fifteen minutes. You need a shell and mawk. No elevated privileges are needed because the workflow creates a temporary working file and writes its report to standard output. Do not run a report against a path you have not checked first: mawk reads each named input file, and a misspelled path ends the command with status 2.

1. Confirm the interpreter you will run

Check the executable and version before relying on implementation-specific details:

$ command -v mawk
/usr/bin/mawk
$ mawk -W version
mawk 1.3.4 20240123
Copyright 2008-2023,2024, Thomas E. Dickey
Copyright 1991-1996,2014, Michael D. Brennan

random-funcs:       arc4random_stir/arc4random
regex-funcs:        internal

compiled limits:
sprintf buffer      8192
maximum-integer     9223372036854775808

mawk implements the AWK language, including its pattern and action model. Short programs can follow the command options in single quotes. The shell then passes the program as one argument, while mawk interprets its own braces, fields and regular expressions.

2. Create a known input file

Use a harmless fixture so the output is reproducible. This changes state only in the current directory and can be removed afterwards:

$ printf '%s\n' \
  'service,status,requests' \
  'api,ok,120' \
  'worker,failed,7' \
  'web,ok,310' \
  'backup,failed,2' > service-checks.csv
$ sed -n '1,5p' service-checks.csv
service,status,requests
api,ok,120
worker,failed,7
web,ok,310
backup,failed,2

The > redirection creates or replaces service-checks.csv. Before using this pattern on a real file, replace the placeholder path and check it with ls -l --. If you accidentally overwrite a file, mawk cannot undo the shell redirection. The fixture can be removed with rm -- service-checks.csv after the exercise.

3. Select records with fields

By default, mawk splits each input record at whitespace. This file is comma-separated, so set FS, the field separator, with -F,. The first line is a header, so skip it with NR > 1:

$ mawk -F, 'NR > 1 && $2 == "failed" { print $1, $3 }' service-checks.csv
worker 7
backup 2

Each line is a record. $1, $2 and $3 are its fields, NF is the number of fields, and NR is the record number across all input. The pattern is true only for data rows whose second field is exactly failed. When the action is just print, mawk inserts OFS, which defaults to one space.

4. Produce a report with BEGIN, actions and END

For a useful report, initialise a counter before input, add the numeric third field for failed rows, and print the result after the last record:

$ mawk -F, '
BEGIN { OFS = "\t"; print "service", "failed requests" }
NR > 1 && $2 == "failed" { failed += 1; requests += $3; print $1, $3 }
END { print "TOTAL", requests; print "failed services:", failed }
' service-checks.csv
service	failed requests
worker	7
backup	2
TOTAL	9
failed services:	2

BEGIN runs before the first input record, ordinary pattern-action pairs run for matching records, and END runs after input is exhausted. Variables come into existence on first use and start with numeric value zero and string value empty, so explicit initialisation of failed and requests is not required here. Setting OFS changes the separator used by a multi-expression print; it does not alter the source data.

Keep numeric input numeric. The addition operator converts the third field for the calculation, while print formats the resulting number. If a field may contain untrusted or malformed text, validate it before adding it rather than silently treating a non-numeric prefix as a number.

5. Move a longer program into a file

Quoting a multi-line command is useful for a quick test, but a report you will rerun belongs in a program file. Create failed-report.awk with the same logic:

$ cat > failed-report.awk <<'AWK'
BEGIN { OFS = "\t"; print "service", "failed requests" }
NR > 1 && $2 == "failed" {
    failed += 1
    requests += $3
    print $1, $3
}
END {
    print "TOTAL", requests
    print "failed services:", failed
}
AWK
$ mawk -F, -f failed-report.awk service-checks.csv
service	failed requests
worker	7
backup	2
TOTAL	9
failed services:	2

The -f option tells mawk to read program text from the named file. It does not make that file an input data file. Multiple -f options are allowed, which can separate shared functions from a report, but keeping this first script in one file makes its execution order easier to inspect.

6. Add a safe exit check

For a shell job, capture mawk's status immediately. A missing input file is an error and returns status 2; a normal run returns zero:

$ mawk -F, -f failed-report.awk service-checks.csv > failed-report.tsv
$ status=$?
$ printf 'mawk status: %s\n' "$status"
mawk status: 0
$ test "$status" -eq 0 && sed -n '1,6p' failed-report.tsv
service	failed requests
worker	7
backup	2
TOTAL	9
failed services:	2

Do not place another command between mawk and status=$?, or you will capture the wrong status. To test the error path without changing the fixture, use a path that does not exist:

$ mawk -F, -f failed-report.awk does-not-exist.csv
mawk: cannot open does-not-exist.csv (No such file or directory)
$ printf 'mawk status: %s\n' "$?"
mawk status: 2

The exact diagnostic prefix can vary with the installed build, but the non-zero status is the part a script should test. A report that is incomplete or based on a missing file must not be treated as successful.

7. Clean up the exercise files

These files are disposable. Confirm the targets before removing them, then delete only the three paths created in this guide:

$ ls -l -- service-checks.csv failed-report.awk failed-report.tsv
$ rm -- service-checks.csv failed-report.awk failed-report.tsv
$ test ! -e service-checks.csv && test ! -e failed-report.awk && test ! -e failed-report.tsv
$ printf '%s\n' 'exercise files removed'
exercise files removed

This cleanup is irreversible for those files. If you need the report later, copy it to an intentional destination before running the removal command. No privilege escalation is appropriate for these paths.