Turn Delimited Text into a Reliable Report with mawk

A pile of comma-separated log data is not a report until mawk turns it into one, and this walkthrough shows exactly how. You will finish with a small mawk report that reads comma-separated service data, selects failed checks, calculates a total, and leaves the input file untouched. The examples use mawk 1.3.4 (package version 1.3.4.20240123-1build1) on this machine.

Allow about fifteen minutes. You need a shell and mawk. No elevated privileges are needed, because the workflow creates a temporary working file and writes its report to standard output. Do not point a report at a path you have not checked first: mawk reads every named input file, and a misspelled path ends the command with status 2.

1. Confirm the interpreter you will run

Check the executable and version before relying on implementation-specific details:

$ command -v mawk
/usr/bin/mawk
$ mawk -W version
mawk 1.3.4 20240123
Copyright 2008-2023,2024, Thomas E. Dickey
Copyright 1991-1996,2014, Michael D. Brennan

random-funcs:       arc4random_stir/arc4random
regex-funcs:        internal

compiled limits:
sprintf buffer      8192
maximum-integer     9223372036854775808

mawk implements the AWK language, including its pattern and action model. Short programs can follow the command options in single quotes: the shell passes the program as one argument, and mawk interprets its own braces, fields and regular expressions from there.

2. Create a known input file

Use a harmless fixture so the output is reproducible. This only changes state in the current directory and can be removed afterwards:

$ printf '%s\n' \
  'service,status,requests' \
  'api,ok,120' \
  'worker,failed,7' \
  'web,ok,310' \
  'backup,failed,2' > service-checks.csv
$ sed -n '1,5p' service-checks.csv
service,status,requests
api,ok,120
worker,failed,7
web,ok,310
backup,failed,2

3. Select records with fields

By default, mawk splits each input record at whitespace. This file is comma-separated, so set FS, the field separator, with -F,. The first line is a header, so skip it with NR > 1:

$ mawk -F, 'NR > 1 && $2 == "failed" { print $1, $3 }' service-checks.csv
worker 7
backup 2

Each line is a record. $1, $2 and $3 are its fields, NF is the number of fields, and NR is the record number across all input. The pattern is only true for data rows whose second field is exactly failed. When the action is just print, mawk inserts OFS, which defaults to a single space.

4. Produce a report with BEGIN, actions and END

For a useful report, set up a counter before input, add the numeric third field for failed rows, and print the result after the last record:

$ mawk -F, '
BEGIN { OFS = "\t"; print "service", "failed requests" }
NR > 1 && $2 == "failed" { failed += 1; requests += $3; print $1, $3 }
END { print "TOTAL", requests; print "failed services:", failed }
' service-checks.csv
service	failed requests
worker	7
backup	2
TOTAL	9
failed services:	2

• BEGIN runs first. It executes before the first input record.
• Pattern-action pairs run per matching record. Ordinary rules only fire for the rows they match.
• END runs last. It executes once input is exhausted.

Variables spring into existence on first use, starting at numeric value zero and string value empty, so explicit initialisation of failed and requests is not required here. Setting OFS changes the separator a multi-expression print uses; it does not touch the source data at all.

Keep numeric input numeric. The addition operator converts the third field for the calculation, while print formats the resulting number. If a field might contain untrusted or malformed text, validate it before adding it rather than letting a non-numeric prefix get silently treated as a number.

5. Move a longer program into a file

Quoting a multi-line command is fine for a quick test, but a report you will rerun belongs in a program file. Create failed-report.awk with the same logic:

$ cat > failed-report.awk <<'AWK'
BEGIN { OFS = "\t"; print "service", "failed requests" }
NR > 1 && $2 == "failed" {
    failed += 1
    requests += $3
    print $1, $3
}
END {
    print "TOTAL", requests
    print "failed services:", failed
}
AWK
$ mawk -F, -f failed-report.awk service-checks.csv
service	failed requests
worker	7
backup	2
TOTAL	9
failed services:	2

The -f option tells mawk to read program text from the named file; it does not make that file an input data file. Multiple -f options are allowed, which can separate shared functions from a report, but keeping this first script in one file makes its execution order easier to check.

6. Add a safe exit check

For a shell job, capture mawk's status immediately. A missing input file is an error and returns status 2; a normal run returns zero:

$ mawk -F, -f failed-report.awk service-checks.csv > failed-report.tsv
$ status=$?
$ printf 'mawk status: %s\n' "$status"
mawk status: 0
$ test "$status" -eq 0 && sed -n '1,6p' failed-report.tsv
service	failed requests
worker	7
backup	2
TOTAL	9
failed services:	2

Do not run another command between mawk and status=$?, or you will capture the wrong status. To test the error path without touching the fixture, point at a path that does not exist:

$ mawk -F, -f failed-report.awk does-not-exist.csv
mawk: cannot open does-not-exist.csv (No such file or directory)
$ printf 'mawk status: %s\n' "$?"
mawk status: 2

The exact diagnostic prefix can vary with the installed build, but the non-zero status is the part a script should test. A report that is incomplete or built on a missing file must never be treated as a success.

7. Clean up the exercise files

These files are disposable. Confirm the targets before removing them, then delete only the three paths created in this guide:

$ ls -l -- service-checks.csv failed-report.awk failed-report.tsv
$ rm -- service-checks.csv failed-report.awk failed-report.tsv
$ test ! -e service-checks.csv && test ! -e failed-report.awk && test ! -e failed-report.tsv
$ printf '%s\n' 'exercise files removed'
exercise files removed