11.1. Conceptual Overview

awk is a pattern-matching program for processing files, especially when each line has a simple field-oriented layout. The new version of awk, called nawk, provides additional capabilities.^[*] Every modern Unix system comes with a version of new awk, and its use is recommended over old awk. The GNU version of awk, called gawk, implements new awk and provides a number of additional features.

^[*] It really isn't so new. The additional features were added in 1984, and it was first shipped with System V Release 3.1 in 1987.

Different systems vary in what new and old awk are called. Some have oawk and awk, for the old and new versions, respectively. Others have awk and nawk. Still others only have awk, which is the new version. This example shows what happens if your awk is the old one:

     $ awk 1 /dev/null     awk: syntax error near line 1     awk: bailing out near line 1

awk will exit silently if it is the new version.

The POSIX standard for awk is based on new awk, and the standard uses the simple designation awk for that language. Thus, we do also. If your system's awk is the old one, find the new one, and use it for your programs.

Solaris is the only modern Unix system that persists in having old awk as the default version. You should be sure to put /usr/xpg4/bin in your shell's search path before /usr/bin, so that you will get a POSIX-compliant version of awk. Alternatively, just install the GNU version.

Items described here as "common extensions" are often available in different versions of new awk, as well as in gawk, but should not be used if strict portability of your programs is important to you.

The freely available versions of awk described in the section "Source Code," later in this chapter, all implement new awk.

With awk, you can:

• Think of a text file as made up of records and fields in a textual database.

• Perform arithmetic and string operations.

• Use programming constructs such as loops and conditionals.

• Produce formatted reports.

• Define your own functions.

• Execute Unix commands from a script.

• Process the results of Unix commands.

• Process command-line arguments gracefully.

• Work easily with multiple input streams.

• Flush open output files and pipes (with the latest Bell Laboratories version of awk).

In addition, with GNU awk (gawk), you can:

• Use regular expressions to separate records, as well as fields.

• Skip to the start of the next file, not just the next record.

• Perform more powerful string substitutions.

• Sort arrays.

• Retrieve and format system time values.

• Use octal and hexadecimal constants in your program.

• Do bit manipulation.

• Internationalize your awk programs, allowing strings to be translated into a local language at runtime.

• Perform two-way I/O to a coprocess.

• Open a two-way TCP/IP connection to a socket.

• Dynamically add built-in functions.

• Profile your awk programs.