Chapter 6. The awk Utility: awk Programming Constructs
 | CONTENTS |  |
Chapter 6. The awk Utility: awk Programming Constructs
- 6.1 Comparison Expressions
- 6.2 Review
- UNIX TOOLS LAB EXERCISE
6.1 Comparison Expressions
Comparison expressions match lines where if the condition is true, the action is performed. These expressions use relational operators and are used to compare numbers or strings. Table 6.1 provides a list of the relational operators. The value of the expression is 1 if the expression evaluates true, and 0 if false.
6.1.1 Relational Operators
| Operator | Meaning | Example |
|---|---|---|
| < | Less than. | x < y |
| <= | Less than or equal to. | x <= y |
| == | Equal to. | x == y |
| != | Not equal to. | x != y |
| >= | Greater than or equal to. | x >= y |
| > | Greater than. | x > y |
| ~ | Matched by regular expression. | x ~ /y/ |
| !~ | Not matched by regular expression. | x !~ /y/ |
Example 6.1
(The Database) % cat employee Tom Jones 4423 5/12/66 543354 Mary Adams 5346 11/4/63 28765 Sally Chang 1654 7/22/54 650000 Billy Black 1683 9/23/44 336500 (The Command Line) 1 % nawk '$3 == 5346' employees Mary Adams 5346 11/4/63 28765 2 % nawk '$3 > 5000{print $1} ' employees Mary 3 % nawk '$2 ~ /Adam/ ' employees Mary Adams 5346 11/4/63 28765 4 % nawk '$2 !~ /Adam/ ' employees Tom Jones 4423 5/12/66 543354 Sally Chang 1654 7/22/54 650000 Billy Black 1683 9/23/44 336500
EXPLANATION
|
6.1.2 Conditional Expressions
A conditional expression uses two symbols, the question mark and the colon, to evaluate expressions. It is really just a short way to achieve the same result as doing an if/else statement. The general format is shown below.
FORMATconditional expression1 ? expression2 : expression3 |
This produces the same result as the if/else shown below. (A complete discussion of the if/else construct is given later.)
{ if (expression1) expression2 else expression3 } Example 6.2
nawk '{max=($1 > $2) ? $1 : $2; print max}' filename EXPLANATIONIf the first field is greater than the second field, the value of the expression after the question mark is assigned to max, otherwise the value of the expression after the colon is assigned to max. This is comparable to if ($1 > $2) max=$1 else max=$2 |
6.1.3 Computation
Computation can be performed within patterns. Awk performs all arithmetic in floating point. The arithmetic operators are provided in Table 6.2.
| Operator | Meaning | Example |
|---|---|---|
| + | Add | x+y |
| - | Subtract | x-y |
| * | Multiply | x*y |
| / | Divide | x/y |
| % | Modulus | x%y |
| ^ | Exponentiation | x^y |
Example 6.3
nawk '$3 * $4 > 500' filename EXPLANATIONAwk will multiply the third field ($3) by the fourth field ($4), and if the result is greater than 500, it will display those lines. (filename is assumed to be a file containing the input.) |
6.1.4 Compound Patterns
Compound patterns are expressions that combine patterns with logical operators (see Table 6.3). An expression is evaluated from left to right.
| Operator | Meaning | Example |
|---|---|---|
| && | Logical and | a && b |
| || | Logical or | a || b |
| ! | not | ! a |
Example 6.4
nawk '$2 > 5 && $2 <= 15' filename EXPLANATIONAwk will display those lines that match both conditions; that is, where the second field ($2) is greater than 5 and the second field ($2) is also less than or equal to 15. With the && operator, both conditions must be true. (filename is assumed to be a file containing the input.) |
Example 6.5
nawk '$3 == 100 || $4 > 50' filename EXPLANATIONAwk will display those lines that match one of the conditions; that is, where the third field is equal to 100 or the fourth field is greater than 50. With the || operator, only one of the conditions must be true. (filename is assumed to be a file containing the input.) |
Example 6.6
nawk '!($2 < 100 && $3 < 20)' filename EXPLANATIONIf both conditions are true, awk will negate the expression and display those lines. So the lines displayed will have one or both conditions false. The unary ! operator negates the result of the condition so that if the expression yields a true condition, the not will make it false, and vice versa. (filename is assumed to be a file containing the input.) |
6.1.5 Range Patterns
Range patterns match from the first occurrence of one pattern to the first occurrence of the second pattern, then match for the next occurrence of the first pattern to the next occurrence of the second pattern, etc. If the first pattern is matched and the second pattern is not found, awk will display all lines to the end of the file.
Example 6.7
nawk '/Tom/,/Suzanne/' filename EXPLANATIONAwk will display all lines, inclusive, that range between the first occurrence of Tom and the first occurrence of Suzanne. If Suzanne is not found, awk will continue processing lines until the end of file. If, after the range between Tom and Suzanne is printed, Tom appears again, awk will start displaying lines until another Suzanne is found or the file ends. |
6.1.6 A Data Validation Program
Using the awk commands discussed so far, the following password-checking program from the book, The AWK Programming Language,[1] illustrates how the data in a file can be validated.
Example 6.8
(The Password Database) 1 % cat /etc/passwd tooth:pwHfudo.eC9sM:476:40:Contract Admin.:/home/rickenbacker/tooth:/bin/csh lisam:9JY7OuS2f3lHY:4467:40:Lisa M. Spencer:/home/fortune1/lisam:/bin/csh goode:v7Ww.nWJCeSIQ:32555:60:Goodwill Guest User:/usr/goodwill:/bin/csh bonzo:eTZbu6M2jM7VA:5101:911: SSTOOL Log account :/home/sun4/bonzo:/bin/csh info:mKZsrioPtW9hA:611:41:Terri Stern:/home/chewie/info:/bin/csh cnc:IN1IVqVj1bVv2:10209:41:Charles Carnell:/home/christine/cnc:/bin/csh bee:*:347:40:Contract Temp.:/home/chanel5/bee:/bin/csh friedman:oyuIiKoFTV0TE:3561:50:Jay Friedman:/home/ibanez/friedman:/bin/csh chambers:Rw7R1k77yUY4.:592:40:Carol Chambers:/usr/callisto2/chambers:/bin/csh gregc:nkLulOg:7777:30:Greg Champlin FE Chicago ramona:gbDQLdDBeRc46:16660:68:RamonaLeininge MWA CustomerService Rep:/ home/forsh: (The Awk Commands) 2 % cat /etc/passwd | nawk F: '\ 3 NF != 7{\ 4 printf("line %d, does not have 7 fields: %s\n",NR,$0)} \ 5 $1 !~ /[A Za z0 9]/{printf("line %d, nonalphanumeric user id: %s\n",NR,$0)} \ 6 $2 == "*" {printf("line %d, no password: %s\n",NR,$0)} ' (The Output) line 7, no password: bee:*:347:40:Contract Temp.:/home/chanel5/bee:/bin/csh line 10, does not have 7 fields: gregc:nk2EYi7kLulOg:7777:30:Greg Champlin FE Chicago line 11, does not have 7 fields: ramona:gbDQLdDBeRc46:16660:68:Ramona Leininger MWA Customer Service Rep:/home/forsh:
EXPLANATION
|
6.2 Review
6.2.1 Equality Testing
% cat datafile northwest NW Joe Craig 3.0 .98 3 4 western WE Sharon Kelly 5.3 .97 5 23 southwest SW Chris Foster 2.7 .8 2 18 southern SO May Chin 5.1 .95 4 15 southeast SE Derek Johnson 4.0 .7 4 17 eastern EA Susan Beal 4.4 .84 5 20 northeast NE TJ Nichols 5.1 .94 3 13 north NO Val Shultz 4.5 .89 5 9 central CT Sheri Watson 5.7 .94 5 13
Example 6.9
nawk '$7 == 5' datafile western WE Sharon Kelly 5.3 .97 5 23 eastern EA Susan Beal 4.4 .84 5 20 north NO Val Shultz 4.5 .89 5 9 central CT Sheri Watson 5.7 .94 5 13
EXPLANATIONIf the seventh field ($7) is equal to the number 5, the record is printed. |
Example 6.10
nawk '$2 == "CT"{print $1, $2}' datafile central CT
EXPLANATIONIf the second field is equal to the string CT, fields one and two ($1, $2) are printed. Strings must be quoted. |
% cat datafile northwest NW Joel Craig 3.0 .98 3 4 western WE Sharon Kelly 5.3 .97 5 23 southwest SW Chris Foster 2.7 .8 2 18 southern SO May Chin 5.1 .95 4 15 southeast SE Derek Johnson 4.0 .7 4 17 eastern EA Susan Beal 4.4 .84 5 20 northeast NE TJ Nichols 5.1 .94 3 13 north NO Val Shultz 4.5 .89 5 9 central CT Sheri Watso 5.7 .94 5 13
6.2.2 Relational Operators
Example 6.11
nawk '$7 != 5' datafile northwest NW Joel Craig 3.0 .98 3 4 southwest SW Chris Foster 2.7 .8 2 18 southern SO May Chin 5.1 .95 4 15 southeast SE Derek Johnson 4.0 .7 4 17 northeast NE TJ Nichols 5.1 94 3 13
EXPLANATIONIf the seventh field ($7) is not equal to the number 5, the record is printed. |
Example 6.12
nawk '$7 < 5 {print $4, $7}' datafile Craig 3 Foster 2 Chin 4 Johnson 4 Nichols 3
EXPLANATIONIf the seventh field ($7) is less than 5, fields 4 and 7 are printed. |
Example 6.13
nawk '$6 > .9 {print $1, $6}' datafile northwest .98 western .97 southern .95 northeast .94 central .94
EXPLANATIONIf the sixth field ($6) is greater than .9, fields 1 and 6 are printed. |
Example 6.14
nawk '$8 <= 17 { print $8}' datafile 4 15 17 13 9 13
EXPLANATIONIf the eighth field ($8) is less than or equal to 17, it is printed. |
Example 6.15
nawk '$8 >= 17 {print $8}' datafile 23 18 17 20
EXPLANATIONIf the eighth field is greater than or equal to 17, the eighth field is printed. |
% cat datafile northwest NW Joel Craig 3.0 .98 3 4 western WE Sharon Kelly 5.3 .97 5 23 southwest SW Chris Foster 2.7 .8 2 18 southern SO May Chin 5.1 .95 4 15 southeast SE Derek Johnson 4.0 .7 4 17 eastern EA Susan Beal 4.4 .84 5 20 northeast NE TJ Nichols 5.1 .94 3 13 north NO Val Shultz 4.5 .89 5 9 central CT Sheri Watson 5.7 .94 5 13
6.2.3 Logical Operators
Example 6.16
nawk '$8 > 10 && $8 < 17' datafile southern SO May Chin 5.1 .95 4 15 northeast NE TJ Nichols 5.1 .94 3 13 central CT Sheri Watson 5.7 .94 5 13
EXPLANATIONIf the eighth field ($8) is greater than 10 and less than 17,the record is printed. The record will be printed only if both expressions are true. |
Example 6.17
nawk '$2 == "NW" || $1 ~ /south/{print $1, $2}' datafile northwest NW southwest SW southern SO southeast SE
EXPLANATIONIf the second field ($2) is equal to the string NW or the first field ($1) contains the pattern south, the first and second fields ($1, $2) are printed. The record will be printed if only one of the expressions is true. |
6.2.4 Logical not Operator
Example 6.18
nawk '!($8 == 13){print $8}' datafile 4 23 18 15 17 20 9
EXPLANATIONIf the eighth field ($8) is equal to 13, the ! (not operator) nots the expression and prints the eighth field ($8). The ! is a unary negation operator. |
6.2.5 Arithmetic Operators
Example 6.19
nawk '/southern/{print $5 + 10}' datafile 15.1
EXPLANATIONIf the record contains the regular expression southern, 10 is added to the value of the fifth field ($5) and printed. Note that the number prints in floating point. |
Example 6.20
nawk '/southern/{print $8 + 10}' datafile 25
EXPLANATIONIf the record contains the regular expression southern, 10 is added to the value of the eighth field ($8) and printed. Note that the number prints in decimal. |
% cat datafile northwest NW Joel Craig 3.0 .98 3 4 western WE Sharon Kelly 5.3 .97 5 23 southwest SW Chris Foster 2.7 .8 2 18 southern SO May Chin 5.1 .95 4 15 southeast SE Derek Johnson 4.0 .7 4 17 eastern EA Susan Beal 4.4 .84 5 20 northeast NE TJ Nichols 5.1 .94 3 13 north NO Val Shultz 4.5 .89 5 9 central CT Sheri Watson 5.7 .94 5 13
Example 6.21
nawk '/southern/{print $5 + 10.56}' datafile 15.66
EXPLANATIONIf the record contains the regular expression southern, 10.56 is added to the value of the fifth field ($5) and printed. |
Example 6.22
nawk '/southern/{print $8 - 10}' datafile 5
EXPLANATIONIf the record contains the regular expression southern, 10 is subtracted from the value of the eighth field ($8) and printed. |
Example 6.23
nawk '/southern/{print $8 / 2}' datafile 7.5
EXPLANATIONIf the record contains the regular expression southern, the value of the eighth field ($8) is divided by 2 and printed. |
Example 6.24
nawk '/northeast/{print $8 / 3}' datafile 4.33333
EXPLANATIONIf the record contains the regular expression northeast, the value of the eighth field ($8) is divided by 3 and printed. The precision is five places to the right of the decimal point. |
Example 6.25
nawk '/southern/{print $8 * 2}' datafile 30
EXPLANATIONIf the record contains the regular expression southern, the eighth field ($8) is multiplied by 2 and printed. |
Example 6.26
nawk '/northeast/ {print $8 % 3}' datafile 1
EXPLANATIONIf the record contains the regular expression northeast, the eighth field ($8) is divided by 3 and the remainder (modulus) is printed. |
Example 6.27
nawk '$3 ~ /^Susan/\ {print "Percentage: "$6 + .2 " Volume: " $8}' datafile Percentage: 1.04 Volume: 20
EXPLANATIONIf the third field ($3) begins with the regular expression Susan, the print function prints the result of the calculations and the strings in double quotes. |
% cat datafile northwest NW Joel Craig 3.0 .98 3 4 western WE Sharon Kelly 5.3 .97 5 23 southwest SW Chris Foster 2.7 .8 2 18 southern SO May Chin 5.1 .95 4 15 southeast SE Derek Johnson 4.0 .7 4 17 eastern EA Susan Beal 4.4 .84 5 20 northeast NE TJ Nichols 5.1 .94 3 13 north NO Val Shultz 4.5 .89 5 9 central CT Sheri Watson 5.7 .94 5 13
6.2.6 Range Operator
Example 6.28
nawk '/^western/,/^eastern/' datafile western WE Sharon Kelly 5.3 .97 5 23 southwest SW Chris Foster 2.7 .8 2 18 southern SO May Chin 5.1 .95 4 15 southeast SE Derek Johnson 4.0 .7 4 17 eastern EA Susan Beal 4.4 .84 5 20
EXPLANATIONAll records within the range beginning with the regular expression western are printed until a record beginning with the expression eastern is found. Records will start being printed again if the pattern western is found and will continue to print until eastern or end of file is reached. |
6.2.7 Conditional Operator
Example 6.29
nawk '{print ($7 > 4 ? "high "$7 : "low "$7)}' datafile low 3 high 5 low 2 low 4 low 4 high 5 low 3 high 5 high 5
EXPLANATIONIf the seventh field ($7) is greater than 4, the print function gets the value of the expression after the question mark (the string high and the seventh field), else the print function gets the value of the expression after the colon (the string low and the value of the seventh field). |
6.2.8 Assignment Operators
Example 6.30
nawk '$3 == "Chris"{ $3 = "Christian"; print}' datafile southwest SW Christian Foster 2.7 .8 2 18
EXPLANATIONIf the third field ($3) is equal to the string Chris, the action is to assign Christian to the third field ($3) and print the record. The double equal tests its operands for equality, whereas the single equal is used for assignment. |
Example 6.31
nawk '/Derek/{$8 += 12; print $8}' datafile 29
EXPLANATIONIf the regular expression Derek is found, 12 is added and assigned to (+=) the eighth field ($8), and that value is printed. Another way to write this is: $8 = $8 + 12. |
Example 6.32
nawk '{$7 %= 3; print $7}' datafile 0 2 2 1 1 2 0 2 2
EXPLANATIONFor each record, the seventh field ($7) is divided by 3, and the remainder of that division (modulus) is assigned to the seventh field and printed. |
UNIX TOOLS LAB EXERCISE
Lab 4: awk Exercise
Mike Harrington:(510) 548-1278:250:100:175
Christian Dobbins:(408) 538-2358:155:90:201
Susan Dalsass:(206) 654-6279:250:60:50
Archie McNichol:(206) 548-1348:250:100:175
Jody Savage:(206) 548-1278:15:188:150
Guy Quigley:(916) 343-6410:250:100:175
Dan Savage:(406) 298-7744:450:300:275
Nancy McNeil:(206) 548-1278:250:80:75
John Goldenrod:(916) 348-4278:250:100:175
Chet Main:(510) 548-5258:50:95:135
Tom Savage:(408) 926-3456:250:168:200
Elizabeth Stachelin:(916) 440-1763:175:75:300
(Refer to the database called lab4.data on the CD.)
The database above contains the names, phone numbers, and money contributions to the party campaign for the past three months.
| 1: | Print the first and last names of those who contributed over $100 in the first month. |
| 2: | Print the names and phone numbers of those who contributed less than $60 in the first month. |
| 3: | Print those who contributed between $90 and $150 in the third month. |
| 4: | Print those who contributed more than $800 over the three-month period. |
| 5: | Print the names and phone numbers of those with an average monthly contribution greater than $150. |
| 6: | Print the first name of those not in the 916 area code. |
| 7: | Print each record preceded by the number of the record. |
| 8: | Print the name and total contribution of each person. |
| 9: | Add $10 to Elizabeth's second contribution. |
| 10: | Change Nancy McNeil's name to Louise McInnes. |
[1] Alfred V. Aho, Brian W. Kernighan, and Peter J. Weinberger, The AWK Programming Language (Boston: Addison-Wesley, 1988). 1988 Bell Telephone Laboratories, Inc. Reprinted by permission of Pearson Education, Inc.
| CONTENTS | |
EAN: 2147483647
Pages: 18