4.12. Transforming FilesThere are several utilities that perform a transformation on the contents of a file, including the following:
The next few subsections contain a description of each utility. 4.12.1. Compressing Files: gzip and gunzipThe GNU file compression utility is called gzip. The utility to uncompress the file, as you might guess, is gunzip. Figure 4-24 resolves these utilities.
This is how to use gzip and gunzip: $ ls -lG palindrome.c reverse.c -rw-r--r-- 1 ables 224 Jul 1 14:14 palindrome.c -rw-r--r-- 1 ables 266 Jul 1 14:14 reverse.c $ gzip -v palindrome.c reverse.c palindrome.c: 34.3% -- replaced with palindrome.c.gz reverse.c: 39.4% -- replaced with reverse.c.gz $ ls -lG palindrome.c.gz reverse.c.gz -rw-r--r-- 1 ables 178 Jul 1 14:14 palindrome.c.gz -rw-r--r-- 1 ables 189 Jul 1 14:14 reverse.c.gz $ gunzip -v *.gz palindrome.c.gz: 34.3% -- replaced with palindrome.c reverse.c.gz: 39.4% -- replaced with reverse.c $ ls -lG palindrome.c reverse.c -rw-r--r-- 1 ables 224 Jul 1 14:14 palindrome.c -rw-r--r-- 1 ables 266 Jul 1 14:14 reverse.c $ _ 4.12.2. Stream Editing: sedThe stream editor utility sed scans one or more files and performs an editing action on all of the lines that match a particular condition. The actions and conditions may be stored in a sed script. sed is useful for performing simple repetitive editing tasks. sed is a fairly comprehensive utility. Because of this, I've only attempted to describe the main features and options of sed; however, the material in this section will allow you to write a good number of useful sed scripts. Figure 4-25 gives a synopsis of sed.
4.12.2.1. sed commandsA sed script is a list of one or more of the commands shown in Figure 4-26, separated by newlines.
The following rules apply:
4.12.2.2. Substituting TextIn the following example, I supplied the sed script on the command line. The script inserted a couple of spaces at the start of every line. $ cat arms ...look at the original file. People just like me, Are all around the world, Waiting for the loved ones that they need. And with my heart, I make a simple wish, Plain enough for anyone to see. $ sed 's/^/ /' arms > arms.indent ...indent the file. $ cat arms.indent ...look at the result. People just like me, Are all around the world, Waiting for the loved ones that they need. And with my heart, I make a simple wish, Plain enough for anyone to see. $ _ To remove all of the leading spaces from a file, use the substitute operator in the reverse fashion: $ sed 's/^ *//' arms.indent ...remove leading spaces. People just like me, Are all around the world, Waiting for the loved ones that they need. And with my heart, I make a simple wish, Plain enough for anyone to see. $ _ 4.12.2.3. Deleting TextThe next example illustrates a script that deleted all of the lines that contained the character 'a': $ sed '/a/d' arms ...remove all lines containing an 'a'. People just like me, $ _ To delete only those lines that contain the word 'a', I surrounded the regular expression by escaped angled brackets (\< and \>): $ sed '/\<a\>/d' arms People just like me, Are all around the world, Waiting for the loved ones that they need. And with my heart, Plain enough for anyone to see. $ _ 4.12.2.4. Inserting TextIn the next example, I inserted a copyright notice at the top of the file by using the insert command. Notice that I stored the sed script in a file and executed it by using the -f option. $ cat sed5 ...look at the sed script. 1i\ Copyright 1992, 1998, & 2002 by Graham Glass\ All rights reserved\ $ sed -f sed5 arms ...insert a copyright notice. Copyright 1992, 1998, & 2002 by Graham Glass All rights reserved People just like me, Are all around the world, Waiting for the loved ones that they need. And with my heart, I make a simple wish, Plain enough for anyone to see. $ _ 4.12.2.5. Replacing TextTo replace lines, use the change function. In the following example, I replaced the group of lines 1..3 with a censored message: $ cat sed6 ...list the sed script. 1,3c\ Lines 1-3 are censored. $ sed -f sed6 arms ...execute the script. Lines 1-3 are censored. And with my heart, I make a simple wish, Plain enough for anyone to see. $ _ To replace individual lines with a message rather than an entire group, supply a separate command for each line: $ cat sed7 ...list the sed script. 1c\ Line 1 is censored. 2c\ Line 2 is censored. 3c\ Line 3 is censored. $ sed -f sed7 arms ...execute the script. Line 1 is censored. Line 2 is censored. Line 3 is censored. And with my heart, I make a simple wish, Plain enough for anyone to see. $ _ 4.12.2.6. Inserting FilesIn the following example, I inserted a message after the last line of the file: $ cat insert ...list the file to be inserted. The End $ sed '$r insert' arms ...execute the script. People just like me, Are all around the world, Waiting for the loved ones that they need. And with my heart, I make a simple wish, Plain enough for anyone to see. The End $ _ 4.12.2.7. Multiple sed CommandsThis last example illustrates the use of multiple sed commands. I inserted a "<<" sequence at the start of each line, and appended a ">>" sequence to the end of each line: $ sed -e 's/^/<< /' -e 's/$/ >>/' arms << People just like me, >> << Are all around the world, >> << Waiting for the loved ones that they need. >> << And with my heart, >> << I make a simple wish, >> << Plain enough for anyone to see. >> $ _ 4.12.3. Translating Characters: trThe tr utility maps the characters in a file from one character set to another (Figure 4-27).
Here are some examples of tr in action: $ cat go.cart ...list the sample input file. go cart racing $ tr a-z A-Z < go.cart ...translate lower to uppercase. GO CART RACING $ tr a-c D-E < go.cart ...replace abc by DEE. go EDrt rDEing $ tr -c a X < go.cart ...replace every non-a with X. XXXXaXXXXXaXXXXX$ ...even last newline is replaced. $ tr -c a-z '\012' < go.cart ...replace non-alphas with go ...ASCII 12 (newline). 4.12.4. Converting Underline Sequences: ulThe ul utility transforms a file that contains underlining characters so that it appears correctly on a particular terminal type. This is useful with commands like man that generate underlined text. Figure 4-28 describes how ul works.
For example, let's say that you want to use the man utility to produce a document that you wish to print on a simple ASCII-only printer. The man utility generates underline characters for your current terminal, so to filter the output so that it's suitable for a dumb printer, pipe the output of man through ul with the "dumb" terminal setting. Here's an example: $ man who | ul -tdumb > man.txt $ head man.txt ...look at the first 10 lines. WHO(1) User Commands WHO(1) NAME who - show who is logged on SYNOPSIS who [OPTION]... [ FILE | ARG1 ARG2 ] DESCRIPTION $ _ |