Introduction

The sed utility is a very useful utility, but it can also be one of the most cryptic. sed is a stream editor that alters text that flows through it using a number of types of transformations. sed does not alter the original file provided to it but instead provides the transformed text to stdout . sed is one of the oldest tools in UNIX, written in the early 1970s by Lee McMahon. sed operations are stream operations (as the name implies) where simple scripts provide filtering and transformation of text. Consider Figure 21.1, which shows a simple example of sed use with a graphical illustration.

Figure 21.1: The sed model as a text filter.

Let s pick apart this sed command and look at it a little further to understand how sed works for this case (the substitute command, s ).

The full sed command (for the case shown in Figure 21.1) is illustrated in Figure 21.2.

Figure 21.2: Anatomy of a simple sed invocation.

We invoke sed using the sed command and then provide a script to use on the input stream (defined by the third parameter,  file.txt ). The sed script represents a substitute transformation where a pattern is searched in the input stream (pattern string) and, when found, is replaced by the replacement string. Note that the command ( s ), pattern search string, and replacement string are all delimited by the / character. This character can be used for search and replacement; we ll look at how this is done shortly.

We end our sed script with a g to indicate that we wish to perform the search and replace over the entire stream. If g were not provided, only the first occurrence would be replaced. We could instead replace g with a number, which would indicate the particular occurrence to change. For example, ending with /2 would indicate to change only the second occurrence.

Anatomy of a Simple Script

To complete our introduction, let s look at this script in action. We ll use the file shown in Listing 21.1 to illustrate our first simple script.

Listing 21.1: Sample File for sed Illustration (on the CD-ROM at ./source/ch21/_file.txt )

  1:  This is a sample text string which is going to be used.  2:   3:  This is one more string that can be used.  4:   5:  Finally, this is the last string in the test set. 

 

Now using our previous sed script on Listing 21.1, we see the following:

 # sed 's/is/IS/g' file.txt     ThIS IS a sample text string which IS going to be used.          ThIS IS one more string that can be used.          Finally, thIS IS the last string in the test set.          # 

Each occurrence of is is replaced with IS over the entire file. Consider now what happens if we omit the final g :

 # sed 's/is/IS/' file.txt     ThIS is a sample text string which is going to be used.          ThIS is one more string that can be used.          Finally, thIS is the last string in the test set.          # 

Note that instead of replacing only the first occurrence in the file, it replaced only the first occurrence in each line.

We can create two transforms by using the -e option (which is implied in our earlier example). Consider this example:

 # sed -e 's/is/IS/g' -e 's/IS/is/g' file.txt     This is a sample text string which is going to be used.          This is one more string that can be used.          Finally, this is the last string in the test set.          # 

In this case, we convert is to IS , but then we follow this with the reverse transform. The result is the same as the original file. The key here is the use of the -e (script or expression) to provide multiple transformations on the same input text.

Note  

sed makes only one pass over the input file and is therefore very efficient. sed operates by reading a single line from stdin , executes the series of editing commands on it ( potentially a series of commands), and then writes the output to stdout .

Now that we have a quick introduction to sed , let s look at some of sed s other capabilities. The sed utility can be quite broad and complex. Therefore, we ll focus in this chapter on the more useful aspects of sed .