Project76.Use Bash Parameter Expansion

Project 76. Use Bash Parameter Expansion

"How do I perform string manipulation in Bash?"

This project covers the topic of parameter expansion. Parameter expansion is most often used to expand variables and arguments by means of the familiar $ notation: $length or $1. Parameter expansion, however, is more than simply the expansion of a variable or an argument into its value; it also involves manipulation of the value, such as pattern replacement and default initialization.

Basic Expansion

By now, the basics of parameter expansion are probably familiar. We give a variable a value.

$ title="101 Projects"

Later, we expand the variable to expose its value.

$ echo $title 101 Projects

Bash uses the terms parameter and parameter expansion not only for variables, but also for arguments passed to a script or function. Where Bash refers specifically to arguments such as $1, it uses the terms positional parameter and positional parameter expansion.

Positional parameter expansion works as follows: The first argument passed to a Bash script or function is available in the variable $1; the second, in $2; and the nth, in the variable $n. The special expansion $* expands to a list of all arguments passed, and $# expands to the number of arguments passed.

The special expansion $@ is useful when enclosed in double quotes. To illustrate this, suppose that we pass two arguments to a script, both of which contain spaces.

$ ./tst "param one" "param two"

Whereas both $* and $@ expand to four items "param", "one", "param", and "two" the quoted versions behave differently.

• "$*" expands to one item: "param one param two".

• "$@" expands more usefully to two items: "param one" and "param two".

Tip

The special parameter $$ expands to the process ID of the shell. It provides an easy way to generate a uniquely named temporary file in a script. For example:

echo "Test" > $0$$.tmp

Complex Expansion

More complex parameter expansion lets us assign a default value to a parameter or change its value by cutting and replacing portions of its contents. Complex expansion uses the notation

${parameter-name<expansion-type>}

Tip

Use the following technique to embed parameter expansion in text that might otherwise be confused with the name of the parameter. To expand an abbreviated day name, where day="Tues", to Tuesday, we type

${day}day

Set Default Values

Suppose that we have a script that takes one optional argument. We want to assign the value of argument (parameter) 1 to the variable level, but only if parameter 1 is given a value. If no argument is given when the script is called, we want the value of level set to equal the text string normal. We can take the conventional, long-handed approach and use an if statement.

if [ "$1" = "" ]; then   level="normal" else   level="$1" fi

Better, we can use the functionally equivalent complex expansion.

level=${1:-"normal"}

An alternate approach also initializes a parameter to a default value if the parameter is null. This technique doesn't apply to positional parameters (arguments), only to parameters (variables).

new_level=${level:="normal"}

A third method causes a script to exit if a compulsory argument is not supplied. The following expansion displays an error message and aborts the script if no value is passed to $1; otherwise, it assigns the passed value to the variable level.

$ cat tst level=${1:?Please supply a value}

If we run the script and fail to supply an argument, it displays the error message and aborts.

$ ./tst ./tst: line 1: 1: Please supply a value

Slice Strings

Bash can expand a string variable (a variable that contains text) to a fragment of the text. This process is called slicing. We might expand the variable $string by taking a slice starting from the 8th character (that's character 7, because the first character is 0) and returning the next 6 characters.

$ echo $hi Hello, please slice me $ echo ${hi:7:6} please

Here's a practical application of slicinga script that checks each of its arguments to see whether it's an option flag (an argument that starts with a - character). This technique is commonly used in connection with scripts for which option flags have been defined. Our example is part of a script that has two legal option flags: -p, which prompts the user for a password; and -v, which sets verbose mode. Any other option will cause the script to exit and report an error to the user.

We want to extract and compare the first character of each argument by slicing a substring one character long, beginning with the first character (character position 0). The following script uses the expansion ${1:0:1} to slice $1, where :0 specifies the start position and :1 specifies the number of characters to extract.

while [ "$1" != "" ]; do   if [ "${1:0:1}" = "-" ]; then     case "$1" in       "-p") stty -echo; read -p "Password:" password             stty echo; echo;;       "-v") verbose=yes;;       *) echo "invalid option $1"; exit     esac   else     echo "Here we process non-option arguments..."   fi   shift done

Tip

Use the command stty -echo to stop the user's input from being echoed to the screen as she types. This is useful when a password or other such sensitive information needs to be input. The command stty echo puts things back to normal.

The script then tests to see whether the character is a dash (-). If so, it issues instructions depending on whether the dash is followed by p, v, or any other character (denoted by *); if not, it writes a message to the screen: Here we process non-option arguments...

The script demonstrates a few other useful techniques, too. It loops, processing each argument in turn. At the end of the loop, it uses the shift command to shift all positional parameters down one place, so $n becomes $n-1, $2 becomes $1, and $1 (which we just processed) drops off the end.

Tip

Use the special expansion ${#var} to return the length of a parameter, in characters.

Top and Tail Strings

Bash provides a way to remove the head or tail of a string. We'll illustrate a few useful techniques on a Unix pathname written to the variable fullpath.

$ fullpath="/usr/local/bin/backup.user.sh"

In our first example, we remove the head of the string by specifying a parameter expansion in the form

${parameter##word}

The character combination ## instructs Bash to remove all characters, starting from the left (the start) of the specified parameter, that match word. We'll specify word as */, where * is matched by zero or more occurrences of any character and / represents itself. The star symbol is interpreted exactly as it would be for shell globbing. Our pattern, therefore, matches any string of characters from the start of the string, ending with /.

Learn More

Refer to Project 11 for a full explanation of globbing.

$ echo ${fullpath##*/} backup.user.sh

You'll notice that the pattern matched the longest string it could, up to the last /. TRy the same command, but type a single # to match the shortest stringup to the first /.

To extract the file extension, we type

$ echo ${fullpath##*.} sh

To remove all characters starting from the right (the end) of the string instead of the left, specify % instead of #. The same convention of % versus %% applies. To remove the extension part (.sh) from fullpath, we require the shortest match, starting from the right (%), for the word.*.

$ echo ${fullpath%.*} /usr/local/bin/backup.user

Here's an example script that splits a pathname into its component directories and the filename. We match the shortest string from the left and the longest from the right. Contrast this with the previous two examples. If you can figure out how it works, you've got topping and tailing down to a tee.

$ cat tst #!/bin/bash pathname=${1}"/" while [ ! -z ${pathname#*/} ]; do   pathname=${pathname#*/}   echo ${pathname%%/*} done

Let's try it on a pathname.

$ ./tst /usr/local/bin/command usr local bin command

Search and Replace

Bash gives us a means to search a parameter for a pattern, replacing each occurrence of that pattern with a new string. The syntax is

${parameter/match-pattern/replace-pattern}

Here are some examples in which we use the echo command to demonstrate search and replace.

Search for the first occurrence of Hello, and replace it with Goodbye.

$ message="Hello, Hello World" $ echo ${message/Hello/Goodbye} Goodbye, Hello World

Only the first occurrence of Hello is replaced: To replace all occurrences, specify a double slash instead if a single slash.

$ echo ${message//Hello/Goodbye} Goodbye, Goodbye World

To match a pattern that must be at the very start of the string, introduce the search-and-replace expression with the character sequence /#.

$ echo ${message/#Hello/Goodbye} Goodbye, Hello World

Similarly, to specify that the pattern must be at the end of the string, introduce the search-and-replace expression with the character sequence /%.

$ echo ${message/%World/Earth} Hello, Hello Earth