Pattern Matching Using SAS Regular Expressions (RX) and Perl Regular Expressions (PRX)


Definition of Pattern Matching and Regular Expressions

Pattern matching enables you to search for and extract multiple matching patterns from a character string in one step, as well as to make several substitutions in a string in one step. The DATA step supports two kinds of pattern-matching functions and CALL routines:

  • SAS regular expressions (RX)

  • Perl regular expressions (PRX).

Regular expressions are a pattern language which provides fast tools for parsing large amounts of text. Regular expressions are composed of characters and special characters that are called metacharacters.

The asterisk (*) and the question mark (?) are two examples of metacharacters. The asterisk (*) matches zero or more characters, and the question mark (?) matches one or zero characters. For example, if you issue the ls data*.txt command from a UNIX shell prompt, UNIX displays all the files in the current directory that begin with the letters 'data' and end with the file extension 'txt'.

The asterisk (*) and the question mark (?) are a limited form of regular expressions. Perl regular expressions build on the asterisk and the question mark to make searching more powerful and flexible.

Definition of SAS Regular Expression (RX) Functions and CALL Routines

SAS Regular expression (RX) functions and CALL routines refers to a group of functions and CALL routines that uses SAS' regular expression pattern matching to parse character strings. You can search for character strings that have a specific pattern that you specify, and change a matched substring to a different substring.

SAS regular expressions are part of the character string matching category for functions and CALL routines. For a short description of these functions and CALL routines, see the 'Functions and CALL Routines by Category' on page 270.

Definition of Perl Regular Expression (PRX) Functions and CALL Routines

Perl regular expression (PRX) functions and CALL routines refers to a group of functions and CALL routines that uses a modified version of Perl as a pattern matching language to parse character strings. You can

  • search for a pattern of characters within a string

  • extract a substring from a string

  • search and replace text with other text

  • parse large amounts of text, such as Web logs or other text data, more quickly than with SAS regular expressions.

Perl regular expressions are part of the character string matching category for functions and CALL routines. For a short description of these functions and CALL routines, see the 'Functions and CALL Routines by Category' on page 270.

Benefits of Using Perl Regular Expressions in the DATA Step

Using Perl regular expressions in the DATA step enhances search and replace options in text. You can use Perl regular expressions to

  • validate data

  • replace text

  • extract a substring from a string

  • write Perl debug output to the SAS log.

You can write SAS programs that do not use regular expressions to produce the same results as you do when you use Perl regular expressions. The code without the regular expressions, however, requires more function calls to handle character positions in a string and to manipulate parts of the string.

Perl regular expressions combine most, if not all, of these steps into one expression. The resulting code is

  • less prone to error

  • easier to maintain

  • clearer to read

  • more efficient in terms of improving system performance.

Using Perl Regular Expressions in the DATA Step

License Agreement

The following paragraph complies with sections 3 (a) and 4 (c) of the artistic license:

The PRX functions use a modified version of Perl 5.6.1 to perform regular expression compilation and matching. Perl is compiled into a library for use with SAS. The modified and original Perl 5.6.1 files are freely available from the SAS Web site at http:/ /support.sas.com/rnd/base. Each of the modified files has a comment block at the top of the file describing how and when the file was changed. The executables were given non-standard Perl names . The standard version of Perl can be obtained from http:// www.perl.com.

Only Perl regular expressions are accessible from the PRX functions. Other parts of the Perl language are not accessible. The modified version of Perl regular expressions does not support

  • Perl variables .

  • the regular expression options /c, /g, and /o and the /e option with substitutions.

  • named characters, which use the \N{ name } syntax.

  • the metacharacters \pP, \PP, and \X.

  • executing Perl code within a regular expression. This includes the syntax (?{code}), (??{code}), and (?p{code}).

  • unicode pattern matching.

  • using ?PATTERN?. ? is treated like an ordinary regular expression start and end delimiter .

  • the metacharacter \G.

  • Perl comments between a pattern and replacement text. For example: s{regexp} # perl comment {replacement}.

  • matching backslashes with m/\\\\/. Instead m/\\/ should be used to match a backslash.

Syntax of Perl Regular Expressions

Perl regular expressions are composed of characters and special characters that are called metacharacters. When performing a match, SAS searches a source string for a substring that matches the Perl regular expression that you specify. Using metacharacters enables SAS to perform special actions when searching for a match:

  • If you use the metacharacter \d, SAS matches a digit between 0-9.

  • If you use /\dt/, SAS finds the digits in the string 'Raleigh, NC 27506'.

  • If you use /world/, SAS finds the substring 'world' in the string 'Hello world!'.

The following table contains a short list of Perl regular expression metacharacters that you can use when you build your code. You can find a complete list of metacharacters at http://www.perldoc.com/perl5.6.1/pod/perlre.html.

Metacharacter

Description

\

marks the next character as either a special character, a literal, a back reference, or an octal escape:

  • "n" matches the character "n"

  • "\n" matches a new line character

  • "\\" matches "\"

  • "\(" matches "("

specifies the or condition when you compare alphanumeric strings.

^

matches the position at the beginning of the input string.

$

matches the position at the end of the input string.

*

matches the preceding subexpression zero or more times:

  • zo* matches "z" and "zoo"

  • * is equivalent to {0}

+

matches the preceding subexpression one or more times:

  • "zo+" matches "zo" and "zoo"

  • "zo+" does not match "z"

  • + is equivalent to {1,}

?

matches the preceding subexpression zero or one time:

  • "do(es)?" matches the "do" in "do" or "does"

  • ? is equivalent to {0,1}

{n}

n is a non-negative integer that matches exactly n times:

  • "o{2}" matches the two o's in "food"

  • "o{2}" does not match the "o" in "Bob"

{n,}

n is a non-negative integer that matches n or more times:

  • "o{2,}" matches all the o's in "foooood"

  • "o{2,}" does not match the "o" in "Bob"

  • "o{1,}" is equivalent to "o+"

  • "o{0,}" is equivalent to "o*"

{n,m}

m and n are non-negative integers, where n<=m. They match at least n and at most m times:

  • "o{1,3}" matches the first three o's in "fooooood"

  • "o{0,1}" is equivalent to "o?"

Note: You cannot put a space between the comma and the numbers .

period (.)

matches any single character except newline. To match any character including newline, use a pattern such as "[.\n]".

(pattern)

matches a pattern and creates a capture buffer for the match. To retrieve the position and length of the match that is captured, use CALL PRXPOSN. To retrieve the value of the capture buffer, use the PRXPOSN function. To match parentheses characters, use "\(" or "\)".

xy

matches either x or y:

  • "zfood" matches "z" or "food"

  • "(zf)ood" matches "zood" or "food"

[xyz]

specifies a character set that matches any one of the enclosed characters:

  • "[abc]" matches the "a" in "plain"

[^xyz]

specifies a negative character set that matches any character that is not enclosed:

  • "[^abc]" matches the "p" in "plain"

[a-z]

specifies a range of characters that matches any character in the range:

  • "[a-z]" matches any lowercase alphabetic character in the range

  • "a" through "z"

[^a-z]

specifies a range of characters that does not match any character in the range:

  • "[^a-z]" matches any character that is not in the range "a" through "z"

\b

matches a word boundary (the position between a word and a space):

  • "er\b" matches the "er" in "never"

  • "er\b" does not match the "er" in "verb"

\B

matches a non-word boundary:

  • "er\B" matches the "er" in "verb"

  • "er\B" does not match the "er" in "never"

\d

matches a digit character that is equivalent to [0-9].

\D

matches a non-digit character that is equivalent to [^0-9].

\s

matches any white space character including space, tab, form feed, and so on, and is equivalent to [\f\n\r\t\v].

\S

matches any character that is not a white space character and is equivalent to [^\f\n\r\t\v].

\t

matches a tab character and is equivalent to "\x09".

\w

matches any word character including the underscore and is equivalent to [A-Za-z0-9_].

\W

matches any non-word character and is equivalent to [^A-Za-z0-9_].

\num

matches num, where num is a positive integer. This is a reference back to captured matches:

  • "(.)\1" matches two consecutive identical characters.

Example 1: Validating Data

You can test for a pattern of characters within a string. For example, you can examine a string to determine whether it contains a correctly formatted telephone number. This type of test is called data validation.

The following example validates a list of phone numbers. To be valid, a phone number must have one of the following forms: (XXX) XXX-XXXX or XXX-XXX-XXXX .

 data _null_;    [1]     if _N_ = 1 then        do;           paren = "\([2-9]\d\d\) ?[2-9]\d\d-\d\d\d\d";    [2]           dash = "[2-9]\d\d-[2-9]\d\d-\d\d\d\d";    [3]           regexp = "/("  paren  ")("  dash  ")/";    [4]           retain re;           re = prxparse(regexp);    [5]           if missing(re) then    [6]              do;                 putlog "ERROR: Invalid regexp " regexp;    [7]                 stop;              end;        end;     length first last home business $ 16;     input first last home business;     if ^prxmatch(re, home) then    [8]        putlog "NOTE: Invalid home phone number for " first last home;     if ^prxmatch(re, business) then    [9]        putlog "NOTE: Invalid business phone number for " first last business;     datalines;  Jerome Johnson (919)319-1677 (919)846-2198  Romeo Montague 800-899-2164 360-973-6201  Imani Rashid (508)852-2146 (508)366-9821  Palinor Kent . 919-782-3199  Ruby Archuleta . .  Takei Ito 7042982145 .  Tom Joad 209/963/2764 2099-66-8474  ; 

The following items correspond to the lines that are numbered in the DATA step that is shown above.

[1]  

Create a DATA step.

[2]  

Build a Perl regular expression to identify a phone number that matches (XXX)XXX-XXXX, and assign the variable PAREN to hold the result. Use the following syntax elements to build the Perl regular expression:

\(

matches the open parenthesis in the area code.

[2-9]

matches the digits 2-9. This is the first number in the area code.

\d

matches a digit. This is the second number in the area code.

\d

matches a digit. This is the third number in the area code.

\)

matches the closed parenthesis in the area code.

?

matches the space (which is the preceding subexpression) zero or one time. Spaces are significant in Perl regular expressions. They match a space in the text that you are searching. If a space precedes the question mark metacharacter (as it does in this case), the pattern matches either zero spaces or one space in this position in the phone number.

[3]  

Build a Perl regular expression to identify a phone number that matches XXX-XXX-XXXX, and assign the variable DASH to hold the result.

[4]  

Build a Perl regular expression that concatenates the regular expressions for (XXX)XXX-XXXX and XXX-XXX-XXXX. The concatenation enables you to search for both phone number formats from one regular expression.

The PAREN and DASH regular expressions are placed within parentheses. The bar metacharacter () that is located between PAREN and DASH instructs the compiler to match either pattern. The slashes around the entire pattern tell the compiler where the start and end of the regular expression is located.

[5]  

Pass the Perl regular expression to PRXPARSE and compile the expression. PRXPARSE returns a value to the compiled pattern. Using the value with other Perl regular expression functions and CALL routines enables SAS to perform operations with the compiled Perl regular expression.

[6]  

Use the MISSING function to check whether the regular expression was successfully compiled.

[7]  

Use the PUTLOG statement to write an error message to the SAS log if the regular expression did not compile.

[8]  

Search for a valid home phone number. PRXMATCH uses the value from PRXPARSE along with the search text and returns the position where the regular expression was found in the search text. If there is no match for the home phone number, the PUTLOG statement writes a note to the SAS log.

[9]  

Search for a valid business phone number. PRXMATCH uses the value from PRXPARSE along with the search text and returns the position where the regular expression was found in the search text. If there is no match for the business phone number, the PUTLOG statement writes a note to the SAS log.

The following lines are written to the SAS log:

 NOTE: Invalid home phone number for Palinor Kent  NOTE: Invalid home phone number for Ruby Archuleta  NOTE: Invalid business phone number for Ruby Archuleta  NOTE: Invalid home phone number for Takei Ito 7042982145  NOTE: Invalid business phone number for Takei Ito  NOTE: Invalid home phone number for Tom Joad 209/963/2764  NOTE: Invalid business phone number for Tom Joad 2099-66-8474 
Example 2: Replacing Text

You can use Perl regular expressions to find specific characters within a string. You can then remove the characters or replace them with other characters. In this example, the two occurrences of the less-than character (<) are replaced by &lt; and the two occurrences of the greater-than character (>) are replaced by &gt;.

 data _null_;    [1]     if _N_ = 1 then        do;           retain lt_re gt_re;           lt_re = prxparse('s/</&lt;/');    [2]           gt_re = prxparse('s/>/&gt;/');  [3]           if missing(lt_re) or missing(gt_re) then    [4]              do;                 putlog "ERROR: Invalid regexp.";    [5]                 stop;              end;        end;     input;     call prxchange(lt_re, -1, _infile_);    [6]     call prxchange(gt_re, -1, _infile_);    [7]     put _infile_;     datalines4;  The bracketing construct (...) creates capture buffers. To refer to  the digit'th buffer use \<digit> within the match. Outside the match  use "$" instead of "\". (The \<digit> notation works in certain  circumstances outside the match. See the warning below about  vs   for details.) Referring back to another part of the match is called  backreference.  ;;;; 

The following items correspond to the numbered lines in the DATA step that is shown above.

[1]  

Create a DATA step.

[2]  

Use metacharacters to create a substitution syntax for a Perl regular expression, and compile the expression. The substitution syntax specifies that a less-than character (<) in the input is replaced by the value & lt; in the output.

[3]  

Use metacharacters to create a substitution syntax for a Perl regular expression, and compile the expression. The substitution syntax specifies that a greater-than character (>) in the input is replaced by the value & gt; in the output.

[4]  

Use the MISSING function to check whether the Perl regular expression compiled without error.

[5]  

Use the PUTLOG statement to write an error message to the SAS log if neither of the regular expressions was found.

[6]  

Call the PRXCHANGE routine. Pass the LT_RE pattern-id , and search for and replace all matching patterns. Put the results in _INFILE_ and write the observation to the SAS log.

[7]  

Call the PRXCHANGE routine. Pass the GT_RE pattern-id , and search for and replace all matching patterns. Put the results in _INFILE_ and write the observation to the SAS log.

The following lines are written to the SAS log:

 The bracketing construct (...) creates capture buffers. To refer to  the digit'th buffer use \&lt;digit&gt; within the match. Outside the match  use "$" instead of "\". (The \&lt;digit&gt; notation works in certain  circumstances outside the match. See the warning below about  vs   for details.) Referring back to another part of the match is called a  backreference. 
Example 3: Extracting a Substring from a String

You can use Perl regular expressions to find and easily extract text from a string. In this example, the DATA step creates a subset of North Carolina business phone numbers. The program extracts the area code and checks it against a list of area codes for North Carolina.

 data _null_;    [1]     if _N_ = 1 then        do;           paren = "\(([2   9]\d\d)\) ?[2   9]\d\d   \d\d\d\d";    [2]           dash = "([2   9]\d\d)   [2   9]\d\d-\d\d\d\d";    [3]           regexp = "/("  paren  ")("  dash  ")/";    [4]           retain re;           re = prxparse(regexp);    [5]           if missing(re) then    [6]              do;                 putlog "ERROR: Invalid regexp " regexp;    [7]                 stop;              end;           retain areacode_re;           areacode_re = prxparse("/828336704910919252/");    [8]           if missing(areacode_re) then              do;                 putlog "ERROR: Invalid area code regexp";                 stop;              end;        end;     length first last home business $ 16;     length areacode $ 3;     input first last home business;     if ^prxmatch(re, home) then        putlog "NOTE: Invalid home phone number for " first last home;     if prxmatch(re, business) then    [9]        do;           which_format = prxparen(re);    [10]           call prxposn(re, which_format, pos, len);    [11]           areacode = substr(business, pos, len);           if prxmatch(areacode_re, areacode) then    [12]              put "In North Carolina: " first last business;        end;        else           putlog "NOTE: Invalid business phone number for " first last business;     datalines;  Jerome Johnson (919)319-1677 (919)846-2198  Romeo Montague 800-899-2164 360-973-6201  Imani Rashid (508)852-2146 (508)366-9821  Palinor Kent 704-782-4673 704-782-3199  Ruby Archuleta 905-384-2839 905-328-3892  Takei Ito 704-298-2145 704-298-4738  Tom Joad 515-372-4829 515-389-2838  ; 

The following items correspond to the numbered lines in the DATA step that is shown above.

[1]  

Create a DATA step.

[2]  

Build a Perl regular expression to identify a phone number that matches (XXX)XXX-XXXX, and assign the variable PAREN to hold the result. Use the following syntax elements to build the Perl regular expression:

\(

matches the open parenthesis in the area code. The open parenthesis marks the start of the submatch.

[2-9]

matches the digits 2-9. This is the first number in the area code.

\d

matches a digit. This is the second number in the area code.

\d

matches a digit. This is the third number in the area code.

\)

matches the closed parenthesis in the area code. The closed parenthesis marks the end of the submatch.

?

matches the space (which is the preceding subexpression) zero or one time. Spaces are significant in Perl regular expressions. They match a space in the text that you are searching. If a space precedes the question mark metacharacter (as it does in this case), the pattern matches either zero spaces or one space in this position in the phone number.

[3]  

Build a Perl regular expression to identify a phone number that matches XXX-XXX-XXXX, and assign the variable DASH to hold the result.

[4]  

Build a Perl regular expression that concatenates the regular expressions for (XXX)XXX-XXXX and XXX-XXX-XXXX. The concatenation enables you to search for both phone number formats from one regular expression.

The PAREN and DASH regular expressions are placed within parentheses. The bar metacharacter () that is located between PAREN and DASH instructs the compiler to match either pattern. The slashes around the entire pattern tell the compiler where the start and end of the regular expression is located.

[5]  

Pass the Perl regular expression to PRXPARSE and compile the expression. PRXPARSE returns a value to the compiled pattern. Using the value with other Perl regular expression functions and CALL routines enables SAS to perform operations with the compiled Perl regular expression.

[6]  

Use the MISSING function to check whether the Perl regular expression compiled without error.

[7]  

Use the PUTLOG statement to write an error message to the SAS log if the regular expression did not compile.

[8]  

Compile a Perl regular expression that searches a string for a valid North Carolina area code.

[9]  

Search for a valid business phone number.

[10]  

Use the PRXPAREN function to determine which submatch to use. PRXPAREN returns the last submatch that was matched. If an area code matches the form (XXX), PRXPAREN returns the value 2. If an area code matches the form XXX, PRXPAREN returns the value 4.

[11]  

Call the PRXPOSN routine to retrieve the position and length of the submatch.

[12]  

Use the PRXMATCH function to determine whether the area code is a valid North Carolina area code, and write the observation to the log.

The following lines are written to the SAS log:

 In North Carolina: Jerome Johnson (919)846-2198  In North Carolina: Palinor Kent 704-782-3199  In North Carolina: Takei Ito 704-298-4738 

Writing Perl Debug Output to the SAS Log

The DATA step provides debugging support with the CALL PRXDEBUG routine. CALL PRXDEBUG enables you to turn on and off Perl debug output messages that are sent to the SAS log.

The following example writes Perl debug output to the SAS log.

 data _null_;        /* CALL PRXDEBUG(1) turns on Perl debug output. */     call prxdebug(1);     putlog 'PRXPARSE: ';     re = prxparse('/[bc]d(ef*g)+h[ij]k$/');     putlog 'PRXMATCH: ';     pos = prxmatch(re, 'abcdefg_gh_');        /* CALL PRXDEBUG(0) turns off Perl debug output. */     call prxdebug(0);  run; 

SAS writes the following output to the log.

Output 4.3: SAS Debugging Output
start example
 PRXPARSE:  Compiling REx '[bc]d(ef*g)+h[ij]k$'  size 41 first at 1  rarest char g at 0  rarest char d at 0     1: ANYOF[bc](10)    10: EXACT <d>(12)    12: CURLYX[0] {1,32767}(26)    14:   OPEN1(16)    16:     EXACT <e>(18)    18:     STAR(21)    19:       EXACT <f>(0)    21:     EXACT <g>(23)    23:    CLOSE1(25)    25:    WHILEM[1/1](0)    26: NOTHING(27)    27: EXACT <h>(29)    29: ANYOF[ij](38)    38: EXACT <k>(40)    40: EOL(41)    41: END(0)  anchored 'de' at 1 floating 'gh' at 3..2147483647 (checking floating) stclass  'ANYOF[bc]' minlen 7  PRXMATCH:  Guessing start of match, REx '[bc]d(ef*g)+h[ij]k$' against 'abcdefg_gh_'...  Did not find floating substr 'gh'...  Match rejected by optimizer 
end example
 

For a detailed explanation of Perl debug output, see 'CALL PRXDEBUG Routine' on page 356.




SAS 9.1 Language Reference Dictionary, Volumes 1, 2 and 3
SAS 9.1 Language Reference Dictionary, Volumes 1, 2 and 3
ISBN: N/A
EAN: N/A
Year: 2004
Pages: 704

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net