Section 8.12. List Comprehensions


8.12. List Comprehensions

List comprehensions (or "list comps" for short) come to us from the functional programming language Haskell. They are an extremely valuable, simple, and flexible utility tool that helps us create lists on the fly. They were added to Python in version 2.0.

Up ahead in Functions (Chapter 11), we will be discussing long-time Python functional programming features like lambda, map(), and filter(). They have been around in Python for quite a while, but with list comprehensions, they have simplified their use to only requiring a list comp instead. map() is a function that applies an operation to list members, and filter() filters out list members based on a conditional expression. Finally, lambda allows you to create one-line function objects on the fly. It is not important that you learn them now, but you will see examples of them in this section because we are discussing the merits of list comps. Let us take a look at the simpler list comprehension syntax first:

[expr for iter_var in iterable]


The core of this statement is the for loop, which iterates over each item of iterable. The prefixed expr is applied for each member of the sequence, and the resulting values comprise the list that the expression yields. The iteration variable need not be part of the expression.

Here is a sneak preview of some code from Chapter 11. It has a lambda function that squares the members of a sequence:

>>> map(lambda x: x ** 2, range(6)) [0, 1, 4, 9, 16, 25]


We can replace this code with the following list comprehension statement:

>>> [x ** 2 for x in range(6)] [0, 1, 4, 9, 16, 25]


In the new statement, only one function call (range()) is made (as opposed to threerange(), map(), and the lambda function). You may also use parentheses around the expression if [(x ** 2) for x in range(6)] is easier for you to read. This syntax for list comprehensions can be a substitute for and is more efficient than using the map() built-in function along with lambda.

List comprehensions also support an extended syntax with the if statement:

[expr for iter_var in iterable if cond_expr]


This syntax will filter or "capture" sequence members only if they meet the condition provided for in the cond_expr conditional expression during iteration.

Recall the following odd() function below, which determines whether a numeric argument is odd or even (returning 1 for odd numbers and 0 for even numbers):

def odd(n):     return n % 2


We were able to take the core operation from this function, and use it with filter() and lambda to obtain the set of odd numbers from a sequence:

>>> seq = [11, 10, 9, 9, 10, 10, 9, 8, 23, 9, 7, 18, 12, 11, 12] >>> filter(lambda x: x % 2, seq) [11, 9, 9, 9, 23, 9, 7, 11]


As in the previous example, we can bypass the use of filter() and lambda to obtain the desired set of numbers with list comprehensions:

>>> [x for x in seq if x % 2] [11, 9, 9, 9, 23, 9, 7, 11]


Let us end this section with a few more practical examples.

Matrix Example

Do you want to iterate through a matrix of three rows and five columns? It is as easy as:

>>> [(x+1,y+1) for x in range(3) for y in range(5)] [(1, 1), (1, 2), (1, 3), (1, 4), (1, 5), (2, 1), (2, 2), (2,  3), (2, 4), (2, 5), (3, 1), (3, 2), (3, 3), (3, 4), (3, 5)]


Disk File Example

Now let us say we have the following data file and want to count the total number of non-whitespace characters in the file hhga.txt:

 And the Lord spake, saying, "First shalt thou take out the Holy Pin. Then shalt thou count to three, no more, no less. Three shall be the number thou shalt count, and the number of the counting shall be three. Four shalt thou not count, neither count thou two, excepting that thou then proceed to three. Five is right out. Once the number three, being the third number, be reached, then lobbest thou thy Holy Hand Grenade of Antioch towards thy foe, who, being naughty in My sight, shall snuff it." 

We know that we can iterate through each line with for line in data, but more than that, we can also go and split each line up into words, and we can sum up the number of words to get a total like this:

>>> f = open('hhga.txt', 'r') >>> len([word for line in f for word in line.split()]) 91


Let us get a quick total file size:

   import os    >>> os.stat('hhga.txt').st_size    499L


Assuming that there is at least one whitespace character in the file, we know that there are fewer than 499 non-whitespace characters in the file. We can sum up the length of each word to arrive at our total:

>>> f.seek(0) >>> sum([len(word) for line in f for word in line.split()]) 408


Note we have to rewind back to the beginning of the file each time through because the iterator exhausts it. But wow, a non-obfuscated one-liner now does something that used to take many lines of code to accomplish!

As you can see, list comps support multiple nested for loops and more than one if clause. The full syntax can be found in the official documentation. You can also read more about list comprehensions in PEP 202.



Core Python Programming
Core Python Programming (2nd Edition)
ISBN: 0132269937
EAN: 2147483647
Year: 2004
Pages: 334
Authors: Wesley J Chun

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net