8.11. The itertools Module The iterools module offers many powerful, high-performance building blocks to build or manipulate iterator objects. Manipulating iterators is often better than manipulating lists thanks to iterators' intrinsic "lazy evaluation" approach: items of an iterator are produced one at a time, as needed, while all the items of a list (or other sequence) must exist in memory at the same time (this "lazy" approach even makes it feasible to build and manipulate unbounded iterators, while all lists must always have finite numbers of items). This section documents the most frequently used attributes of module itertools; each of them is an iterator type, which you can call to get an instance of the type in question. chain | chain (*iterables )
Builds and returns an iterator whose items are all those from the first iterable passed, followed by all those from the second iterable passed, and so on until the end of the last iterable passed, just like the generator expression: (item for iterable in iterables for item in iterable)
| count | count(firstval=0)
Builds and returns an unbounded iterator whose items are consecutive integers starting from firstval, just like the generator: def count(firstval=0): while True: yield firstval firstval += 1
| cycle | count(iterable)
Builds and returns an unbounded iterator whose items are the items of iterable, endlessly repeating the items from the beginning each time the cycle reaches the end, just like the generator: def cycle(iterable): buffer = [] for item in iterable: yield item buffer.append(item) while True: for item in buffer: yield item
| ifilter | ifilter(func,iterable)
Builds and returns an iterator whose items are those items of iterable for which func is true, just like the generator expression: (item for item in iterable if func(item))
func can be any callable object that accepts a single argument, or None. When func is None, ifilter tests for true items, just like the generator expression: (item for item in iterable if item)
| imap | imap(func,*iterables)
Builds and returns an iterator whose items are the results of func, called with one corresponding argument from each of the iterables; stops when the shortest of the iterables is exhausted. Just like the generator. def imap(func,*iterables): next_items = [iter(x).next for x in iterables] while True: yield func(*(next( ) for next in next_items))
| islice | islice(iterable[,start], stop[,step])
Builds and returns an iterator whose items are items of iterable, skipping the first start ones (default is 0) until the stopth one excluded, and advancing by steps of step (default is 1) at a time. All arguments must be nonnegative integers, and step must be >0. Apart from checks and optionality of the arguments, just like the generator. def islice(iterable,start,stop,step=1): en = enumerate(iterable) for n, item in en: if n>=start: break while n<stop: yield item for x in range(step): n, item = en.next( )
| izip | izip(*iterables)
Builds and returns an iterator whose items are tuples with one corresponding item from each of the iterables; stops when the shortest of the iterables is exhausted. Just like imap(tuple,*iterables). | repeat | repeat(item[,times])
Builds and returns an iterator whose times items are all the object item, just like the generator expression: (item for x in xrange(times))
When times is absent, the iterator is unbound, with a potentially infinite number of items, which are all the object item. Just like the generator. def repeat_unbounded(item): while True: yield item
| tee | tee(iterable,n=2)
Builds and returns a tuple of n independent iterators, each of whom has items that are the same as the items of iterable. While the returned iterators are independent from each other, they are not independent from iterable; therefore, you must avoid altering object iterable in any way, as long as you're still using any of the returned iterators (this caveat is roughly the same as for the result of iter(iterable)). New in 2.4. Python's online documentation has abundant examples and explanations about each of these types of module itertools, as well as about other types of itertools that I do not cover in this book. One surprising fact is the sheer speed of itertools types. To take a trivial example, consider repeating some action 10 times. On my laptop: for x in itertools.repeat(0, 10): pass
turns out to be about 10 percent faster than the second-fastest alternative: for x in xrange(10): pass
and almost twice as fast as using range instead of xrange in this case. |
|
