Author avatar


Python Tricks - Iterable - Part 1


  • Nov 23, 2019
  • 8 Min read
  • Nov 23, 2019
  • 8 Min read


Editor's note: This guide is part of a series on useful Python tricks. Read more about the series and find links the other guides here.

In this guide, we will begin learning iterable tricks in Python, including advanced skills related to iterables. Each Python trick is explained in two steps:

  1. A brief description of the concept and an interactive code example
  2. Some inspiring examples to enlighten you

Construct Iterator

Here are some refined definitions of iterators picked up from Hackaday:

An iterator is an object which represents a stream of data. Technically, it is an object that has an __iter__ and a __next__ method.

An iterable is something which is able to iterate. In practice, an iterable is an object which has an __iter__ method, which returns an iterator. strings, lists, files, and dictionaries are all examples of iterables.

Inspiring Example

This example is a very creative usage for an iterator. in function loops through the same iterator to make sure every character is checked just once sequentially.

1"""construct iterator to make sure check every char only once by 'in' """
2# L392: check if s is subsequence of t. s = "abc", t = "ahbgdc", return true.
3def is_subsequence(s: str, t: str) -> bool:
4    t = iter(t)
5    return all(c in t for c in s)

Construct Generator

Generators provide an easy, built-in way to create instances of iterators. We can build a generator by calling a function that has one or more yield expressions. We can also construct a generator with agenerator expression in a comprehension way, which we will discuss below.

Generators do not store all the values in memory; they generate values on the fly. This is the lazy evaluation technique we discussed in the Python Tricks - Basic guide.

Inspiring Examples

To simplify the logic, we can construct the result as a generator style.

1"""generate all combinations of coins which sum to target amount"""
2def get_coin_changes(coins: List[int], amount[int]) -> List[List[int]]:
3    for i, coin in enumerate(coins):
4        if coin == amount:
5            yield (coin,)
6        elif coin < amount:
7            yield from ((coin,) + x for x in change(coins[i:], amount - coin))
9"""generate all combinations of well-formed n pairs of parentheses"""
10# L22: generate all combinations of well-formed n pairs of parentheses.
11def generate_parenthesis(n: int) -> List[str]:
12    def generate(p, left, right):
13        if 0 <= left <= right:
14            if not right:
15                yield p
16            for q in generate(p + '(', left-1, right): yield q
17            for q in generate(p + ')', left, right-1): yield q
18    return list(generate('', n, n))


The comprehension technique provides a short and concise way to construct new sequences (such as lists, set, dictionary, etc.) based on existing sequences.

Python provides the following four types of comprehensions:

1"""list comprehension"""
2odds_list = [x for x in range(20) if x % 2]
5odds_generator = (x for x in range(20) if x % 2)
7"""set comprehension"""
8odds_set = {x for x in range(20) if x % 2}
10"""dict comprehension"""
11square_dict = {x: x**2 for x in range(10)}

Inspiring Examples

The following examples demonstrate the advanced skills of constructing a list comprehension. They also show various application scenarios.

1"""bfs in list comprehension way"""
2# L515: find the largest value in each row of a binary tree
3def find_largest_in_binary_tree_rows(root: TreeNode) -> List[int]:
4    maxes, row = [], [root]
5    while any(row):
6        maxes.append(max(node.val for node in row))
7        row = [kid for node in row for kid in (node.left, node.right) if kid]
8    return maxes
10"""rotate image by list comprehension"""
11# L48: rotate the image by 90 degrees (clockwise)
12def rotate_image_90_degrees(matrix: List[List[int]]) -> None:
13    matrix[:] = [[row[i] for row in matrix[::-1]] for i in range(len(A))]
15"""generate combinations by list comprehension recursively"""
16# L77: Given two integers n and k, return all possible combinations of k numbers out of 1 ... n.
17def combine(n: int, k: int) -> List[List[int]]:
18    if k == 0:
19        return [[]]
20    # add last element
21    return [pre + [i] for i in range(k, n + 1) for pre in combine(i - 1, k - 1)]
23"""generate binary trees by list comprehension recursively"""
24# given an integer n, generate all structurally unique BST's (binary search trees) that store values 1 ... n.
25def generate_binary_search_trees(last: int, first: int=1) -> List[TreeNode]:
26    def node(val, left, right):
27        node = TreeNode(val)
28        node.left = left
29        node.right = right
30        return node
31    return [node(root, left, right) 
32            for root in range(first, last+1)
33            for left in generate_binary_search_trees(root-1, first)
34            for right in generate_binary_search_trees(last, root+1)] or [None]


lambda function, also called anonymous function, is a single expression with implicit return. lambda functions can have any number of arguments but only one expression.

1lambda arguments : expression

Inspiring Examples

The following examples show various application scenarios for lambda.

1"""lambda used in lexical closure"""
2def make_adder(n):
3    return lambda x: x + n
5plus_3 = make_adder(3)
7# output: 7
9"""lambda used in key"""
10# sort dict by value
11d = {'apple': 18, 'orange': 20, 'banana': 5, 'rotten tomato': 1}
12sorted(d.items(), key=lambda x: x[1])
13# alternative
14from operator import itemgetter
15sorted(d.items(), key=itemgetter(1))
16# output: [('rotten tomato', 1), ('banana', 5), ('apple', 18), ('orange', 20)]
18for pos in range(len(S)):
19    """lambda used in filter"""
20    matched_words = list(filter(lambda x: S[pos:].startswith(x), words))
21    if matched_words:
22        """lambda used in max"""
23        add_interval([pos, pos + len(max(matched_words, key=lambda x: len(x)))])
25"""call lambda in nested"""
26# L422: given a sequence of words, check whether it forms a valid word square.
27def valid_word_square(words: List[str]) -> bool:
28    f = lambda x: map(None, *x)  # python 2
29    return f(f(words)) == f(words)
31"""use lambda to wrap parameter as callable"""
32def wrapped_callable(f):
33    if callable(f): return f
34    else: return lambda: f


In this guide, we have learned many iterable Python tricks, such as iterator, generator, comprehension, and lambda. I hope some of them will be useful for you.

In Part 2, we will continue to learn about other iterable Python tricks.

You can also download an example notebook, iterables.ipynb, from Github.

This guide is one of a series of Python tricks guides:

I hope you enjoyed it. If you have any questions, you're welcome to contact me at [email protected]