Generators and Generator Expressions

Generators are a powerful feature of Python that allow you to create iterable objects that can be used to iterate over a sequence of values. They are similar to lists and tuples, but unlike lists and tuples, generators do not store all of the values in memory at once. Instead, they generate the values one at a time as needed, which makes them more efficient and memory-friendly when working with large datasets.

Creating a Generator

To create a generator, you use the yield keyword instead of return in a function definition. When the function is called, it returns a generator object that can be used to iterate over the values. For example:

def count_up_to(max):
    count = 1
    while count <= max:
        yield count
        count += 1

counter = count_up_to(5)
print(next(counter))  # Output: 1
print(next(counter))  # Output: 2
print(next(counter))  # Output: 3
print(next(counter))  # Output: 4
print(next(counter))  # Output: 5
print(next(counter))  # Output: StopIteration

You can also use a for loop to iterate over the values in a generator. The for loop will automatically catch the StopIteration exception and stop iterating when the generator is exhausted.

def count_up_to(max):
    count = 1
    while count <= max:
        yield count
        count += 1

for number in count_up_to(5):
    print(number)  # Output: 1 2 3 4 5

Generator Expressions

Generator expressions are a concise way to create generators using a similar syntax to list comprehensions. To create a generator expression, you enclose an expression in parentheses and use the same syntax as a list comprehension, but without the square brackets. For example:

# Create a generator expression that generates the squares of the numbers from 1 to 5
squares = (n ** 2 for n in range(1, 6))

print(next(squares))  # Output: 1
print(next(squares))  # Output: 4
print(next(squares))  # Output: 9
print(next(squares))  # Output: 16
print(next(squares))  # Output: 25
print(next(squares))  # Output: StopIteration

Generator expressions are particularly useful when you need to perform a transformation on a large dataset, but don’t need to store the entire transformed dataset in memory. They are also useful when you need to perform a complex transformation that would be difficult to do with a traditional for loop.

Conclusion

In summary, generators and generator expressions are a powerful and efficient way to create and work with iterable objects in Python. They allow you to iterate over large datasets without using up a lot of memory, and allow you to write concise and expressive code to transform and manipulate data. By learning how to use generators and generator expressions effectively, you’ll be able to write more efficient and powerful Python programs.

Exercises

To review these concepts, we will go through a series of exercises designed to test your understanding and apply what you have learned.

def squares(numbers):
    for n in numbers:
        yield n ** 2

numbers = [1, 2, 3, 4, 5]
squares_gen = squares(numbers)
print(next(squares_gen))  # Output: 1
print(next(squares_gen))  # Output: 4
print(next(squares_gen))  # Output: 9
print(next(squares_gen))  # Output: 16
print(next(squares_gen))  # Output: 25
print(next(squares_gen))  # Output: StopIteration

words = ['racecar', 'hello', 'world', 'madam', 'level']
palindromes = (w for w in words if w == w[::-1])
print(next(palindromes))  # Output: 'racecar'
print(next(palindromes))  # Output: 'madam'
print(next(palindromes))  # Output: 'level'
print(next(palindromes))  # Output: StopIteration

def second_elements(tuples):
    for x in tuples:
        yield x[1]

tuples = [(1, 2), (3, 4), (5, 6)]
second_elements_gen = second_elements(tuples)
print(next(second_elements_gen))  # Output: 2
print(next(second_elements_gen))  # Output: 4
print(next(second_elements_gen))  # Output: 6
print(next(second_elements_gen))  # Output: StopIteration

lists = [['a', 'b', 'c'], ['d', 'e', 'f'], ['g', 'h', 'i']]
strings = (' '.join(x) for x in lists)
print(next(strings))  # Output: 'a b c'
print(next(strings))  # Output: 'd e f'
print(next(strings))  # Output: 'g h i'
print(next(strings))  # Output: StopIteration

def divisible_by_three(numbers):
    for n in numbers:
        if n % 3 == 0:
            yield n

numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
divisible_by_three_gen = divisible_by_three(numbers)
print(next(divisible_by_three_gen))  # Output: 3
print(next(divisible_by_three_gen))  # Output: 6
print(next(divisible_by_three_gen))  # Output: 9
print(next(divisible_by_three_gen))  # Output: StopIteration