10.1 Generators & Iterators

Understanding generators and iterators for efficient management of large data sequences, saving memory.

Generators and iterators are advanced yet fundamental concepts in Python for efficiently managing data sequences, especially when they are very large or potentially infinite. Their core idea is "lazy evaluation": they produce items one by one and only when needed, instead of storing the entire sequence in memory.

Iterators

An iterator is an object that implements the "iterator protocol," which consists of two special methods: `__iter__()` and `__next__()`. `__iter__()` returns the iterator object itself, while `__next__()` returns the next item in the sequence. When there are no more items, `__next__()` raises a `StopIteration` exception. Python's `for` loop works internally using this protocol.

# Simple Iterator Example
my_list = [10, 20, 30]
my_iterator = iter(my_list) # iter() returns an iterator

print("Getting items from the iterator with next():")
print(next(my_iterator)) # 10
print(next(my_iterator)) # 20
print(next(my_iterator)) # 30

try:
    print(next(my_iterator)) # Raises StopIteration
except StopIteration:
    print("End of iteration: The iterator is exhausted.")

Generators

Generators are a much simpler and more elegant way to create iterators. Instead of writing a whole class with `__iter__()` and `__next__()`, you simply define a function that uses the `yield` keyword instead of `return`. Each time the function encounters `yield`, it "pauses" its execution, returns the value, and maintains its state until the next call to `next()`, at which point it resumes where it left off.

def generate_squares(limit):
    """A generator that yields the squares of numbers up to a limit."""
    for i in range(1, limit + 1):
        print(f"(Generating {i*i})")
        yield i * i # Yields the square and "pauses"

print("\nSquares of numbers up to 5:")
for square in generate_squares(5):
    print(f"  Received the square: {square}")

Explore More with AI

Use AI to generate new examples, delve deeper into theory, or get your questions answered.