Generators in Python: Lazy Evaluation and Streaming

When an API returns a large response — text spanning thousands of tokens — waiting for the full response is inconvenient. Instead, the server sends data in chunks, and Python receives them via a generator.

What Is a Generator

A regular function returns all the data at once:

def get_numbers():
    return [1, 2, 3, 4, 5]  # entire list in memory at once

for n in get_numbers():
    print(n)

A generator returns values one at a time, without loading everything into memory:

def get_numbers():
    for i in range(1, 6):
        yield i  # ← keyword yield instead of return

for n in get_numbers():
    print(n)  # 1, 2, 3, 4, 5 — one at a time

yield pauses the function and returns the current value. On the next iteration, the function resumes from where it left off.

Iterating Over a Generator

A generator is an iterable object, just like a list or string:

gen = get_numbers()   # create the generator (nothing computed yet)
print(next(gen))      # 1 — fetch the first value
print(next(gen))      # 2 — fetch the next
print(next(gen))      # 3

# Or in a loop:
for value in gen:
    print(value)      # 4, 5 (1, 2, 3 already consumed)

Generators in Streaming APIs

The Gemini SDK returns a generator object when stream=True:

response = model.generate_content("Напиши рассказ", stream=True)

# response — generator of chunks
for chunk in response:
    print(chunk.text, end="", flush=True)
print()  # newline at the end

Each chunk is a piece of the response. Text appears gradually, just like in a chat.

flush=True — Why?

Python buffers output for efficiency. flush=True flushes the buffer immediately — text appears on screen right away, without waiting for the buffer to fill.

print("Hello", end="", flush=True)   # visible immediately
print("Hello", end="")               # may be delayed in buffer

Generator Expression

The equivalent of a list comprehension, but lazy:

squares_list = [x**2 for x in range(1000)]   # list: all 1000 elements in memory
squares_gen  = (x**2 for x in range(1000))   # generator: computed one at a time

# Usage is identical:
for sq in squares_gen:
    print(sq)

() instead of [] — and it’s already a generator.

When to Use Generators

Use a generator when:
- Data arrives gradually (streaming API)
- The dataset is large and doesn’t fit in memory
- You need a lazy data-processing pipeline

Use a list when:
- You need random access by index (items[5])
- The data is needed multiple times
- The dataset is small

Summary

A generator is a function with yield that returns values one at a time. In the context of streaming APIs, generators let you process the response as data arrives — without waiting for the server to send everything at once.