A generator is created by defining a function containing at least one yield statement. Unlike normal functions, it returns a generator object instead of executing the entire function and returning a single result.
Syntax
def generator_function(parameters):
yield expression
Example: Here we will create a generator that produces number from 1 to 5.
def count_numbers():
for num in range(1, 6):
yield num
for value in count_numbers():
print(value)
Output
1 2 3 4 5
Explanation:
- count_numbers() function contains a yield statement, making it a generator function.
- Each execution of yield produces the current value and pauses the function.
- Next iteration resumes execution from where it stopped.
- for loop automatically retrieves values until the generator is exhausted.
Working of Generators
They work through lazy execution. When a generator function is called, Python creates a generator object without immediately executing its function body. Execution starts when a value is requested using next() or a for loop.
Working process involves the following steps:
- Calling a generator function returns a generator object.
- Function starts executing when the first value is requested.
- Yield statement produces a value and pauses execution.
- Local variables and the execution position are preserved.
- Next request resumes execution from the previous yield statement.
- When the function finishes, the generator raises StopIteration to indicate that no more values are available.
Example:
def simple_generator():
print("Generator Started")
yield 10
print("Generator Resumed")
yield 20
print("Generator Finished")
gen = simple_generator()
print("Generator Created")
print(next(gen))
print(next(gen))
for value in gen:
print(value)
Output
Generator Created Generator Started 10 Generator Resumed 20 Generator Finished
Explanation:
- Calling simple_generator() creates a generator object but does not execute its body.
- First next() call starts execution and produces 10.
- Second next() call resumes execution and produces 20.
- for loop requests another value, allowing the remaining statement to execute.
- Once the function finishes, iteration stops.
Yield vs Return
Both yield and return are used to produce values from functions, but their execution behaviour is different.
yield | return |
|---|---|
Produces a value and pauses execution. | Produces a final result and terminates execution. |
Preserves the function state. | Does not preserve execution for resumption. |
Can produce multiple values. | Normal function execution returns a single result. |
Used in generator functions. | Used in normal and generator functions. |
Example:
def normal_function():
return 10
return 20
def generator_function():
yield 10
yield 20
print(normal_function())
for value in generator_function():
print(value)
Output
10 10 20
Explanation:
- normal_function() terminates after executing its first return statement. The second statement is never executed.
- generator_function() pauses after each yield statement.
- Calling the generator again resumes execution and produces the next value.
Accessing Values From Generators
Values produced by generators can be accessed using a for loop or the next() function.
1. Using for Loop
It automatically retrieves values from a generator until it is exhausted.
Example:
def generate_numbers():
yield 10
yield 20
yield 30
for num in generate_numbers():
print(num)
Output
10 20 30
2. Using next() Function
It retrieves the next available value from a generator. Each call resumes execution until another yield statement is reached.
Example:
def generate_numbers():
yield 10
yield 20
yield 30
gen = generate_numbers()
print(next(gen))
print(next(gen))
print(next(gen))
Output
10 20 30
Explanation:
- first next() call returns 10.
- Second call resumes execution and returns 20.
- Third call returns 30.
- Calling next() again without a default value raises StopIteration because no more values are available.
Generator Expression
They provide a shorter way to create generators without defining a separate function. They use parentheses instead of square brackets and generate values lazily.
Syntax
(expression for item in iterable)
Example: We will generate the squares of numbers from 1 to 5 using a generator expression.
squares = (num * num for num in range(1, 6))
for value in squares:
print(value)
Output
1 4 9 16 25
Explanation:
- Expression num * num calculates the square of each number.
- range() function provides numbers from 1 to 5.
- Parentheses create a generator expression instead of a list.
- Values are calculated only when requested by the for loop.
Generator Expressions vs List Comprehension
Generator Expression | List Comprehension |
|---|---|
Uses parentheses. | Uses square brackets. |
Produces values lazily. | Creates the complete list immediately. |
Generally uses less memory for large sequences. | Stores all generated values in memory. |
Values can be consumed only once. | Elements can be accessed repeatedly. |
Use Cases
They are useful in situations where values need to be produced or processed gradually.
1. Processing Large Files
They can read files line by line instead of loading their complete contents into memory.
Example:
def read_file(file_path):
with open(file_path, "r") as file:
for line in file:
yield line.strip()
for line in read_file("data.txt"):
print(line)
Explanation:
- Function reads one line at a time.
- Each yield statement produces the current line.
- Remaining file contents are processed only when requested.
- This approach is useful for handling large log files and text datasets.
2. Generating Fibonacci Numbers
They can produce infinite sequences without storing all previous results in a list.
Example:
def fibonacci():
a, b = 0, 1
while True:
yield a
a, b = b, a + b
for num in fibonacci():
if num > 50:
break
print(num, end=" ")
Output
0 1 1 2 3 5 8 13 21 34
Explanation:
- Function maintains two variables to calculate Fibonacci numbers.
- Each yield statement produces the current number.
- While loop continues generating values indefinitely.
- Break statement stops iteration when a value exceeds 50.