How does a function in Python remember its values after it returns? [duplicate]

Question

Warning: extreme newbie question

I seem to have been thinking of functions as a recipe. In my world, the program is a recipe box and some of the recipes (functions) call for other recipes (other functions). The processor starts executing the master recipe by writing the instructions into RAM and working through them. Like, breakfast crepes. You call the breakfast crepes recipe from Julia Childs. You have to do make the crepe batter once. Then, while you still have crepe batter, you iteratively make crepes. Concurrently, there are various fruit preparations you can make.

Well, I apparently don't understand. I just ran the python wiki solution to Project Euler Problem 2 (sum of even Fibonacci numbers less than 4 million) through pythontutor.com. And I think something occurred to me. It seems like every time you conjure a recipe, you don't just use the same processor, you get a gnome with some pots to work on that function. The pots are variables, the gnome works out his recipe, and, if the calling function was expecting return values, the gnome shows the contents of those pots to the caller. The caller may then go back, figure out some more things, and show return values to his caller.

So lets say Al calls Bob to make crepes. Bob makes the batter and calls Charlie to cook them. Charlie cooks a crepe, serves that crepe to Bob, Bob gives it to Al, and goes back to Charlie. Who still exists! Al is unaware that Bob has Charlie stashed in the kitchen, but even after Charlie makes that first crepe, he's still in the kitchen, knows how to make a crepe, and knows how much crepe batter he has left. Even though he already returned the first crepe.

Can someone help clear this up for me?

Here's the code from the Python wiki

 def fib():
    x,y = 0,1
    while True:
        yield x
        x,y = y, x+y

def even(seq):
    for number in seq:
        if not number % 2:
            yield number

def under_a_million(seq):
    for number in seq:
        if number > 1000000:
            break
        yield number   

print sum(even(under_a_million(fib())))

And here's http://pythontutor.com/visualize.html

Answer 1

A simplified answer.

If you have a function that generates a sequence of values, you may transform it in a generator by using yield.

def func():
    for i in range(3):
        yield i

list(func()) ==> [0,1,2]

for el in func():
    print(el) # 0
              # 1
              # 2

Each time yield is called the function is frozen somewhere. When it is called again it continues from his last state, and doesn't start anew (unless it has finished to consume elements).

If you call the function you get a generator, which is something you can iterate over.

Note that this way you may iterate over infinite sequences without the need to use infinite memory.

def inf():
    x = -1
    while True:
        x = x + 1
        yield x

for i in inf():
    if i < 10:
        print(i)
    else:
        break

Answer 2

This is because those functions aren't functions, but generators. The yield statement returns a value, but doesn't return from the function. Each time you call next on the generator, it continues the execution of the generator from the last call, until it reaches yield statement again.

Answer 3

In principle your first assumption was right: the variables of a function are only available while the function is executed. In your case however, you are using the yield statement. As a result the function call returns an iterator, which, when called, returns the value of the next yield statement.

Check this post for further explanations on what iterators are and do.