Why yield returns an iterator?

Question

I'm trying to understand how yield works and after I had read this text I believed that it's quite understandable.

However I still don't understand what's the connection between yield and __iter__ because I've just found out that this code works:

class Vect():
    def __init__(self, *args):
        self.__a = list(args)
    def print_(self):
        print self.__a
    def __iter__(self):
        yield self.__a

asd = Vect(1,2,3,4,5)
for foo in asd:
    print foo

I thought that when I have a generator (a function which returns a single argument at the time but returns as many arguments as it can until it hits the end) yield works like: "OK, let's return this argument, but maybe we can still return another". However in my example I don't have any generator, yield "returns" a list and in some way gets access to list's iterator. I have absolutely no idea what's going on.

Answer 1

yield returns whatever object is passed to it, even if that object is a sequence or a generator or other iterator. Thus:

>>> def g():
...     yield [1,2,3]
...     yield 1
...     yield 2
...     yield 3
... 
>>> gen = g()
>>> gen.next()
[1, 2, 3]
>>> gen.next()
1
>>> gen.next()
2
>>> gen.next()
3
>>> gen.next()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration
>>>

__iter__ is called on an object when an iterator over the object's contents is required (as in a when it is part of a for x in obj construct). You can use yield to create a generator (since generators are iterators), but in the present example you don't need to. The following will work as well:

def __iter__(self):
   return iter(self.__a)

If you want to use yield, and you want the iterator for Vect objects to move through the contents of the vector, you have to yield each value separately:

def __iter__(self):
    for i in self.__a:
        yield i

The yield means that __iter__ will return a generator, and calling next() on the generator object will resume the function at the point where it last left off, as it iterates through __a.

=======

In response to the additional question about how Python tracks where in the execution of the generator it is, I believe it uses the f_lasti (== "last instruction") of the gi_frame attribute of the generator (Generators, unlike ordinary functions, carry a frame of execution around with them). Here's a bit of tooling around that shows how the values change:

>>> import dis
>>> def g():
...     yield 1
...     for i in range(10):
...             yield i*2
... 
>>> gen = g() 
>>> dis.dis(gen.gi_code)
  2           0 LOAD_CONST               1 (1)
              3 YIELD_VALUE         
              4 POP_TOP             

  3           5 SETUP_LOOP              29 (to 37)
              8 LOAD_GLOBAL              0 (range)
              11 LOAD_CONST               2 (10)
              14 CALL_FUNCTION            1
              17 GET_ITER            
         >>   18 FOR_ITER                15 (to 36)
              21 STORE_FAST               0 (i)

   4          24 LOAD_FAST                0 (i)
              27 LOAD_CONST               3 (2)
              30 BINARY_MULTIPLY     
              31 YIELD_VALUE         
              32 POP_TOP             
              33 JUMP_ABSOLUTE           18
         >>   36 POP_BLOCK           
         >>   37 LOAD_CONST               0 (None)
              40 RETURN_VALUE        
>>> gen.gi_frame.f_lasti ## -1 because we haven't started yet
-1 
>>> gen.next()  
1
>>> gen.gi_frame.f_lasti
3
>>> gen.gi_frame.f_locals
{}
>>> gen.next() 
0
>>> gen.gi_frame.f_lasti , gen.gi_frame.f_locals
(31, {'i': 0})
>>> gen.next()
2
>>> gen.gi_frame.f_lasti , gen.gi_frame.f_locals
(31, {'i': 1})
>>> 

Note how the f_lasti value corresponds to the numbered line in the disassembled code that that the last yield was on: it restarts from that point when the generator is reentered.

Answer 2

if you check your print you'll see the iter is returning the list as a whole and when you print foo it only prints once

check this out:

for index, foo in enumerate(asd):
    print index, foo

Answer 3

yield works exactly as you understood it - it returns one object which in your case is a list.

Maybe this will make it more clear - change your code to:

i = 1
for foo in asd:
    print i
    i += 1
    print foo

As you can see there is only one iteration - the iterator contains one element - the list [1,2,3,4,5].