Return to Answer

From a programming viewpoint, the iterators are implemented as thunks.

To implement concepts such as iterators, generators, concurrent execution via messages, etc., one uses messages sent to a closure object, which has a dispatcher, and the dispatcher answers to "messages" (this concept comes from Simula and is the central part of Smalltalk).

"next" is a message sent to a closure, created by the "iter" call.

There are lots of ways to implement this computation. I used mutation, but it is possible to do this kind of computation without mutation, by returning the current value and the next yielder (making it referential transparent). Racket uses a sequence of transformations of the initial program in some intermediary languages, one of such rewriting making the yield operator to be transformed in some language with simpler operators.

Here is a demonstration of how yield could be rewritten, which uses the structure of R6RS, but the semantics is identical to Python's. It's the same model of computation, and only a change in syntax is required to rewrite it using yield of Python.

Welcome to Racket v6.5.0.3.

-> (define gen
     (lambda (l)
       (define yield
         (lambda ()
           (if (null? l)
               'END
               (let ((v (car l)))
                 (set! l (cdr l))
                 v))))
       (lambda(m)
         (case m
           ('yield (yield))
           ('init  (lambda (data)
                     (set! l data)
                     'OK))))))
-> (define stream (gen '(1 2 3)))
-> (stream 'yield)
1
-> (stream 'yield)
2
-> (stream 'yield)
3
-> (stream 'yield)
'END
-> ((stream 'init) '(a b))
'OK
-> (stream 'yield)
'a
-> (stream 'yield)
'b
-> (stream 'yield)
'END
-> (stream 'yield)
'END
->

From a programming viewpoint, the iterators are implemented as thunks.

To implement concepts such as iterators, generators, concurrent execution via messages, etc., one uses messages sent to a closure object, which has a dispatcher, and the dispatcher answers to "messages" (this concept comes from Simula and is the central part of Smalltalk).

"next" is a message sent to a closure, created by the "iter" call.

There are lots of ways to implement this computation. I used mutation, but it is possible to do this kind of computation without mutation, by returning the current value and the next yielder (making it referential transparent). Racket uses a sequence of transformations of the initial program in some intermediary languages, one of such rewriting making the yield operator to be transformed in some language with simpler operators.

Here is a demonstration of how yield could be rewritten, which uses the structure of R6RS, but the semantics is identical to Python's. It's the same model of computation, and only a change in syntax is required to rewrite it using yield of Python.

Welcome to Racket v6.5.0.3.

-> (define gen
     (lambda (l)
       (define yield
         (lambda ()
           (if (null? l)
               'END
               (let ((v (car l)))
                 (set! l (cdr l))
                 v))))
       (lambda(m)
         (case m
           ('yield (yield))
           ('init  (lambda (data)
                     (set! l data)
                     'OK))))))
-> (define stream (gen '(1 2 3)))
-> (stream 'yield)
1
-> (stream 'yield)
2
-> (stream 'yield)
3
-> (stream 'yield)
'END
-> ((stream 'init) '(a b))
'OK
-> (stream 'yield)
'a
-> (stream 'yield)
'b
-> (stream 'yield)
'END
-> (stream 'yield)
'END
->

From a programming viewpoint, the iterators are implemented as thunks.

To implement iterators, generators, and thread pools for concurrent execution, etc. as thunks, one uses messages sent to a closure object, which has a dispatcher, and the dispatcher answers to "messages".

"next" is a message sent to a closure, created by the "iter" call.

There are lots of ways to implement this computation. I used mutation, but it is possible to do this kind of computation without mutation, by returning the current value and the next yielder (making it referential transparent). Racket uses a sequence of transformations of the initial program in some intermediary languages, one of such rewriting making the yield operator to be transformed in some language with simpler operators.

Here is a demonstration of how yield could be rewritten, which uses the structure of R6RS, but the semantics is identical to Python's. It's the same model of computation, and only a change in syntax is required to rewrite it using yield of Python.

Welcome to Racket v6.5.0.3.

-> (define gen
     (lambda (l)
       (define yield
         (lambda ()
           (if (null? l)
               'END
               (let ((v (car l)))
                 (set! l (cdr l))
                 v))))
       (lambda(m)
         (case m
           ('yield (yield))
           ('init  (lambda (data)
                     (set! l data)
                     'OK))))))
-> (define stream (gen '(1 2 3)))
-> (stream 'yield)
1
-> (stream 'yield)
2
-> (stream 'yield)
3
-> (stream 'yield)
'END
-> ((stream 'init) '(a b))
'OK
-> (stream 'yield)
'a
-> (stream 'yield)
'b
-> (stream 'yield)
'END
-> (stream 'yield)
'END
->

From a programming viewpoint, the iterators are implemented as thunks.

To implement concepts such as iterators, generators, concurrent execution via messages, etc., one uses messages sent to a closure object, which has a dispatcher, and the dispatcher answers to "messages" (this concept comes from Simula and is the central part of Smalltalk).

"next" is a message sent to a closure, created by the "iter" call.

There are lots of ways to implement this computation. I used mutation, but it is possible to do this kind of computation without mutation, by returning the current value and the next yielder (making it referential transparent). Racket uses a sequence of transformations of the initial program in some intermediary languages, one of such rewriting making the yield operator to be transformed in some language with simpler operators.

Here is a demonstration of how yield could be rewritten, which uses the structure of R6RS, but the semantics is identical to Python's. It's the same model of computation, and only a change in syntax is required to rewrite it using yield of Python.

Welcome to Racket v6.5.0.3.

-> (define gen
     (lambda (l)
       (define yield
         (lambda ()
           (if (null? l)
               'END
               (let ((v (car l)))
                 (set! l (cdr l))
                 v))))
       (lambda(m)
         (case m
           ('yield (yield))
           ('init  (lambda (data)
                     (set! l data)
                     'OK))))))
-> (define stream (gen '(1 2 3)))
-> (stream 'yield)
1
-> (stream 'yield)
2
-> (stream 'yield)
3
-> (stream 'yield)
'END
-> ((stream 'init) '(a b))
'OK
-> (stream 'yield)
'a
-> (stream 'yield)
'b
-> (stream 'yield)
'END
-> (stream 'yield)
'END
->

Behind the yield syntax there must be a semantics. It is important to understand the semantic concept behind the syntax. For example, if you want to create a closure, there is lambda and def. Is a generator simply a closure? Does a generator save the stack? For sure it does not save the stack as call-with-current-continuation does, as python execution model is not a tree... And for sure a generator is not a simple closure, otherwise a function and a generator would be the same concept. So I try to figure out the concept of generator by building it from more basic concept of closure...

To implement iterators, generators, and thread pools for concurrent execution, etc. as thunks, one uses messages sent to a closure object, which has a dispatcher, and the dispatcher answers to "messages".

To implement iterators, generators, and thread pools for concurrent execution, etc. as thunks, one uses messages sent to a closure object, which has a dispatcher, and the dispatcher answers to "messages".