SlideShare a Scribd company logo

a quick Introduction to PyPy

•Download as PPT, PDF•
•
K
Kai Aras

PyPy is a Python implementation that uses a just-in-time (JIT) compiler. It is written in RPython, a restricted subset of Python, and uses a translation framework to generate C code from RPython programs. This allows PyPy to run programs faster than the standard CPython implementation while also supporting features like lazy evaluation, pluggable object spaces, and stackless concurrency.

1 of 52
PyPy Compiling Dynamic Language Implementations
The Python Situation
The Python Situation lots of implementations CPython - Posix Jython - JVM  IronPython - .NET specialized projects Stackless Python unladen-swallow Psyco
What PyPy is  Python Implementation Translation Framework
What PyPy is Python implemented in Python 
What PyPy is Python implemented in Python  looks like this: $> python py.py
What PyPy is Python implemented in Python  looks like this: runs on top of CPython $> python py.py
What PyPy is Python implemented in Python  looks like this: runs on top of CPython about 2000 times slower than CPython  $> python py.py
What PyPy is 2.  Translation Framework  $> python py.py $> ./pypy-c  $> ./pypy-jvm $> ./pypy-cli
some PyPy facts founded in 2003 by Holger Krekel and Armin Rigo 2004 - 2007 EU Project - Sixth Framework since 2007 Open Source Project occasional Funding by Google Sprint-Driven Development
PyPy Components Architecture RPython Interpreter  Translation Framework
PyPy components Architektur
PyPy components RPython RPython = Restricted/Reduced Python 
PyPy components RPython RPython = Restricted/Reduced Python  Restricted = most possible static subset of Python  
PyPy components RPython RPython = Restricted/Reduced Python  Restricted = most possible static subset of Python   required to perform Type Inference 
PyPy components RPython RPython = Restricted/Reduced Python  Restricted = most possible static subset of Python   required to perform Type Inference  input to the Translation Framework
PyPy components RPython RPython = Restricted/Reduced Python  Restricted = most possible static subset of Python   required to perform Type Inference  input to the Translation Framework no real specification - just some hints
PyPy components RPython RPython = Restricted/Reduced Python  Restricted = most possible static subset of Python   required to perform Type Inference  input to the Translation Framework no real specification - just some hints used for writing interpreters
PyPy components RPython RPython = Restricted/Reduced Python  Restricted = most possible static subset of Python   required to perform Type Inference  input to the Translation Framework no real specification - just some hints used for writing interpreters can be used for writing extensions as well
PyPy components Interpreter
PyPy components Interpreter written in RPython
PyPy components Interpreter written in RPython Internals: bytecode compiler -> generates bytecode
PyPy components Interpreter written in RPython Internals: bytecode compiler -> generates bytecode bytecode evaluator -> interprets bytecode 
PyPy components Interpreter written in RPython Internals: bytecode compiler -> generates bytecode bytecode evaluator -> interprets bytecode  object space -> handles operations on objects
Python Interpreter
PyPy components Translation Framework
PyPy components Translation Framework
PyPy components Translation Framework
PyPy components Translation Framework
 
PyPy > CPython High Level Language Implementation JIT Generation Object Spaces Stackless
PyPy > CPython High Level Language Implementation easy to experiment with 
PyPy > CPython High Level Language Implementation easy to experiment with  easy to implement new features
PyPy > CPython High Level Language Implementation easy to experiment with  easy to implement new features lazily computed objects and functions
PyPy > CPython High Level Language Implementation easy to experiment with  easy to implement new features lazily computed objects and functions plug-able  garbage-collection 
PyPy > CPython High Level Language Implementation easy to experiment with  easy to implement new features lazily computed objects and functions plug-able  garbage-collection  runtime replacement of live-objects 
PyPy > CPython High Level Language Implementation easy to experiment with  easy to implement new features lazily computed objects and functions plug-able  garbage-collection  runtime replacement of live-objects stackless concurrency 
PyPy > CPython JIT Generation based on Partial Evaluation via Yoshihiko Futamura (1971). "Partial Evaluation of Computation Process – An Approach to a Compiler-Compiler"  
PyPy > CPython JIT Generation based on Partial Evaluation via Yoshihiko Futamura (1971). "Partial Evaluation of Computation Process – An Approach to a Compiler-Compiler" says:  it's possible to take an interpreter and transform it into a compiler
PyPy > CPython JIT Generation based on Partial Evaluation via Yoshihiko Futamura (1971). "Partial Evaluation of Computation Process – An Approach to a Compiler-Compiler" says:  it's possible to take an interpreter and transform it into a compiler means: having such a thing is pretty cool because writing interpreters is a lot easier than writing compilers. 
PyPy > CPython JIT Generation
PyPy > CPython Object Spaces Flow ObjSpace Thunk ObjSpace Taint ObjSpace Dump ObjSpace Transparent Proxies
Thunk Object Space provides: lazily computed objects  lazily computed functions globally replaceable objects
Thunk Object Space provides: lazily computed objects  lazily computed functions globally replaceable objects is implemented in about 200 lines of code
Thunk Object Space provides: lazily computed objects  lazily computed functions globally replaceable objects is implemented in about 200 lines of code Example: >>>> a = "hello" >>>> b = "world" >>>> a + b 'helloworld' >>>> become(a,b) >>>> a + b 'worldworld'
Taint Object Space provides protection for : sensitive data that should not leak
Taint Object Space provides protection for : sensitive data that should not leak provides protection from : untrusted data that needs to be validated
Taint Object Space provides protection for : sensitive data that should not leak provides protection from : untrusted data that needs to be validated Example: >>>> password = &quot;secret&quot; >>>> password 'secret' >>>> password = taint(&quot;secret&quot;) >>>> password Traceback (application-level):    File &quot;<inline>&quot;, line 1 in <interactive>      password TaintError
PyPy > CPython Stackless infinite Recursion CPython limited to depth=1000 regular PyPy limited to depth=5000
PyPy > CPython Stackless infinite Recursion CPython limited to depth=1000 regular PyPy limited to depth=5000 Microthreads Coroutines Tasklets and Channels Greenlets
PyPy > CPython Stackless infinite Recursion CPython limited to depth=1000 regular PyPy limited to depth=5000 Microthreads Coroutines Tasklets and Channels Greenlets Coroutine Pickling Coroutine Cloning 
Demo Lazy functions and objects Tainted objects Translator Shell

More Related Content

a quick Introduction to PyPy

  • 1. PyPy Compiling Dynamic Language Implementations
  • 3. The Python Situation lots of implementations CPython - Posix Jython - JVM  IronPython - .NET specialized projects Stackless Python unladen-swallow Psyco
  • 4. What PyPy is  Python Implementation Translation Framework
  • 5. What PyPy is Python implemented in Python 
  • 6. What PyPy is Python implemented in Python  looks like this: $> python py.py
  • 7. What PyPy is Python implemented in Python  looks like this: runs on top of CPython $> python py.py
  • 8. What PyPy is Python implemented in Python  looks like this: runs on top of CPython about 2000 times slower than CPython  $> python py.py
  • 9. What PyPy is 2.  Translation Framework  $> python py.py $> ./pypy-c  $> ./pypy-jvm $> ./pypy-cli
  • 10. some PyPy facts founded in 2003 by Holger Krekel and Armin Rigo 2004 - 2007 EU Project - Sixth Framework since 2007 Open Source Project occasional Funding by Google Sprint-Driven Development
  • 11. PyPy Components Architecture RPython Interpreter  Translation Framework
  • 13. PyPy components RPython RPython = Restricted/Reduced Python 
  • 14. PyPy components RPython RPython = Restricted/Reduced Python  Restricted = most possible static subset of Python  
  • 15. PyPy components RPython RPython = Restricted/Reduced Python  Restricted = most possible static subset of Python   required to perform Type Inference 
  • 16. PyPy components RPython RPython = Restricted/Reduced Python  Restricted = most possible static subset of Python   required to perform Type Inference  input to the Translation Framework
  • 17. PyPy components RPython RPython = Restricted/Reduced Python  Restricted = most possible static subset of Python   required to perform Type Inference  input to the Translation Framework no real specification - just some hints
  • 18. PyPy components RPython RPython = Restricted/Reduced Python  Restricted = most possible static subset of Python   required to perform Type Inference  input to the Translation Framework no real specification - just some hints used for writing interpreters
  • 19. PyPy components RPython RPython = Restricted/Reduced Python  Restricted = most possible static subset of Python   required to perform Type Inference  input to the Translation Framework no real specification - just some hints used for writing interpreters can be used for writing extensions as well
  • 21. PyPy components Interpreter written in RPython
  • 22. PyPy components Interpreter written in RPython Internals: bytecode compiler -> generates bytecode
  • 23. PyPy components Interpreter written in RPython Internals: bytecode compiler -> generates bytecode bytecode evaluator -> interprets bytecode 
  • 24. PyPy components Interpreter written in RPython Internals: bytecode compiler -> generates bytecode bytecode evaluator -> interprets bytecode  object space -> handles operations on objects
  • 30.  
  • 31. PyPy > CPython High Level Language Implementation JIT Generation Object Spaces Stackless
  • 32. PyPy > CPython High Level Language Implementation easy to experiment with 
  • 33. PyPy > CPython High Level Language Implementation easy to experiment with  easy to implement new features
  • 34. PyPy > CPython High Level Language Implementation easy to experiment with  easy to implement new features lazily computed objects and functions
  • 35. PyPy > CPython High Level Language Implementation easy to experiment with  easy to implement new features lazily computed objects and functions plug-able  garbage-collection 
  • 36. PyPy > CPython High Level Language Implementation easy to experiment with  easy to implement new features lazily computed objects and functions plug-able  garbage-collection  runtime replacement of live-objects 
  • 37. PyPy > CPython High Level Language Implementation easy to experiment with  easy to implement new features lazily computed objects and functions plug-able  garbage-collection  runtime replacement of live-objects stackless concurrency 
  • 38. PyPy > CPython JIT Generation based on Partial Evaluation via Yoshihiko Futamura (1971). &quot;Partial Evaluation of Computation Process – An Approach to a Compiler-Compiler&quot;  
  • 39. PyPy > CPython JIT Generation based on Partial Evaluation via Yoshihiko Futamura (1971). &quot;Partial Evaluation of Computation Process – An Approach to a Compiler-Compiler&quot; says:  it's possible to take an interpreter and transform it into a compiler
  • 40. PyPy > CPython JIT Generation based on Partial Evaluation via Yoshihiko Futamura (1971). &quot;Partial Evaluation of Computation Process – An Approach to a Compiler-Compiler&quot; says:  it's possible to take an interpreter and transform it into a compiler means: having such a thing is pretty cool because writing interpreters is a lot easier than writing compilers. 
  • 41. PyPy > CPython JIT Generation
  • 42. PyPy > CPython Object Spaces Flow ObjSpace Thunk ObjSpace Taint ObjSpace Dump ObjSpace Transparent Proxies
  • 43. Thunk Object Space provides: lazily computed objects  lazily computed functions globally replaceable objects
  • 44. Thunk Object Space provides: lazily computed objects  lazily computed functions globally replaceable objects is implemented in about 200 lines of code
  • 45. Thunk Object Space provides: lazily computed objects  lazily computed functions globally replaceable objects is implemented in about 200 lines of code Example: >>>> a = &quot;hello&quot; >>>> b = &quot;world&quot; >>>> a + b 'helloworld' >>>> become(a,b) >>>> a + b 'worldworld'
  • 46. Taint Object Space provides protection for : sensitive data that should not leak
  • 47. Taint Object Space provides protection for : sensitive data that should not leak provides protection from : untrusted data that needs to be validated
  • 48. Taint Object Space provides protection for : sensitive data that should not leak provides protection from : untrusted data that needs to be validated Example: >>>> password = &quot;secret&quot; >>>> password 'secret' >>>> password = taint(&quot;secret&quot;) >>>> password Traceback (application-level):    File &quot;<inline>&quot;, line 1 in <interactive>      password TaintError
  • 49. PyPy > CPython Stackless infinite Recursion CPython limited to depth=1000 regular PyPy limited to depth=5000
  • 50. PyPy > CPython Stackless infinite Recursion CPython limited to depth=1000 regular PyPy limited to depth=5000 Microthreads Coroutines Tasklets and Channels Greenlets
  • 51. PyPy > CPython Stackless infinite Recursion CPython limited to depth=1000 regular PyPy limited to depth=5000 Microthreads Coroutines Tasklets and Channels Greenlets Coroutine Pickling Coroutine Cloning 
  • 52. Demo Lazy functions and objects Tainted objects Translator Shell