Lcna example-2012
•Download as ODP, PDF•
1 like•512 views
This document summarizes a presentation about building a negative lookup caching translator for GlusterFS. The presentation demonstrates adding caching functionality to speed up lookups by caching previous misses. It shows the steps to hook the translator together, build it, configure it, debug it, and test its performance. Finally, it briefly introduces glupy, a new project for writing GlusterFS translators in Python, and demonstrates a Python implementation of the negative lookup cache.
Report
Share
Lcna example-2012
- 1. GlusterFS Translators Almost Live Tutorial Jeff Darcy August 28, 2012
- 2. Session Overview ● Build a “negative lookup” caching translator ● add functionality ● hook things together ● build ● configure ● debug ● Almost live - see each step as code is written, built, and run
- 3. The Problem Dir A file 4 Dir B file 1 Dir C file 3 Dir D file 2 ● First process misses in A/B/C ● Second process has no “memory” of misses, searches from the beginning again ● Idea: cache the misses, return immediately
- 4. On the Shoulders of Giants ++++
- 8. Boiler Plate
- 9. init
- 10. fini
- 11. Header File
- 12. Updating the Cache (negatives)
- 13. Updating the Cache (positives)
- 15. Final Pieces (create and mkdir)
- 16. Time to Build
- 17. Almost There
- 18. Original Volfile
- 19. Modified Volfile
- 20. Running in a Debugger
- 23. Cache Kicks In
- 24. How Well Does It Work? ● Simulate PHP workload ● 1000 files spread across 10 directories ● 80% of requests to 10% of files ● test time for 10000 searches ● Numbers for comparison only ● debug build ● wimpy test machines (GigE)
- 26. Are You Ready?
- 27. Interlude: glupy ● Write translators in Python ● Uses extending/embedding, ctypes module ● Brand new project ● weekend hacking, mid-August 2012 ● still implementing 70+ operation types ● Article coming soon in $major_publication
- 30. Python Setup
- 32. Scripting Languages Aren't So Slow
- 33. Conclusion ● Even simple translators can produce big effects ● Writing translators isn't that hard ● Writing glupy translators will be even easier ● What would you like GlusterFS to do today?