A quick overview of why to use and how to set up iPython notebooks for research
•
3 likes•2,021 views
A quick overview of why to use and how to set up iPython notebooks for research in the Amaral lab. Example notebook is a gist at: http://nbviewer.ipython.org/gist/anonymous/f8e6d8985d2ea0e4bab1
Report
Share
Related slideshows
A quick overview of why to use and how to set up iPython notebooks for research
- 1. ipython notebook from scratch. for realsies.
- 2. Some basic information to start with iPython notebook is a part of the iPython package (iPython is an improved command line interpreter)
- 3. Some basic information to start with iPython notebook is a part of the iPython package (iPython is an improved command line interpreter) iPython notebook is young, it’s at v2.1 right now.
- 4. Some basic information to start with iPython notebook is a part of the iPython package (iPython is an improved command line interpreter) iPython notebook is young, it’s at v2.1 right now. Only 1 year ago, iPython notebook was at v0.11
- 5. Some basic information to start with iPython notebook is a part of the iPython package (iPython is an improved command line interpreter) iPython notebook is young, it’s at v2.1 right now. Only 1 year ago, iPython notebook was at v0.11 In any case iPython notebook has pretty much become *the* tool for data analysis and analysis sharing on the inter webs
- 6. Why is that?
- 7. Why is that? Easy - Like using the interactive interpreter but now there is the ability to edit, fix, and refactor code.!
- 8. Why is that? Easy - Like using the interactive interpreter but now there is the ability to edit, fix, and refactor code.! Shareable - Easily exported, results are in-line with the page so that it’s easy to see how code turns out.!
- 9. Why is that? Easy - Like using the interactive interpreter but now there is the ability to edit, fix, and refactor code.! Shareable - Easily exported, results are in-line with the page so that it’s easy to see how code turns out.! Documented - The code and results are both there without necessitating that someone else run it first!
- 10. Why is that? Easy - Like using the interactive interpreter but now there is the ability to edit, fix, and refactor code.! Shareable - Easily exported, results are in-line with the page so that it’s easy to see how code turns out.! Polyglot - Something works better with bash? R? Ruby? Perl? No problem, run it from iPython notebook! Documented - The code and results are both there without necessitating that someone else run it first!
- 12. The basics with an assumption of pyenv We’ll assume you have the basics, now do: $ pyenv virtualenv 2.7.6 notebook $ pyenv global notebook $ pip install ipython pyzmq tornado (Using pip then using ipython will install notebook too!)
- 13. The basics with an assumption of pyenv We’ll assume you have the basics, now do: $ pyenv virtualenv 2.7.6 notebook $ pyenv global notebook $ pip install ipython pyzmq tornado (Using pip then using ipython will install notebook too!) Or follow this handy blog post: http://amaral-lab.org/blog/i-wish-i-knew-then-what-i-know-now-lab-notebooks-a
- 14. So how can I use this… Start me up on the command line now $ ipython notebook
- 15. So how can I use this… Start me up on the command line now $ ipython notebook Navigate to localhost:8888 in your preferred web browser
- 16. So how can I use this… Start me up on the command line now $ ipython notebook Navigate to localhost:8888 in your preferred web browser VOILA!
- 17. So how can I use this… From the web interface you can create a new notebook
- 18. So how can I use this… From the web interface you can create a new notebook
- 19. So how can I use this… From the web interface you can create a new notebook Folders have to be created on the filesystem (terminal or nautilus/finder)
- 20. So what is the best way to use this? Data Exploration
- 21. So what is the best way to use this? Data Exploration Learning new libraries/techniques
- 22. So what is the best way to use this? Data Exploration Learning new libraries/techniques Documenting code/workflow/results
- 23. So what is the best way to use this? Data Exploration Learning new libraries/techniques Trying different visualizations Documenting code/workflow/results
- 24. So what is the best way to use this? Data Exploration Learning new libraries/techniques Trying different visualizations Documenting code/workflow/results Looking at small/modular parts
- 25. What is not a good way to use it? Greater than 15-20 images in a single notebook
- 26. What is not a good way to use it? Greater than 15-20 images in a single notebook Poor programming practices (i.e. no functions or modules)
- 27. What is not a good way to use it? Greater than 15-20 images in a single notebook Poor programming practices (i.e. no functions or modules) Extensive amounts of *production* code (Think hundreds or thousands of lines)
- 28. So how can I set this up… Holy crap! So many ways!
- 29. So how can I set this up… Holy crap! So many ways! As a one-off as described!
- 30. So how can I set this up… Holy crap! So many ways! As a one-off as described! As a server!
- 31. So how can I set this up… Holy crap! So many ways! As a one-off as described! As a server! As a constantly updating resource!
- 32. Initial caveats first iPython notebook currently has no method of handling two individuals editing the same notebook simultaneously
- 33. As a one-off Simply do: $ ipython notebook
- 34. As a one-off Simply do: $ ipython notebook The cool thing is that iPython notebook can now traverse directories
- 35. As a one-off Simply do: $ ipython notebook The cool thing is that iPython notebook can now traverse directories So *ANY* directory that is a child of the directory where you start the service you can reach it!
- 36. As a one-off Simply do: $ ipython notebook The cool thing is that iPython notebook can now traverse directories So *ANY* directory that is a child of the directory where you start the service you can reach it! But! While iPython notebook can create notebooks from the web page it can’t create folders
- 37. As a one-off So this is pretty simple then, you can embed the ipython notebooks in your project directories and navigate to them from your project root $ cd ~ $ nohup ipython notebook & ! Now navigate to the project code in the browser
- 38. But I have to keep switching computers …and I can’t remember to keep committing repositories.
- 39. But I have to keep switching computers …and I can’t remember to keep committing repositories. BAM! Solution!
- 40. But I have to keep switching computers …and I can’t remember to keep committing repositories. BAM! Solution! Make one folder for ipython notebooks in dropbox!
- 41. But I have to keep switching computers …and I can’t remember to keep committing repositories. BAM! Solution! Make one folder for ipython notebooks in dropbox! But how do I have ipython notebook start in dropbox?
- 42. Creating a custom startup directory Create your profile $ ipython profile create $ vi ~/.ipython/profile_default/ipython_notebook_config Change line 551 from: #c.NotebookManager.notebook_dir = u’’ to: c.NotebookManager.notebook_dir = u’/home/staff/$USER/Dropbox/Notebooks/’
- 43. Kick it up a notch What if I told you about a world where you could always access your programming environment so long as you had internet?
- 44. Kick it up a notch What if I told you about a world where you could always access your programming environment so long as you had internet? Or refer to your graphs?
- 45. Kick it up a notch What if I told you about a world where you could always access your programming environment so long as you had internet? Or refer to your graphs? Or show your partners/collaborators?
- 46. Make a server! Okay, but hold on there. Here is the HUGE NUMBER ONE RULE
- 47. Make a server! YOU MUST PASSWORD PROTECT IT!
- 48. Make a server! YOU MUST PASSWORD PROTECT IT! AND NOT WITH YOUR USER PASSWORD!
- 49. Make a server! YOU MUST PASSWORD PROTECT IT! AND NOT WITH YOUR USER PASSWORD! MAKE IT A VERYYYY LONG PASSPHRASE
- 50. So let’s make a password Open iPython from your command line (NOT the regular python interpreter!) $ ipython From Thomas Sileo’s blog
- 51. So let’s make a password Open iPython from your command line (NOT the regular python interpreter!) $ ipython From Thomas Sileo’s blog
- 52. Now set up the profile WITH the password From Thomas Sileo’s blog
- 53. Now set up the profile WITH the password From Thomas Sileo’s blog
- 54. And start that baby up! Neat idea: Use the server with the notebook directory in your dropbox.
- 55. And start that baby up! Neat idea: Use the server with the notebook directory in your dropbox. Just copy new data files into dropbox and bam! instant transfer and analysis.