SlideShare a Scribd company logo

Docker at DevTable

•Download as PPTX, PDF•
•
D
dotCloud

This document discusses how DevTable uses Docker containers to securely run development tools and services for users. Some key points: - DevTable runs each user's code in a separate Docker container to isolate potentially dangerous tools like debuggers or terminals. - It uses the Python Docker API and runs an SSH "command and control" process to execute commands inside containers quickly and securely. - When files change, notifications are passed between the backend and container server to keep the container and hosted code in sync. - Dynamic port mapping through HAProxy allows container services to be accessed publicly through subdomains while maintaining isolation. - Docker provides a major speed improvement over their previous LXC solution, booting containers in

1 of 44
Docker at DevTable
What is DevTable? DevTable is a browser-based, hosted, collaborative IDE Develop in the cloud with the same power as your desktop applications
Code
Collaborate
Debug and Test
Deploy ● Google App Engine ● REST ● SCP ● Git (Heroku and other providers)
Sealed evil in a can There are a lot of neat things that we run for our users, but they are all potentially very dangerous: ● App Engine Development Server ● Debuggers and Emulators ● REPLs (Python, etc) ● Terminal support (which means all of the above as well)
Why this is a problem ● Without a containment system of some kind, any of these awesome features would allow users to cause mayhem: ○ A REPL use could open any file ○ A DevServer can execute arbitrary code ○ A terminal could allow anything to happen
Why not simply use permissions? ● Permissions solve the file access problem ● Permissions do not prevent users from causing other system issues: instability, exhaustion, escalation, etc
Solution: containers! To contain the insecurity of running live code, we run all non-custom code in a container, with only the user’s project mounted and available Evil (not to scale) Project data Container
Ideal container properties ● Lightweight ● Secure ● Easy to manage ● FAST
Originally we used LXC... ● Lightweight (sort of…) ● Secure ● Easy to manage (sort of...) ● FAST
In the beginning, there was LXC... … and it was slow. ● Typical startup times for our containers were on the order of minutes ● Starting a debugger or shell is not fun at those speeds ● Getting the security and management just right was quite painful
Then the community said “let there be Docker”... Yo!
… and it made things amazing. Our average startup time for a container has dropped from over a minute to just under four seconds. LXC Docker Go make a cup of coffee and play swords on office chairs Go!
Before Docker
But, but Docker is just... LXC... Almost, Docker does some things that make starting up single processes lightning quick: ● Incremental by default ● Replace distro init process with lightweight version ● No DHCP, upstart, dnsmasq, etc. ● Aufs seems to be faster than OverlayFS ● Build process is MUCH better (Dockerfiles)
Docker at DevTable The fun technical details!
DevTable overview Clients Web browsers Clients - Web browsers Frontends Python Clients - Web browsers Backends C# WebSocket Socket DFS Clients - Web browsers Container Servers Python Thrift SSH HTTP ? Images
Things we’ll discuss today Clients Web browsers Clients - Web browsers Frontends Python Clients - Web browsers Backends C# WebSocket Socket DFS Clients - Web browsers Container Servers Python Thrift HTTP SSH ? Images
How we use docker now ● Python Docker API bindings ● Run a single instance per project ● Mount only the files relevant to the project in the container ● Run an SSH “command and control” process ● Execute user processes through SSH ● Dynamic version of Docker port forwarding
Backend <-> Container server Backends C# Container Servers Python Thrift
Container server The container server is the server in charge of managing all aspects related to the Docker containers ● Written in Python ● Conforms to a Thrift interface ● Called by the Backends to start containers, stop containers, run commands, mount file systems in containers, etc
Container server startContainer Starts a new container for a project. runCommand Runs a command inside a container stopCommand Stops a command inside a container notifyFilesModifed Notifies a container that a file has been modified by the backend stopContainer Stops a container
Handling file changes ● Changes made by the container or the backend to the DFS are propagated automatically ● However, both sides have code that depends on notification of changes ● Each server notifies the other about changes that occur via a notification service
DFS change notifications Backend C# Container Server Python Hey, a user added file “test.txt” in container 1234 Backend C# Container Server Python Hey, the user changed file “foo.py” in container 1
How we handle file changes in Docker ● The container server watches changes inside the container using inotify, and reports changes to the backend ● The backend reports changes to the container server which will touch files that have been added or changed
Container server <-> Docker Clients - Web browsers Container Servers Python SSH
Container server <-> Docker We use the Python Docker bindings to create a new image and load it with a temporary ssh key New container requests bring up the container with the known session SSH key and issue commands to the container via SSH Much better than LXC issuing commands via subprocess
Docker <-> Outside world For many services we run (such as the App Engine Development Server), we need to expose the server running inside Docker to the outside world
Docker <-> Outside world HTTP Clients Web browsers HTTP Container Server HAProxy
Docker <-> Outside world Services inside of Docker as exposed via dynamic port mapping to a HAProxy running on the container server The HAProxy exposes the port by remapping it to the external port and a custom subdomain
Docker <-> Outside world Container Server 93nx83ndsc34mn.c4.devtable.io:80Clients Web browsers Port 38563 HAProxy
Example: running a dev server 1. Backend requests a container from the server Backend C# Container Server Python I need a container for project “testapplication” Container “container1234” started for project
Example: running a dev server 2. Backend registers for file notification events Backend C# Container Server Python Let me know if any files change Duly noted
Example: running a dev server 3. Backend asks for the dev server to be started and port 80 to be forwarded Backend C# Container Server Python Please start the dev server and forward port 80 Dev server started and port is forwarded at subdomain foobarbaz
Example: running a dev server 1. Container server tells Docker to start a container Container Server Python create_container, mount_filesystem, forward_port, start_ssh Done. Port exposed: 84639
Example: running a dev server 2. Container server tells HAProxy to forward the port returned by docker Container Server Python Forward port 84639 as subdomain foobarbaz HAProxy
Example: running a dev server 3. Container server tells Docker to run the dev server Container Server Python ssh command_for_devserver
Summary Docker has allowed DevTable to run amazing tools securely and fast, without a large management overhead
Future opportunities Docker presents some amazing new opportunities for DevTable and the community: ● Ability to quickly load (and save) complete development environments, securely ● Ability to quickly write custom plugins and run them in our IDE (want to analyze and build Go? just give us a URL or a Dockerfile!)
But wait… There’s something that has been bugging us… How should we distribute our private images in production?
Quay Demo At this point in the live talk we unveiled and gave a demo of our hosted private docker registry called Quay.io.
Questions? Comments? Witty anecdotes? devtable.com Jacob Moshenko - jake@devtable.com Joseph Schorr - jschorr@devtable.com

More Related Content

Docker at DevTable

  • 2. What is DevTable? DevTable is a browser-based, hosted, collaborative IDE Develop in the cloud with the same power as your desktop applications
  • 6. Deploy ● Google App Engine ● REST ● SCP ● Git (Heroku and other providers)
  • 7. Sealed evil in a can There are a lot of neat things that we run for our users, but they are all potentially very dangerous: ● App Engine Development Server ● Debuggers and Emulators ● REPLs (Python, etc) ● Terminal support (which means all of the above as well)
  • 8. Why this is a problem ● Without a containment system of some kind, any of these awesome features would allow users to cause mayhem: ○ A REPL use could open any file ○ A DevServer can execute arbitrary code ○ A terminal could allow anything to happen
  • 9. Why not simply use permissions? ● Permissions solve the file access problem ● Permissions do not prevent users from causing other system issues: instability, exhaustion, escalation, etc
  • 10. Solution: containers! To contain the insecurity of running live code, we run all non-custom code in a container, with only the user’s project mounted and available Evil (not to scale) Project data Container
  • 11. Ideal container properties ● Lightweight ● Secure ● Easy to manage ● FAST
  • 12. Originally we used LXC... ● Lightweight (sort of…) ● Secure ● Easy to manage (sort of...) ● FAST
  • 13. In the beginning, there was LXC... … and it was slow. ● Typical startup times for our containers were on the order of minutes ● Starting a debugger or shell is not fun at those speeds ● Getting the security and management just right was quite painful
  • 14. Then the community said “let there be Docker”... Yo!
  • 15. … and it made things amazing. Our average startup time for a container has dropped from over a minute to just under four seconds. LXC Docker Go make a cup of coffee and play swords on office chairs Go!
  • 17. But, but Docker is just... LXC... Almost, Docker does some things that make starting up single processes lightning quick: ● Incremental by default ● Replace distro init process with lightweight version ● No DHCP, upstart, dnsmasq, etc. ● Aufs seems to be faster than OverlayFS ● Build process is MUCH better (Dockerfiles)
  • 18. Docker at DevTable The fun technical details!
  • 19. DevTable overview Clients Web browsers Clients - Web browsers Frontends Python Clients - Web browsers Backends C# WebSocket Socket DFS Clients - Web browsers Container Servers Python Thrift SSH HTTP ? Images
  • 20. Things we’ll discuss today Clients Web browsers Clients - Web browsers Frontends Python Clients - Web browsers Backends C# WebSocket Socket DFS Clients - Web browsers Container Servers Python Thrift HTTP SSH ? Images
  • 21. How we use docker now ● Python Docker API bindings ● Run a single instance per project ● Mount only the files relevant to the project in the container ● Run an SSH “command and control” process ● Execute user processes through SSH ● Dynamic version of Docker port forwarding
  • 22. Backend <-> Container server Backends C# Container Servers Python Thrift
  • 23. Container server The container server is the server in charge of managing all aspects related to the Docker containers ● Written in Python ● Conforms to a Thrift interface ● Called by the Backends to start containers, stop containers, run commands, mount file systems in containers, etc
  • 24. Container server startContainer Starts a new container for a project. runCommand Runs a command inside a container stopCommand Stops a command inside a container notifyFilesModifed Notifies a container that a file has been modified by the backend stopContainer Stops a container
  • 25. Handling file changes ● Changes made by the container or the backend to the DFS are propagated automatically ● However, both sides have code that depends on notification of changes ● Each server notifies the other about changes that occur via a notification service
  • 26. DFS change notifications Backend C# Container Server Python Hey, a user added file “test.txt” in container 1234 Backend C# Container Server Python Hey, the user changed file “foo.py” in container 1
  • 27. How we handle file changes in Docker ● The container server watches changes inside the container using inotify, and reports changes to the backend ● The backend reports changes to the container server which will touch files that have been added or changed
  • 28. Container server <-> Docker Clients - Web browsers Container Servers Python SSH
  • 29. Container server <-> Docker We use the Python Docker bindings to create a new image and load it with a temporary ssh key New container requests bring up the container with the known session SSH key and issue commands to the container via SSH Much better than LXC issuing commands via subprocess
  • 30. Docker <-> Outside world For many services we run (such as the App Engine Development Server), we need to expose the server running inside Docker to the outside world
  • 31. Docker <-> Outside world HTTP Clients Web browsers HTTP Container Server HAProxy
  • 32. Docker <-> Outside world Services inside of Docker as exposed via dynamic port mapping to a HAProxy running on the container server The HAProxy exposes the port by remapping it to the external port and a custom subdomain
  • 33. Docker <-> Outside world Container Server 93nx83ndsc34mn.c4.devtable.io:80Clients Web browsers Port 38563 HAProxy
  • 34. Example: running a dev server 1. Backend requests a container from the server Backend C# Container Server Python I need a container for project “testapplication” Container “container1234” started for project
  • 35. Example: running a dev server 2. Backend registers for file notification events Backend C# Container Server Python Let me know if any files change Duly noted
  • 36. Example: running a dev server 3. Backend asks for the dev server to be started and port 80 to be forwarded Backend C# Container Server Python Please start the dev server and forward port 80 Dev server started and port is forwarded at subdomain foobarbaz
  • 37. Example: running a dev server 1. Container server tells Docker to start a container Container Server Python create_container, mount_filesystem, forward_port, start_ssh Done. Port exposed: 84639
  • 38. Example: running a dev server 2. Container server tells HAProxy to forward the port returned by docker Container Server Python Forward port 84639 as subdomain foobarbaz HAProxy
  • 39. Example: running a dev server 3. Container server tells Docker to run the dev server Container Server Python ssh command_for_devserver
  • 40. Summary Docker has allowed DevTable to run amazing tools securely and fast, without a large management overhead
  • 41. Future opportunities Docker presents some amazing new opportunities for DevTable and the community: ● Ability to quickly load (and save) complete development environments, securely ● Ability to quickly write custom plugins and run them in our IDE (want to analyze and build Go? just give us a URL or a Dockerfile!)
  • 42. But wait… There’s something that has been bugging us… How should we distribute our private images in production?
  • 43. Quay Demo At this point in the live talk we unveiled and gave a demo of our hosted private docker registry called Quay.io.
  • 44. Questions? Comments? Witty anecdotes? devtable.com Jacob Moshenko - jake@devtable.com Joseph Schorr - jschorr@devtable.com