SlideShare a Scribd company logo

Managing and Scaling Puppet - PuppetConf 2014

•Download as PPTX, PDF•
•
Puppet
Puppet

Miguel Zuniga presented on managing and scaling Puppet. The presentation covered using a Puppet master with a web cluster for scaling, adding caching to reduce load, using source control with Puppet, multi-datacenter configurations, masterless Puppet in the cloud, and future directions including search capabilities and dynamic configurations. Zuniga took questions at the end.

Related slideshows

Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails
Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in RailsRails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails
Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails
 
Cassandra
CassandraCassandra
Cassandra
 
The Zen of High Performance Messaging with NATS (Strange Loop 2016)
The Zen of High Performance Messaging with NATS (Strange Loop 2016)The Zen of High Performance Messaging with NATS (Strange Loop 2016)
The Zen of High Performance Messaging with NATS (Strange Loop 2016)
 
1 of 18
Managing and Scaling Puppet
Who is this guy? Name: Miguel Zuniga Job: Computer guy @ Symantec Past: Ebay, Paypal, EA, Rackspace and many more Puppet user since: 0.22 mostly 0.24 Not much of a social network user but just in case: @mikezuniga +MiguelZuniga
Agenda ● Puppet and Puppetmaster ● Scaling with a web cluster ● Less load more cache ● SCM and puppet ● Multi datacenter ● Masterless and the cloud ● Moving forward ● Questions?
Puppet and Puppetmaster Puppet: ● Client - Server (with puppetmaster) ● Client Only (puppet apply) ● Applies changes to nodes Puppetmaster (Puppet server) ● CA authority ● Runs functions ● Keeps tracks of nodes ● Store data (facters)
Puppet and Puppetmaster
Puppet and Puppetmaster
Scaling with a web cluster
Scaling with web cluster Pros ● You can scale if you have money ● Simple configuration, almost drag and drop ● Puppet CA to rule them all Cons ● More complexity ● If not SSL termination in use you need to share certs across all puppetmasters ● More clients = more load = more money
Scaling with web cluster Usual setup Apache + Passenger for puppetmasters Haproxy or Physical LB Nginx + Passenger for puppetmasters Apache reverse proxy + mod_ssl for LB Nginx + Passenger for puppetmasters Nginx loadbalancing + ssl for LB
Less load more cache Puppet with passenger works as a Rack web application Almost all web applications can benefit from having a caching layer Will it work?
Less load more cache server { listen 8140 ssl; server_name puppet <%= @puppet_server %>; ssl_certificate /var/lib/puppet/ssl/certs/<%= @puppet_server %>.pem; ssl_certificate_key /var/lib/puppet/ssl/private_keys/<%= @puppet_server %>.pem; ssl_crl /var/lib/puppet/ssl/ca/ca_crl.pem; ssl_client_certificate /var/lib/puppet/ssl/certs/ca.pem; ssl_ciphers SSLv2:-LOW:-EXPORT:RC4+RSA; ssl_prefer_server_ciphers on; ssl_verify_client optional; ssl_verify_depth 1; ssl_session_cache shared:SSL:128m; ssl_session_timeout 5m; access_log /var/log/nginx-puppet_access.log headerlog; error_log /var/log/nginx-puppet_error.log; location ~* /certificate.*? { proxy_pass http://puppetca; } location ~* /node/ { return 404; } location / { proxy_pass http://puppetmaster; proxy_cache one; proxy_cache_methods GET POST; proxy_cache_valid 200 7d; } }
Less load more cache Note: Puppet > 3 use nginx with POST cache
SCM and Puppet Use any SCM to keep track of your changes. The less environments you have, the better. Make logical decisions on classes. Categorize your clients by roles. Use requires instead of includes. Virtual resources are always fun. Manage dependencies.
Multi Data Center Distribute the cache servers as endpoints Use the SCM to replicate code One central source of code and CA Use foreman, cobbler, razor... to generate your node configurations. Define downtime windows to pull new changes from SCM Configure a class specifically to clear the cache for that downtime window Remember standardization is your friend
Masterless and the Cloud Create a bootstrap script which loads the basic needs of your environment through puppet apply. Connect your clients to the puppet master at the end of bootstrap Maintain certs through query the cloud or cmdb. If certs are really a problem generate one cert for all (not recommended).
Moving Forward ● Search function ○ Do queries against a CMDB, PuppetDB, Ldap Nodes, Foreman, X, Y, Z ● Dynamic configurations ○ Based on the result modify catalogs through variables which could allow nodes to change them selves.
Use cases of search ● Discover new nodes ● Semi-orchestrate ● Create dynamic configurations ● Notification based on dynamic resources Example: Let know HAproxy that a new node is ready to be added.
Thank you Questions?

More Related Content

Managing and Scaling Puppet - PuppetConf 2014

  • 2. Who is this guy? Name: Miguel Zuniga Job: Computer guy @ Symantec Past: Ebay, Paypal, EA, Rackspace and many more Puppet user since: 0.22 mostly 0.24 Not much of a social network user but just in case: @mikezuniga +MiguelZuniga
  • 3. Agenda ● Puppet and Puppetmaster ● Scaling with a web cluster ● Less load more cache ● SCM and puppet ● Multi datacenter ● Masterless and the cloud ● Moving forward ● Questions?
  • 4. Puppet and Puppetmaster Puppet: ● Client - Server (with puppetmaster) ● Client Only (puppet apply) ● Applies changes to nodes Puppetmaster (Puppet server) ● CA authority ● Runs functions ● Keeps tracks of nodes ● Store data (facters)
  • 7. Scaling with a web cluster
  • 8. Scaling with web cluster Pros ● You can scale if you have money ● Simple configuration, almost drag and drop ● Puppet CA to rule them all Cons ● More complexity ● If not SSL termination in use you need to share certs across all puppetmasters ● More clients = more load = more money
  • 9. Scaling with web cluster Usual setup Apache + Passenger for puppetmasters Haproxy or Physical LB Nginx + Passenger for puppetmasters Apache reverse proxy + mod_ssl for LB Nginx + Passenger for puppetmasters Nginx loadbalancing + ssl for LB
  • 10. Less load more cache Puppet with passenger works as a Rack web application Almost all web applications can benefit from having a caching layer Will it work?
  • 11. Less load more cache server { listen 8140 ssl; server_name puppet <%= @puppet_server %>; ssl_certificate /var/lib/puppet/ssl/certs/<%= @puppet_server %>.pem; ssl_certificate_key /var/lib/puppet/ssl/private_keys/<%= @puppet_server %>.pem; ssl_crl /var/lib/puppet/ssl/ca/ca_crl.pem; ssl_client_certificate /var/lib/puppet/ssl/certs/ca.pem; ssl_ciphers SSLv2:-LOW:-EXPORT:RC4+RSA; ssl_prefer_server_ciphers on; ssl_verify_client optional; ssl_verify_depth 1; ssl_session_cache shared:SSL:128m; ssl_session_timeout 5m; access_log /var/log/nginx-puppet_access.log headerlog; error_log /var/log/nginx-puppet_error.log; location ~* /certificate.*? { proxy_pass http://puppetca; } location ~* /node/ { return 404; } location / { proxy_pass http://puppetmaster; proxy_cache one; proxy_cache_methods GET POST; proxy_cache_valid 200 7d; } }
  • 12. Less load more cache Note: Puppet > 3 use nginx with POST cache
  • 13. SCM and Puppet Use any SCM to keep track of your changes. The less environments you have, the better. Make logical decisions on classes. Categorize your clients by roles. Use requires instead of includes. Virtual resources are always fun. Manage dependencies.
  • 14. Multi Data Center Distribute the cache servers as endpoints Use the SCM to replicate code One central source of code and CA Use foreman, cobbler, razor... to generate your node configurations. Define downtime windows to pull new changes from SCM Configure a class specifically to clear the cache for that downtime window Remember standardization is your friend
  • 15. Masterless and the Cloud Create a bootstrap script which loads the basic needs of your environment through puppet apply. Connect your clients to the puppet master at the end of bootstrap Maintain certs through query the cloud or cmdb. If certs are really a problem generate one cert for all (not recommended).
  • 16. Moving Forward ● Search function ○ Do queries against a CMDB, PuppetDB, Ldap Nodes, Foreman, X, Y, Z ● Dynamic configurations ○ Based on the result modify catalogs through variables which could allow nodes to change them selves.
  • 17. Use cases of search ● Discover new nodes ● Semi-orchestrate ● Create dynamic configurations ● Notification based on dynamic resources Example: Let know HAproxy that a new node is ready to be added.