Overview of Chef - Fundamentals Webinar Series Part 1
•
40 likes•103,661 views
This is an Overview of Chef. After viewing this webinar you will be able to: - Describe how Chef thinks about Infrastructure Automation - Define the following terms: - Resource - Recipe - Node - Run List - Search - Login to Hosted Chef - Run `knife` commands from your workstation Video of this webinar can be found at the following URL https://www.youtube.com/watch?v=S5lHUpzoCYo&list=PL11cZfNdwNyPnZA9D1MbVqldGuOWqbumZ
Report
Share
Overview of Chef - Fundamentals Webinar Series Part 1
- 1. Chef Fundamentals training@getchef.com Copyright (C) 2014 Chef Software, Inc.
- 2. Nathen Harvey • Community Director • Co-host of the Food Fight Show Podcast • @nathenharvey
- 3. Webinar Objectives and Style 3
- 4. Multi-week Webinar Series • After completing of this webinar series you will be able to • Automate common infrastructure tasks with Chef • Describe Chef’s architecture • Describe Chef’s various tools • Apply Chef’s primitives to solve your problems
- 5. How to learn Chef • You bring the domain expertise about your business and infrastructure • Chef provides a framework for automating your infrastructure • Our job is to work together to teach you how to model and automate your infrastructure with Chef
- 6. Chef is a Language • Learning Chef is like learning the basics of a language • 80% fluency will be reached very quickly • The remaining 20% just takes practice • The best way to learn Chef is to use Chef
- 7. Agenda 7
- 8. Topics • Overview of Chef • Workstation Setup • Node Setup • Chef Resources and Recipes • Introducing the Node object • Setting Node attributes • Roles • Community Cookbooks
- 10. Lesson Objectives • After completing the lesson, you will be able to • Describe how Chef thinks about Infrastructure Automation • Define the following terms: • Resource • Recipe • Node • Run List • Search
- 12. Items of Manipulation (Resources) • Networking • Files • Directories • Symlinks • Mounts • Registry Keys • Powershell Scripts • Users • Groups • Packages • Services • Filesystems 12
- 13. Application A tale of growth... 13
- 14. Application Application Database Add a database 14
- 15. Application App Databases Make database redundant 15
- 16. App Servers App Databases Application server redundancy 16
- 17. App LB App Servers App Databases Add a load balancer 17
- 18. App LBs App Servers App Databases Webscale! 18
- 19. App LBs App Servers App DB Cache App DBs Now we need a caching layer 19
- 20. App LBs App Servers App DB Cache App DBs Infrastructure has a Topology 20
- 21. Round Robin DNS App Servers App DB Cache App DBs Floating IP? Your Infrastructure is a Snowflake 21
- 22. App LBs App Servers < Shiny! DB slaves Cache DB Cache DBs Complexity Increases Quickly Are we monitoring?? 22
- 23. App LBs App Servers DB slaves Cache DB Cache DBs ...and change happens! 23
- 24. App LBs App Servers DB slaves Cache DB Cache DBs ...and change happens! Add a Central Log Host Central Log Host 24
- 25. App LBs App Servers DB slaves Cache DB Cache DBs ...and change happens! Add a Central Log Host Update syslog.conf on all Nodes Central Log Host 25
- 26. Chef Solves This Problem • But you already guessed that, didn’t you? 26
- 27. Chef is Infrastructure as Code • Programmatically provision and configure components http://www.flickr.com/photos/louisb/4555295187/ 27
- 28. Chef is Infrastructure as Code • Treat like any other code base http://www.flickr.com/photos/louisb/4555295187/ 28
- 29. Chef is Infrastructure as Code • Reconstruct business from code repository, data backup, and compute resources http://www.flickr.com/photos/louisb/4555295187/ 29
- 30. Chef is Infrastructure as Code • Programmatically provision and configure components • Treat like any other code base • Reconstruct business from code repository, data backup, and compute resourceshttp://www.flickr.com/photos/louisb/4555295187/ 30
- 31. Configuration Code • Chef ensures each Node complies with the policy • Policy is determined by the configurations in each Node’s run list • Reduce management complexity through abstraction • Store the configuration of your infrastructure in version control 31
- 32. Declarative Interface to Resources • You define the policy in your Chef configuration • Your policy states what state each resource should be in, but not how to get there • Chef-client will pull the policy from the Chef Server and enforce the policy on the Node 32
- 33. Managing Complexity • Resources • Recipes • Nodes • Search 33
- 34. Resources • A Resource represents a piece of the system and its desired state • A package that should be installed • A service that should be running • A file that should be generated • A cron job that should be configured • A user that should be managed • and more 34
- 35. Resources in Recipes • Resources are the fundamental building blocks of Chef configuration • Resources are gathered into Recipes • Recipes ensure the system is in the desired state 35
- 36. Recipes • Configuration files that describe resources and their desired state • Recipes can: • Install and configure software components • Manage files • Deploy applications • Execute other recipes • and more 36
- 37. Example Recipe package "httpd" do action :start end template "/etc/httpd/conf/httpd.conf" do source "httpd.conf.erb" owner "root" group "root" mode "0644" variables(:allow_override => "All") notifies :reload, "service[httpd]" end service "httpd" do action [:enable,:start] supports :reload => true end 37
- 38. Nodes 38
- 39. Nodes Adhere to Policy • The chef-client application runs on each node, which • Gathers the current system configuration of the node • Downloads the desired system configuration policies from the Chef server for that node • Configures the node such that it adheres to those policies 39
- 40. Run List Node Enterprise Chef chef-client What policy should I follow? 40
- 41. Run List Node Enterprise Chef chef-client What policy should I follow? "recipe[ntp::client]" "recipe[users]" "role[webserver]" 41
- 42. Run List Enterprise Chef chef-client What policy should I follow? "recipe[ntp::client]" "recipe[users]" "role[webserver]" 42
- 43. Run List Specifies Policy • The Run List is an ordered collection of policies that the Node should follow • Chef-client obtains the Run List from the Chef Server • Chef-client ensures the Node complies with the policy in the Run List 43
- 44. Search • Search for nodes with Roles • Find Topology Data • IP addresses • Hostnames • FQDNs http://www.flickr.com/photos/kathycsus/268677262544
- 45. Search for Nodes pool_members = search("node","role:webserver") template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root" group "root" mode 0644 variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]" end 45
- 47. Webservers HAProxy Load Balancer 47 HA ProxyEnterprise Chef pool_members = search("node","role:webserver")
- 48. Webservers HAProxy Load Balancer 48 HA ProxyEnterprise Chef Webservers? pool_members = search("node","role:webserver")
- 49. Webservers HAProxy Load Balancer 49 HA ProxyEnterprise Chef Webservers? pool_members = search("node","role:webserver")
- 50. Webservers HAProxy Load Balancer 50 HA ProxyEnterprise Chef Webservers? pool_members = search("node","role:webserver") { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } }
- 51. Webservers HAProxy Load Balancer 51 HA ProxyEnterprise Chef Webservers? pool_members = search("node","role:webserver") { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } }
- 52. Webservers HAProxy Load Balancer 52 HA ProxyEnterprise Chef Webservers? pool_members = search("node","role:webserver") { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } } pool_members { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } }
- 53. Search for Nodes pool_members = search("node","role:webserver") template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root" group "root" mode 0644 variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]" end 53
- 54. Pass results into Templates # Set up application listeners here. listen application 0.0.0.0:80 balance roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> <% if node["haproxy"]["enable_admin"] -%> listen admin 0.0.0.0:22002 mode http stats uri / <% end -%> 54
- 55. Webservers HAProxy Configuration 55 HA Proxypool_members { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } } <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%>
- 56. Webservers HAProxy Configuration 56 HA Proxypool_members { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } } <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> haproxy.cfg server web01 10.1.1.1 weight 1 maxconn 1 check
- 57. Webservers HAProxy Configuration 57 HA Proxypool_members { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } } <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> haproxy.cfg server web01 10.1.1.1 weight 1 maxconn 1 check server web02 10.1.1.2 weight 1 maxconn 1 check
- 58. Webservers HAProxy Configuration 58 HA Proxypool_members { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } } <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> haproxy.cfg server web01 10.1.1.1 weight 1 maxconn 1 check server web02 10.1.1.2 weight 1 maxconn 1 check server web03 10.1.1.3 weight 1 maxconn 1 check
- 59. Webservers HAProxy Configuration 59 HA Proxypool_members { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } } <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> haproxy.cfg server web01 10.1.1.1 weight 1 maxconn 1 check server web02 10.1.1.2 weight 1 maxconn 1 check server web03 10.1.1.3 weight 1 maxconn 1 check server web04 10.1.1.4 weight 1 maxconn 1 check
- 60. Webservers HAProxy Configuration 60 HA Proxypool_members { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } } <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> haproxy.cfg server web01 10.1.1.1 weight 1 maxconn 1 check server web02 10.1.1.2 weight 1 maxconn 1 check server web03 10.1.1.3 weight 1 maxconn 1 check server web04 10.1.1.4 weight 1 maxconn 1 check server web05 10.1.1.5 weight 1 maxconn 1 check
- 61. Webservers HAProxy Configuration 61 HA Proxypool_members { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } } <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> haproxy.cfg server web01 10.1.1.1 weight 1 maxconn 1 check server web02 10.1.1.2 weight 1 maxconn 1 check server web03 10.1.1.3 weight 1 maxconn 1 check server web04 10.1.1.4 weight 1 maxconn 1 check server web05 10.1.1.5 weight 1 maxconn 1 check server web06 10.1.1.1 weight 1 maxconn 1 check
- 62. Jboss App Memcache Postgres Slaves Postgres Master NagiosGraphite So when this... 62
- 63. Jboss App Memcache Postgres Slaves Postgres Master NagiosGraphite ...becomes this 63
- 64. Memcache Postgres Slaves Postgres Master NagiosGraphite Jboss App Memcache Postgres Slaves Postgres Master NagiosGraphite ...this can happen automatically 64
- 65. NagiosGraphite NagiosGraphite Memcache Postgres Slaves • Load balancer config • Nagios host ping • Nagios host ssh • Nagios host HTTP • Nagios host app health • Graphite CPU • Graphite Memory • Graphite Disk • Graphite SNMP • Memcache firewall • Postgres firewall • Postgres authZ config • 12+ resource changes for 1 node addition Count the Resources Jboss App 65
- 66. Manage Complexity • Determine the desired state of your infrastructure • Identify the Resources required to meet that state • Gather the Resources into Recipes • Compose a Run List from Recipes • Apply a Run List to each Node in your environment • Your infrastructure adheres to the policy modeled in Chef 66
- 67. Configuration Drift • Configuration Drift happens when: • Your infrastructure requirements change • The configuration of a server falls out of policy • Chef makes it easy to manage • Model the new requirements in your Chef configuration files • Run the chef-client to enforce your policies 67
- 68. Recap • In this section, we have • Described how Chef thinks about Infrastructure Automation • Defined the following terms: • Resource • Recipe • Node • Run List • Search
- 69. What Questions Do You Have? Nathen Harvey Community Director nharvey@getchef.com @nathenharvey 69
- 71. Sign-up for Hosted Chef • http://getchef.com • Click “Get Chef” • Select “Hosted Chef” • Complete the registration form • Create an Organization
- 72. Their Infrastructure Organizations My Infrastructure Your Infrastructure 72
- 73. Organizations • Provide multi-tenancy in Enterprise Chef • Nothing is shared between Organizations - they're completely independent • May represent different • Companies • Business Units • Departments 73
- 74. Configure Workstation • Download and extract Chef starter kit • Install chef-client • http://getchef.com/chef/install 74
- 75. • Your version may be different, that’s ok! Verify Knife $ knife --version Chef: 11.12.4 $ knife client list ORGNAME-validator 75
- 76. OPEN IN EDITOR: chef-repo/.chef/knife.rb current_dir = File.dirname(__FILE__) log_level :info log_location STDOUT node_name "USERNAME" client_key "#{current_dir}/USERNAME.pem" validation_client_name "ORGNAME-validator" validation_key "#{current_dir}/ORGNAME-validator.pem" chef_server_url "https://api.opscode.com/organizations/ORGNAME" cache_type 'BasicFile' cache_options( :path => "#{ENV['HOME']}/.chef/checksums" ) cookbook_path ["#{current_dir}/../cookbooks"] knife.rb 76
- 77. 1.Reads the chef_server_url from knife.rb 2.Invokes HTTP GET to #{chef_server_url}/clients 3.Displays the result 77 knife client list
- 79. Six Week Series • May 20 - Overview of Chef • May 27 - Node Setup, Chef Resources & Recipes • June 3 - Working with the Node object • June 10 - Common configuration data with Databags • June 17 - Using Roles and Environments • June 24 - Community Cookbooks and Further Resources • * Topics subject to change, schedule unlikely to change
- 80. Sign-up for Webinar • http://pages.getchef.com/ cheffundamentalsseries.html
- 81. Additional Resources • Chef Fundamentals Webinar Series • https://www.youtube.com/watch? v=S5lHUpzoCYo&list=PL11cZfNdwNyPnZA9D1MbVqldGuOWqbum Z • Discussion group for webinar participants • https://groups.google.com/d/forum/learnchef-fundamentals-webinar 81
- 82. Additional Resources • Learn Chef • http://learnchef.com • Documentation • http://docs.opscode.com 82