SlideShare a Scribd company logo

Using Puppet and Cobbler to Automate Your Infrastructure

•
•
Phil Windley
Phil Windley

The document discusses using Puppet and Cobbler to automate infrastructure. It describes how Cobbler can be used for machine provisioning, managing images and repositories, and kicking off installs. Puppet is then used to configure systems by specifying desired configurations and ensuring services and files are in the proper state. A deployment system is also described that allows controlled deployments of new code or configurations to servers using a simple command line tool. The key benefits highlighted are improved reliability, scalability, repeatability and reduced mistakes through automation of the provisioning and configuration process.

1 of 89
Download to read offline
Using Puppet and Cobbler to Automate Your Infrastructure Phillip J. Windley, Ph.D Founder and CTO Kynetx www.kynetx.com Monday, October 12, 2009 1
Sleeping Through the Night Phillip J. Windley, Ph.D Founder and CTO Kynetx www.kynetx.com Monday, October 12, 2009 2
(afford|scal|reli)ability Monday, October 12, 2009 3
hire fewer people Monday, October 12, 2009 4
meet demand quickly Monday, October 12, 2009 5
make fewer mistakes Monday, October 12, 2009 6
Monday, October 12, 2009 7
Monday, October 12, 2009 8
Monday, October 12, 2009 8
Monday, October 12, 2009 8
Monday, October 12, 2009 9
1. machine provisioning Monday, October 12, 2009 9
1. machine provisioning 2. system configuration Monday, October 12, 2009 9
1. machine provisioning 2. system configuration 3. deployment Monday, October 12, 2009 9
Monday, October 12, 2009 10
provisioning Monday, October 12, 2009 11
machine provisioning Monday, October 12, 2009 12
machine provisioning manage images & repositories Monday, October 12, 2009 12
machine provisioning manage images & repositories kickstart machines Monday, October 12, 2009 12
machine provisioning manage images & repositories kickstart machines handle physical and virtual hardware Monday, October 12, 2009 12
machine provisioning manage images & repositories kickstart machines handle physical and virtual hardware set up DHCP and DNS Monday, October 12, 2009 12
Monday, October 12, 2009 13
cobbler is a collection of tools that support machine provisioning Monday, October 12, 2009 13
Monday, October 12, 2009 14
cobblerd dns power dhcp Monday, October 12, 2009 14
cobblerd dns power dhcp cobbler web repos images kickstart Monday, October 12, 2009 14
koan cobblerd dns power dhcp cobbler web repos images kickstart Monday, October 12, 2009 14
cobbler uses a collection of specifications that define your systems Monday, October 12, 2009 15
Monday, October 12, 2009 16
distro Monday, October 12, 2009 16
profile repo distro Monday, October 12, 2009 16
system profile repo distro Monday, October 12, 2009 16
import a distro cobbler import --mirror ~/fc8 --name fc8 Monday, October 12, 2009 17
import a distro cobbler import --mirror ~/fc8 --name fc8 create a profile cobbler profile add --name=base-fc8 --distro=fc8-xen-i386 --kickstart=/root/base-fc8.ks --repos=fc8-newkey-repo Monday, October 12, 2009 17
import a distro cobbler import --mirror ~/fc8 --name fc8 create a profile cobbler profile add --name=base-fc8 --distro=fc8-xen-i386 --kickstart=/root/base-fc8.ks --repos=fc8-newkey-repo define a system cobbler system add --name=log0 --mac=00:16:3E:4B:40:00 --ip=192.168.122.180 --profile=base-fc8 --hostname=log0 Monday, October 12, 2009 17
building a machine koan --server=cobbler.kobj.net --virt --nogfx --system=log0 Monday, October 12, 2009 18
configuration Monday, October 12, 2009 19
system configuration Monday, October 12, 2009 20
system configuration critical services on or off Monday, October 12, 2009 20
system configuration critical services on or off security systems configured correctly Monday, October 12, 2009 20
system configuration critical services on or off security systems configured correctly users created Monday, October 12, 2009 20
system configuration critical services on or off security systems configured correctly users created necessary libraries in place Monday, October 12, 2009 20
system configuration critical services on or off security systems configured correctly users created necessary libraries in place right packages built & installed Monday, October 12, 2009 20
Monday, October 12, 2009 21
puppet is a language for specifying desired system configuration Monday, October 12, 2009 21
install package Monday, October 12, 2009 22
install package configure Monday, October 12, 2009 22
install package configuration should be modified after package installation configure Monday, October 12, 2009 22
install package configuration should be modified after package installation configure service Monday, October 12, 2009 22
install package configuration should be modified after package installation configure service should restart whenever configuration changes service Monday, October 12, 2009 22
the hard way yum install openssh-server vi /etc/ssh/sshd_config service sshd start Monday, October 12, 2009 23
the puppet way class ssh { package { ssh: ensure => installed } file { sshd_config: name => “/etc/ssh/sshd_config”, owner=> root, source => “puppet://server/apps/ssh/…”, after => Package[ssh] } service { sshd: ensure => running, subscribe => [Package[ssh], File[sshd_config]] } } Monday, October 12, 2009 24
the puppet way class ssh { package { ssh: ensure => installed } file { sshd_config: name => “/etc/ssh/sshd_config”, owner=> root, source => “puppet://server/apps/ssh/…”, after => Package[ssh] } service { sshd: ensure => running, subscribe => [Package[ssh], File[sshd_config]] } } Monday, October 12, 2009 24
the puppet way class ssh { package { ssh: ensure => installed } file { sshd_config: name => “/etc/ssh/sshd_config”, owner=> root, source => “puppet://server/apps/ssh/…”, after => Package[ssh] } service { sshd: ensure => running, subscribe => [Package[ssh], File[sshd_config]] } } Monday, October 12, 2009 24
wait a minute… that looks like a lot more lines to me! Monday, October 12, 2009 25
deployment Monday, October 12, 2009 26
requirements Monday, October 12, 2009 27
requirements deployment happens over & over again Monday, October 12, 2009 27
requirements deployment happens over & over again controlled, not continuous Monday, October 12, 2009 27
requirements deployment happens over & over again controlled, not continuous role-based Monday, October 12, 2009 27
requirements deployment happens over & over again controlled, not continuous role-based remotable Monday, October 12, 2009 27
now for deployment... Monday, October 12, 2009 28
now for deployment... Monday, October 12, 2009 28
now for deployment... Monday, October 12, 2009 28
now for deployment... Monday, October 12, 2009 28
in the end… I just wrote it in Perl in a few hours Monday, October 12, 2009 29
[root@ops deploy]# ./deploy.pl -d The following tasks are configured: deploy | Export a new copy of the code install | deploy, initialize, restart uninstall | rollback code, initialize,restart start_httpd | Start the HTTP server rollback | Rollback to the deploy stop_httpd | Stop the HTTP server test_server | Run the appropriate server test cleanup | Remove old copies of code test_code | Run the all tests configure_httpd| Build the httpd.conf file install_init | Install the init JS files restart_httpd | Restart the HTTP server Monday, October 12, 2009 30
[root@ops deploy]# ./deploy.pl -s server | version -----------------------|---------------- init0.kobj.net | 340M init1.kobj.net | 340M log.kobj.net | 340 log0.kobj.net | 340 log1.kobj.net | 340 krl.kobj.net | 340 cs0.kobj.net | 341 cs1.kobj.net | 341 cs2.kobj.net | 341 cs3.kobj.net | 341 Monday, October 12, 2009 31
[root@ops deploy]# ./deploy.pl -m krl -t install Performing install on krl with role krl... A /web/lib/releases/perl_0910091229/ops ... A /web/lib/releases/perl_0910091229/startup.pl A /web/lib/releases/perl_0910091229/Kynetx.pm A /web/lib/releases/perl_0910091229/README Checked out revision 342. Writing /web/conf/httpd.conf Stopping httpd: [ OK ] Starting httpd: [ OK ] Testing RuleManager.....ok All tests successful. Files=1, Tests=73, 8 wallclock secs ... Result: PASS Monday, October 12, 2009 32
TODO Monday, October 12, 2009 33
TODO configuration database Monday, October 12, 2009 33
TODO configuration database (more) automated testing Monday, October 12, 2009 33
TODO configuration database (more) automated testing continuous integration Monday, October 12, 2009 33
results Monday, October 12, 2009 34
Monday, October 12, 2009 35
kynetx can stand up a new server in < 30 minutes Monday, October 12, 2009 36
our servers stay up downtime* 0.00229% uptime 99.99772% * includes scheduled maintenance Monday, October 12, 2009 37
Warning! Monday, October 12, 2009 38
Warning! Monday, October 12, 2009 38
lessons learned Monday, October 12, 2009 39
lessons learned architect for (afford|scal|reli)ability Monday, October 12, 2009 39
lessons learned architect for (afford|scal|reli)ability insist on consistency & repeatability Monday, October 12, 2009 39
lessons learned architect for (afford|scal|reli)ability insist on consistency & repeatability document process with code Monday, October 12, 2009 39
lessons learned architect for (afford|scal|reli)ability insist on consistency & repeatability document process with code rolling releases and change control Monday, October 12, 2009 39
lessons learned architect for (afford|scal|reli)ability insist on consistency & repeatability document process with code rolling releases and change control put ops procedures online Monday, October 12, 2009 39
learning more Introduction to Cobbler Derek Carter 2:30 Puppet Workshop Andrew Shafer 3:00 Managing your minions with func Daniel Hanks 3:45 Cobbler power tools Derek Carter 5:00 Monday, October 12, 2009 40
Nov 18-19, 2009, Provo UT Monday, October 12, 2009 41
Nov 18-19, 2009, Provo UT Use discount code Windley50 www.kynetx.com Monday, October 12, 2009 41
Sleeping Through the Night Contact info: pjw@kynetx.com www.windley.com @windley FREE Context Automation White Paper at Kynetx Booth Sign up free: http://www.kynetx.com/signup Monday, October 12, 2009 42

More Related Content

Using Puppet and Cobbler to Automate Your Infrastructure

  • 1. Using Puppet and Cobbler to Automate Your Infrastructure Phillip J. Windley, Ph.D Founder and CTO Kynetx www.kynetx.com Monday, October 12, 2009 1
  • 2. Sleeping Through the Night Phillip J. Windley, Ph.D Founder and CTO Kynetx www.kynetx.com Monday, October 12, 2009 2
  • 4. hire fewer people Monday, October 12, 2009 4
  • 5. meet demand quickly Monday, October 12, 2009 5
  • 6. make fewer mistakes Monday, October 12, 2009 6
  • 12. 1. machine provisioning Monday, October 12, 2009 9
  • 13. 1. machine provisioning 2. system configuration Monday, October 12, 2009 9
  • 14. 1. machine provisioning 2. system configuration 3. deployment Monday, October 12, 2009 9
  • 18. machine provisioning manage images & repositories Monday, October 12, 2009 12
  • 19. machine provisioning manage images & repositories kickstart machines Monday, October 12, 2009 12
  • 20. machine provisioning manage images & repositories kickstart machines handle physical and virtual hardware Monday, October 12, 2009 12
  • 21. machine provisioning manage images & repositories kickstart machines handle physical and virtual hardware set up DHCP and DNS Monday, October 12, 2009 12
  • 23. cobbler is a collection of tools that support machine provisioning Monday, October 12, 2009 13
  • 25. cobblerd dns power dhcp Monday, October 12, 2009 14
  • 26. cobblerd dns power dhcp cobbler web repos images kickstart Monday, October 12, 2009 14
  • 27. koan cobblerd dns power dhcp cobbler web repos images kickstart Monday, October 12, 2009 14
  • 28. cobbler uses a collection of specifications that define your systems Monday, October 12, 2009 15
  • 31. profile repo distro Monday, October 12, 2009 16
  • 32. system profile repo distro Monday, October 12, 2009 16
  • 33. import a distro cobbler import --mirror ~/fc8 --name fc8 Monday, October 12, 2009 17
  • 34. import a distro cobbler import --mirror ~/fc8 --name fc8 create a profile cobbler profile add --name=base-fc8 --distro=fc8-xen-i386 --kickstart=/root/base-fc8.ks --repos=fc8-newkey-repo Monday, October 12, 2009 17
  • 35. import a distro cobbler import --mirror ~/fc8 --name fc8 create a profile cobbler profile add --name=base-fc8 --distro=fc8-xen-i386 --kickstart=/root/base-fc8.ks --repos=fc8-newkey-repo define a system cobbler system add --name=log0 --mac=00:16:3E:4B:40:00 --ip=192.168.122.180 --profile=base-fc8 --hostname=log0 Monday, October 12, 2009 17
  • 36. building a machine koan --server=cobbler.kobj.net --virt --nogfx --system=log0 Monday, October 12, 2009 18
  • 39. system configuration critical services on or off Monday, October 12, 2009 20
  • 40. system configuration critical services on or off security systems configured correctly Monday, October 12, 2009 20
  • 41. system configuration critical services on or off security systems configured correctly users created Monday, October 12, 2009 20
  • 42. system configuration critical services on or off security systems configured correctly users created necessary libraries in place Monday, October 12, 2009 20
  • 43. system configuration critical services on or off security systems configured correctly users created necessary libraries in place right packages built & installed Monday, October 12, 2009 20
  • 45. puppet is a language for specifying desired system configuration Monday, October 12, 2009 21
  • 46. install package Monday, October 12, 2009 22
  • 47. install package configure Monday, October 12, 2009 22
  • 48. install package configuration should be modified after package installation configure Monday, October 12, 2009 22
  • 49. install package configuration should be modified after package installation configure service Monday, October 12, 2009 22
  • 50. install package configuration should be modified after package installation configure service should restart whenever configuration changes service Monday, October 12, 2009 22
  • 51. the hard way yum install openssh-server vi /etc/ssh/sshd_config service sshd start Monday, October 12, 2009 23
  • 52. the puppet way class ssh { package { ssh: ensure => installed } file { sshd_config: name => “/etc/ssh/sshd_config”, owner=> root, source => “puppet://server/apps/ssh/…”, after => Package[ssh] } service { sshd: ensure => running, subscribe => [Package[ssh], File[sshd_config]] } } Monday, October 12, 2009 24
  • 53. the puppet way class ssh { package { ssh: ensure => installed } file { sshd_config: name => “/etc/ssh/sshd_config”, owner=> root, source => “puppet://server/apps/ssh/…”, after => Package[ssh] } service { sshd: ensure => running, subscribe => [Package[ssh], File[sshd_config]] } } Monday, October 12, 2009 24
  • 54. the puppet way class ssh { package { ssh: ensure => installed } file { sshd_config: name => “/etc/ssh/sshd_config”, owner=> root, source => “puppet://server/apps/ssh/…”, after => Package[ssh] } service { sshd: ensure => running, subscribe => [Package[ssh], File[sshd_config]] } } Monday, October 12, 2009 24
  • 55. wait a minute… that looks like a lot more lines to me! Monday, October 12, 2009 25
  • 58. requirements deployment happens over & over again Monday, October 12, 2009 27
  • 59. requirements deployment happens over & over again controlled, not continuous Monday, October 12, 2009 27
  • 60. requirements deployment happens over & over again controlled, not continuous role-based Monday, October 12, 2009 27
  • 61. requirements deployment happens over & over again controlled, not continuous role-based remotable Monday, October 12, 2009 27
  • 62. now for deployment... Monday, October 12, 2009 28
  • 63. now for deployment... Monday, October 12, 2009 28
  • 64. now for deployment... Monday, October 12, 2009 28
  • 65. now for deployment... Monday, October 12, 2009 28
  • 66. in the end… I just wrote it in Perl in a few hours Monday, October 12, 2009 29
  • 67. [root@ops deploy]# ./deploy.pl -d The following tasks are configured: deploy | Export a new copy of the code install | deploy, initialize, restart uninstall | rollback code, initialize,restart start_httpd | Start the HTTP server rollback | Rollback to the deploy stop_httpd | Stop the HTTP server test_server | Run the appropriate server test cleanup | Remove old copies of code test_code | Run the all tests configure_httpd| Build the httpd.conf file install_init | Install the init JS files restart_httpd | Restart the HTTP server Monday, October 12, 2009 30
  • 68. [root@ops deploy]# ./deploy.pl -s server | version -----------------------|---------------- init0.kobj.net | 340M init1.kobj.net | 340M log.kobj.net | 340 log0.kobj.net | 340 log1.kobj.net | 340 krl.kobj.net | 340 cs0.kobj.net | 341 cs1.kobj.net | 341 cs2.kobj.net | 341 cs3.kobj.net | 341 Monday, October 12, 2009 31
  • 69. [root@ops deploy]# ./deploy.pl -m krl -t install Performing install on krl with role krl... A /web/lib/releases/perl_0910091229/ops ... A /web/lib/releases/perl_0910091229/startup.pl A /web/lib/releases/perl_0910091229/Kynetx.pm A /web/lib/releases/perl_0910091229/README Checked out revision 342. Writing /web/conf/httpd.conf Stopping httpd: [ OK ] Starting httpd: [ OK ] Testing RuleManager.....ok All tests successful. Files=1, Tests=73, 8 wallclock secs ... Result: PASS Monday, October 12, 2009 32
  • 71. TODO configuration database Monday, October 12, 2009 33
  • 72. TODO configuration database (more) automated testing Monday, October 12, 2009 33
  • 73. TODO configuration database (more) automated testing continuous integration Monday, October 12, 2009 33
  • 76. kynetx can stand up a new server in < 30 minutes Monday, October 12, 2009 36
  • 77. our servers stay up downtime* 0.00229% uptime 99.99772% * includes scheduled maintenance Monday, October 12, 2009 37
  • 81. lessons learned architect for (afford|scal|reli)ability Monday, October 12, 2009 39
  • 82. lessons learned architect for (afford|scal|reli)ability insist on consistency & repeatability Monday, October 12, 2009 39
  • 83. lessons learned architect for (afford|scal|reli)ability insist on consistency & repeatability document process with code Monday, October 12, 2009 39
  • 84. lessons learned architect for (afford|scal|reli)ability insist on consistency & repeatability document process with code rolling releases and change control Monday, October 12, 2009 39
  • 85. lessons learned architect for (afford|scal|reli)ability insist on consistency & repeatability document process with code rolling releases and change control put ops procedures online Monday, October 12, 2009 39
  • 86. learning more Introduction to Cobbler Derek Carter 2:30 Puppet Workshop Andrew Shafer 3:00 Managing your minions with func Daniel Hanks 3:45 Cobbler power tools Derek Carter 5:00 Monday, October 12, 2009 40
  • 87. Nov 18-19, 2009, Provo UT Monday, October 12, 2009 41
  • 88. Nov 18-19, 2009, Provo UT Use discount code Windley50 www.kynetx.com Monday, October 12, 2009 41
  • 89. Sleeping Through the Night Contact info: pjw@kynetx.com www.windley.com @windley FREE Context Automation White Paper at Kynetx Booth Sign up free: http://www.kynetx.com/signup Monday, October 12, 2009 42