It gilde 20150209
•Download as ODP, PDF•
0 likes•481 views
The CFEngine Roadshow is a multi-platform configuration management training that started in 2007 and is now in its fifth edition. It uses CFEngine to demonstrate how to automate configuration of virtual machines for tasks like installing Java, Tomcat, and deploying applications. A past success story showed how CFEngine was implemented at DirecTV to manage over 3,500 servers, improving quality, enabling patching, and providing an automated CMDB. Key CFEngine concepts covered include desired state configuration, convergence, policy-based versus imperative configuration, and CFEngine's small footprint and ability to manage diverse platforms.
Report
Share
It gilde 20150209
- 1. The CFEngine Roadshow @ITGilde 9 februari 2015 Vijfde editie Martin Simons @webhuis #TheCFEngineRoadshow
- 2. Introduction Martin Simons ● IT since 1984, entrepreneur since 1998 ● Economics Universiteit van Amsterdam ● Until 1994 Mainframe development ● SInce 1998 focus op Linux ● Webhuis established 1999 ● Since 2007 specialism CFEngine ● Initiator CFEngine Debian-team
- 3. Contibutors ● Frits van der Holst ● Willem Ligtenberg ● Antal Lohmann ● Gábor Nyers ● Hans Spaans ● Ted Zlatanov
- 4. Agenda Part one ● Introduction Automation ● CFEngine concepts, example Part two ● Starting CFEngine on VM's ● Demonstrate configurered machines Part three ● Configuring ITGilde webservice machines ● Succes story CFEngine at DirecTV
- 5. Topics part one ● Introduction Automation and concepts ● How and when did The CFEngine Roadshow start? ● What happened? ● CFEngine Roadshow concepts
- 6. Known products The big three: ● Chef ● CFEngine ● Puppet Similarities: ● Open core ● Governance ● Community and Enterprise version
- 7. Differences ● Puppet, model driven. Ruby ● Chef, automate existing practices Ruby, Erlang ● CFEngine, policy driven. Native C, draait op Android, Raspbery Pi Rudder by Normation, complete CFEngine Management environment
- 9. Automation Evolutionary steps ● Scripting ● Golden Image ● Third wave Specialized Automation products
- 11. Concepts CFEngine Guiding principles ● Promise theory ● Desired state ● Convergence ● Declarative vs Imperative ● Pull vs Push Technical ● C ● Footprint
- 12. Someone who understands? Promise Theory Voluntary cooperation between individual, autonomous actors or agents who publish their intentions to one another in the form of promises -- Mark Burgess
- 13. Promise universe
- 14. Status Promise Theory Prof dr Mark Burgess, Prof dr Jan Bergstra ● Promise Theory: Principles and Applications (Volume 1) ● A static theory of promises http://arxiv.org/abs/0810.3294 Zie ook literatuuroverzicht
- 15. A Promise Is A Statement of Intention Promiser Promises to… If not currently kept, CFEngine will A variable… …hold a certain value of a certain type …store the appropriate value in the variable A file …have certain characteristics (permissions, ownership, etc.) …set the desired properties on the file A user account …exist and have certain characteristics (home directory, group, etc.) …create the user account with the desired characteristics A process …be running on the system …run the appropriate command to create the process
- 16. Convergence
- 17. Imperative vs Declarative Imperative ● Perl, Ruby, Python ● Sequential ● Inconsistent when interrupted Declarative ● CFEngine DSL ● Describes the desired stat ● CFEngine: Convergently to Desired state
- 18. Imperative Imperative is sequential ● Step by step in sequence ● Dependencies between steps ● Interruption leads to inconsistency ● Repetition can cause damage, because steps may have intermediate results.
- 19. Declarative Declarative is describing ● Description of the Desired State ● Step are independent ● Deviation Desired State is acceptable ● Deviation Desired State is not inconsistent ● Reiteration of policies always possible
- 21. Technical CFEngine ● Pull mechanism ● Native C, runs where C runs ● Spares resources ● Small footprint ● No dependencies except the OS ● 5.000 nodes per Policy Host, easily
- 22. Example convergence Desired state ● User: cferoadshow ● Group: cfegroup ● Directory: /home/cferoadshow/files ● File: cfe_test_file ● cferoadshow is owner, group cfegroup
- 23. Desired State root@ips-161:/home# ls -laR cferoadshow/ cferoadshow/: total 12 drwxr-xr-x 3 cferoadshow cfegroup 4096 Sep 12 14:07 . drwxr-xr-x 4 root root 4096 Sep 12 14:07 .. drwxr-xr-x 2 cferoadshow cfegroup 4096 Sep 12 14:07 files cferoadshow/files: total 8 drwxr-xr-x 2 cferoadshow cfegroup 4096 Sep 12 14:07 . drwxr-xr-x 3 cferoadshow cfegroup 4096 Sep 12 14:07 .. -rw-r--r-- 1 cferoadshow cfegroup 0 Sep 12 14:07 cfe_test_file
- 24. Anatomie of a Promise
- 25. The CFEgine code bundle agent create_user_file { files: "/home/cfetest/files/cfe_test_file" perms => mog("644","cfetest","cfegroup"), create => "true"; } bundle agent create_user_directory { files: "/home/cfetest/files/." perms => mog("755","cfetest","cfegroup"), create => "true"; } bundle agent adduser { commands: "/usr/sbin/useradd cfetest -d /home/cfetest -g cfegroup -m"; } bundle agent addgroup { commands: "/usr/sbin/groupadd -g 1001 cfegroup"; } body common control { bundlesequence => { "create_user_file", "create_user_directory", "adduser", "addgroup" }; inputs => { "/var/cfengine/inputs/libraries/cfengine_stdlib.cf" }; }
- 26. Topics part two ● How did it start? ● What happened? ● CFEngine Roadshow concepts ● CFEngine Roadshow example ● Succes story CFEngine at DirecTV
- 27. How did it start? ● Need to convey the principle – To Management ● Just for internal use, Debian only ● Describe as many different kinds of servers as needed
- 28. What happened? ● Conveying the principle succeeded ● I was the only Debian only guy ● Suse, Centos and Ubuntu people wanted to take part ● Multi platform integration slowed down the description of roles.
- 29. CFEngine Roadshow concepts ● Dynamic CFEngine3 ● Role based approach ● Hostname determines the role ● Support of all platforms ● Ambition – Androids – Raspberry Pi ● Integration with masterfiles – Multi platform support
- 30. CFEngine Roadshow examples Deploy 7 webservice VM's ● Install Java ● Install Tomcat7 ● Deploy war file from an nfs mount ● Sample, Testweb, Hudson, Jenkins ● Do all the neat stuff necessary on the box
- 31. Help The CFEngine Roadshow How can you help? ● Install xvnc4viewer ● Login as cfetest ssh -Y 10.168.0.7 ● Connect to console to you machine: xvnc4viewer -Autoselect=0 localhost:59xx -LowColourLevel=0 & ● Become root (password=password) ● Bootstrap the box: root@webjen0086:~# cf-agent -B 10.168.0.10 ● Enjoy and monitor through 10.168.0.15/nagios3/
- 32. The CFEngine Roadshow topology ITGilde network 192.168.125/24 The CFEngine Roadshow network 10.168.0.0/16 cfeutl01 10.168.0.10 aptutl01 10.168.0.11 nfsutl01 10.168.0.12 mnmutl0015 10.168.0.15 webapp0080 10.168.0.80 webapp0081 10.168.0.81 webttw0082 10.168.0.82 webttw0083 10.168.0.83 webhud0084 10.168.0.84 webhud0085 10.168.0.85 webjen0086 10.168.0.86 192.168.125.239 10.168.0.1
- 33. CFEngine code example(1) bundle agent debian_8_web { vars: "pkg[openjdk-7-jre]" string => "*"; "pkg[tomcat7]" string => "*"; methods: "any" usebundle => packages("debian_8_web.pkg"); commands: restart_tomcat:: "service tomcat7 restart" handle => "restart_tomcat", comment => "restarting tomcat"; }
- 34. CFEngine code example(2) bundle agent hud { vars: "catalina_base" string => "/var/lib/tomcat7"; files: "/usr/share/tomcat7" perms => mog("775","root","tomcat7"); "$(catalina_base)/webapps/hudson" delete => tidy; "$(catalina_base)/webapps/hudson.war" perms => mog("644","tomcat7","tomcat7"), classes => if_repaired("restart_tomcat"), copy_from => local_cp("/mnt/webapps/hudson/hudson.war"); }
- 35. CFEngine convergently deploys ● Standard stuff (small part) ● Dynamic allocation of OS related ploicies ● Mounting network volumes ● Installing OS versions of Tomcat and Java ● Deploying the app
- 36. ● Turn over $3,5 Billion ● +/- 24 Million Subscribers ● AT&T offered $40 Billion uitgebracht ● > 5.000 servers in Noord and South America ● > 30 different versions RHEL and OEL ● > 50 different applications ● 100 netwerks
- 37. The result ● CFEngine controles > 3.500 servers ● NFL seizon started with CFEngine ● Savings on Technisch application management ● Enhanced quality in production ● Patching is now possible ● CMDB supplied automagicvally
- 38. Why did we succeed? CFEngine implementation at DirecTV. Management commitment: ● Optima facilitation and support, sprints ● Projtection against politics ● Remote in the backyard ● Management war without us ● End speech manager: It will never be the same again ● Dicrete change of the way they work
- 39. What could be done better? You can always improve: ● Education before training on the job ● No time for “CFEngine thinking” ● Pressure for quick result, sprints ● Needed by Ops, Dev trailed ● Time boxing brought quick result, but there was unfished business ● No time to resolve issues ● No time for improvements
- 40. Topics part three ● How did it start? ● What happened? ● CFEngine Roadshow concepts ● CFEngine Roadshow example ● Succes story CFEngine at DirecTV
- 41. CFEngine Je gaat het pas zien as je het door heb -- Johan Cruijff