2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange
- 1. Håkon Drange Head of Infrastructure Operations Chef @ Aptoma Redpill Linpro breakfast seminar February 9th 2016
- 3. Introduction
- 4. ● SaaS provider of tools for online news media. ● 15 employees (80% tech staff) About Aptoma
- 5. Vision: To help the new media with tools that shapes effective, flexible and profitable news organisations. About Aptoma
- 6. About Aptoma - some customers
- 7. The main services we provide About Aptoma
- 8. Good looking editions on mobile devices for “+” subscriptions DrMobile
- 9. Easy and flexible front page editing DrFront
- 10. Efficient article production to any output channel DrPublish
- 11. Video encoding, playback API’s and CDN distribution DrVideo
- 12. Flexible search API for all your media content DrLib
- 13. ● All tools delivered as SaaS ● The customer builds their business logic and presentation on top of our APIs About Aptoma
- 14. Internal PaaS project started in 2011 Aptoma Media Platform (AMP)
- 15. Vendor agnostic ● Cloud providers ● VMware ● KVM ● Physical hardware Aptoma Media Platform (AMP)
- 16. Aptoma Media Platform (AMP) SaaS Dr* PaaS (AMP) Chef AWS Cloudformation IaaS AWS EC2 IBM Softlayer - Zetta - Dell Poweredge
- 17. Consistent and predictable platform for the developers ● Same Ubuntu ● Munin, Nagios, New Relic, log shipping, SSH access, NTP, firewall, swap, etc. ● Same Apache+PHP ● Same MySQL ● Same MongoDB ● etc. .. Achieved with Chef. Aptoma Media Platform (AMP)
- 18. Primary vendor is AWS Primary region eu-central-1 (Frankfurt) Secondary region eu-west-1 (Dublin) IaaS/PaaS providers
- 19. Chef
- 20. Chef is a configuration management and automation platform from Opscode. Chef
- 21. Can be used in multiple ways: ● Chef Server mode ● Chef Solo mode ○ With Vagrant Chef
- 22. Building blocks: ● Cookbooks contains recipes ○ Write your own cookbook or leverage community cookboks from the Supermarket ○ Or a combination, with wrapper cookbooks Chef
- 23. Building blocks: ● Cookbooks contains recipes ● A role consists of one or more re-usable recipes ● chef-client on the node (server) Chef
- 25. Add common bootstrap role first to configure OS platform Chef
- 26. Then add server role ● apache-server ● nginx-server ● varnish-server ● mysql-server ● mongodb-server Chef
- 27. Then add service role for service specific config. Chef
- 28. service-drfront-web ● Adds Apache virtualhosts ● memcached on port 11211 ● Ruby gems ● Specific Node.js version Chef
- 29. Base role common-bootstrap on all manually provisioned servers Chef
- 30. Base role common-bootstrap on all manually provisioned servers knife bootstrap --bootstrap-version 11.18.12 <hostname> --hint ec2 --environment production-v2 -x tech --sudo -r “role[common-bootstrap],role[apache-server],role [service-drfront-web]” Chef
- 31. Base role common-bootstrap-cloud on all automatically provisioned servers with AWS Cloudformation Chef
- 32. “Provision and manage AWS infrastructure as code” https://aws.amazon.com/cloudformation/ AWS Cloudformation
- 33. { "Description" : "Create an EC2 instance running the Amazon Linux 32 bit AMI.", "Parameters" : { "KeyPair" : { "Description" : "The EC2 Key Pair to allow SSH access to the instance", "Type" : "String" } }, "Resources" : { "Ec2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "KeyName" : { "Ref" : "KeyPair" }, "ImageId" : "ami-3b355a52" } } }, "Outputs" : { "InstanceId" : { "Description" : "The InstanceId of the newly created EC2 instance", "Value" : { "Ref" : "Ec2Instance" } } }, "AWSTemplateFormatVersion" : "2010-09-09" } AWS Cloudformation
- 35. 1. Develop a template 2. Define stack specific parameter values AWS Cloudformation
- 41. 1. Develop a template 2. Parameterize input values 3. Define resources a. Launch Configuration for an Auto Scaling Group b. Instance type, storage configuration c. OS via AMI ID d. Elastic Load Balancer to attach to (optional) 4. Scaling thresholds 5. .. etc AWS Cloudformation
- 42. Now you can create a new stack (a collection of resources) AWS Cloudformation
- 43. Everything gets provisioned automatically AWS Cloudformation
- 44. Consistent and predictable configuration AWS Cloudformation
- 45. Provision stack via AWS Console or API AWS Cloudformation
- 46. So how do we kick off the Chef node bootstrap process? AWS Cloudformation + Chef
- 47. In Cloudformation Launch Configuration user-data: ● Download and install publicly available dependencies "apt-get update; apt-get upgrade -yn", "apt-get install -y libffi-dev libssl-dev python-setuptools munin-noden", "easy_install pipn", "pip install pyopenssl ndg-httpsclient pyasn1 requests s3cmd boto3 awsclin", "easy_install https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest. tar.gzn", AWS Cloudformation + Chef
- 48. In Cloudformation Launch Configuration user-data: ● Download and install private stuff from S3 ○ Aptoma Chef bootstrap validator key ○ amp-aws-auto-bootstrap-v1.1.0.py bootstrap script ○ chef-client installer ● If all OK, Cloudformation signal OK to the Auto Scaling Group AWS Cloudformation + Chef
- 49. The ASG should now have instances attached to a load balancer ELB instance health check /status fails because no application is deployed AWS Cloudformation + Chef
- 50. What about application deployments? AWS Cloudformation + Chef
- 51. We use AWS CodeDeploy. AWS CodeDeploy
- 52. CodeDeploy is a deployment tool from AWS. AWS CodeDeploy
- 53. Nice integration with Auto Scaling AWS CodeDeploy
- 54. A framework, not a complete tool AWS CodeDeploy
- 55. AWS CodeDeploy
- 56. 1. Provision servers manually or with Cloudformation 2. Use the same Chef cookbooks for any IaaS 3. Leverage Auto Scaling with Cloudformation and CodeDeploy Summary
- 57. We achieve: ● Automated, repeatable infrastructure with re-usable code ● Predictable and scalable setup ● Easy and efficient for developers to set up new services Summary
- 59. Email: haakon@aptoma.com Twitter: @haakond Slides: http://apto.ma/slides20160209hd WWW: http://aptoma.com/ Thanks
- 60. ● https://docs.chef.io/_images/chef_run.png ● http://docs.aws.amazon.com/gettingstarted/latest/wah-linux/images/architecture_linux.png ● http://media.amazonwebservices.com/architecturecenter/AWS_ac_ra_web_01.pdf ● https://aws.amazon.com/cloudformation/details/ ● http://docs.aws.amazon.com/codedeploy/latest/userguide/deployment-steps.html ● https://www.chef.io/ ● http://www.aptoma.com/ ● http://southparkstudios.mtvnimages.com/shared/characters/adults/chef.png ● http://s3.amazonaws.com/opscode-corpsite/assets/121/pic-chef-logo.png References