SlideShare a Scribd company logo

Stratalux Cloud Formation and Chef Integration Presentation

•
•
Jeremy Przygode
Jeremy Przygode

The document provides an overview of Stratalux's cloud-based managed services and infrastructure as code approach using Chef. Key points include: - Stratalux builds highly scalable and available websites in the public cloud. - Infrastructure is defined as code using Chef for consistent builds, configurations, backups and reduced costs. - Chef is used to define infrastructure including cookbooks, roles, environments and nodes. - CloudFormation templates are used to define the needed AWS resources like IAM roles, security groups, auto scaling groups. - Templates specify parameters, mappings and resources to deploy the application servers configured via Chef.

Related slideshows

[Coscup 2012] JavascriptMVC
[Coscup 2012] JavascriptMVC[Coscup 2012] JavascriptMVC
[Coscup 2012] JavascriptMVC
 
20141002 delapsley-socalangularjs-final
20141002 delapsley-socalangularjs-final20141002 delapsley-socalangularjs-final
20141002 delapsley-socalangularjs-final
 
Full stack development with node and NoSQL - All Things Open - October 2017
Full stack development with node and NoSQL - All Things Open - October 2017Full stack development with node and NoSQL - All Things Open - October 2017
Full stack development with node and NoSQL - All Things Open - October 2017
 
1 of 44
Hi!
Stratalux is a cloud-based Managed Services Provider We build highly scalable & available websites in the public cloud 7/20/2012 2
A match made in the Hall of Justice. 7/20/2012 3
Infrastructure as code ™ 7/20/2012 4
Consistent builds and configurations 7/20/2012 5
Backups! 7/20/2012 6
Things break! 7/20/2012 7
Reduced costs! 7/20/2012 8
7/20/2012 9
• Hosted Chef • Attributes • Cookbooks • Data Bags • Roles • Environments 7/20/2012 10
$ knife node list web01.stratalux.com web02.stratalux.com web03.stratalux.com web04.stratalux.com web05.stratalux.com app01.stratalux.com app02.stratalux.com proxy01.stratalux.com proxy02.stratalux.com 7/20/2012 11
cookbook_versions: amazon_dns_tools: = 0.0.3 amazon_ses: = 0.1.5 apache2: = 0.99.5 apt: = 1.1.2 aws: = 0.99.0 aws_accounts: = 0.0.4 aws_tools: = 0.1.6 backup-manager: = 0.0.2 bluepill: = 0.2.0 build-essential: = 1.0.0 cacti: = 0.0.5 collectd: = 1.0.1 confluence: = 0.0.1 7/20/2012 12
override_attributes: java: install_flavor: openjdk 7/20/2012 13
comment: Justin Sysadmin ftp_enabled: true ftp_passwd: password groups: sysadmin id: justin shell: /bin/bash uid: 2304 vsftpd_configs: guest_username: justin local_root: /home/justin 7/20/2012 14
$knife role show application_server chef_type: role description: Application server role. json_class: Chef::Role name: application_server run_list: role[stratalux_base_prod] recipe[jetty] recipe[java] default_attributes: jetty: admin_server: false bucket: bucket123 caching.servers: server1.stratalux.com:11211 server2.stratalux.com:11211 7/20/2012 15
$ knife environment show Production name: Production cookbook_versions: java: = 1.0.0 users: = 0.3.5 ntp: = 0.0.2 wordpress: = 0.8.4 apache2: = 1.0.1 jetty: = 0.0.8 default_attributes: ntp: servers: 0.pool.ntp.org 1.pool.ntp.org override_attributes: {} 7/20/2012 © 2012 – Stratalux, Inc. 16
• Needed AWS Resources – IAM User – IAM Access Key – Bucket Policy – Auto Scaling Group – Launch Configuration – Security Groups – Load Balancer – Scaling Policy – Cloudwatch Alarms – WaitConditions 7/20/2012 17
• Parameters • Mappings • Resources • Functions 7/20/2012 18
"KeyName": { "Type": "String", "Description": "Name of an existing EC2 KeyPair to enable SSH access to the web server", "Default": “stratalux-key" }, "InstanceType": { "Description": "Application Server EC2 instance type", "Type": "String", "Default": "m1.small", "AllowedValues": ["t1.micro", "m1.small", "m1.medium", "m1.large", "m1.xlarge", "m2.xlarge", "m2.2xlarge", "m2.4xlarge", "c1.medium", "c1.xlarge", "cc1.4xlarge", "cc2.8xlarge", "cg1.4xlarge"], "ConstraintDescription": "must be a valid EC2 instance type." }, 7/20/2012 19
"ChefOrganization": { "Description": "The Organization name used in Hosted Chef.", "Type": "String", "Default": “stratalux" }, "ChefEnvironment": { "Description": "The Chef Environment.", "Type": "String", "Default": “Production" }, "ChefRunList": { "Description": "The Chef runlist.", "Type": "String", "Default": "role[application_server]" }, 7/20/2012 20
"DomainName": { "Description": "Server Domain Name.", "Type": "String", "Default": “stratalux.com" }, "ChefServerPrivateKeyBucket": { "Description": "S3 bucket containing validation private key for Chef Server", "Type": "String", "Default": "stratalux_cloudformation" } 7/20/2012 21
"Mappings" : { "RegionMap" : { "us-east-1" : { "AMI" : "ami-76f0061f" }, "us-west-1" : { "AMI" : "ami-655a0a20" }, "eu-west-1" : { "AMI" : "ami-7fd4e10b" }, "ap-southeast-1" : { "AMI" : "ami-72621c20" } } } 7/20/2012 22
• Resources "Resources" :{ "MySimpleImage" : { "Type" : "AWS::EC2::Image", "Properties" : { "ImageId" : "myLinuxBundle-2011-12-30", } } } 7/20/2012 23
"ChefClientUser": { "Type": "AWS::IAM::User", "Properties": { "Path": "/", "Policies": [{ "PolicyName": "root", "PolicyDocument": { "Statement": [ {"Effect": "Allow", "Action": ["cloudformation:DescribeStackResource", "s3:Get"], "Resource": "*" }]}}]}}, 7/20/2012 24
"HostKeys": { "Type": "AWS::IAM::AccessKey", "DependsOn": "ChefClientUser", "Properties": { "UserName": { "Ref": "ChefClientUser" } }}, 7/20/2012 25
"BucketPolicy": { "Type": "AWS::S3::BucketPolicy", "DependsOn": "HostKeys", "Properties": { "PolicyDocument": { "Statement": [{ "Sid": "ReadAccess", "Action": ["s3:GetObject"], "Effect": "Allow", "Resource": { "Fn::Join": ["", ["arn:aws:s3:::", { "Ref": "ChefServerPrivateKeyBucket"}, "/*"]]}, "Principal": { "AWS": { "Fn::GetAtt": ["ChefClientUser", "Arn"] }}}]}, "Bucket": { "Ref": "ChefServerPrivateKeyBucket"}}}, 7/20/2012 26
"AppServerGroup": { "Type": "AWS::AutoScaling::AutoScalingGroup", "Properties": { "AvailabilityZones": { "Fn::GetAZs": ""}, "LaunchConfigurationName": { "Ref": "ChefClient"}, "LoadBalancerNames": [ {"Ref": “appserverstrataluxcom"} ], "MinSize": "2", "MaxSize": "10" }}, 7/20/2012 27
"ChefClient": { "Type": "AWS::AutoScaling::LaunchConfiguration", "DependsOn": "BucketPolicy", "Metadata": { "AWS::CloudFormation::Init": { "config": { "packages": { "rubygems": { "chef": [],"ohai": ["0.6.4"] }, "apt": { "build-essential": [],"ruby": [],"rubygems": [] }}, 7/20/2012 28
"files": { "/etc/chef/first-boot.json": { "content": { "run_list": { "Ref": "ChefRunList“ }, "mode": "000644", "owner": "root", "group": "root" }}, "/home/ubuntu/.s3cfg": { "content": { "Fn::Join": ["", ["[default]n", "access_key = ", { "Ref": "HostKeys" }, "n", "secret_key = ", { "Fn::GetAtt": ["HostKeys", "SecretAccessKey"] }, "n", "use_https = Truen"]]}, "mode": "000644", "owner": "ubuntu", "group": "ubuntu" }, 7/20/2012 29
"Properties": { "SecurityGroups": [ { "Ref": “AppServerAccess" } ], "ImageId": { "Fn::FindInMap": ["AWSRegionArch2AMI", { "Ref": "AWS::Region" }, { "Fn::FindInMap": ["AWSInstanceType2Arch", { "Ref": "InstanceType" }, "Arch“ ]}]}, "KeyName": { "Ref": "KeyName" }, "InstanceType": { "Ref": "InstanceType" } 7/20/2012 30
"UserData": { "Fn::Base64": { "Fn::Join": ["", ["#!/bin/bash -vn", "function error_exitn", "{n", " cfn-signal -e 1 -r "$1" '", { "Ref": "ChefClientWaitHandle“ }, "chef-client -j /etc/chef/first-boot.json ||”, “error_exit 'Failed to initialize host via chef client' n", "# If all went well, signal successn", "cfn-signal -e $? -r 'Chef Server configuration' '", { "Ref": "ChefClientWaitHandle" }, "'n“ } 7/20/2012 31
“AppServerAccess ": { "Type": "AWS::EC2::SecurityGroup", "Properties": { "GroupDescription": “AppServerAccess", "SecurityGroupIngress": [ { "IpProtocol": "tcp", "FromPort": "8080", "ToPort": "8080", "CidrIp": "0.0.0.0/0"} ]}}, 7/20/2012 32
“appserverstrata1uxit": { "Type": "AWS::ElasticLoadBalancing::LoadBalancer", "Properties": { "AvailabilityZones": { "Fn::GetAZs": ""}, "HealthCheck": { "HealthyThreshold": "10", "Interval": "30", "Target": "HTTP:8080/healthcheck", "Timeout": "2", "UnhealthyThreshold": "2" }, "Listeners": [{ "InstancePort": "8080", "LoadBalancerPort": "80", "Protocol": "HTTP", "PolicyNames": [] }]}}, 7/20/2012 33
"AppServerScaleUpPolicy": { "Type": "AWS::AutoScaling::ScalingPolicy", "Properties": { "AdjustmentType": "ChangeInCapacity", "AutoScalingGroupName": { "Ref": "AppServerGroup" }, "Cooldown": "60", "ScalingAdjustment": "1"} }, 7/20/2012 34
"AppServerScaleDownPolicy": { "Type": "AWS::AutoScaling::ScalingPolicy", "Properties": { "AdjustmentType": "ChangeInCapacity", "AutoScalingGroupName": { "Ref": "AppServerGroup" }, "Cooldown": "60", "ScalingAdjustment": "-1"} }, 7/20/2012 35
"CPUAlarmHigh": { "Type": "AWS::CloudWatch::Alarm", "Properties": { "AlarmDescription": "Scale-up if CPU > 75% for 2 minutes", "MetricName": "CPUUtilization", "Namespace": "AWS/EC2", "Statistic": "Average", "Period": "60", "EvaluationPeriods": "2", "Threshold": "75", "AlarmActions": [{ "Ref": "AppServerScaleUpPolicy"}], "Dimensions": [{ "Name": "AutoScalingGroupName", "Value": { "Ref": "AppServerGroup"}}], "ComparisonOperator": "GreaterThanThreshold“ }}, 7/20/2012 36
"CPUAlarmLow": { "Type": "AWS::CloudWatch::Alarm", "Properties": { "AlarmDescription": "Scale-down if CPU < 75% for 10 minutes", "MetricName": "CPUUtilization", "Namespace": "AWS/EC2", "Statistic": "Average", "Period": "300", "EvaluationPeriods": "2", "Threshold": "75", "AlarmActions": [{ "Ref": "AppServerScaleDownPolicy"}], "Dimensions": [{ "Name": "AutoScalingGroupName", "Value": { "Ref": "AppServerGroup"}}], "ComparisonOperator": "LessThanThreshold" }}, 7/20/2012 37
"ChefClientWaitHandle": { "Type": "AWS::CloudFormation::WaitConditionHandle“ }, "ChefClientWaitCondition": { "Type": "AWS::CloudFormation::WaitCondition", "DependsOn": "ChefClient", "Properties": { "Handle": { "Ref": "ChefClientWaitHandle"}, "Timeout": "1200“} } 7/20/2012 38
7/20/2012 39
7/20/2012 40
7/20/2012 41
7/20/2012 42
7/20/2012 43
Jeremy Przygode jp@stratalux.com www.stratalux.com @stratalux Special thanks to JD Hanson for the awesome images! 7/20/2012 © 2012 – Stratalux, Inc. 44

More Related Content

Stratalux Cloud Formation and Chef Integration Presentation

  • 1. Hi!
  • 2. Stratalux is a cloud-based Managed Services Provider We build highly scalable & available websites in the public cloud 7/20/2012 2
  • 3. A match made in the Hall of Justice. 7/20/2012 3
  • 4. Infrastructure as code ™ 7/20/2012 4
  • 5. Consistent builds and configurations 7/20/2012 5
  • 10. • Hosted Chef • Attributes • Cookbooks • Data Bags • Roles • Environments 7/20/2012 10
  • 11. $ knife node list web01.stratalux.com web02.stratalux.com web03.stratalux.com web04.stratalux.com web05.stratalux.com app01.stratalux.com app02.stratalux.com proxy01.stratalux.com proxy02.stratalux.com 7/20/2012 11
  • 12. cookbook_versions: amazon_dns_tools: = 0.0.3 amazon_ses: = 0.1.5 apache2: = 0.99.5 apt: = 1.1.2 aws: = 0.99.0 aws_accounts: = 0.0.4 aws_tools: = 0.1.6 backup-manager: = 0.0.2 bluepill: = 0.2.0 build-essential: = 1.0.0 cacti: = 0.0.5 collectd: = 1.0.1 confluence: = 0.0.1 7/20/2012 12
  • 13. override_attributes: java: install_flavor: openjdk 7/20/2012 13
  • 14. comment: Justin Sysadmin ftp_enabled: true ftp_passwd: password groups: sysadmin id: justin shell: /bin/bash uid: 2304 vsftpd_configs: guest_username: justin local_root: /home/justin 7/20/2012 14
  • 15. $knife role show application_server chef_type: role description: Application server role. json_class: Chef::Role name: application_server run_list: role[stratalux_base_prod] recipe[jetty] recipe[java] default_attributes: jetty: admin_server: false bucket: bucket123 caching.servers: server1.stratalux.com:11211 server2.stratalux.com:11211 7/20/2012 15
  • 16. $ knife environment show Production name: Production cookbook_versions: java: = 1.0.0 users: = 0.3.5 ntp: = 0.0.2 wordpress: = 0.8.4 apache2: = 1.0.1 jetty: = 0.0.8 default_attributes: ntp: servers: 0.pool.ntp.org 1.pool.ntp.org override_attributes: {} 7/20/2012 © 2012 – Stratalux, Inc. 16
  • 17. • Needed AWS Resources – IAM User – IAM Access Key – Bucket Policy – Auto Scaling Group – Launch Configuration – Security Groups – Load Balancer – Scaling Policy – Cloudwatch Alarms – WaitConditions 7/20/2012 17
  • 18. • Parameters • Mappings • Resources • Functions 7/20/2012 18
  • 19. "KeyName": { "Type": "String", "Description": "Name of an existing EC2 KeyPair to enable SSH access to the web server", "Default": “stratalux-key" }, "InstanceType": { "Description": "Application Server EC2 instance type", "Type": "String", "Default": "m1.small", "AllowedValues": ["t1.micro", "m1.small", "m1.medium", "m1.large", "m1.xlarge", "m2.xlarge", "m2.2xlarge", "m2.4xlarge", "c1.medium", "c1.xlarge", "cc1.4xlarge", "cc2.8xlarge", "cg1.4xlarge"], "ConstraintDescription": "must be a valid EC2 instance type." }, 7/20/2012 19
  • 20. "ChefOrganization": { "Description": "The Organization name used in Hosted Chef.", "Type": "String", "Default": “stratalux" }, "ChefEnvironment": { "Description": "The Chef Environment.", "Type": "String", "Default": “Production" }, "ChefRunList": { "Description": "The Chef runlist.", "Type": "String", "Default": "role[application_server]" }, 7/20/2012 20
  • 21. "DomainName": { "Description": "Server Domain Name.", "Type": "String", "Default": “stratalux.com" }, "ChefServerPrivateKeyBucket": { "Description": "S3 bucket containing validation private key for Chef Server", "Type": "String", "Default": "stratalux_cloudformation" } 7/20/2012 21
  • 22. "Mappings" : { "RegionMap" : { "us-east-1" : { "AMI" : "ami-76f0061f" }, "us-west-1" : { "AMI" : "ami-655a0a20" }, "eu-west-1" : { "AMI" : "ami-7fd4e10b" }, "ap-southeast-1" : { "AMI" : "ami-72621c20" } } } 7/20/2012 22
  • 23. • Resources "Resources" :{ "MySimpleImage" : { "Type" : "AWS::EC2::Image", "Properties" : { "ImageId" : "myLinuxBundle-2011-12-30", } } } 7/20/2012 23
  • 24. "ChefClientUser": { "Type": "AWS::IAM::User", "Properties": { "Path": "/", "Policies": [{ "PolicyName": "root", "PolicyDocument": { "Statement": [ {"Effect": "Allow", "Action": ["cloudformation:DescribeStackResource", "s3:Get"], "Resource": "*" }]}}]}}, 7/20/2012 24
  • 25. "HostKeys": { "Type": "AWS::IAM::AccessKey", "DependsOn": "ChefClientUser", "Properties": { "UserName": { "Ref": "ChefClientUser" } }}, 7/20/2012 25
  • 26. "BucketPolicy": { "Type": "AWS::S3::BucketPolicy", "DependsOn": "HostKeys", "Properties": { "PolicyDocument": { "Statement": [{ "Sid": "ReadAccess", "Action": ["s3:GetObject"], "Effect": "Allow", "Resource": { "Fn::Join": ["", ["arn:aws:s3:::", { "Ref": "ChefServerPrivateKeyBucket"}, "/*"]]}, "Principal": { "AWS": { "Fn::GetAtt": ["ChefClientUser", "Arn"] }}}]}, "Bucket": { "Ref": "ChefServerPrivateKeyBucket"}}}, 7/20/2012 26
  • 27. "AppServerGroup": { "Type": "AWS::AutoScaling::AutoScalingGroup", "Properties": { "AvailabilityZones": { "Fn::GetAZs": ""}, "LaunchConfigurationName": { "Ref": "ChefClient"}, "LoadBalancerNames": [ {"Ref": “appserverstrataluxcom"} ], "MinSize": "2", "MaxSize": "10" }}, 7/20/2012 27
  • 28. "ChefClient": { "Type": "AWS::AutoScaling::LaunchConfiguration", "DependsOn": "BucketPolicy", "Metadata": { "AWS::CloudFormation::Init": { "config": { "packages": { "rubygems": { "chef": [],"ohai": ["0.6.4"] }, "apt": { "build-essential": [],"ruby": [],"rubygems": [] }}, 7/20/2012 28
  • 29. "files": { "/etc/chef/first-boot.json": { "content": { "run_list": { "Ref": "ChefRunList“ }, "mode": "000644", "owner": "root", "group": "root" }}, "/home/ubuntu/.s3cfg": { "content": { "Fn::Join": ["", ["[default]n", "access_key = ", { "Ref": "HostKeys" }, "n", "secret_key = ", { "Fn::GetAtt": ["HostKeys", "SecretAccessKey"] }, "n", "use_https = Truen"]]}, "mode": "000644", "owner": "ubuntu", "group": "ubuntu" }, 7/20/2012 29
  • 30. "Properties": { "SecurityGroups": [ { "Ref": “AppServerAccess" } ], "ImageId": { "Fn::FindInMap": ["AWSRegionArch2AMI", { "Ref": "AWS::Region" }, { "Fn::FindInMap": ["AWSInstanceType2Arch", { "Ref": "InstanceType" }, "Arch“ ]}]}, "KeyName": { "Ref": "KeyName" }, "InstanceType": { "Ref": "InstanceType" } 7/20/2012 30
  • 31. "UserData": { "Fn::Base64": { "Fn::Join": ["", ["#!/bin/bash -vn", "function error_exitn", "{n", " cfn-signal -e 1 -r "$1" '", { "Ref": "ChefClientWaitHandle“ }, "chef-client -j /etc/chef/first-boot.json ||”, “error_exit 'Failed to initialize host via chef client' n", "# If all went well, signal successn", "cfn-signal -e $? -r 'Chef Server configuration' '", { "Ref": "ChefClientWaitHandle" }, "'n“ } 7/20/2012 31
  • 32. “AppServerAccess ": { "Type": "AWS::EC2::SecurityGroup", "Properties": { "GroupDescription": “AppServerAccess", "SecurityGroupIngress": [ { "IpProtocol": "tcp", "FromPort": "8080", "ToPort": "8080", "CidrIp": "0.0.0.0/0"} ]}}, 7/20/2012 32
  • 33. “appserverstrata1uxit": { "Type": "AWS::ElasticLoadBalancing::LoadBalancer", "Properties": { "AvailabilityZones": { "Fn::GetAZs": ""}, "HealthCheck": { "HealthyThreshold": "10", "Interval": "30", "Target": "HTTP:8080/healthcheck", "Timeout": "2", "UnhealthyThreshold": "2" }, "Listeners": [{ "InstancePort": "8080", "LoadBalancerPort": "80", "Protocol": "HTTP", "PolicyNames": [] }]}}, 7/20/2012 33
  • 34. "AppServerScaleUpPolicy": { "Type": "AWS::AutoScaling::ScalingPolicy", "Properties": { "AdjustmentType": "ChangeInCapacity", "AutoScalingGroupName": { "Ref": "AppServerGroup" }, "Cooldown": "60", "ScalingAdjustment": "1"} }, 7/20/2012 34
  • 35. "AppServerScaleDownPolicy": { "Type": "AWS::AutoScaling::ScalingPolicy", "Properties": { "AdjustmentType": "ChangeInCapacity", "AutoScalingGroupName": { "Ref": "AppServerGroup" }, "Cooldown": "60", "ScalingAdjustment": "-1"} }, 7/20/2012 35
  • 36. "CPUAlarmHigh": { "Type": "AWS::CloudWatch::Alarm", "Properties": { "AlarmDescription": "Scale-up if CPU > 75% for 2 minutes", "MetricName": "CPUUtilization", "Namespace": "AWS/EC2", "Statistic": "Average", "Period": "60", "EvaluationPeriods": "2", "Threshold": "75", "AlarmActions": [{ "Ref": "AppServerScaleUpPolicy"}], "Dimensions": [{ "Name": "AutoScalingGroupName", "Value": { "Ref": "AppServerGroup"}}], "ComparisonOperator": "GreaterThanThreshold“ }}, 7/20/2012 36
  • 37. "CPUAlarmLow": { "Type": "AWS::CloudWatch::Alarm", "Properties": { "AlarmDescription": "Scale-down if CPU < 75% for 10 minutes", "MetricName": "CPUUtilization", "Namespace": "AWS/EC2", "Statistic": "Average", "Period": "300", "EvaluationPeriods": "2", "Threshold": "75", "AlarmActions": [{ "Ref": "AppServerScaleDownPolicy"}], "Dimensions": [{ "Name": "AutoScalingGroupName", "Value": { "Ref": "AppServerGroup"}}], "ComparisonOperator": "LessThanThreshold" }}, 7/20/2012 37
  • 38. "ChefClientWaitHandle": { "Type": "AWS::CloudFormation::WaitConditionHandle“ }, "ChefClientWaitCondition": { "Type": "AWS::CloudFormation::WaitCondition", "DependsOn": "ChefClient", "Properties": { "Handle": { "Ref": "ChefClientWaitHandle"}, "Timeout": "1200“} } 7/20/2012 38
  • 39. 7/20/2012 39
  • 40. 7/20/2012 40
  • 41. 7/20/2012 41
  • 42. 7/20/2012 42
  • 43. 7/20/2012 43
  • 44. Jeremy Przygode jp@stratalux.com www.stratalux.com @stratalux Special thanks to JD Hanson for the awesome images! 7/20/2012 © 2012 – Stratalux, Inc. 44

Editor's Notes

  1. Welcome.Introduce HeatherIntroduce AlexIntroduce FlorianIntroduce ColoftSince it’s Summer in Los Angeles Spiderman release last weekBatman release tonightTechnology can be boring.I thought I would spice it up my presentation with some action heros.
  2. We are Amazon partnerOpscode PartnerRightscale PartnerCustomers includeWorld Golf TourActive NetworkCrownPeakServices we provide.Professional services.Build websites on public cloud resources like amazon.Chef development services for all needs. Both windows and Unix.Managed services We provide managed services for systems built by us or any systems in the public cloud.We provide 24/7 NOC services.
  3. Why cloud formation and Chef?In order to leverage public cloud resources to fullest advantage you need to be able to automate your infrastructureCloud formation automates Amazon’s infrastructureChef automates server configurations.
  4. Opscode says “Infrastructure as code” What does this mean? With code and data you should be able to recreate your infrastructure.Apply development methodologies to IT Revision control. (use git, svnetc). Agile methodologies Lean methodologies use only what you need now, scale as necessary laterDeploy oftenImproved quality Ensure consistent dev, test and production environments. Automation you need automation to scale.
  5. Don’t fix it, rebuild it. Cloud formation &amp; Chef let’s you rebuild systems.Improved deploymentsDeploy entire stacks not just new code.No Gumby Servers
  6. Cloud formation and Chef means you can recreate your environment. You now have backups!Rebuild your infrastructure anytime anywhere.
  7. Don’t fix problems, build new servers.Things break in the cloud. Commodity hardware fails. You need automation to rebuild systems.
  8. Using cloud formation and chef let’s you implement technologies like auto scaling.You can scale your service with customer traffic.You can build DR systems without the cost of running the entire stack.
  9. We use hosted chef to configure our servers.
  10. We use hosted chef and configure our servers.You can get a free hosted chef account by visiting opscode.comHosted chef is a centralized and searchable database where all your chef configurations reside and are made available to your nodes. Hosted chef holds all your attributes, cookbooks, recipes and data bags.ExampleApp servers search hosted chef for names of DB servers and configure themselves appropriately.
  11. Cookbooks are the fundamental units of distribution in Chef. They encapsulate all the resources you need to automate your infrastructure and are easily sharable with other Chef users.We will use the jetty and java cookbooks to configure our application server.
  12. Attributes are Node data such as the IP address, hostname, loaded kernel modules, version of programming languages available on the system and more. New attributes can be dynamically added to the node in a variety of ways.Attributes are used by chef recipes to configure systems. Example:You may use a Java cookbook but set flavor of Java to openJDK or Oracle Java depending upon your server’s role.
  13. Data bags provide an arbitrary stores of globally available JSON data.Data Bags are not directly associated with Node or Role attributes. When using Data Bags with a chef server, they are stored on the server and indexed for searching. When using Data Bags with chef-solo, data bags are stored in a directory hierarchy on the machine running chef-solo. Example:the users cookbook searches the user data bag for users to add to the system. You need a new user? Update the data_bag and run chef-client.
  14. A role provides a means of grouping similar features of similar nodes, providing a mechanism for easily composing sets of functionality.Roles allow you to scale…We have created an application_server role. That is configured to install jetty and java. Among other things.
  15. With environments, you can specify per environment run lists in roles, per environment cookbook versions, and environment attributes.Environments allows us to protect production cookbooks and configurations from cookbooks in development. Environments allows us to promote updates in our cookbooks in a controlled fashion.Review how Chef-client runs. Connects to hosted chef.Get’s role,Runs recipes, etc.
  16. In order to build WGT an auto scaling group using Cloud Formation we need to create the following Cloud Formation resources.
  17. Cloud Formation is JSON.Consisting of Parameters mappings resources and functions.
  18. Parameters are inputs that allow you to control your stack.Here we Define the Host KeypairAnd the instance type.
  19. In order for Cloud formation to boot strap the server using chef we have to define some Parameters that are used to communicate with Hosted chef.OrganizationEnvironmentRunlist (basically the role)
  20. Next we define the DomainName for the server. The hostname is automatically generated based upon what Amazon provides.Then the name of the bucket that holds Chef’s validator.pem and encrypted data bag secrets. Basically keys to authenticate with Hosted chef.
  21. Mappings match a key to a corresponding set of named values. For example, if you want to set values based on a region, you can create a mapping that uses the region name as a key and contains the values you want to specify for each specific region.Here we are matching Amazon Regions with AMI’s we want to use for our application servers.
  22. Resources are the actual amazon services that we will need to configure.More later but they include everything from Instances to SQS, SES, Load balancers, security groups etc.
  23. Here we setup a chefclient user to access our S3 bucket with the chef validator.pem and other files.
  24. Here we define the hostkeyNotice the dependson directive.
  25. Bucket policy sets permissions to only allow the chefclient access to our bucket of secured items.Notice the function Join and Ref.
  26. Here we define the autoscaling group.Availability Zones.Launch configurationLoad balancer.Min/max
  27. Launch configuration.Here we configure the instance to install packages and gems necessary for chef to run.
  28. Here we lay down configuration files for chef to use And for S3cmd to get the files from the bucket.
  29. Here we set the security group, instnance type and Image id.Ie which AMI and what size instance we are going to use.
  30. Userdata allows us to run arbitrary code on the instance once it’s up and running.Here we are actually going to run chef-client. Which connects to hosted chef.Notice the cfn-signal command. This gives status information to cloudformation on if command ran successfully or not.Notice the WaitHandle. WE are telling cloudformation to wait for chef run to finish before continuing.
  31. Security groups.
  32. Create our load balancers.Choses which Azs to use.Healthcheck and other things.
  33. Scale up policy.How many servers to add as we scale up.
  34. Scale down does the oposite.
  35. Now we configure our envrionment to monitor the servers.We are monitoring CPU utilization and then sacaling up and down as encessary. CPU &gt; 75% scales up.
  36. Here we scale down.
  37. Here is the wait definitiondefinition used earlier.
  38. Now that we have a complete cloud formation template we need to load it into amazon.
  39. Load your Template.
  40. Here are the default values for parameters for input.You can update those inputs now if necessary.
  41. Templates define stacks
  42. Templates define stacks
  43. Thank you!There are many things I didn’t mention like. Chef:Chef search, Resources, primitives.AWS cloud formation: Simplified JSONfunctions