AWS Infrastructure as Code - September 2016 Webinar Series

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Chris Munns
Business Development Manager – DevOps
Amazon Web Services
September 2016
Infrastructure as Code: Best
Practices with AWS CloudFormation

CloudFormation concepts and technology
JSON/YAML formatted file
Parameter definition
Resource creation
Configuration actions
Framework
Stack creation
Stack updates
Error detection and rollback
Configured AWS resources
Comprehensive service support
Service event aware
Customizable
Template CloudFormation Stack

Infrastructure as Code workflow
code
version
control
code
review
integrate deploy

code
version
control
code
review
integrate deploy
Text Editor
Git/SVN/
Perforce
Review
Tools
Syntax
Validation
Tools
AWS
Services

code
version
control
code
review
integrate deploy
“It’s all software”
Text Editor
Git/SVN/
Perforce
Review
Tools
Syntax
Validation
Tools
AWS
Services

In-place Blue-Green
Traffic
• Quicker and more cost efficient
• Updates or replaces resources
• Simpler state and data
migration
• Working stack stays intact for
easy roll backs
• Potentially need to sync data
• Shift traffic to new stack
Templates
Stacks
Update like software
Traffic

{
"AWSTemplateFormatVersion" : "2010-09-09",
"Description" : "AWS CloudFormation Sample Template EC2InstanceSample: **WARNING** This template an Amazon EC2 instances. You will be billed for the AWS resources used if
you create a stack from this template.",
"Parameters" : {
"KeyName" : {
"Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instance",
"Type" : "String"
},
"Environment": {
"Type" : "String",
"Default" : ”Dev",
"AllowedValues" : [”Mgmt", "Dev", ”Staging", "Prod"],
"Description" : "Environment that the instances will run in.”
}
},
"Mappings" : {
"RegionMap" : {
"us-east-1" : { "AMI" : "ami-7f418316" },
"us-west-2" : { "AMI" : "ami-16fd7026" }
}
},
"Conditions" : {
”EnableEBSOptimized" : {"Fn::Equals" : [{"Ref" : " Environment "}, ”Prod"]},
},
"Resources" : {
"Ec2Instance" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"KeyName" : { "Ref" : "KeyName" },
"EbsOptimized " : {"Fn::If": [ " EnableEBSOptimized ", {“true”}, {“false”}]},
"ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]},
"UserData" : { "Fn::Base64" : "80" }
}
}
},
"Outputs" : {
"InstanceId" : {
"Description" : "InstanceId of the newly created EC2 instance",
"Value" : { "Ref" : "Ec2Instance" }
},
"PublicDNS" : {
"Description" : "Public DNSName of the newly created EC2 instance",
"Value" : { "Fn::GetAtt" : [ "Ec2Instance", "PublicDnsName" ] }
}
}
}

{
"Parameters" : {
"KeyName" : {
"Type" : "String"
},
"Environment": {
"Type" : "String",
}
},
"Mappings" : {
"RegionMap" : {
"us-east-1" : { "AMI" : "ami-7f418316" },
"us-west-2" : { "AMI" : "ami-16fd7026" }
}
},
"Conditions" : {
},
"Resources" : {
"Ec2Instance" : {
"Properties" : {
"UserData" : { "Fn::Base64" : "80" }
}
}
},
"Outputs" : {
"InstanceId" : {
},
"PublicDNS" : {
}
}
}
HEADERS
PARAMETERS
MAPPINGS
RESOURCES
OUTPUTS
CONDITIONALS

{
"Parameters" : {
"KeyName" : {
"Type" : "String"
},
"Environment": {
"Type" : "String",
}
},
"Mappings" : {
"RegionMap" : {
"us-east-1" : { "AMI" : "ami-7f418316" },
"us-west-2" : { "AMI" : "ami-16fd7026" }
}
},
"Conditions" : {
},
"Resources" : {
"Ec2Instance" : {
"Properties" : {
"UserData" : { "Fn::Base64" : "80" }
}
}
},
"Outputs" : {
"InstanceId" : {
},
"PublicDNS" : {
}
}
}
HEADERS
PARAMETERS
MAPPINGS
RESOURCES
OUTPUTS
CONDITIONALS
Description of what your stack does, contains, etc
Provision time values that add structured flexibility and
customization
Pre-defined conditional case statements
Conditional values set via evaluations of passed references
AWS resource definitions
Resulting attributes of stack resource creation

{
"Description" : "Create an EC2 instance.”,
"Parameters": {
"KeyName": {
"Description" : "Name of an existing EC2 KeyPair to enable SSH
access into the WordPress web server",
"Type": "AWS::EC2::KeyPair::KeyName"
},
"EC2InstanceType" : {
"Description" : "EC2 instance type",
"Type" : "String",
"Default" : "t2.micro",
"AllowedValues" : [ "t2.micro", "t2.small", "t2.medium" ],
"ConstraintDescription" : "Must be t2.micro, t2.small, t2.medium"
},
},
Template Anatomy - Parameters

{
"Description" : "Create an EC2 instance.”,
"Resources" : {
"Ec2Instance" : {
"Properties" : {
"KeyName" : “my-key-pair”,
"ImageId" : "ami-6869aa05”,
“InstanceType” : “m3.medium”
}
}
}
}
Template Anatomy - Resources

"Outputs" : {
"WebsiteURL" : {
"Description" : ”DNS name of the website",
"Value" : {
"Fn::GetAtt" : [ “LoadBalancer”, “DNSName” ]
}
}
}
Template Anatomy - Outputs

 Visualize template resources
 Modify template with drag-drop gestures
 Customize sample templates
CloudFormation Designer

Avoid manual resource modifications
 Avoid making quick-fixes out of band
 Update your stacks with CloudFormation
 Do not manually change resources
 Consider using resource based permissions to
limit ability to make changes directly

Preview updates with Change Sets

"Mappings" : {
"RegionMap" : {
"us-east-1" : { "32" : "ami-6411e20d", "64" : "ami-7a11e213" },
"us-west-1" : { "32" : "ami-c9c7978c", "64" : "ami-cfc7978a" },
"eu-west-1" : { "32" : "ami-37c2f643", "64" : "ami-31c2f645" },
"ap-southeast-1" : { "32" : "ami-66f28c34", "64" : "ami-60f28c32" },
"ap-northeast-1" : { "32" : "ami-9c03a89d", "64" : "ami-a003a8a1" }
}
},
Fn::FindInMap

Fn::FindInMap
"Resources" : {
"myEC2Instance" : {
"Properties" : {
"ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region"
}, "32"]},
"InstanceType" : "m1.small"
}
}
}

"Resources" : {
"Ec2Instance" : {
"Properties" : {
"SecurityGroups" : [ { "Ref" : "InstanceSecurityGroup" } ],
"UserData" : { "Fn::Base64" : { "Fn::Join" : ["",[
"#!/bin/bash -ex","n",
"yum -y install gcc-c++ make","n",
"yum -y install mysql-devel sqlite-devel","n",
"yum -y install ruby-rdoc rubygems ruby-mysql ruby-devel","n",
"gem install --no-ri --no-rdoc rails","n",
"gem install --no-ri --no-rdoc mysql","n",
"gem install --no-ri --no-rdoc sqlite3","n",
"rails new myapp","n",
"cd myapp","n",
"rails server -d","n"]]}}
}
}
Use EC2 UserData, which is available as a property of AWS::EC2::Instance resources
Bootstrap your applications using EC2 UserData

 AWS CloudFormation provides helper scripts
for deployment within your EC2 instances
 Metadata Key —
AWS::CloudFormation::Init
 Cfn-init reads this metadata key and installs
the packages listed in this key (e.g., httpd,
mysql, and php). Cfn-init also retrieves and
expands files listed as sources.
Amazon EC2
AWS CloudFormation
cfn-init
cfn-hup
cfn-signal
cfn-get-
metadata
Bootstrap your applications using helper scripts

"Metadata": {
"AWS::CloudFormation::Init" : {
"config" : {
"packages" : {
},
"sources" : {
},
"commands" : {
},
"files" : {
},
"services" : {
},
"users" : {
},
"groups" : {
}
}
}
“WebAppHost" : {
"Metadata" : {
"AWS:CloudFormation::Init" : {
"config" : {
"packages" : {
"yum" : {
"gcc" : [],
"gcc-c++" : [],
"make" : [],
"automake" : [],
Use AWS::CloudFormation::Init with cfn-init to help bootstrap instances:
Bootstrapping example

{
"Statement" : [
{
"Effect" : "Allow",
"Action" : "Update:*",
"Principal": "*",
"Resource" : "*"
},
{
"Effect" : "Deny",
"Action" : "Update:*",
"Principal": "*",
"Resource" : "LogicalResourceId/ProductionDatabase"
}
]
}
Prevent stack updates to protected resources using Stack policies
Protect your resources using Stack policies

Ownership based template design
 Limit one template to a single service
 Aim for reusability across environments (development, testing, production)
 Use nested stacks and cross-stack reference to break up large templates
 Organize templates according to team structure/job function/line of business

Ownership based template design

Template File
Defining Stack
The entire infrastructure can be
represented in an AWS
CloudFormation template.
Many Stacks & Environments from One Template

Template File
Defining Stack
Use the version
control system of
your choice to
store and track
changes to this
template
Git
Perforce
SVN
…

Template File
Defining Stack
Git
Perforce
SVN
…
Dev
Test
Prod
Use the version
control system of
your choice to
store and track
changes to this
template
Build out multiple
environments, such
as for Development,
Test, Production and
even DR using the
same template

Web-SG
Ownership – cross-stack references
App-SG
App-SG
DB-SG

Re-usable Templates – across AWS Regions
 Consider environmental or regional differences
 Amazon EC2 image Ids
 VPC environment or “classic” environment
 Available instance types
 IAM policy principals
 Endpoint names
 Amazon Resource Names (arns)

"LogsBucketPolicy": {
"Type": "AWS::S3::BucketPolicy",
"Properties": {
"Bucket": {"Ref": "LogsBucket”},
"PolicyDocument": {
"Version": "2008-10-17",
"Statement": [{
"Sid": "ELBAccessLogs",
"Effect": "Allow",
"Resource": {
"Fn::Join": [ "", [ “arn:aws:s3:::",
{ "Ref": "LogsBucket" }, "/", "Logs",
"/AWSLogs/", { "Ref": "AWS::AccountId" }, "/*” ]]
},
"Principal": …,
"Action": [ "s3:PutObject" ]
}]
}
}
},
 Use “pseudo-parameters” to retrieve environmental data
 Account Id
 Region
 Stack Name and Id
Re-usable Templates – “Pseudo-Parameters”

Re-usable Templates - Using mappings
"LogsBucketPolicy": {
"Type": "AWS::S3::BucketPolicy",
"Properties": {
"Bucket": {"Ref": "LogsBucket”},
"PolicyDocument": {
"Version": "2008-10-17",
"Statement": [{
"Sid": "ELBAccessLogs",
"Effect": "Allow",
"Resource": {
"Fn::Join": [ "", [
{ "Fn::FindInMap" : ["RegionalConfig",
{"Ref" : "AWS::Region"},
"ArnPrefix”]},
"s3:::”, { "Ref": "LogsBucket" }, "/",
"Logs",
"/AWSLogs/”,
{ "Ref": "AWS::AccountId" }, "/*" ] ]
},
}
“Mappings” : {
“RegionalConfig” : {
“us-east-1” : {
“AMI” :
“ami-12345678”,
”ELBAccountId":
"127311923021”,
“ArnPrefix” :
“arn:aws:”
},
“us-west-1” : {
“AMI” :
“ami-98765432”
”ELBAccountId":
“027434742980"
“ArnPrefix” :
“arn:aws:”
},
:
}
}
 Use mappings to define variables
 Single place for configuration
 Re-usable within the template

Use conditionals to customize
resources and parameters
"DBEC2SG": {
"Type": "AWS::EC2::SecurityGroup",
"Condition" : "Is-EC2-VPC",
"Properties" : {…}
},
"DBSG": {
"Type": "AWS::RDS::DBSecurityGroup",
"Condition" : "Is-EC2-Classic",
"Properties": {…}
},
"MySQLDatabase": {
"Type": "AWS::RDS::DBInstance",
"Properties": {
:
"VPCSecurityGroups": { "Fn::If" : [ "Is-EC2-VPC",
[ { "Fn::GetAtt": [ "DBEC2SG", "GroupId" ] } ],
{ "Ref" : "AWS::NoValue"}]},
"DBSecurityGroups": { "Fn::If" : [ "Is-EC2-Classic",
[ { "Ref": "DBSG" } ],
{ "Ref" : "AWS::NoValue"}]}
}
}
}
"Conditions" : {
"Is-EC2-VPC” : { "Fn::Or" : [
{"Fn::Equals" : [
{"Ref” : "AWS::Region"},
"eu-central-1" ]},
{"Fn::Equals" : [
{"Ref" : "AWS::Region"},
"cn-north-1" ]}]},
"Is-EC2-Classic" : { "Fn::Not" : [
{ "Condition" : "Is-EC2-VPC"}]}
},
Re-usable Templates – Using conditionals

Best Practices Summary
 CloudFormation Designer
 Avoid manual resource modifications
 Preview updates with Change Sets
 Learn the intrinsic functions
 Bootstrap your applications using UserData and helper scripts
 Protect critical resources using stack policies
 Ownership based template design
 Plan for environment and multi-region
 Use Pseudo-Parameters
 Use Mappings
 Use Conditionals

 YAML formatted templates
 Overview of template structure / basics
 New function formatting (!Ref / !GetAZs / !FindInMap)
 New Intrinsic Function ( Fn::Sub )
 Cross Stack References
 New function Fn::ImportValue
 Allows use of outputs from unrelated stacks with custom resource
New
Key new features
New

CloudFormation - YAML
Why YAML?
 Better authoring and readability of templates
 Native comment support
 Simplification as templates get more and more complex
New

Cloudformation - YAML
Resources:
VPC1:
Type: "AWS::EC2::VPC"
Properties:
CidrBlock: !Ref VPC1Cidr
Tags:
-
Key: "Name"
Value: "TroubleShooting"
 Structure is shown through indentation (one or more spaces).
 Sequence items are denoted by a dash
 Key value pairs within a map are separated by a colon.
 Tips: Use a monospace font, don’t use Tab, save using UTF-8

All sections are the same as in a JSON template
---
AWSTemplateFormatVersion: "version date"
Description:
String
Metadata:
template metadata
Parameters:
set of parameters
Mappings:
set of mappings
Conditions:
set of conditions
Resources:
set of resources
Outputs:
set of outputs
CloudFormation – YAML Template Structure

 Two ways to declare Intrinsic functions: Long and Short
 Short Form:
 !FindInMap [ MapName, TopLevelKey, SecondLevelKey ]
 Long Form:
 "Fn::FindInMap" : [ "MapName", "TopLevelKey", "SecondLevelKey"]
 Tag = ! (Its not Negation operator)
 Few things to note with Tags
 You cannot use one tag immediately after another
 !Base64 !Sub…
 Instead, you can do this
 "Fn::Base64": !Sub...
 !Select [ !Ref Value, [1,2,3]]
CloudFormation – YAML Function Declaration

Fn::Base64 Fn::And
Short !Base64 valueToEncode Short !And [condition]
Long "Fn::Base64": valueToEncode Long "Fn::And": [condition]
Fn::Equals Fn::If
Short !Equals [value_1, value_2] Short !If [condition_name, value_if_true, value_if_false]
Long "Fn::Equals": [value_1, value_2] Long "Fn::If": [condition_name, value_if_true, value_if_false]
Fn::Not Fn::Or
Short !Not [condition] Short !Or [condition, ...]
Long "Fn::Not": [condition] Long "Fn::Or": [condition, ...]
CloudFormation – Intrinsic Functions

NewCloudFormation – Fn::Sub
 Substitute variables in an input string with values
 Function accepts a string or a map as a parameter.
 Usage
 VarName: ${MyVariableValue}
 Literal: ${!LiteralValue}
 Use | if you are spanning multiple lines
 Available in JSON as well

/tmp/create-wp-config:
content: !Sub |
#!/bin/bash -xe
cp /var/www/html/wordpress/wp-config-sample.php /var/www/html/wordpress/wp-config.php
sed -i "s/'database_name_here'/'${DBName}'/g" wp-config.php
sed -i "s/'username_here'/'${DBUser}'/g" wp-config.php
sed -i "s/'password_here'/'${DBPassword}'/g" wp-config.php
mode: '000500'
owner: root
group: root
configure_wordpress:
commands:
01_set_mysql_root_password:
command: !Sub |
mysqladmin -u root password '${DBRootPassword}'
test: !Sub |
$(mysql ${DBName} -u root --password='${DBRootPassword}' >/dev/null 2>&1 </dev/null); (( $? != 0 ))
02_create_database:
command: !Sub |
mysql -u root --password='${DBRootPassword}' < /tmp/setup.mysql
test: !Sub |
$(mysql ${DBName} -u root --password='${DBRootPassword}' >/dev/null 2>&1 </dev/null); (( $? !=0))
03_configure_wordpress:
command: /tmp/create-wp-config
cwd: /var/www/html/wordpress
CloudFormation – Fn::Sub Examples

CloudFormation – Cross Stack References
 Sharing resources made easy
 IAM roles, VPC, Security groups
 Add an explicit “Export” declaration to stack output
 Use the resource in another stack using a new intrinsic function,
Fn::ImportValue`
 Few guidelines:
 Export names must be unique within an account and region
 Cannot create references across regions
 Cannot delete a stack that is referenced by another stack (Dependencies
are communicated in errors)
 Outputs cannot be modified or removed as long as it is referenced by a
current stack
New

The new intrinsic function for accessing exported outputs.
JSON
{ "Fn::ImportValue" : sharedValueToImport }
YAML
"Fn::ImportValue": sharedValueToImport
!ImportValue sharedValueToImport
CloudFormation – Fn::ImportValue

Stack A
Stack B
"Outputs": {
"WebServerSecurityGroup": {
"Description": "TheIDofthesecuritygroup",
"Value": {"Fn: : GetAtt": ["WebServerSecurityGroup", "GroupId"]},
"Export": { "Name": "AccountSecGroup"}
}
}
"Resources" : {
"WebServerInstance" : {
"Properties" : {
"InstanceType" : "ts.micro",
"ImageId" : "ami-a1b23456",
"NetworkInterfaces" : [{
"GroupSet" : [{ "Fn::ImportValue" : "AccountSecGroup" ]}
]}
}
}
}
CloudFormation – Cross Stack Examples

AWS Infrastructure as Code - September 2016 Webinar Series

Related slideshows

More Related Content

AWS Infrastructure as Code - September 2016 Webinar Series