SlideShare a Scribd company logo
Baking in the Cloud
with Packer and
Puppet
Alan
Parkinson
@alan_parkinso
n
Baking in the cloud with packer and puppet
Large jumps in demand, we
have to scale fast
Load Balancer Behave Pro App
Our problems with
provisioning on start-up

Recommended for you

(2016-06-11) Packer: Make Multi-Platform Images
(2016-06-11) Packer: Make Multi-Platform Images(2016-06-11) Packer: Make Multi-Platform Images
(2016-06-11) Packer: Make Multi-Platform Images

Packer is software that allows you to create machine images for multiple platforms from a single template configuration. It uses builders to create images for platforms like AWS, VirtualBox, Docker etc. and provisioners can be used to configure the images. Packer provides benefits like consistency between development and production environments, and ease of sharing and reuse of images.

devopshashicorpself
Ansible and AWS
Ansible and AWSAnsible and AWS
Ansible and AWS

This document summarizes an Ansible and AWS meetup. It discusses using Ansible to provision and configure AWS resources like EC2 instances, security groups, ELBs, and more through idempotent playbooks. Key points covered include Ansible's agentless architecture, dynamic AWS inventory plugin, core modules like ec2 and cloudformation, templates, roles for reuse, and examples of provisioning playbooks that launch instances and apply configurations. It also briefly mentions NetflixOSS projects that use Ansible like Aminator for AMIs and Asgard for provisioning.

awsansibleamazon web services
Jenkins and ansible reference
Jenkins and ansible referenceJenkins and ansible reference
Jenkins and ansible reference

This document provides an introduction to Ansible, describing it as a simple and lightweight automation tool that can be used to execute one-time tasks, perform system administration tasks, and configure servers and routers. It discusses Ansible's key features including being written in Python, being open source, and being easy to install and use. It also provides information on installing and configuring Ansible on various operating systems as well as how to use ad-hoc commands and playbooks with Ansible.

Reliability
Many single points of
failure
Require Puppet
Master redundancy
and datacentre
replication
Mirror dependant
software repositories
Provisioning Latency…
9 minutes for puppet to prepare a app server
Total 15 minute response time to a scaling request
Baking images like
Produce a
preconfigured
machine image
that is rolled out
to the autoscaling
groups
Packer is a tool for
creating identical
machine images
for multiple
platforms from a
single source
configuration
PACKER.IO A HASHICORP PROJECT

Recommended for you

Cyansible
CyansibleCyansible
Cyansible

Blue/Green deployments have been an important, if rarely implemented, technique in the Continuous Delivery playbook for years. Their aim is simple: provision, deploy, test — and optionally roll-back — your application before it's served to the public. Betterment's deployment architecture takes a similar, but more straightforward approach, accomplishing the important goals sought out by Blue/Green practitioners. Dubbed 'Cyan' (a mixture of Blue/Green), Betterment uses Ansible to provision new instances, push the latest artifacts to them, and ensure that they're healthy before marking them ready for production. All this ensures fast, stable, zero-downtime rollout with minimal human interaction. We'll discuss Betterment's philosophical approach to shipping new code and then dive into the nitty-gritty Ansible that powers the whole thing.

ansiblefinancedevops
Ansible 101 - Presentation at Ansible STL Meetup
Ansible 101 - Presentation at Ansible STL MeetupAnsible 101 - Presentation at Ansible STL Meetup
Ansible 101 - Presentation at Ansible STL Meetup

Jeff Geerling, author of Ansible for DevOps, demonstrates basic Ansible usage on the Dramble, a cluster of six Raspberry Pi 2 computers. This presentation was delivered on July 8, 2015, at the Ansible St. Louis meetup, at Riot Games in Clayton, MO.

deploymentmeetuporchestration
Ansible presentation
Ansible presentationAnsible presentation
Ansible presentation

This document provides an overview of Ansible, an open source tool for configuration management and application deployment. It discusses how Ansible aims to simplify infrastructure automation tasks through a model-driven approach without requiring developers to learn DevOps tools. Key points: - Ansible uses YAML playbooks to declaratively define server configurations and deployments in an idempotent and scalable way. - It provides ad-hoc command execution and setup facts gathering via SSH. Playbooks can target groups of servers to orchestrate complex multi-server tasks. - Variables, templates, conditionals allow playbooks to customize configurations for different environments. Plugins support integration with cloud, monitoring, messaging tools. - Ansible aims to reduce complexity compared

ansibledevops
PACKER.IO A HASHICORP PROJECT
packer.json
{
"variables": {
"aws_access_key": "",
"aws_secret_key": ""
},
"builders": [{
"type": "amazon-ebs",
"access_key": "{{user `aws_access_key`}}",
"secret_key": "{{user `aws_secret_key`}}",
"region": "us-east-1",
"source_ami": "ami-9eaa1cf6",
"instance_type": "t2.micro",
"ssh_username": "ubuntu",
"ami_name": "packer-example {{timestamp}}"
}]
}
$ packer build -var 'aws_access_key=YOUR ACCESS KEY' 
-var 'aws_secret_key=YOUR SECRET KEY' 
example.json
==> amazon-ebs: amazon-ebs output will be in this color.
==> amazon-ebs: Creating temporary keypair for this instance...
==> amazon-ebs: Creating temporary security group for this instance...
==> amazon-ebs: Authorizing SSH access on the temporary security group...
==> amazon-ebs: Launching a source AWS instance...
==> amazon-ebs: Waiting for instance to become ready...
==> amazon-ebs: Connecting to the instance via SSH...
==> amazon-ebs: Stopping the source instance...
==> amazon-ebs: Waiting for the instance to stop...
==> amazon-ebs: Creating the AMI: packer-example 1371856345
==> amazon-ebs: AMI: ami-19601070
==> amazon-ebs: Waiting for AMI to become ready...
==> amazon-ebs: Terminating the source AWS instance...
==> amazon-ebs: Deleting temporary security group...
==> amazon-ebs: Deleting temporary keypair...
==> amazon-ebs: Build finished.
==> Builds finished. The artifacts of successful builds are:
--> amazon-ebs: AMIs were created:
us-east-1: ami-19601070
Add some provisioners
{
"variables": {…},
"builders": […],
"provisioners": [
{
"type": "shell",
"script": "../common/install-puppet.sh"
},
…
{
"inline": [
"sudo apt-get purge --yes puppet",
"sudo apt-get autoremove --yes"
],
"type": "shell"
}
]
}
No need for a Puppet Master
"provisioners": [
…
{
"type": "puppet-masterless",
"hiera_config_path": "../puppet/hiera.yaml",
"manifest_file": "../puppet/manifests/default.pp",
"module_paths": [ "../puppet/modules" ]
},
…
]
Manifests, modules and hiera data can all be
stored in git and git submodules

Recommended for you

A quick intro to Ansible
A quick intro to AnsibleA quick intro to Ansible
A quick intro to Ansible

A revamped version of the Ansible intro talk from February 2015, brought up-to-date for the January Ansible meetup in Berlin. Join our group: https://www.meetup.com/Ansible-Berlin

configuration managementansibleautomation
Network Automation with Ansible
Network Automation with AnsibleNetwork Automation with Ansible
Network Automation with Ansible

Anas Tarsha presented on using Ansible for network automation. Ansible is an open source automation tool that is agentless and uses simple YAML files called playbooks to execute tasks sequentially. It can be used to generate device configurations, push configurations, collect running configs, upgrade devices, and more. Ansible modules run Python code directly on network devices to perform tasks. The demo showed using Ansible modules like ping, ios_command, and junos_command to execute show commands and change the hostname on both IOS and Junos devices. Additional resources were provided to learn more about using Ansible for network automation.

ansiblenetwork automation
Ansible Introduction - Ansible Brno #1 - David Karban
Ansible Introduction - Ansible Brno #1 - David KarbanAnsible Introduction - Ansible Brno #1 - David Karban
Ansible Introduction - Ansible Brno #1 - David Karban

Ansible is an agentless configuration management and provisioning tool that is easy to use and secure. It uses an inventory file to define hosts and groups, and facts to gather information about hosts. Playbooks are written in YAML format to define tasks like provisioning, deploying applications, and configuration using modules. Playbooks can include roles and tasks. Ansible has over 250 modules for various tasks like packaging, source control, cloud services, and operating system functions. Additional tools include Vault for encrypting variables and Galaxy for sharing roles.

devopslinuxansible
How do we protect
sensitive configuration
data?
hiera-eyaml
backend for Hiera that provides per-value
encryption of sensitive data within yaml files
---
duo-security-skey: ENC[PKCS7,MIIBmQYJKoZIh……Anc=]
behave_pro:
logentries_api_key: ENC[PKCS7,MIIBmQYJKoZIh……uW8=]
application_secret: ENC[PKCS7, MIIBmQYJKoZIh……FRg==]
common.eyaml
hiera.yaml
---
:backends:
- eyaml
- yaml
:hierarchy:
- "%{environment}"
- common
:yaml:
:datadir: '/tmp/hieradata'
:eyaml:
:datadir: '/tmp/hieradata'
:pkcs7_private_key: /tmp/hierakeys/private_key.pkcs7.pem
:pkcs7_public_key: /tmp/hierakeys/public_key.pkcs7.pem
Note: Use a temporary folders or data will be baked into the final
image
Basic asymmetric encryption
(PKCS#7)
Private key decrypts data
Puppet Master or agent only needs this at runtime
$ eyaml encrypt -s 'hello there'
Public key encrypts data
Safe to distribute to developers and ops engineers
Git diff allows peer review without decrypting values

Recommended for you

Ansible 2 and Ansible Galaxy 2
Ansible 2 and Ansible Galaxy 2Ansible 2 and Ansible Galaxy 2
Ansible 2 and Ansible Galaxy 2

- Ansible 2 includes an architecture overhaul, new YAML parsing engine, and 100% backwards compatibility for playbooks. It introduces task blocks for error handling and dynamic includes. Execution strategies allow running tasks linearly or in parallel. Many new modules were added. - Galaxy 2 provides better metrics, role management, and Travis CI integration. It allows importing roles from organizations and individuals. The ansible-galaxy CLI was improved with features like role scaffolding and authentication.

softwareinfrastructuredevops
Ansible
AnsibleAnsible
Ansible

Introduces Ansible as DevOps favorite choice for Configuration Management and Server Provisioning. Enables audience to get started with using Ansible. Developed in Python which only needs YAML syntax knowledge to automate using this tool.

ansibleconfiguration managementdeployment
Using Ansible for Deploying to Cloud Environments
Using Ansible for Deploying to Cloud EnvironmentsUsing Ansible for Deploying to Cloud Environments
Using Ansible for Deploying to Cloud Environments

Andrew Hamilton presented on using Ansible for deploying to cloud environments. He discussed that Ansible was chosen to provide a simple and repeatable way to build services and deploy them to configured environments. It allows deploying to multiple languages and cloud platforms. Ansible uses a simple execution model and YAML files. Its key advantages include being agentless over SSH, supporting dynamic inventories, and having modules for common tasks. Hamilton provided tips for using Ansible effectively in cloud environments, such as using dynamic inventories, separating variables, and testing changes thoroughly.

ansibleclouddevops
The eyaml keys are stored in a private S3
bucket with access controlled by a IAM
Policy
Distributing the keys
to the Bakery
Use a IAM Role in Packer to access the S3
bucket
"builders": [{
"type": "amazon-ebs",
…
"iam_instance_profile" : "puppet-provisioner",
…
}]
IAM Policy
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1425244502000",
"Effect": "Allow",
"Action": [
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::puppet.behave.pro/*"
]
}
]
}
install-heira-key.sh
Download the keys to the EC2
Instance
sudo apt-get install --yes python-pip
sudo pip install s3cmd
s3cmd get s3://puppet.behave.pro/private_key.pkcs7.pem
/tmp/hierakeys/private_key.pkcs7.pem
s3cmd get s3://puppet.behave.pro/public_key.pkcs7.pem
/tmp/hierakeys/public_key.pkcs7.pem
"provisioners": [
{
"type": "shell",
"script": "../common/install-hiera-key.sh"
},
]
packer.json
Summary
If scaling fast or reliably are
important, bake images
Git makes a great alternative to
Puppet Master when baking
Secure data with hiera-eyaml
https://github.com/TomPoulton/hiera-eyaml

Recommended for you

ansible why ?
ansible why ?ansible why ?
ansible why ?

This document contains information about the sys/net/sec admin Yashar Esmaildokht, including their contact information and websites. It then provides a brief overview of the open-source automation tool Ansible, describing its main features and uses for configuration management, application deployment, and cloud provisioning. Requirements and versions of Ansible are listed. The document concludes with examples of Ansible concepts including playbooks, tasks, modules, variables, and host inventory organization.

Ansible basics workshop
Ansible basics workshopAnsible basics workshop
Ansible basics workshop

Slides from ansible workshop on devconf.cz, youtube link: https://www.youtube.com/watch?v=ISu9YV2eoug

linuxautomationansible
DevOps for Humans - Ansible for Drupal Deployment Victory!
DevOps for Humans - Ansible for Drupal Deployment Victory!DevOps for Humans - Ansible for Drupal Deployment Victory!
DevOps for Humans - Ansible for Drupal Deployment Victory!

Everyone knows it's a Good Idea™ to use a configuration management system (e.g. Puppet, Chef) to manage your Drupal infrastructure. But many people (myself included) have run into a wall of #wtfmoments when trying to learn the vagaries of traditional CM systems and their vendor-specific syntaxes. In 2012, Ansible was released, enabling normal human beings to manage their servers with an easy, but powerful, CM system that uses YAML (just like Drupal 8!) to define configuration and Jinja2 (very much like Twig!) for templates. Not only that, but Ansible is also an incredibly simple and very flexible Drupal deployment and continuous delivery tool. Learn how you can use Ansible to manage your infrastructure—including local development environments—and stop letting servers and deployments get in the way of development.

drupaldeploymentconfiguration management
Questions

More Related Content

What's hot

OpenSource ToolChain for the Hybrid Cloud
OpenSource ToolChain for the Hybrid CloudOpenSource ToolChain for the Hybrid Cloud
OpenSource ToolChain for the Hybrid Cloud
Isaac Christoffersen
 
Automation with Packer and TerraForm
Automation with Packer and TerraFormAutomation with Packer and TerraForm
Automation with Packer and TerraForm
Wesley Charles Blake
 
Build automated Machine Images using Packer
Build automated Machine Images using PackerBuild automated Machine Images using Packer
Build automated Machine Images using Packer
Marek Piątek
 
(2016-06-11) Packer: Make Multi-Platform Images
(2016-06-11) Packer: Make Multi-Platform Images(2016-06-11) Packer: Make Multi-Platform Images
(2016-06-11) Packer: Make Multi-Platform Images
garrett honeycutt
 
Ansible and AWS
Ansible and AWSAnsible and AWS
Ansible and AWS
Peter Sankauskas
 
Jenkins and ansible reference
Jenkins and ansible referenceJenkins and ansible reference
Jenkins and ansible reference
laonap166
 
Cyansible
CyansibleCyansible
Cyansible
Alan Norton
 
Ansible 101 - Presentation at Ansible STL Meetup
Ansible 101 - Presentation at Ansible STL MeetupAnsible 101 - Presentation at Ansible STL Meetup
Ansible 101 - Presentation at Ansible STL Meetup
Jeff Geerling
 
Ansible presentation
Ansible presentationAnsible presentation
Ansible presentation
John Lynch
 
A quick intro to Ansible
A quick intro to AnsibleA quick intro to Ansible
A quick intro to Ansible
Dan Vaida
 
Network Automation with Ansible
Network Automation with AnsibleNetwork Automation with Ansible
Network Automation with Ansible
Anas
 
Ansible Introduction - Ansible Brno #1 - David Karban
Ansible Introduction - Ansible Brno #1 - David KarbanAnsible Introduction - Ansible Brno #1 - David Karban
Ansible Introduction - Ansible Brno #1 - David Karban
ansiblebrno
 
Ansible 2 and Ansible Galaxy 2
Ansible 2 and Ansible Galaxy 2Ansible 2 and Ansible Galaxy 2
Ansible 2 and Ansible Galaxy 2
Jeff Geerling
 
Ansible
AnsibleAnsible
Ansible
Vishal Yadav
 
Using Ansible for Deploying to Cloud Environments
Using Ansible for Deploying to Cloud EnvironmentsUsing Ansible for Deploying to Cloud Environments
Using Ansible for Deploying to Cloud Environments
ahamilton55
 
ansible why ?
ansible why ?ansible why ?
ansible why ?
Yashar Esmaildokht
 
Ansible basics workshop
Ansible basics workshopAnsible basics workshop
Ansible basics workshop
David Karban
 
DevOps for Humans - Ansible for Drupal Deployment Victory!
DevOps for Humans - Ansible for Drupal Deployment Victory!DevOps for Humans - Ansible for Drupal Deployment Victory!
DevOps for Humans - Ansible for Drupal Deployment Victory!
Jeff Geerling
 
Introduction to ansible
Introduction to ansibleIntroduction to ansible
Introduction to ansible
Dharmit Shah
 
Introducing Ansible
Introducing AnsibleIntroducing Ansible
Introducing Ansible
Francesco Pantano
 

What's hot (20)

OpenSource ToolChain for the Hybrid Cloud
OpenSource ToolChain for the Hybrid CloudOpenSource ToolChain for the Hybrid Cloud
OpenSource ToolChain for the Hybrid Cloud
 
Automation with Packer and TerraForm
Automation with Packer and TerraFormAutomation with Packer and TerraForm
Automation with Packer and TerraForm
 
Build automated Machine Images using Packer
Build automated Machine Images using PackerBuild automated Machine Images using Packer
Build automated Machine Images using Packer
 
(2016-06-11) Packer: Make Multi-Platform Images
(2016-06-11) Packer: Make Multi-Platform Images(2016-06-11) Packer: Make Multi-Platform Images
(2016-06-11) Packer: Make Multi-Platform Images
 
Ansible and AWS
Ansible and AWSAnsible and AWS
Ansible and AWS
 
Jenkins and ansible reference
Jenkins and ansible referenceJenkins and ansible reference
Jenkins and ansible reference
 
Cyansible
CyansibleCyansible
Cyansible
 
Ansible 101 - Presentation at Ansible STL Meetup
Ansible 101 - Presentation at Ansible STL MeetupAnsible 101 - Presentation at Ansible STL Meetup
Ansible 101 - Presentation at Ansible STL Meetup
 
Ansible presentation
Ansible presentationAnsible presentation
Ansible presentation
 
A quick intro to Ansible
A quick intro to AnsibleA quick intro to Ansible
A quick intro to Ansible
 
Network Automation with Ansible
Network Automation with AnsibleNetwork Automation with Ansible
Network Automation with Ansible
 
Ansible Introduction - Ansible Brno #1 - David Karban
Ansible Introduction - Ansible Brno #1 - David KarbanAnsible Introduction - Ansible Brno #1 - David Karban
Ansible Introduction - Ansible Brno #1 - David Karban
 
Ansible 2 and Ansible Galaxy 2
Ansible 2 and Ansible Galaxy 2Ansible 2 and Ansible Galaxy 2
Ansible 2 and Ansible Galaxy 2
 
Ansible
AnsibleAnsible
Ansible
 
Using Ansible for Deploying to Cloud Environments
Using Ansible for Deploying to Cloud EnvironmentsUsing Ansible for Deploying to Cloud Environments
Using Ansible for Deploying to Cloud Environments
 
ansible why ?
ansible why ?ansible why ?
ansible why ?
 
Ansible basics workshop
Ansible basics workshopAnsible basics workshop
Ansible basics workshop
 
DevOps for Humans - Ansible for Drupal Deployment Victory!
DevOps for Humans - Ansible for Drupal Deployment Victory!DevOps for Humans - Ansible for Drupal Deployment Victory!
DevOps for Humans - Ansible for Drupal Deployment Victory!
 
Introduction to ansible
Introduction to ansibleIntroduction to ansible
Introduction to ansible
 
Introducing Ansible
Introducing AnsibleIntroducing Ansible
Introducing Ansible
 

Viewers also liked

Building Windows Images with Packer
Building Windows Images with PackerBuilding Windows Images with Packer
Building Windows Images with Packer
Matt Wrock
 
Chasing AMI - Building Amazon machine images with Puppet, Packer and Jenkins
Chasing AMI - Building Amazon machine images with Puppet, Packer and JenkinsChasing AMI - Building Amazon machine images with Puppet, Packer and Jenkins
Chasing AMI - Building Amazon machine images with Puppet, Packer and Jenkins
Tomas Doran
 
Fabric: A Capistrano Alternative
Fabric:  A Capistrano AlternativeFabric:  A Capistrano Alternative
Fabric: A Capistrano Alternative
Panoptic Development, Inc.
 
Gestionando tu infraestructura con Ansible
Gestionando tu infraestructura con Ansible Gestionando tu infraestructura con Ansible
Gestionando tu infraestructura con Ansible
Moisés Maciá
 
EC2 AMI Factory with Chef, Berkshelf, and Packer
EC2 AMI Factory with Chef, Berkshelf, and PackerEC2 AMI Factory with Chef, Berkshelf, and Packer
EC2 AMI Factory with Chef, Berkshelf, and Packer
George Miranda
 
Immutable servers
Immutable serversImmutable servers
Immutable servers
Alan Chalmers
 
Dev ops infraestructura agil con open source
Dev ops   infraestructura agil con open sourceDev ops   infraestructura agil con open source
Dev ops infraestructura agil con open source
scrumecuador
 

Viewers also liked (7)

Building Windows Images with Packer
Building Windows Images with PackerBuilding Windows Images with Packer
Building Windows Images with Packer
 
Chasing AMI - Building Amazon machine images with Puppet, Packer and Jenkins
Chasing AMI - Building Amazon machine images with Puppet, Packer and JenkinsChasing AMI - Building Amazon machine images with Puppet, Packer and Jenkins
Chasing AMI - Building Amazon machine images with Puppet, Packer and Jenkins
 
Fabric: A Capistrano Alternative
Fabric:  A Capistrano AlternativeFabric:  A Capistrano Alternative
Fabric: A Capistrano Alternative
 
Gestionando tu infraestructura con Ansible
Gestionando tu infraestructura con Ansible Gestionando tu infraestructura con Ansible
Gestionando tu infraestructura con Ansible
 
EC2 AMI Factory with Chef, Berkshelf, and Packer
EC2 AMI Factory with Chef, Berkshelf, and PackerEC2 AMI Factory with Chef, Berkshelf, and Packer
EC2 AMI Factory with Chef, Berkshelf, and Packer
 
Immutable servers
Immutable serversImmutable servers
Immutable servers
 
Dev ops infraestructura agil con open source
Dev ops   infraestructura agil con open sourceDev ops   infraestructura agil con open source
Dev ops infraestructura agil con open source
 

Similar to Baking in the cloud with packer and puppet

Burn down the silos! Helping dev and ops gel on high availability websites
Burn down the silos! Helping dev and ops gel on high availability websitesBurn down the silos! Helping dev and ops gel on high availability websites
Burn down the silos! Helping dev and ops gel on high availability websites
Lindsay Holmwood
 
Immutable AWS Deployments with Packer and Jenkins
Immutable AWS Deployments with Packer and JenkinsImmutable AWS Deployments with Packer and Jenkins
Immutable AWS Deployments with Packer and Jenkins
Manish Pandit
 
Bootstrap your Cloud Infrastructure using puppet and hashicorp stack
Bootstrap your Cloud Infrastructure using puppet and hashicorp stackBootstrap your Cloud Infrastructure using puppet and hashicorp stack
Bootstrap your Cloud Infrastructure using puppet and hashicorp stack
Bram Vogelaar
 
Puppet and Apache CloudStack
Puppet and Apache CloudStackPuppet and Apache CloudStack
Puppet and Apache CloudStack
Puppet
 
Zero to Sixty: AWS CloudFormation (DMG201) | AWS re:Invent 2013
Zero to Sixty: AWS CloudFormation (DMG201) | AWS re:Invent 2013Zero to Sixty: AWS CloudFormation (DMG201) | AWS re:Invent 2013
Zero to Sixty: AWS CloudFormation (DMG201) | AWS re:Invent 2013
Amazon Web Services
 
Running Vue Storefront in production (PWA Magento webshop)
Running Vue Storefront in production (PWA Magento webshop)Running Vue Storefront in production (PWA Magento webshop)
Running Vue Storefront in production (PWA Magento webshop)
Vendic Magento, PWA & Marketing
 
OSGi Feature Model - Where Art Thou - David Bosschaert (Adobe)
OSGi Feature Model - Where Art Thou - David Bosschaert (Adobe)OSGi Feature Model - Where Art Thou - David Bosschaert (Adobe)
OSGi Feature Model - Where Art Thou - David Bosschaert (Adobe)
mfrancis
 
Automation with Ansible and Containers
Automation with Ansible and ContainersAutomation with Ansible and Containers
Automation with Ansible and Containers
Rodolfo Carvalho
 
Puppet: Eclipsecon ALM 2013
Puppet: Eclipsecon ALM 2013Puppet: Eclipsecon ALM 2013
Puppet: Eclipsecon ALM 2013
grim_radical
 
Puppet and CloudStack
Puppet and CloudStackPuppet and CloudStack
Puppet and CloudStack
ke4qqq
 
AWS EC2
AWS EC2AWS EC2
AWS EC2
whiskybar
 
Puppet at Pinterest
Puppet at PinterestPuppet at Pinterest
Puppet at Pinterest
Puppet
 
Infrastructure-as-code: bridging the gap between Devs and Ops
Infrastructure-as-code: bridging the gap between Devs and OpsInfrastructure-as-code: bridging the gap between Devs and Ops
Infrastructure-as-code: bridging the gap between Devs and Ops
Mykyta Protsenko
 
Puppetpreso
PuppetpresoPuppetpreso
Puppetpreso
ke4qqq
 
Web APIs & Apps - Mozilla
Web APIs & Apps - MozillaWeb APIs & Apps - Mozilla
Web APIs & Apps - Mozilla
Robert Nyman
 
Puppet and Apache CloudStack
Puppet and Apache CloudStackPuppet and Apache CloudStack
Puppet and Apache CloudStack
Puppet
 
Infrastructure as code with Puppet and Apache CloudStack
Infrastructure as code with Puppet and Apache CloudStackInfrastructure as code with Puppet and Apache CloudStack
Infrastructure as code with Puppet and Apache CloudStack
ke4qqq
 
Rapid scaling in_the_cloud_with_puppet
Rapid scaling in_the_cloud_with_puppetRapid scaling in_the_cloud_with_puppet
Rapid scaling in_the_cloud_with_puppet
Carl Caum
 
"Scaling ML from 0 to millions of users", Julien Simon, AWS Dev Day Kyiv 2019
"Scaling ML from 0 to millions of users", Julien Simon, AWS Dev Day Kyiv 2019"Scaling ML from 0 to millions of users", Julien Simon, AWS Dev Day Kyiv 2019
"Scaling ML from 0 to millions of users", Julien Simon, AWS Dev Day Kyiv 2019
Provectus
 
AWS September Webinar Series - Running Microservices with Amazon EC2 Contain...
AWS September Webinar Series -  Running Microservices with Amazon EC2 Contain...AWS September Webinar Series -  Running Microservices with Amazon EC2 Contain...
AWS September Webinar Series - Running Microservices with Amazon EC2 Contain...
Amazon Web Services
 

Similar to Baking in the cloud with packer and puppet (20)

Burn down the silos! Helping dev and ops gel on high availability websites
Burn down the silos! Helping dev and ops gel on high availability websitesBurn down the silos! Helping dev and ops gel on high availability websites
Burn down the silos! Helping dev and ops gel on high availability websites
 
Immutable AWS Deployments with Packer and Jenkins
Immutable AWS Deployments with Packer and JenkinsImmutable AWS Deployments with Packer and Jenkins
Immutable AWS Deployments with Packer and Jenkins
 
Bootstrap your Cloud Infrastructure using puppet and hashicorp stack
Bootstrap your Cloud Infrastructure using puppet and hashicorp stackBootstrap your Cloud Infrastructure using puppet and hashicorp stack
Bootstrap your Cloud Infrastructure using puppet and hashicorp stack
 
Puppet and Apache CloudStack
Puppet and Apache CloudStackPuppet and Apache CloudStack
Puppet and Apache CloudStack
 
Zero to Sixty: AWS CloudFormation (DMG201) | AWS re:Invent 2013
Zero to Sixty: AWS CloudFormation (DMG201) | AWS re:Invent 2013Zero to Sixty: AWS CloudFormation (DMG201) | AWS re:Invent 2013
Zero to Sixty: AWS CloudFormation (DMG201) | AWS re:Invent 2013
 
Running Vue Storefront in production (PWA Magento webshop)
Running Vue Storefront in production (PWA Magento webshop)Running Vue Storefront in production (PWA Magento webshop)
Running Vue Storefront in production (PWA Magento webshop)
 
OSGi Feature Model - Where Art Thou - David Bosschaert (Adobe)
OSGi Feature Model - Where Art Thou - David Bosschaert (Adobe)OSGi Feature Model - Where Art Thou - David Bosschaert (Adobe)
OSGi Feature Model - Where Art Thou - David Bosschaert (Adobe)
 
Automation with Ansible and Containers
Automation with Ansible and ContainersAutomation with Ansible and Containers
Automation with Ansible and Containers
 
Puppet: Eclipsecon ALM 2013
Puppet: Eclipsecon ALM 2013Puppet: Eclipsecon ALM 2013
Puppet: Eclipsecon ALM 2013
 
Puppet and CloudStack
Puppet and CloudStackPuppet and CloudStack
Puppet and CloudStack
 
AWS EC2
AWS EC2AWS EC2
AWS EC2
 
Puppet at Pinterest
Puppet at PinterestPuppet at Pinterest
Puppet at Pinterest
 
Infrastructure-as-code: bridging the gap between Devs and Ops
Infrastructure-as-code: bridging the gap between Devs and OpsInfrastructure-as-code: bridging the gap between Devs and Ops
Infrastructure-as-code: bridging the gap between Devs and Ops
 
Puppetpreso
PuppetpresoPuppetpreso
Puppetpreso
 
Web APIs & Apps - Mozilla
Web APIs & Apps - MozillaWeb APIs & Apps - Mozilla
Web APIs & Apps - Mozilla
 
Puppet and Apache CloudStack
Puppet and Apache CloudStackPuppet and Apache CloudStack
Puppet and Apache CloudStack
 
Infrastructure as code with Puppet and Apache CloudStack
Infrastructure as code with Puppet and Apache CloudStackInfrastructure as code with Puppet and Apache CloudStack
Infrastructure as code with Puppet and Apache CloudStack
 
Rapid scaling in_the_cloud_with_puppet
Rapid scaling in_the_cloud_with_puppetRapid scaling in_the_cloud_with_puppet
Rapid scaling in_the_cloud_with_puppet
 
"Scaling ML from 0 to millions of users", Julien Simon, AWS Dev Day Kyiv 2019
"Scaling ML from 0 to millions of users", Julien Simon, AWS Dev Day Kyiv 2019"Scaling ML from 0 to millions of users", Julien Simon, AWS Dev Day Kyiv 2019
"Scaling ML from 0 to millions of users", Julien Simon, AWS Dev Day Kyiv 2019
 
AWS September Webinar Series - Running Microservices with Amazon EC2 Contain...
AWS September Webinar Series -  Running Microservices with Amazon EC2 Contain...AWS September Webinar Series -  Running Microservices with Amazon EC2 Contain...
AWS September Webinar Series - Running Microservices with Amazon EC2 Contain...
 

More from Alan Parkinson

Test Data Builder Pattern
Test Data Builder PatternTest Data Builder Pattern
Test Data Builder Pattern
Alan Parkinson
 
Testing requirements with BDD
Testing requirements with BDDTesting requirements with BDD
Testing requirements with BDD
Alan Parkinson
 
Decapitating Selenium with JavaScript
Decapitating Selenium with JavaScriptDecapitating Selenium with JavaScript
Decapitating Selenium with JavaScript
Alan Parkinson
 
Pull requests and testers can be friends
Pull requests and testers can be friendsPull requests and testers can be friends
Pull requests and testers can be friends
Alan Parkinson
 
What is Continuous Delivery?
What is Continuous Delivery?What is Continuous Delivery?
What is Continuous Delivery?
Alan Parkinson
 
Cross browser Testing JavaScript
Cross browser Testing JavaScriptCross browser Testing JavaScript
Cross browser Testing JavaScript
Alan Parkinson
 
Why we used Feature Branching
Why we used Feature BranchingWhy we used Feature Branching
Why we used Feature Branching
Alan Parkinson
 
Test automation with Cucumber-JVM
Test automation with Cucumber-JVMTest automation with Cucumber-JVM
Test automation with Cucumber-JVM
Alan Parkinson
 
A Quick overview of Behaviour Driven Development (BDD)
A Quick overview of Behaviour Driven Development (BDD)A Quick overview of Behaviour Driven Development (BDD)
A Quick overview of Behaviour Driven Development (BDD)
Alan Parkinson
 
Cross-browser unit testing JavaScript
Cross-browser unit testing JavaScriptCross-browser unit testing JavaScript
Cross-browser unit testing JavaScript
Alan Parkinson
 
Creating data with the test data builder pattern
Creating data with the test data builder patternCreating data with the test data builder pattern
Creating data with the test data builder pattern
Alan Parkinson
 
Acceptance test styles - Imperative vs Declarative
Acceptance test styles - Imperative vs DeclarativeAcceptance test styles - Imperative vs Declarative
Acceptance test styles - Imperative vs Declarative
Alan Parkinson
 
Continuous Deployment Pipeline with maven
Continuous Deployment Pipeline with mavenContinuous Deployment Pipeline with maven
Continuous Deployment Pipeline with maven
Alan Parkinson
 
Feature branching a release day painkiller
Feature branching   a release day painkillerFeature branching   a release day painkiller
Feature branching a release day painkiller
Alan Parkinson
 

More from Alan Parkinson (14)

Test Data Builder Pattern
Test Data Builder PatternTest Data Builder Pattern
Test Data Builder Pattern
 
Testing requirements with BDD
Testing requirements with BDDTesting requirements with BDD
Testing requirements with BDD
 
Decapitating Selenium with JavaScript
Decapitating Selenium with JavaScriptDecapitating Selenium with JavaScript
Decapitating Selenium with JavaScript
 
Pull requests and testers can be friends
Pull requests and testers can be friendsPull requests and testers can be friends
Pull requests and testers can be friends
 
What is Continuous Delivery?
What is Continuous Delivery?What is Continuous Delivery?
What is Continuous Delivery?
 
Cross browser Testing JavaScript
Cross browser Testing JavaScriptCross browser Testing JavaScript
Cross browser Testing JavaScript
 
Why we used Feature Branching
Why we used Feature BranchingWhy we used Feature Branching
Why we used Feature Branching
 
Test automation with Cucumber-JVM
Test automation with Cucumber-JVMTest automation with Cucumber-JVM
Test automation with Cucumber-JVM
 
A Quick overview of Behaviour Driven Development (BDD)
A Quick overview of Behaviour Driven Development (BDD)A Quick overview of Behaviour Driven Development (BDD)
A Quick overview of Behaviour Driven Development (BDD)
 
Cross-browser unit testing JavaScript
Cross-browser unit testing JavaScriptCross-browser unit testing JavaScript
Cross-browser unit testing JavaScript
 
Creating data with the test data builder pattern
Creating data with the test data builder patternCreating data with the test data builder pattern
Creating data with the test data builder pattern
 
Acceptance test styles - Imperative vs Declarative
Acceptance test styles - Imperative vs DeclarativeAcceptance test styles - Imperative vs Declarative
Acceptance test styles - Imperative vs Declarative
 
Continuous Deployment Pipeline with maven
Continuous Deployment Pipeline with mavenContinuous Deployment Pipeline with maven
Continuous Deployment Pipeline with maven
 
Feature branching a release day painkiller
Feature branching   a release day painkillerFeature branching   a release day painkiller
Feature branching a release day painkiller
 

Recently uploaded

BITCOIN HEIST RANSOMEWARE ATTACK PREDICTION
BITCOIN HEIST RANSOMEWARE ATTACK PREDICTIONBITCOIN HEIST RANSOMEWARE ATTACK PREDICTION
BITCOIN HEIST RANSOMEWARE ATTACK PREDICTION
ssuser2b426d1
 
Responsibilities of Fleet Managers and How TrackoBit Can Assist.pdf
Responsibilities of Fleet Managers and How TrackoBit Can Assist.pdfResponsibilities of Fleet Managers and How TrackoBit Can Assist.pdf
Responsibilities of Fleet Managers and How TrackoBit Can Assist.pdf
Trackobit
 
ThaiPy meetup - Indexes and Django
ThaiPy meetup - Indexes and DjangoThaiPy meetup - Indexes and Django
ThaiPy meetup - Indexes and Django
akshesh doshi
 
Software development... for all? (keynote at ICSOFT'2024)
Software development... for all? (keynote at ICSOFT'2024)Software development... for all? (keynote at ICSOFT'2024)
Software development... for all? (keynote at ICSOFT'2024)
miso_uam
 
Folding Cheat Sheet #7 - seventh in a series
Folding Cheat Sheet #7 - seventh in a seriesFolding Cheat Sheet #7 - seventh in a series
Folding Cheat Sheet #7 - seventh in a series
Philip Schwarz
 
Independence Day Hasn’t Always Been a U.S. Holiday.pdf
Independence Day Hasn’t Always Been a U.S. Holiday.pdfIndependence Day Hasn’t Always Been a U.S. Holiday.pdf
Independence Day Hasn’t Always Been a U.S. Holiday.pdf
Livetecs LLC
 
dachnug51 - All you ever wanted to know about domino licensing.pdf
dachnug51 - All you ever wanted to know about domino licensing.pdfdachnug51 - All you ever wanted to know about domino licensing.pdf
dachnug51 - All you ever wanted to know about domino licensing.pdf
DNUG e.V.
 
Leading Project Management Tool Taskruop.pptx
Leading Project Management Tool Taskruop.pptxLeading Project Management Tool Taskruop.pptx
Leading Project Management Tool Taskruop.pptx
taskroupseo
 
introduction of Ansys software and basic and advance knowledge of modelling s...
introduction of Ansys software and basic and advance knowledge of modelling s...introduction of Ansys software and basic and advance knowledge of modelling s...
introduction of Ansys software and basic and advance knowledge of modelling s...
sachin chaurasia
 
Top 10 Tips To Get Google AdSense For Your Website
Top 10 Tips To Get Google AdSense For Your WebsiteTop 10 Tips To Get Google AdSense For Your Website
Top 10 Tips To Get Google AdSense For Your Website
e-Definers Technology
 
MVP Mobile Application - Codearrest.pptx
MVP Mobile Application - Codearrest.pptxMVP Mobile Application - Codearrest.pptx
MVP Mobile Application - Codearrest.pptx
Mitchell Marsh
 
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...
karim wahed
 
dachnug51 - HCL Sametime 12 as a Software Appliance.pdf
dachnug51 - HCL Sametime 12 as a Software Appliance.pdfdachnug51 - HCL Sametime 12 as a Software Appliance.pdf
dachnug51 - HCL Sametime 12 as a Software Appliance.pdf
DNUG e.V.
 
What is OCR Technology and How to Extract Text from Any Image for Free
What is OCR Technology and How to Extract Text from Any Image for FreeWhat is OCR Technology and How to Extract Text from Any Image for Free
What is OCR Technology and How to Extract Text from Any Image for Free
TwisterTools
 
React Native vs Flutter - SSTech System
React Native vs Flutter  - SSTech SystemReact Native vs Flutter  - SSTech System
React Native vs Flutter - SSTech System
SSTech System
 
Addressing the Top 9 User Pain Points with Visual Design Elements.pptx
Addressing the Top 9 User Pain Points with Visual Design Elements.pptxAddressing the Top 9 User Pain Points with Visual Design Elements.pptx
Addressing the Top 9 User Pain Points with Visual Design Elements.pptx
Sparity1
 
Break data silos with real-time connectivity using Confluent Cloud Connectors
Break data silos with real-time connectivity using Confluent Cloud ConnectorsBreak data silos with real-time connectivity using Confluent Cloud Connectors
Break data silos with real-time connectivity using Confluent Cloud Connectors
confluent
 
React vs Next js: Which is Better for Web Development? - Semiosis Software Pr...
React vs Next js: Which is Better for Web Development? - Semiosis Software Pr...React vs Next js: Which is Better for Web Development? - Semiosis Software Pr...
React vs Next js: Which is Better for Web Development? - Semiosis Software Pr...
Semiosis Software Private Limited
 
Cisco Live Announcements: New ThousandEyes Release Highlights - July 2024
Cisco Live Announcements: New ThousandEyes Release Highlights - July 2024Cisco Live Announcements: New ThousandEyes Release Highlights - July 2024
Cisco Live Announcements: New ThousandEyes Release Highlights - July 2024
ThousandEyes
 
Ported to Cloud with Wing_ Blue ZnZone app from _Hexagonal Architecture Expla...
Ported to Cloud with Wing_ Blue ZnZone app from _Hexagonal Architecture Expla...Ported to Cloud with Wing_ Blue ZnZone app from _Hexagonal Architecture Expla...
Ported to Cloud with Wing_ Blue ZnZone app from _Hexagonal Architecture Expla...
Asher Sterkin
 

Recently uploaded (20)

BITCOIN HEIST RANSOMEWARE ATTACK PREDICTION
BITCOIN HEIST RANSOMEWARE ATTACK PREDICTIONBITCOIN HEIST RANSOMEWARE ATTACK PREDICTION
BITCOIN HEIST RANSOMEWARE ATTACK PREDICTION
 
Responsibilities of Fleet Managers and How TrackoBit Can Assist.pdf
Responsibilities of Fleet Managers and How TrackoBit Can Assist.pdfResponsibilities of Fleet Managers and How TrackoBit Can Assist.pdf
Responsibilities of Fleet Managers and How TrackoBit Can Assist.pdf
 
ThaiPy meetup - Indexes and Django
ThaiPy meetup - Indexes and DjangoThaiPy meetup - Indexes and Django
ThaiPy meetup - Indexes and Django
 
Software development... for all? (keynote at ICSOFT'2024)
Software development... for all? (keynote at ICSOFT'2024)Software development... for all? (keynote at ICSOFT'2024)
Software development... for all? (keynote at ICSOFT'2024)
 
Folding Cheat Sheet #7 - seventh in a series
Folding Cheat Sheet #7 - seventh in a seriesFolding Cheat Sheet #7 - seventh in a series
Folding Cheat Sheet #7 - seventh in a series
 
Independence Day Hasn’t Always Been a U.S. Holiday.pdf
Independence Day Hasn’t Always Been a U.S. Holiday.pdfIndependence Day Hasn’t Always Been a U.S. Holiday.pdf
Independence Day Hasn’t Always Been a U.S. Holiday.pdf
 
dachnug51 - All you ever wanted to know about domino licensing.pdf
dachnug51 - All you ever wanted to know about domino licensing.pdfdachnug51 - All you ever wanted to know about domino licensing.pdf
dachnug51 - All you ever wanted to know about domino licensing.pdf
 
Leading Project Management Tool Taskruop.pptx
Leading Project Management Tool Taskruop.pptxLeading Project Management Tool Taskruop.pptx
Leading Project Management Tool Taskruop.pptx
 
introduction of Ansys software and basic and advance knowledge of modelling s...
introduction of Ansys software and basic and advance knowledge of modelling s...introduction of Ansys software and basic and advance knowledge of modelling s...
introduction of Ansys software and basic and advance knowledge of modelling s...
 
Top 10 Tips To Get Google AdSense For Your Website
Top 10 Tips To Get Google AdSense For Your WebsiteTop 10 Tips To Get Google AdSense For Your Website
Top 10 Tips To Get Google AdSense For Your Website
 
MVP Mobile Application - Codearrest.pptx
MVP Mobile Application - Codearrest.pptxMVP Mobile Application - Codearrest.pptx
MVP Mobile Application - Codearrest.pptx
 
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...
 
dachnug51 - HCL Sametime 12 as a Software Appliance.pdf
dachnug51 - HCL Sametime 12 as a Software Appliance.pdfdachnug51 - HCL Sametime 12 as a Software Appliance.pdf
dachnug51 - HCL Sametime 12 as a Software Appliance.pdf
 
What is OCR Technology and How to Extract Text from Any Image for Free
What is OCR Technology and How to Extract Text from Any Image for FreeWhat is OCR Technology and How to Extract Text from Any Image for Free
What is OCR Technology and How to Extract Text from Any Image for Free
 
React Native vs Flutter - SSTech System
React Native vs Flutter  - SSTech SystemReact Native vs Flutter  - SSTech System
React Native vs Flutter - SSTech System
 
Addressing the Top 9 User Pain Points with Visual Design Elements.pptx
Addressing the Top 9 User Pain Points with Visual Design Elements.pptxAddressing the Top 9 User Pain Points with Visual Design Elements.pptx
Addressing the Top 9 User Pain Points with Visual Design Elements.pptx
 
Break data silos with real-time connectivity using Confluent Cloud Connectors
Break data silos with real-time connectivity using Confluent Cloud ConnectorsBreak data silos with real-time connectivity using Confluent Cloud Connectors
Break data silos with real-time connectivity using Confluent Cloud Connectors
 
React vs Next js: Which is Better for Web Development? - Semiosis Software Pr...
React vs Next js: Which is Better for Web Development? - Semiosis Software Pr...React vs Next js: Which is Better for Web Development? - Semiosis Software Pr...
React vs Next js: Which is Better for Web Development? - Semiosis Software Pr...
 
Cisco Live Announcements: New ThousandEyes Release Highlights - July 2024
Cisco Live Announcements: New ThousandEyes Release Highlights - July 2024Cisco Live Announcements: New ThousandEyes Release Highlights - July 2024
Cisco Live Announcements: New ThousandEyes Release Highlights - July 2024
 
Ported to Cloud with Wing_ Blue ZnZone app from _Hexagonal Architecture Expla...
Ported to Cloud with Wing_ Blue ZnZone app from _Hexagonal Architecture Expla...Ported to Cloud with Wing_ Blue ZnZone app from _Hexagonal Architecture Expla...
Ported to Cloud with Wing_ Blue ZnZone app from _Hexagonal Architecture Expla...
 

Baking in the cloud with packer and puppet

  • 1. Baking in the Cloud with Packer and Puppet Alan Parkinson @alan_parkinso n
  • 3. Large jumps in demand, we have to scale fast Load Balancer Behave Pro App
  • 5. Reliability Many single points of failure Require Puppet Master redundancy and datacentre replication Mirror dependant software repositories
  • 6. Provisioning Latency… 9 minutes for puppet to prepare a app server Total 15 minute response time to a scaling request
  • 7. Baking images like Produce a preconfigured machine image that is rolled out to the autoscaling groups
  • 8. Packer is a tool for creating identical machine images for multiple platforms from a single source configuration PACKER.IO A HASHICORP PROJECT
  • 9. PACKER.IO A HASHICORP PROJECT packer.json { "variables": { "aws_access_key": "", "aws_secret_key": "" }, "builders": [{ "type": "amazon-ebs", "access_key": "{{user `aws_access_key`}}", "secret_key": "{{user `aws_secret_key`}}", "region": "us-east-1", "source_ami": "ami-9eaa1cf6", "instance_type": "t2.micro", "ssh_username": "ubuntu", "ami_name": "packer-example {{timestamp}}" }] }
  • 10. $ packer build -var 'aws_access_key=YOUR ACCESS KEY' -var 'aws_secret_key=YOUR SECRET KEY' example.json ==> amazon-ebs: amazon-ebs output will be in this color. ==> amazon-ebs: Creating temporary keypair for this instance... ==> amazon-ebs: Creating temporary security group for this instance... ==> amazon-ebs: Authorizing SSH access on the temporary security group... ==> amazon-ebs: Launching a source AWS instance... ==> amazon-ebs: Waiting for instance to become ready... ==> amazon-ebs: Connecting to the instance via SSH... ==> amazon-ebs: Stopping the source instance... ==> amazon-ebs: Waiting for the instance to stop... ==> amazon-ebs: Creating the AMI: packer-example 1371856345 ==> amazon-ebs: AMI: ami-19601070 ==> amazon-ebs: Waiting for AMI to become ready... ==> amazon-ebs: Terminating the source AWS instance... ==> amazon-ebs: Deleting temporary security group... ==> amazon-ebs: Deleting temporary keypair... ==> amazon-ebs: Build finished. ==> Builds finished. The artifacts of successful builds are: --> amazon-ebs: AMIs were created: us-east-1: ami-19601070
  • 11. Add some provisioners { "variables": {…}, "builders": […], "provisioners": [ { "type": "shell", "script": "../common/install-puppet.sh" }, … { "inline": [ "sudo apt-get purge --yes puppet", "sudo apt-get autoremove --yes" ], "type": "shell" } ] }
  • 12. No need for a Puppet Master "provisioners": [ … { "type": "puppet-masterless", "hiera_config_path": "../puppet/hiera.yaml", "manifest_file": "../puppet/manifests/default.pp", "module_paths": [ "../puppet/modules" ] }, … ] Manifests, modules and hiera data can all be stored in git and git submodules
  • 13. How do we protect sensitive configuration data?
  • 14. hiera-eyaml backend for Hiera that provides per-value encryption of sensitive data within yaml files --- duo-security-skey: ENC[PKCS7,MIIBmQYJKoZIh……Anc=] behave_pro: logentries_api_key: ENC[PKCS7,MIIBmQYJKoZIh……uW8=] application_secret: ENC[PKCS7, MIIBmQYJKoZIh……FRg==] common.eyaml
  • 15. hiera.yaml --- :backends: - eyaml - yaml :hierarchy: - "%{environment}" - common :yaml: :datadir: '/tmp/hieradata' :eyaml: :datadir: '/tmp/hieradata' :pkcs7_private_key: /tmp/hierakeys/private_key.pkcs7.pem :pkcs7_public_key: /tmp/hierakeys/public_key.pkcs7.pem Note: Use a temporary folders or data will be baked into the final image
  • 16. Basic asymmetric encryption (PKCS#7) Private key decrypts data Puppet Master or agent only needs this at runtime $ eyaml encrypt -s 'hello there' Public key encrypts data Safe to distribute to developers and ops engineers Git diff allows peer review without decrypting values
  • 17. The eyaml keys are stored in a private S3 bucket with access controlled by a IAM Policy Distributing the keys to the Bakery Use a IAM Role in Packer to access the S3 bucket "builders": [{ "type": "amazon-ebs", … "iam_instance_profile" : "puppet-provisioner", … }]
  • 18. IAM Policy { "Version": "2012-10-17", "Statement": [ { "Sid": "Stmt1425244502000", "Effect": "Allow", "Action": [ "s3:GetObject" ], "Resource": [ "arn:aws:s3:::puppet.behave.pro/*" ] } ] }
  • 19. install-heira-key.sh Download the keys to the EC2 Instance sudo apt-get install --yes python-pip sudo pip install s3cmd s3cmd get s3://puppet.behave.pro/private_key.pkcs7.pem /tmp/hierakeys/private_key.pkcs7.pem s3cmd get s3://puppet.behave.pro/public_key.pkcs7.pem /tmp/hierakeys/public_key.pkcs7.pem "provisioners": [ { "type": "shell", "script": "../common/install-hiera-key.sh" }, ] packer.json
  • 20. Summary If scaling fast or reliably are important, bake images Git makes a great alternative to Puppet Master when baking Secure data with hiera-eyaml https://github.com/TomPoulton/hiera-eyaml