Hot tutorials

Kanagaraj.Manickam@huawei.com
huangtianhua@huawei.com
OpenStack Summit, Austin, TX
April 27th 2016
Heat Orchestration Template
(HOT) Learning

About Authors
Kanagaraj Manickam
Sr. System Architect @ Huawei, Bangalore, India.
• Core-reviewer @ Heat, OpenStack.
• Active participant @ Open-O an Global SDN & NFV-Orchestrator .
• Expert in Data-center Server, Storage management and automation.
• Founder of namos – OpenStack Manager)
Huang Tianhua
Software engineer @ Huawei, Xi’an, China
Core-reviewer @ Heat, Openstack.
Having 3+ years of experience in Openstack Cloud.
2

Start to prepare setup
• Download https://goo.gl/ne29yO
• Install devstack with https://github.com/mkr1481/hot-
tutorials/blob/master/localrc (devstack with heat, ceilometer, aodh)
• Upload the image to glace
• (Ctrl+C from https://etherpad.openstack.org/p/austin-heat)
3

Agenda
• Heat
• HOT schematics
• Validation & Pre-view
• Heat features
– Auto-scaling
– Software deployment
4

Heat
• An orchestration service to create and manage the
lifecycle of OpenStack cloud application i.e,
anything-as-a-service (XaaS)
• Uses JSON/YAML based parameterized HOT
template to model the customizable cloud application made of
software, configurations and infrastructure.
• Represent each provisioned cloud application as
Stack
• Similar to AWS cloud-formation (CFN) service
5

HOT schematic
• Sections
• Parameter constraints
• Pseudo parameters
• Intrinsic functions
• Resource Type
• Environment
• Provider template
6

HOT schematic : Parameter Constraints
length: (String)
{ min: <lower limit>, max: <upper limit> }
range: (number)
{ min: <lower limit>, max: <upper limit> }
allowed_values:
[ <value>, <value>, ... ]
allowed_pattern:
<regular expression>
custom_constraint: <constraint name>
8

HOT schematic : Pseudo parameters
Available by default
- OS::Stack_id : Id of Stack , the template belongs to
- OS::stack_name: Name of Stack , the template belongs to
-OS::project_id: User project Id
How to get the value of these parameters?
-Use ‘openstack stack show <stack name>’ and refer from the
parameters in the output
9

HOT schematic : Intrinsic functions
10

get_file
files: {
“file:///path/to/userdata.sh”:
<file content of userdata.sh>
}
resources:
my_instance:
type: OS::Nova::Server
properties:
user_data: {get_file: userdata.sh }
user_data: <file content of userdata.sh>
NOTE:
- OpenStack heat CLI or REST API provides the ‘files’ map used here
- file path could either in absolute file path or url format
Data Reference Intrinsic functions
11

Data Reference Intrinsic functions
get_param
instance_type : "m1.tiny",
server_data : {
"metadata": { "foo": "bar"},
"keys": ["a_key","other_key"]}
}
parameters:
instance_type:
type: string
server_data:
type: json
resources:
my_instance:
properties:
flavor: { get_param: instance_type}
metadata: { get_param: [ server_data, metadata ] }
key_name: { get_param: [ server_data, keys, 0 ] }
flavor: “m1.tiny “
metadata: {"foo": "bar"}
key_name: "a_key"
12

Data Manipulation Intrinsic functions
str_split: [',', 'string,to,split', 0] ‘string’
list_join: [‘- ', [‘a', b‘, c'], [‘d’, ‘e’]] a-b-c-d-e
str_replace:
template: http://host:port/v1
params:
host: 192.168.10.1
port: 8088
http://192.168.10.1:8088/v1
map_merge: [{'k1': 'v1', 'k2': 'v2'}, {'k1': 'v2'}] {'k1': 'v2', 'k2': 'v2‘}
Data Manipulation functions
repeat:
for_each:
<%port%>: [80, 443]
<%protocol%>: [‘udp’, ‘tcp’]
template:
protocol: <%protocol%>
port_range_min: <%port%>
protocol:udp port_range_min:80
protocol:tcp port_range_min:80
protocol:udb port_range_min:443
protocol:tcp port_range_min:80
Utility functions
digest: ['sha512‘, ‘password’]
(supports md5, sha1, sha224, sha256, Sha384 also)
B109f3bbbc244eb82441917ed06d618b9008dd0
9b3befd1b5e07394c706……
13

get_resource
Dependency reference Intrinsic functions
resources:
instance_port:
type: OS::Neutron::Port
properties:
...
instance:
properties:
networks:
port: { get_resource: instance_port }
resources:
server:
properties:
…
member:
type: OS::Neutron::PoolMember
properties:
address: {get_attr: [server, first_address]}
outputs:
ip:
value: {get_attr: [server, networks, private, 0] }
get_attr
resource_fecade
Child.yaml
------------------
resources:
instance:
properties:
…
metadata: {resource_facade: metadata}
update_policy: {resource_facade: update_policy}
deletion_policy: {resource_facade: deletion_policy}
depends_on(Not an intrinsic function)
resources:
instance1:
depends_on: [instance2, instance3]
instance2:
instance3:
parent.yaml
------------------
resources:
parent:
type: child.yaml
metadata: …
update_policy: …
deletion_policy: …
14

Resource type
• Helps to model OpenStack elements such as nova instance, cinder volume, neutron network, etc in
template
• Represented by namespace Cloud Provider :: Service :: Resource (Examples: OS::Nova::Server,
OS::Cinder::Volume)
• Supported with Life cycle operations
(inline with stack life cycle operations)
– Create / Update / Delete
– Snapshot / restore
– Abandon / adopt
– Check
• Has Typed Properties with built-in constraints
– Types: Integer, String, Number, Boolean, Map, List
– Updatable (y/n, default:yes)
– Required (y/n, default: no)
– Metadata
– deletion_policy
– update_policy
• Provides Typed Output
– Types: Integer, String, Boolean, Map, List
– Special one ‘show’
– get_attr() helps
OS::Nova::Server
properties:
image:
type:String
constraints: glance.image
attributes:
first_address:
type: String
description: first IP
resources:
sample:
image: cirros
outputs:
ip:
value: {get_attr: [sample_doamin, serial]}15

Environment
• Global (/etc/heat/heat.conf:environment_dir)
• User env (heat stack-create –e env.yaml)
• Parameters
• resource_registry
– Resource mapping
– Override resource
– restricted_actions
– hooks
• event_sinks
parameters
flavor: m1.large
image: cirros
resource_registry
“OS::Quantaum*”: “OS::neutron*”
"AWS::EC2::Instance": file:///path/to/my_instance.yaml
“db_server”
"OS::DBInstance": file:///path/to/db.yaml
restricted_actions : [update, replace]
hooks : [ pre-create, pre-update, pre-delete,
post-create, post-update, post-delete]
event_sinks
-type: zaqar-queue
-target: myqueue
-ttl: 1200
*
16

Provider template
heat_template_version: 2015-04-30
resources:
my_server:
type: my_nova.yaml
properties:
key_name: my_key
resource_registry:
"OS::Nova::Server": my_nova.yaml
resources:
my_server:
properties:
key_name: my_key
outputs:
test_out:
value: {get_attr: [ my_server, resource.server, first_address]}
OS::stack_id:
value: {get_resource: my_server}
parameters:
key_name:
type: string
description: Name of a KeyPair
resources:
server:
properties:
key_name: {get_param: key_name}
flavor: m1.small
image: ubuntu-trusty-x86_64
my_nova.yaml•Bifurcate complex template into easy-manageable-small template
• New Resourc e type
• Direct reference
• using environment
• Access the provider template’s resource attributes
• Make provider template based resources as transparent
17

Validation & Preview
Validation
heat template-validate –template-file <template file>
openstack orchestration template validate -t <template file>
- Produces list of input parameters for this template.
Pre-view
heat stack-preview –template-file <template file> <stack_name>
openstack stack create -t <template file> --dry-run <stack_name>
- Produces the pre-view of all resources in the template
18

Heat features: by means of HOT
• Using Resource Type
– auto-scaling
– software configuration & deployment
– resource grouping
– remote stack
– none resource
– random string
– wait condition
• Using Environment
– hook/breakpoint
19

Auto-scaling
• Scale up/down compute/storage/network
capabilities of cloud application based on the
pre-defined threshold level, which is
monitored continuously
• For defining threshold and monitoring, user
could choose either Ceilometer or Monasca
20

Auto-scaling: based on Monasca
OS::Heat::AutoScalingGroup
 Properties
 resource (scaling element)
 desired_capacity (initial count)
 max_size
 min_szie
 Outputs
 current_size
 outputs/output_list
• Scale Group
• Scaling policy
• Alarm Notification
• Alarm definition
• Metadata-Dimension
OS::Heat::AutoScalingGroupOS::Heat::ScalingPolicy
 Properties
 auto_scaling_group_id
 cooldown (timing-window for scaling)
 adjustement_type
 Scaling_adjustment (+/- 1, n, x%)
 Outputs
 alarm_url ( CFN signal type)
 signal_url (HOT signal type)
OS::Heat::ScalingPolicy
OS::Monasca::Notification
 Properties
 type (webhook, email)
 address
signal / alarm url
OS::Monasca::Notification
signal / alarm url
OS::Monasca::AlarmDefinition
 Properties
 name
 expression
 match_by=scale_group
 alarm_actions
resource:
properties:
metadata: {"scale_group": <stack-id>}
expression: avg(vm.cpu.utilization_perc{scale_group=<stack-id>}) > 90
Monasca alarm definition wiki
Presented an demo @ OpenStack Summit, Tokyo, JP
21

Auto-scaling: based on Ceilometer
 Properties
 resource (scaling element)
 desired_capacity (initial count)
 max_size
 min_szie
 Outputs
 current_size
 outputs/output_list
• Scale Group
• Scaling policy
• Alarm
• Metadata-Dimension
OS::Heat::AutoScalingGroupOS::Heat::ScalingPolicy
 Properties
 auto_scaling_group_id
 cooldown (timing-window for scaling)
 adjustement_type
 Scaling_adjustment (+/- 1, n, x%)
 Outputs
 alarm_url ( CFN signal type)
 signal_url (HOT signal type)
OS::Heat::ScalingPolicy
signal / alarm url
OS::Ceilometer::Alarm
signal / alarm url
OS::Monasca::Alarm
 Properties
 meter_name
 statistic
 period
 evalutation_periods
 threshold
 Comparision_operator
 alarm_actions
resource:
properties:
metadata: {"metering.stack": {get_param: "OS::stack_id"}}
OS::Monasca::Alarm
Maticing_metadata: {'metadata.user_metadata.stack': {get_param:
"OS::stack_id"}}
Ceilometer alarm definition wiki
22

Auto-scaling :work-flow
Heat
Monasca/
Ceilometer
VMVMVM
resources:
group:
type: OS::Heat::AutoScalingGroup
scaleup_policy:
type: OS::Heat::ScalingPolicy
notification:
type: OS::Monasca::Notification
cpu_alarm_high:
type: OS::Monasca::AlarmDefinition
1
3
4
5
6
2 7
1. Call Heat with a cloud application HOT template
2. Heat creates cloud application stack with scale group and
scale policy
3. Heat creates alarm definitions and scaling notifications in
Ceilometer/Monasca
4. Ceilometer/Monasca starts to monitor the scale group
elements
[[CLOUD APPLICATION AUTO-SCALING SETUP IS DONE]]
5. When Scale group reaches the alarm threshold,
Ceilometer/Monasca detects and generates alarm
6. Ceilometer/Monasca signals cloud application stack in heat
7. Heat either scale up or scale down based on the signal
(here, increment scale group elements by 1)
5-7 runs for ever ! (auto-scale)
5
23

• Custom image building
• User-data boot scripts and cloud-init
• Software deployment resources
24
Software Config & Deployment

25
Custom image building
• Boot speed
• Boot reliability
• Test verification
• Configuration dependencies
Tool: diskimage-builder

26
User-data boot scripts and cloud-init
resources:
my_instance:
properties:
user data: |
#!/bin/bash
echo “Running boot script”
# ...
user_data_format: RAW
user_data:
str_replace:
template: {get_file:the_server_boot.sh}
params:
$FOO: {get_param: foo}
user_data:
str_replace:
template: |
#!/bin/bash
echo “Script with $FOO"
# ...
params:
$FOO: {get_param: foo}
#!
#cloud-config
#part-handler
#cloud-boothook
……
HEAT_CFNTOOLS
RAW
SOFTWARE_CONFIG
my_server:
properties:
user_data_format: RAW
user_data: {get_resource: xxx}
boot_script:
type: OS::Heat::SoftwareConfig
properties:
group: ungrouped
config: |
#!/bin/bash
echo "Running boot script“
# ...
OS::Heat::SoftwareConfig
OS::Heat::CloudConfig
boot_config:
type: OS::Heat::CloudConfig
properties:
cloud_config:
write_files:
- path: /tmp/file
content: {get_param: file_content}
OS::Heat::MultipartMime
server_init:
type: OS::Heat::MultipartMime
properties:
parts:
- config: {get_resource:boot_config}
- config: {get_resource: boot_script}
- config: {get_param: other_config}

Software deployment resources
deployment:
type: OS::Heat::SoftwareDeployment
properties:
config:
get_resource: config
server:
get_resource: server
input_values:
foo: fooooo
bar: baaaaa
actions: [CREATE, UPDATE, DELETE…]
signal_transport: HEAT_SIGNAL/
ZAQAR_SIGNAL/
TEMP_URL_SIGNAL…
OS::Heat::SoftwareDeployment
boot_script:
properties:
group: ungrouped
config: |
#!/bin/bash
echo "Running boot script“
# ...
OS::Heat::SoftwareConfig
OS::Nova::Server
my_server:
properties:
user_data_format: SOFTWARE_CONFIG

Software deployment – build custom image
pip install git+https://git.openstack.org/openstack/diskimage-builder
git clone https://git.openstack.org/openstack/tripleo-image-elements
git clone https://git.openstack.org/openstack/heat-templates
export ELEMENTS_PATH=tripleo-image-elements/elements:heat-
templates/hot/software-config/elements
disk-image-create vm
fedora selinux-permissive
heat-config
os-collect-config
os-refresh-config
os-apply-config
heat-config-cfn-init
heat-config-puppet
heat-config-script
-o fedora-software-config.qcow2

29
Software Deployment :work-flow
Heat
resources:
server:
config:
deployment:
1
1. Call Heat with a cloud application HOT template
2. Heat creates cloud application stack with
software config, software deployment and server
3. VM is booting, cloud-init consumes the user-data
4. os-collect-config starts to pull the configuration
metadata to vm
5. Corresponding hooks to perform configuration
6. Heat config tools signals software deployment
resource to tell the result
7. Call Heat update-stack to update the
configuration
8. Step 4-6 repeats
2
os-collect-config
os-apply-config
os-refresh-config
Hooks…
VM
4
6
resources:
server:
config_update:
deployment:
5

THANK YOU
Useful references:
http://docs.openstack.org/developer/heat/template_guide/index.html
http://docs.openstack.org/developer/heat/template_guide/software_deployment.htm
https://github.com/mkr1481/hot-tutorials
https://github.com/openstack/heat-templates
Kanagaraj Manickam
@mrkanag
IRC: KanagarajM
Huang Tianhua
IRC: tiantian
30

Hot tutorials

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to Hot tutorials

Similar to Hot tutorials (20)

Recently uploaded

Recently uploaded (20)

Hot tutorials

Editor's Notes