A Continuous Delivery Safety Net for Databases

© 2013 IBM Corporation
A Continuous Delivery
Safety Net for Databases

2 © 2015 IBM Corporation
Speaking Today
• Eric is a DevOps Evangelist with IBM where
he helps customers get the most out of their
build, deploy and release processes.
• Today he works with customers and industry
leaders to find the best ways of adopting
continuous delivery and DevOps.
• Eric is the author of Application Deploy and
Release for Dummies.
• @EricMinick
Eric Minick
• Yaniv is the Co-Founder and CTO of
DBmaestro, the leading provider of DevOps
for Database solutions which enable control
of database development and deployment.
• Yaniv has spent the last years raising
awareness about the challenges around
database development and deployment,
and how to support database Continuous
Delivery.
• @DBmaestro
Yaniv
Yehuda

Why Waterfall Fails

Continuous Delivery helps us learn faster
Automated flow from
Build to “ready for prod”
Push button release to
production
Lots of feedback
Emphasis on always
shippable
Themes

“Classic” CD
build
dev
test
system
test
UAT sign-off staging prod

The Build Pipeline
build
dev
test
system
test
for (env in testEnvironments) {
deploy( build, env );
runTests (env.testType, env);
}

Composite apps: many tiers & components

An Application might have
dozens of components

An Application might have
dozens of components
Delivered by Different Teams

Which build does “Login” test?

Build pipelines in composite applications
build
dev
test
system
test
build
dev
test
system
test
build
dev
test
system
test
build
dev
test
system
test
build
dev
test
system
test
build
dev
test
system
test
build
dev
test
system
test
build
dev
test
system
test
build
dev
test
system
test
build
dev
test
system
test
build
dev
test
system
test
build
dev
test
system
test

Combine Versioned Stuff into Versioned App
• Manage a set of tested component versions as a single unit
• Guarantee the validated versions of related components are
deployed together
• Ease of promoting applications between environments
1
1
2
3
UAT Stage PROD
Promote Promote
Web
Mid. Code
Mid. Config
DB
Snapshot

Expanding “All the Stuff”
 Systems of Record
– z/OS
– iSeries
 Mobile
– iOS / Android
– Mobile Testing
 Infrastructure
– Treating environment as part of Application

Combine Versioned Stuff into Versioned App
• Manage a set of tested component versions as a single unit
• Guarantee the validated versions of related components are
deployed together
• Ease of promoting applications between environments
1
1
2
3
UAT Stage PROD
Promote Promote
Web
Mid. Code
Mid. Config
DB
Snapshot

• Old adage but true
• The database is often neglected and therefore can become the weakest
link
• Manual processes
• Database/Code Silos exist…
• Don’t always communicate effectively
• Need to follow same procedures & best practices
• Essential from a compliance and business point of view
• Should be the strongest link
The Weakest Link In a Chain ???

 Recently Conducted DBmaestro Survey
– Over 150 Companies
– Over 200 participants
 Why is that happening?
Continuous Delivery Moving Ahead!

Why Continuous Delivery?

 Why?
What about the database?

 Mistrust…
So why not move forward?

Mistrust…

What is the problem?
 Root Causes for issues:
– Challenging manual version control process
– Static deployments code overrides
– Dynamic deployments unaware of version control
– No release automation red-flags – don’t know when to
stop the line…

Mistrust in version control…
Version Control Process
(file based)
Development Process
Check-Out
Script
Modify Script
Get updated
Script from
DB
Check-In
Script
Compile
Script
in DB
Debug Script
in DB

(file based)
Development Process
Check-Out
Script
Modify Script
Get updated
Script from
DB
Check-In
Script
Compile
Script
in DB
Debug Script
in DB
A
A’

(file based)
Development Process
Check-Out
Script
Modify Script
Get updated
Script from
DB
Check-In
Script
Compile
Script
in DB
Debug Script
in DB
?
?
?
?
A
A’
No single source of truth…

Scripts & Version Control
Challenges…
– Code-overrides
– Working on the wrong revisions
– Scripts do not always find their way to the version control solution
– Out of process updates go unnoticed
– Hard to locate outdated update scripts
Playing safe? what we really need:
– The actual code of the object
– The upgrade script
– A roll-back script
Scripts
– Hard to test in their entirely (holistically)
– Hard to test due to colliding dependencies
– Need to run in a specific order…
– Much harder to deal with project scope changes

1.11.1.11.11.21.31.41.51.61.7
Mistrust in static scripting automation…
Int QA Stage Prod
Database Deploy Script
Environment
Re-Base (due to
defects)
Dev
Dev
Dev
Model
1.1 1.2
1.2 1.3
1.3 1.4
1.11.11.41.7 ? 1.1.1

1.11.1.11.11.21.31.41.51.61.7
Int QA Stage Prod
Environment
Re-Base (due to
defects)
Dev
Dev
Dev
Model
1.1 1.2
1.2 1.3
1.3 1.4
1.11.11.41.7
1.1 1.2
1.2 1.3
1.3 1.4
? 1.1.1

1.11.1.11.11.21.31.41.51.61.7
Int QA Stage Prod
Environment
Re-Base (due to
defects)
Dev
Dev
Dev
Model
1.1 1.2
1.2 1.3
1.3 1.4
1.4 1.5
1.5 1.6
1.6 1.7
1.11.11.41.7
1.1 1.2
1.2 1.3
1.3 1.4
1.4 1.5
1.5 1.6
1.6 1.7
? 1.1.1

1.11.1.11.11.21.31.41.51.61.7
Int QA Stage Prod
Environment
Re-Base (due to
defects)
Dev
Dev
Dev
Model
1.1 1.2
1.2 1.3
1.3 1.4
1.4 1.5
1.5 1.6
1.6 1.7
1.11.11.41.7
1.1 1.2
1.2 1.3
1.3 1.4
1.4 1.5
1.5 1.6
1.6 1.7
Out of
Process
Change
? 1.1.1

X
1.11.1.11.11.21.31.41.51.61.7
Int QA Stage Prod
Environment
Re-Base (due to
defects)
Dev
Dev
Dev
Model
1.1 1.2
1.2 1.3
1.3 1.4
1.4 1.5
1.5 1.6
1.6 1.7
1.11.11.41.7
1.1 1.2
1.2 1.3
1.3 1.4
1.4 1.5
1.5 1.6
1.6 1.7
1.1 1.2
1.2 1.3
1.3 1.4
1.4 1.5
1.5 1.6
1.6 1.7
Out of
Process
Change
X
X
X
X
X
? 1.1.1
X

Scripts are static…
Scripts, unless super sophisticated:
– Unaware of changes made in the target environment
– Time passed from their coding to the time they are run
– Potentially overriding production hot-fixes or work done in parallel by
another team
Content changes are very hard to manage
– Metadata & lookup content does not practically fit into the VC
– In most cases they are simply not managed

Gaining Trust!
Coordinated ProcessTraceability
Start in the Beginning
No Out-of-Process Changes
Impact Analysis
Automation
Task Based Development
Well Defined Processes

Version Control - One Enforced Process

Dealing with challenges…
Integrated Database Version Control process
– Leverage proven version control best practices
• Forcing check in & out for changes
• Labels
• etc..
– No code-overrides
– Always working with the correct revision
– All changes are documented
– Always know who did what, when, why and from where
– No out-of-process changes
– Supporting structure, code and content
No time spent on manual coding of the change scripts

Bonus points – dealing with scope changes
Task based development…
– Correlate each database change with a change request
• Task ID
• Work Item
• Trouble Ticket
• CR
• etc…
…enables task based deployments
– Partial deployments (a feature, a collection of bugs, etc…)
– Scope changes easily synced between code and database

For deployment automation we need:
Repeatability & Safety
Using tools make sense …

Build & Deploy On Demand
1.11.21.31.41.51.61.7
Int QA Stage Prod
Environment
Re-Base (due to
defects)
Dev
Dev
Dev
Model
1.1 1.2
1.2 1.3
1.3 1.4
1.1 1.4
1.1 1.1 1.11.41.7 1.1.11.7 1.1.11.7

1.11.21.31.41.51.61.7
Int QA Stage Prod
Environment
Re-Base (due to
defects)
Dev
Dev
Dev
Model
1.1 1.2
1.2 1.3
1.3 1.4
1.4 1.5
1.5 1.6
1.6 1.7
1.1 1.4
1.4 1.7
1.1 1.1 1.11.41.7 1.1.11.7 1.1.11.7

1.11.21.31.41.51.61.7
Int QA Stage Prod
Environment
Re-Base (due to
defects)
Dev
Dev
Dev
Model
1.1 1.2
1.2 1.3
1.3 1.4
1.4 1.5
1.5 1.6
1.6 1.7
1.1 1.4
1.4 1.7
1.1 1.1 1.11.41.7
Out of
Process
Change
1.1.11.7 1.1.11.7

1.11.21.31.41.51.61.7
Int QA Stage Prod
Environment
Re-Base (due to
defects)
Dev
Dev
Dev
Model
1.1 1.2
1.2 1.3
1.3 1.4
1.4 1.5
1.5 1.6
1.6 1.7
1.1 1.4
1.4 1.7
1.1.1 1.7
1.1 1.1 1.11.41.7
File Based
Version
Control
Out of
Process
Change
1.1.11.7 1.1.11.7

1.11.21.31.41.51.61.7
*
Int QA Stage Prod
Environment
Re-Base (due to
defects)
Dev
Dev
Dev
Model
1.1 1.2
1.2 1.3
1.3 1.4
1.4 1.5
1.5 1.6
1.6 1.7
1.1 1.4
1.4 1.7
1.1.1 1.7
1.1 1.1 1.11.41.7
File Based
Version
Control
Out of
Process
Change
1.1.11.7 1.1.11.7

Using tools
Test cases using compare & sync tools:
An index exists in source (QA) but not in target (Production)
What should we do? Add the index or not?

Compare & Sync tools
Safe to automate?
Sure… (?)

Compare & Sync tools
Safe to automate?
No. Requires manual inspection…

Safe?
Simple, right?
NO! we are going to BREAK production without even
knowing…

Why break production???
A compare & sync tool:
 Is unaware of any changes that occurred before the time it ran
 Has no knowledge of changes that took place at the target environment
 Does not leverage version control for more information
 Unable to deal with conflicts & merges between different teams
 Requires manual inspection
 Requires detailed knowledge regarding each change as part of the
process
Mistrust AGAIN…
So…no automation… as we fear automating problems into
production and a major risk!!!

We need to leverage version control
into deployment decisions…

Safety Net For Deployment Automation
Static script based deployment

Deploying Changes if Needed
Development Baseline
Previous Label /
Production Golden Copy
Production
If we had the index in the baseline =>
we should take it down from production…
(Deploy Change)

Or Protecting Target Environment…
Development Baseline
Previous Label /
Production Golden Copy
Production
BUT… If no index in baseline =>
we should protect the NEW index on production!!!
(Protect Target)

Deployment Automation
And Merge!!!

Conflict Resolving – Database Code

Impact Analysis – Code & Content

Impact Analysis! not Damage Control…

Safety Net For Deployment Automation
Database Safe Deployment Automation:
 Leverages version control (baselines & previous revisions)
 Has a flexible scope (deploy multi schema to single task or work item)
 Can be run as a batch process (repeatable & consistent)
 Integrates to ALM (labels, CRs, Continuous Integration & Delivery)
 Deals with conflicts & merges to match code agility
Can raise red flags to stop the line…
if requires human intervention

What is DBmaestro TeamWork?
 Database Enforced Change Management solution
+ Database version control
+ Enforce best practices
+ Plugs into the ALM (change request, tickets & work items)
+ Database merge & change impact analysis
+ Know who can do what, where, when & why
 DevOps Solution for databases
+ Baseline aware deployment automation, rollback &
recovery
+ Reduce database deployment issues
+ Plugs into release management & Continuous Delivery

IBM UrbanCode Deploy
Visibility and automated control of your application deployment process
Mobile App
Team
Middleware
team
Backend Systems of
Record
• Manage application
components and versions
• Manage configurations across
all environments
• Offer secure ‘self-service’
capabilities
• Increase transparency
• Ensure governance and
compliancy

www.ibm.com/devops

© Copyright IBM Corporation 2014. All rights reserved.
The information contained in these materials is provided for
informational purposes only, and is provided AS IS without
warranty of any kind, express or implied. IBM shall not be
responsible for any damages arising out of the use of, or
otherwise related to, these materials. Nothing contained in
these materials is intended to, nor shall have the effect of,
creating any warranties or representations from IBM or its
suppliers or licensors, or altering the terms and conditions of
the applicable license agreement governing the use of IBM
software. References in these materials to IBM products,
programs, or services do not imply that they will be available
in all countries in which IBM operates. Product release dates
and/or capabilities referenced in these materials may change
at any time at IBM’s sole discretion based on market
opportunities or other factors, and are not intended to be a
commitment to future product or feature availability in any
way. IBM, the IBM logo, Rational, the Rational logo, and
other IBM products and services are trademarks of the
International Business Machines Corporation, in the United
States, other countries or both. Other company, product, or
service names may be trademarks or service marks of others.
www.ibm.com/devops

A Continuous Delivery Safety Net for Databases

More Related Content

A Continuous Delivery Safety Net for Databases