Continuous Delivery in the Enterprise
- 1. © 2013 IBM Corporation
Continuous Delivery in the Enterprise
Eric Minick, Technical Evangelist,
UrbanCode
and
Damon Poole, Chief Agilist,
Eliassen Group
- 2. © 2013 IBM Corporation
Our Speakers
2
Eric Minick
Technical Evangelist
Damon Poole
Chief Agilist
- 3. © 2012 Eliassen Group. All Rights Reserved -3-
Damon Poole
• Chief Agilist, Eliassen Group’s Agile Practice
– Coaching: Transformation and Tune-ups
– Training
• 20 years of process change: small co-located teams to
multi-hundred team global enterprises
• Founder and past CTO and CEO of AccuRev
• Creator of multiple Jolt-award winning products
• Past President of Agile New England
• Author of “DIY Agile Kickstart”
• Consulted with Ford IT, Orbitz, Fidelity, Capital
One, ING Direct, and many others
• Taught Agile techniques to thousands of people
- 4. © 2012 Eliassen Group. All Rights Reserved -4-
What does it take to get a
hotfix/patch to your customer?
- 5. © 2012 Eliassen Group. All Rights Reserved -5-
Cycle Time, aka, Measuring Delay
- 6. © 2012 Eliassen Group. All Rights Reserved -6-
2 4 61 3 5 7 9 11 138 10 12 14 15 1716 18
months
$0 $300K $600K $900K
Project A – Old Cycle Time
- 7. © 2012 Eliassen Group. All Rights Reserved -7-
Example: receive $300K 3 months early
2 4 61 3 5 7 9 11 138 10 12 14 15 1716 18
months
$0 $300K $1.2M$600K $900K
Project A
- 8. © 2012 Eliassen Group. All Rights Reserved -8-
Cycle Time
2 4 61 3 5 7
Development
Integration and
testingPreparation
9 11 138 10 12 14 15 1716 18
months
- 9. © 2012 Eliassen Group. All Rights Reserved -9-
A Typical Risk Mitigation Strategy That Can Increase Risk
• A Project is Prioritized after gathering
requirements and doing estimation for
multiple projects.
• This usually takes 3, 6, or more months.
• Requirement gathering and scoping for
multiple projects takes time away from
working on funded projects.
- 10. © 2012 Eliassen Group. All Rights Reserved -10-
Cycle Time
2 4 61 3 5 7
Development
Integrate, test,
& releasePreparation
9 11 138 10 12 14 15 1716 18
Proposing
Funding
(picking)
months
18 month cycle time
Doing
- 11. © 2012 Eliassen Group. All Rights Reserved -11-
Cycle Time
2 4 61 3 5 7
1 month iterations
Prep
Do
Ship
Prep
Do
Ship
Prep
Do
Ship
Prep
Do
Ship
Prep
Do
Ship
Prep
Do
Ship
Pick Pick Pick Pick Pick Pick
- 12. © 2012 Eliassen Group. All Rights Reserved -12-
Cycle Time
2 4 61 3 5 7
1 month iterations
Prep
Do
Ship
3 month cycle time
Pick
- 13. © 2012 Eliassen Group. All Rights Reserved -13-
Dev
Design/Code
Test/Debug
AutomatableCreative
Test
Execution
ReleaseData
Gathering
Test Deploy
Product
Mgmt
Business
Planning
Test
Design
Releng
- 14. © 2012 Eliassen Group. All Rights Reserved -14-
Kanban
on
deck
coding testing qccept
1) Work is managed visually
2) Limited work in progress
3) Flow is pull-based
backlog done
Admin wants a
report of site-
wide activity
5
Bob
Seller wants to
remove an ad
5
Traveller wants
to e-mail a hotel
booking
2
Traveller wants
to e-mail a car
booking
2
Traveller wants
to e-mail an
airline booking
2
Traveller wants
to link to on-line
check-in
2
Traveller wants
to link to cancel
a booking
2
Hotel owner
wants to check
usage data
2
Airline wants to
check usage data
2
Traveller wants
to register with
the system
3
Sue
Traveller wants
to see their
upcoming trips
2
Tom
Traveller wants
to copy a
booking
2
Bob
Traveller wants
to edit a booking
2
Tom
Traveller wants
to delete a
booking
1
Sue
Traveller wants
to enter a
booking
3
Bob
Rental agency
wants to check
usage data
2
Seller wants to
show an ad
5
Sue
- 15. © 2012 Eliassen Group. All Rights Reserved -15-
Kanban in Action
Customers / Market
Product
Mgmt
$
- 16. © 2012 Eliassen Group. All Rights Reserved -16-
AutomatableCreative
Build/Test ReleaseData
Gathering
Team
Design/Code
Business
Planning
Test
Design Releng
- 17. © 2012 Eliassen Group. All Rights Reserved -17-
Relationship of Agile Maturity to Benefits
Agile Maturity
Benefit
- 18. © 2012 Eliassen Group. All Rights Reserved -18-
CYCLE TIME 1-CLICK
DEPLOY
Enterprise Agility Model
AGILE OFFICE
I3I2I1
LOB
CUSTOMERS
ESCALATION
ARCHITECTURE
RELEASE TEAM
/ OPS
EPICS
DELIVERY BASED
MANAGEMENT
LOB
BUSINESS
LEADERS
DELIVERY
BASED
METRICS
PORTFOLIOOFPROGRAMS
CAPACITYBASEDINVESTMENT
AGILE
PROJECT
MGMT
AGILE SCM
- 19. IBM Corporation ©2013
Lead Consultant & Tech Evangelist
Eric is Lead Consultant at Urbancode
where I help customers get the most
out of their build, deploy and
release processes.
Today he works with customers and
industry leaders to figure out this
DevOps thing.
Eric Minick
eric@urbancode.com
@EricMinick
- 20. IBM Corporation ©2013
An updated toolchain
Source
Repos
Continuous
Integration & Build
Mgmt.
Build
Package
Repo
Build Registered
with Package
Repo
Triggers
Version
Meta-data
Deployment
System
Builds
Test 1
Test ...
Test n
Prod
Functional
Tests
Performance
Tests
Monitoring
- 21. IBM Corporation ©2013
The basic flow
1 The Build
a. Create the package
b. Run tests and scans
2 For each test environment
a. Deploy
b. Run some tests
c. Determine worthiness for next env.
3 Release to Production
- 22. IBM Corporation ©2013
Two big bottlenecks
• Manual regression tests are
too slow
• Never allocated enough time
Testing
• Manual / semi-scripted
deployments are slow
• Errors in deployment are risky
and waste QA time
Deployment
- 23. IBM Corporation ©2013
Testing tools
Source
• Coventions / Compliance
• Flow Analysis / Security Scans
Unit
• xUnit
• MSTest
Functional
• Script Driven
• Playback / Record
Perf
• Load Testing
• Func Tests + Monitoring
- 24. IBM Corporation ©2013
Deployments should be automated
Differences in dev and
ops environments and
procedures cause failures
Time to market pressure
for more frequent releases
Manual (tribal) processes
for release lack
repeatability/speed
Major releases take days,
100 people and are
organized by spreadsheet
Daily
Build
Release
Who did
this last
time?
Dave…
Dave’s not
here
man…
Dev
Prod
I’ll order
breakfast
- 25. IBM Corporation ©2013
Deployment: Two approaches
Build Pipeline Tool
- Perform a build, promote build through environments
CI tool + Application Deployment Automation Tool
- Applications are made up of several builds
- 26. IBM Corporation ©2013
Source
Repos
Continuous
Integration & Build
Mgmt.
Build
Package
Repo
Build Registered
with Package
Repo
Triggers
Version
Meta-data
Deployment
System
Builds
Test 1
Test ...
Test n
Prod
Functional
Tests
Performance
Tests
Monitoring
These are 1 tool in a pipeline model
These are 2-3 tools in CI + ADA model
- 27. IBM Corporation ©2013
When is each approach appropriate?
Pipeline
Simple apps
Low coupling between
components / services
Tests validate ONE version
of ONE thing
Shared tool ownership ok
Build + ADA
More complex applications
Higher coupling between
components / services
Tests validate that the larger
system is working
Dev wants to own build,
Ops wants to own deploy
- 29. IBM Corporation ©2013
Reviewing the updated toolchain
Source
Repos
Continuous
Integration & Build
Mgmt.
Build
Package
Repo
Build Registered
with Package
Repo
Triggers
Version
Meta-data
Deployment
System
Builds
Test 1
Test ...
Test n
Prod
Functional
Tests
Performance
Tests
Monitoring
- 30. IBM Corporation ©2013
Bonus bottleneck: Provisioning
Agile leads to:
- More small teams
- More changes, and automated tests to use
- Places to test becomes the next bottleneck
Package
Repo
Deployment
Chain
Test 1
Test ...
Test n
Prod
Version
Meta-data
Provisioning
Controller
Source
Repos Env. Def
Images
IaaS
Provisioning
Server
Configuration
Environment
Config
Environment
Inventory
CMDB
- 31. IBM Corporation ©2013
CD Self Assessment
Continuous Delivery Maturity Model
- 1 pager
- Full White paper
Consider Maturity
- Current vs Desired
- Build, Deploy Test, Decide
- 32. IBM Corporation ©2013
Here to help
Eliassen: Urbancode: tools guys
uBuild
- Build automation on an
enterprise scale
uDeploy
- Application Release
Automation
uRelease
- Release management and
release weekend execution
The leading consulting
services and technology
staffing firm with practices
in Agile, Government
Services, Workforce
Management and Life
Sciences.
Services include:
development
advisory, implementation
services, and on demand
support.
- 34. © 2013 IBM Corporation34
© Copyright IBM Corporation 2013. All rights reserved. The information contained in
these materials is provided for informational purposes only, and is provided AS IS without
warranty of any kind, express or implied. IBM shall not be responsible for any damages
arising out of the use of, or otherwise related to, these materials. Nothing contained in
these materials is intended to, nor shall have the effect of, creating any warranties or
representations from IBM or its suppliers or licensors, or altering the terms and conditions
of the applicable license agreement governing the use of IBM software. References in
these materials to IBM products, programs, or services do not imply that they will be
available in all countries in which IBM operates. Product release dates and/or capabilities
referenced in these materials may change at any time at IBM’s sole discretion based on
market opportunities or other factors, and are not intended to be a commitment to future
product or feature availability in any way. IBM, the IBM logo, Rational, the Rational logo,
Telelogic, the Telelogic logo, and other IBM products and services are trademarks of the
International Business Machines Corporation, in the United States, other countries or both.
Other company, product, or service names may be trademarks or service marks of others.
Editor's Notes
- Hey! Hotfixes/patches are like that and all organizations do those!How long does it take to do a hotfix? What gets in the way? What are the risks?How would you feel about doing one 10 times a day?Continuous Delivery is treating all work as a hotfix.
- Cycle time is not just how long it takes to develop a set of requirements and get it to “gold master” status. It is the entire process from “I think we may want to do X” to “X is now in production and usable by real customers.”That includes the funding process, requirements gathering, and getting it from “code complete” to production. It is very, very, very hard to shrink just development (aka “construction”) without shrinking all of the rest. They are surprisingly interdependent.
- This has always been considered to be “the most efficient” and to “reduce the most risk”
- Cycle time is not just how long it takes to develop a set of requirements and get it to “gold master” status. It is the entire process from “I think we may want to do X” to “X is now in production and usable by real customers.”That includes the funding process, requirements gathering, and getting it from “code complete” to production. It is very, very, very hard to shrink just development (aka “construction”) without shrinking all of the rest. They are surprisingly interdependent.
- You may or may not have a backlog.