They don't call it Continuous Integration for nothing!

May 28, 2015
South Florida Agile Association – Fort Lauderdale, FL
“It’s not called Continuous Integration for Nothing!”
Dan Boutin – Senior Product Evangelist
dboutin@soasta.com
Mobile (404) 304-9529
@DanBoutinSOASTA

© 2014 SOASTA. All rights reserved. 2
In This Discussion Today
Agenda:
• SOASTA Introduction
• The need for “Continuous”
• The “Big 3”
• CI with mobile and performance testing

SOASTA – Performance Analytics
100 Billion
User Experiences
Tested
10
Million
Tests
Performed
5 Million
Failures
Avoided
Actual CloudTest
view

Agenda:
• The “Big 3”

CI Offers Speed of Delivery
What’s not to like?

o Small Batch Sizes

o Small Batch Sizes
o Comprehensive Version Control

o Small Batch Sizes
o Simple Branch Strategies

o Small Batch Sizes
o Automation

o Small Batch Sizes
o Automation
o Continuous Feedback

o Small Batch Sizes
o Automation
o Emphasis on working builds

o Small Batch Sizes
o Automation
o Consistent environments

o Small Batch Sizes
o Automation
o Consistent environments
o Developer – Tester Collaboration

Agenda:
• The “Big 3”

What’s Been Missing?

So, let’s break down DevOps – What is it, really?
o It’s not what’s within the silos
o It’s about the relationships

A typical silo’d organization lacks end-to-end integration
Performance Engineering & Continuous Integration-Full Lifecycle
Operational Configuration
Continuous Integration touchpoints
ADLC
QA/Test Pre-
Production
Production
QA/Test
Environment
Operational
Production
Environment
Application
Development
Environment

Development Test
Configuration management
Release management
Change management
Performance management
CI Focus Areas – The assembly line Hand-offs
Key Processes

Software is simple: It’s Manufacturing & BOM
o Bill of Materials (BOM!)
o The “Big 3” => Change, Configuration & Release
• That is your software assembly line conveyor belt

So, what?
• We can only test as fast as the conveyor belt allows.
• If the conveyor belt is moving slow, testing is delayed.
• The Big 3 drives testing cycle.
o Why is CI critical?
• Speed
• New features, functions, etc. need to get to the customer, ASAP!

So, let’s break that down…
o CI is a key component of Performance Engineering
o Performance Engineering spans the silos
BIG 3
Testin
g
Applicatio
nDevelopmen
t Life
Cycle
IT
Operation
s
Capacit
yManagemen
t
Performanc
eManagemen
t
Performanc
eEngineering

Laying the Foundation
A factory streamlines for quality and speed
Does it work as
designed?
Does it perform
under extreme
circumstances?
Approve materials Real Drivers
What happens
in actual
conditions?
Assembly Line

Laying the Foundation
CI streamlines for continuous quality
Functional validation
Performance,
reliability,
scalability tests
Real Users
Build
What happens
in actual
conditions?
Unit-tested code

Agenda:
• The “Big 3”

Continuous Quality
SOASTA’s Lifecycle

Continuous Quality
SOASTA’s LifeCycle
I JUST MADE A
CODE CHANGE
Jenkins runs any
unit tests as usual

Continuous Quality
Run iterative load
tests via Jenkins
Spin up suitable, consistent
test environments in the cloud
or the lab
I JUST MADE A
CODE CHANGE
Jenkins runs any
unit tests as usual

Continuous Quality
Run iterative load
tests via Jenkins
Mobile? Run functional
automation on real devices
for functional & performance
validation
or the lab
I JUST MADE A
CODE CHANGE
Jenkins runs any
unit tests as usual

Continuous Quality
Run iterative load
tests via Jenkins
Daily (or more frequent) updates
keep teams quality focused
Embedded failure
details & bottleneck
data isolate issues
validation
or the lab
I JUST MADE A
CODE CHANGE
Jenkins runs any
unit tests as usual

Continuous Quality
Run iterative load
tests via Jenkins
Embedded failure
data isolate issues
Manage to a
performance
baseline
validation
or the lab
I JUST MADE A
CODE CHANGE
Jenkins runs any
unit tests as usual

Continuous Quality
Use real user data to
iterate dev & test plans
Run iterative load
tests via Jenkins
Embedded failure
data isolate issues
Manage to a
performance
baseline
validation
or the lab
I JUST MADE A
CODE CHANGE
Jenkins runs any
unit tests as usual

They don't call it Continuous Integration for nothing!

SOASTA – Branch Build from CloudTest

SOASTA – Drill Down (Test Failures)

SOASTA – UI Testing Bundle Results

SOASTA – UI Testing Failure Drill down into
SOASTA CloudTest from Jenkins

SOASTA – Successful regression test results

Continuous Quality for Continuous Delivery
o SOASTA’s Process Metrics
• 3 Main Products
• (2) Builds per day of each product (Dev branch + Customer branch)
• 8,000 tests executed per build
• For Mobile (TouchTest), Over 300 fully-automated tests each for iOS & Android
SOASTA - Takeaways

The Path to Continuous Testing
o Understand your requirements
o Identify the cloud advantages for you
o Automate the obvious and most critical
o Connect to a continuous process
o Align teams with actionable information
SOASTA - Takeaways

SOASTA - Takeaways

We will close with some Mobile Trivia
@DanBoutinSOASTA

Questions?
Dan Boutin – Senior Product Evangelist
dboutin@soasta.com
Mobile (404) 304-9529
@DanBoutinSOASTA
o SOASTA TouchTest
• FREE DownLoad: http://goo.gl/6CErMN
o Visit our Blog: http://goo.gl/TA2HKP
o Questions? @DanBoutinSOASTA

They don't call it Continuous Integration for nothing!

Related slideshows

More Related Content

They don't call it Continuous Integration for nothing!