Stress Testing for the Digital Economy: Are YOU Ready to Deliver High Performance APIs, Streaming, and Mobile Applications

IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Sven Hammar
CEO
Apica
Twitter: @svenhammar
Stress Testing for the
Digital Economy
Julie Craig
Research Director
Enterprise Management Associates
Twitter: @julie_craig
Are YOU Ready to Deliver High Performance APIs,
Streaming, and Mobile Applications

Today’s Speakers
2 © 2016, Enterprise Management Associates
Julie Craig – Research Director
Julie has over 20 years of experience in software engineering, IT
infrastructure engineering, and enterprise management. At EMA, Julie’s
focus areas include DevOps, application performance and availability
management, Application Discovery and Dependency Mapping, software
development, and software as a service.
Sven Hammar – CEO
Sven founded Apica in 2005. As CEO, he has led the company from concept
through development to becoming a premier provider of performance testing
and monitoring. Prior to founding Apica, he was CEO & co-founder of Celo
Communications. He also served as VP of Marketing at Gemplus (GEMP),
leading the market launch of Gemplus security solutions/products for banking
and PC-OEM with major players like RSA, VISA, MasterCard, ACI,
Panasonic, and Target.

Logistics for Today’s Webinar
An archived version of the event recording will be
available at www.enterprisemanagement.com
• Log questions in the Q&A panel located on the lower
right corner of your screen
• Questions will be addressed during the Q&A session
of the event
> QUESTIONS
> EVENT RECORDING

Stress Testing for the
Digital Economy
Julie Craig
Research Director
Enterprise Management Associates
Twitter: @julie_craig
Are YOU Ready to Deliver High Performance APIs,
Streaming, and Mobile Applications

Overview
• A new technology landscape
• New technologies and use cases
• Continuous Delivery Supporting Digital Transformation
• Automating for Digital Transformation
• Wrap-up

A New Technology Landscape

“Incorporating new technologies” IT’s #1 challenge
7 © 2016 Enterprise Management Associates, Inc.
40%
36%
36%
34%
26%
24%
23%
21%
21%
18%
18%
Incorporating new technologies such as cloud and mobile into the technology…
Acquiring personnel with the right skills
High cost of administration ties up too much budget
Business alignment
Lack of application-focused versus silo-focused monitoring/mgmt tools
Internet of Things (IoT) considerations such as sensor networks, etc.
Difficulty meeting demands of the business in a timely manner
Poor application performance and/or availability
Lack of organizational cloud strategy
"Shadow IT"
Lack of executive vision or leadership
What are the top three (3) overall concerns/challenges affecting your IT department?

We are in the midst of an explosion of innovation….

Yet IT teams are still struggling with the “same old, same
old” in terms of performance & availability mgt.
“Supporting these applications requires skills which we don’t have.”
“We are unable to pinpoint sources and causes of problems.”
“Our applications are unstable.”
“We can’t see transactions or data flow between tiers”
“We have too many applications running on different devices and no way to monitor them all.”
“We want to become more proactive in detecting and fixing problems – but we can’t.”
“We are seeing performance issues with our web applications – such as slow loading times.”
“We had ten people on a 32 hour conference call.”
“We can’t get less than 30 second response time.”

Technology innovation is driving growth of a new breed
of massively interconnected, complex applications
Virtualization
IoT SaaS
PaaS
IaaS
APIs/Int
Mobile
Big Data Streaming
Agile
Docker
Continuous
Delivery

New Technologies and Use Cases

Mobile applications: high value, high risk
• Volume of mobile Business to Consumer (B2C)
interactions increases each year
 % of online shoppers using mobile devices
continues to rise
• High risk of poor performance
 User experience varies depending on device type,
network connection type/speed, & performance of
back-end systems
• Users perceive poor performance as the fault of
the app provider
#3
Technology-
related initiative
supporting
“Digital
Transformation”

APIs: Critical to doing business in an interconnected
marketplace
• APIs are the “de facto standard” for integrating
within & across organizational boundaries
• Essential to delivery of container-based
microservices
• Both consumer and provider APIs essential to
revenue growth
• Each integration point creates a performance
risk
• API consumption means being dependent on
third-party services for performance/availability
90% say providing
APIs TO partners, etc.
“critical” or “important” to
revenue generation
98% say consuming
APIs FROM partners, etc.
“critical” or “important” to
revenue generation

Streaming: A whole different animal
• Endpoint devices (i.e. set-top boxes, game
stations, etc.) each have specific Operating
System and hardware implementations
• “Players” for streamed content are different
for an iPhone, a PlayStation (PS4™), or a set-
top box
• Hi bandwidth, latency sensitive
• Network dependent
• Connection speed varies depending on
device, location, and service provider
50% report
“significant
amounts” of
streaming traffic on
their organization's
network in the
course of an
average day

20%
21%
21%
24%
25%
27%
29%
32%
32%
35%
38%
45%
51%
Omnichannel (mobile, mobile web, web) for internal non-IT stakeholders
Omnichannel (mobile, mobile web, web) for external business consumers
Omnichannel (mobile, mobile web, web) for IT stakeholders
Social IT in support of IT stakeholders
Social IT in support of IT to IT consumer dialog
New or enhanced CMDB/CMS/application dependency mapping
Social IT in support of business/company consumers
Cross-domain IT asset management and optimization
Improved third-party service oversight/management
Enhanced automation for self-service
New or enhanced project management capabilities
Improved application performance management
Improved user/customer experience management
Which four of the following functional capabilities are (will be) most important for your organization’s digital/IT
transformation initiative in the coming 12 months?
Improved user/customer experience, APM are top
“functional” priorities for Digital Transformation

Continuous Delivery Supporting
Digital Transformation

Top 4 drivers for Continuous Delivery relate to business-
and customer- focused requirements
© 2016 Enterprise Management Associates, Inc.
42%
40%
39%
34%
33%
31%
30%
29%
22%
Business demands for new products or services
Customer demands for better application quality
Customer demands for new products or services
Business demands to better compete against competitors
Agile development practices
CIO-driven requirements for increased productivity
Executive-driven desire to rapidly implement new ideas
Regulatory or compliance requirements/changes
Industry vertical specific changes
Which of the following are the top drivers for the
Continuous Delivery initiative at your company?

Accelerating delivery of IT services:
Good for business, Hard on IT…
52%
48%
45%
36%
31%
Operations is spending more time on
production troubleshooting
Development is drawn into production
troubleshooting more often
Service levels have degraded as rates
of change increased
We are experiencing a higher number
of performance and/or availability
problems
Minimal production impact
How has this increase in frequency of code
delivery impacted production environments?
49%
44%
41%
35%
33%
Higher levels of customer satisfaction
Faster revenue growth
Minimal business impact
We are more agile in competing within
our industry
Less negative fallout on social media
How has this increase in frequency of code
delivery impacted the business as a whole?

“Manual troubleshooting” of production environments top
bottleneck slowing Continuous Delivery
44%
37%
37%
37%
36%
34%
27%
24%
23%
Manual troubleshooting processes dealing with problems arising from…
Time spent testing code
Manual processes supporting package builds and production deployment
Fear/risk related to adverse production impacts of new code
Time spent creating code
Manual processes supporting code movement through lifecycle stages
Lack of testing/QA personnel
Lack of testing facilities, developers and/or QA must "wait their turn" to test
Lack of workflow/automation tools to drive the Continuous Delivery pipeline
Which of the following factors are the biggest "bottlenecks"
slowing down your Continuous Delivery pipeline?

Automating for
Digital Transformation

24%
25%
27%
29%
32%
32%
35%
38%
45%
51%
Social IT in support of IT stakeholders
Social IT in support of IT to IT consumer dialog
New or enhanced CMDB/CMS/application dependency mapping
Social IT in support of business/company consumers
Cross-domain IT asset management and optimization
Improved third-party service oversight/management
Enhanced automation for self-service
New or enhanced project management capabilities
Improved application performance management
Improved user/customer experience management
Which four of the following functional capabilities are (will be) most important for your organization’s
digital/IT transformation initiative in the coming 12 months?
Top priorities for functional Digital Transformation:
Improved user experience, app performance

Top priorities for organizational Digital Transformation:
Delivering new services faster and at higher quality
42%
42%
39%
38%
36%
36%
35%
33%
Compressed delivery time frames for IT services
Improvements in quality of the services delivered to IT and business
Improvements in IT-business alignment
Organizational shifts in IT focus from silo and technology-based to customer-
and business-aligned
Cost reduction associated with delivery of IT services
Agility in terms of "time to innovation", i.e. time required to transform ideas into
full-blown organizational processes
Customer satisfaction (external customers)
Customer satisfaction (internal customers)
From the organizational perspective, which of the following are the
top focus areas for your Digital Transformation initiatives?

Wrap-up

Wrap up
• Vast majority of companies report they are actively engaged in
Digital Transformation.
• Influx of new technologies is #1 challenge IT organizations are
facing
• New development practices– Agile, Continuous Delivery– are
impacting IT and business with high rates of change
• Automation focused on performance/availability testing across the
lifecycle is the answer

Apica
A Performance-as-a-Service Company
Stress Testing for the Digital Economy
2016

2626 www.apicasystems.com
Agenda
• Apica Overview
• Apica LoadTest Suite
• Stress & Loadtesting 1-1
• Customer Cases
• Q&A

Apica Overview
• Founded in 2005
• Sweden, US and the UK
• Leading global platform, network & tools for
performance testing and monitoring
• 550+ global customers
• We test and monitor web and mobile applications

Apica’s Product Offering
Max
Synthetic Monitoring LoadTesting Test Tool & Experts1 2 3
OK

Unified Performance Monitoring & Testing Platform
One GUI - One Platform

200+ Monitoring
Locations
50+ LoadTest
Clusters

Elite - social platform
 Capacity planing
 Stress Test
 Test Automation
 Device
– iPhone
– Android App
– Web

Objectives We Help Address
• Site launch
• Stress test / Max Load
• Monitoring scripts
• Baseline test
• Test Automation
Test Types
Technology
covered
• Mobile
• Streaming
• Web applications
• API / Authentication
• DDoS / SDoS
Capacity
Planing
• Cloud, on-premise, hybrid
• Infrastructure plan test
• Mega Test
• DDoS Test
• Fail over and back up test

Apica LoadTest Suite
• Apica Zebratester Tool
• Apica SaaS Portal
• Cloud Capacity
• Team
• Test Automation
• Software
• SaaS & On Premise

How It Works
Record Usage
Scenarios
Script Functions Generate Load APM & Client Side Analyze Report
Fast, via GUI,
Any Device
Dynamic Data
& Device
Scale in Apica
Portal & Network
Compile and
compare result files Status &
Re-runs

Project result
 Load - Normal and Max
– How many users ,
can we handle?
 Behavior in ”Danger Zone”
– become unstable
above maximum load?
 Problem Analysis ?
– Where are the bottlenecks?
– Fix them via A/B test?

Mega Test Ready

Plan a head
Know your numbers !
• Traffic
• Behavior
• Devices
Integrations
Test plan
• Baseline (90%)
• Mega (10%)
High Volume Testing

 Record real user scenarios
– User interaction
– API endpoint vs App Usage
 Any device Iphone, Android, Apple TV
 Match Speed & Location
 Combine with Apica Monitoring
Testing mobile applications

Test Automation
POST Action
Execute Apica Loadtest
• Continuous Delivery service.
• Customize release process model with API
• Apica’ test platform is full API driven and integrate direct with AWS
CodePipeline, Jenkins and Team City

Continuous Deployment

DDOS – Validation Test
• High Traffic…
• Network & Layer 7 Attacs
– Simple URL
– Full Scripts
– Slowloris ...
– Network only modules
• Risk & Function & Protection

Launch with Confidence
Set Your Targets-> Use Metrics
Validate Baseline -> Design
Max Test -> Go / NoGo
Set a Process ->
Test Automation
1
2
3
4

Thank you - Questions?
Sven.Hammar@apicasystem.com
Peter.Tollgård@apicasystem.com

Stress Testing for the Digital Economy: Are YOU Ready to Deliver High Performance APIs, Streaming, and Mobile Applications

Related slideshows

More Related Content

Stress Testing for the Digital Economy: Are YOU Ready to Deliver High Performance APIs, Streaming, and Mobile Applications