SlideShare a Scribd company logo
1
© 2013 SOASTA CONFIDENTIAL - All rights reserved.
Sharing the keys to retail performance success
TODAY’S SOASTA PRESENTERS






Peter Galvin: 
 
Brad Johnson: 
 
Mike Ostenberg

SVP Marketing 
 
VP Product Marketing 
Dir. Systems Engineering

Agenda: 
•  Retail case studies
•  Key considerations gleaned from retail performance optimization
•  SOASTA’s solutions for measuring and testing user performance

Questions: 
Submit in chat box during event
Poll Question
3
© 2013 SOASTA CONFIDENTIAL - All rights reserved.
4
© 2013 SOASTA CONFIDENTIAL - All rights reserved.
o  Inspection to Prevention
o  Approximation to Reality


o  Periodic to Continuous
5
© 2013 SOASTA CONFIDENTIAL - All rights reserved.
6
© 2013 SOASTA CONFIDENTIAL - All rights reserved.
7
© 2013 SOASTA CONFIDENTIAL - All rights reserved.
Tested to 7x Initial load test goals


Reached 9x previous year’s peak

 


 Validated dynamic caching investment

 
 


 Reduced hardware capacity needs

 


 
 Validated Akamai to 4x beyond peak

 
 
 
 

“One of the best investments
to our website this year .” 
Nancy Jordan,
IT Middleware 
& Integration 
Manager
8
© 2013 SOASTA CONFIDENTIAL - All rights reserved.
9
© 2013 SOASTA CONFIDENTIAL - All rights reserved.
“We are 100% focused 
on our consumer
satisfaction…Given the
revenue it’s protecting for
us, it’s money very well
spent.”
Michael Hart, Bonobos
CTO

Iterative testing of purchase cycle

Easy test creation and results


Systematically tuned app to infrastructure



 Compares test/test, year/year

 
 
 
 
 


 
 Revenue Protection

 
 


 
 64x normal traffic, 15X sales, no issues
10
© 2013 SOASTA CONFIDENTIAL - All rights reserved.
11
© 2013 SOASTA CONFIDENTIAL - All rights reserved.
Tests for flash sales with unknown spikes

mPulse analyzes real user performance


Continuous tests of mobile and web traffic

 


 
 50% mobile traffic – 160% more visitors

 


 
 $5M AUD in one flash sale – zero issues

 
 




 

“SOASTA was very
easy to work with
and could
accommodate our 
requirements in a
four-week
timeframe.”
Gavin Cliffe, 
OZSale CTO.
12
© 2013 SOASTA CONFIDENTIAL - All rights reserved.
13
© 2013 SOASTA CONFIDENTIAL - All rights reserved.
“Website performance really
affects your brand image
when you have problems…
The inherent savings to our
brand by using CloudTest is
priceless.”
—Cheryl Davis, 
digital technical 
operations manager
Hallmark Cards, Inc.
Tested to 120% of expected load


Saved 50% on testing expenses



 
 Reduced staff costs by 66%

 


 Eliminated test software and hardware costs
14
© 2013 SOASTA CONFIDENTIAL - All rights reserved.
2008
 2013
15
© 2013 SOASTA CONFIDENTIAL - All rights reserved.
16
© 2013 SOASTA CONFIDENTIAL - All rights reserved.
Meaningful scale &
appropriate concurrency
Most accurate models
of real user behavior
Control to start, stop & modify
tests without delays
Test on your schedule
17
© 2013 SOASTA CONFIDENTIAL - All rights reserved.
Failover Scenarios
Success Disasters
End to End
18
© 2013 SOASTA CONFIDENTIAL - All rights reserved.
19
© 2013 SOASTA CONFIDENTIAL - All rights reserved.
20
© 2013 SOASTA CONFIDENTIAL - All rights reserved.
s
Memory leaks
Test Scale
Stage/Team
Dev
& Test
Release
& Deploy
Network
& Ops
CDN file placement
Load Balancer configuration
Network bandwidth
Network configuration
DNS routing
Inadequate server resources
Default configuration settings
Unbalanced web serversAuto-scaling failures
Latency between systems
Slow third-party plug-ins
Garbage collection
Database thread counts
Inefficient database queriesSlow pages
Conflict with other apps
Test Lab Staging Production (100% +++)
Search technology limits
Method-level tuning
Max sockets exceeded
Firewall max capacity
Global latency variance
Security bottlenecks
Continuous tuning for full-scale…and well beyond
21
© 2013 SOASTA CONFIDENTIAL - All rights reserved.
ü Build and modify your tests based on real usage
ü Test for spikes with the cloud
ü Test what you don’t think you need to
ü Measure, test and act in real time
ü Test more (compress processes)
ü Test in production
22
© 2013 SOASTA CONFIDENTIAL - All rights reserved.
Mike Ostenberg
Thanks
Contact SOASTA:
www.soasta.com/cloudtest/
info@soasta.com
866.344.8766
Follow us:

twitter.com/cloudtest
facebook.com/cloudtest
RESOURCES

Retail Readiness Page: http://www.soasta.com/retail-readiness/

Case studies: http://www.soasta.com/customers/ecommerce/

Webinars future and past: http://www.soasta.com/knowledge-center/webinars/

Need a test, fast? CloudTest On-Demand.
- email info@soasta.com
- You’ll hear from us by tomorrow
www.SOASTA.com

More Related Content

What Retailers Know About Performance and User Experience

  • 1. 1 © 2013 SOASTA CONFIDENTIAL - All rights reserved. Sharing the keys to retail performance success TODAY’S SOASTA PRESENTERS Peter Galvin: Brad Johnson: Mike Ostenberg SVP Marketing VP Product Marketing Dir. Systems Engineering Agenda: •  Retail case studies •  Key considerations gleaned from retail performance optimization •  SOASTA’s solutions for measuring and testing user performance Questions: Submit in chat box during event
  • 3. 3 © 2013 SOASTA CONFIDENTIAL - All rights reserved.
  • 4. 4 © 2013 SOASTA CONFIDENTIAL - All rights reserved. o  Inspection to Prevention o  Approximation to Reality o  Periodic to Continuous
  • 5. 5 © 2013 SOASTA CONFIDENTIAL - All rights reserved.
  • 6. 6 © 2013 SOASTA CONFIDENTIAL - All rights reserved.
  • 7. 7 © 2013 SOASTA CONFIDENTIAL - All rights reserved. Tested to 7x Initial load test goals Reached 9x previous year’s peak Validated dynamic caching investment Reduced hardware capacity needs Validated Akamai to 4x beyond peak “One of the best investments to our website this year .” Nancy Jordan, IT Middleware & Integration Manager
  • 8. 8 © 2013 SOASTA CONFIDENTIAL - All rights reserved.
  • 9. 9 © 2013 SOASTA CONFIDENTIAL - All rights reserved. “We are 100% focused on our consumer satisfaction…Given the revenue it’s protecting for us, it’s money very well spent.” Michael Hart, Bonobos CTO Iterative testing of purchase cycle Easy test creation and results Systematically tuned app to infrastructure Compares test/test, year/year Revenue Protection 64x normal traffic, 15X sales, no issues
  • 10. 10 © 2013 SOASTA CONFIDENTIAL - All rights reserved.
  • 11. 11 © 2013 SOASTA CONFIDENTIAL - All rights reserved. Tests for flash sales with unknown spikes mPulse analyzes real user performance Continuous tests of mobile and web traffic 50% mobile traffic – 160% more visitors $5M AUD in one flash sale – zero issues “SOASTA was very easy to work with and could accommodate our requirements in a four-week timeframe.” Gavin Cliffe, OZSale CTO.
  • 12. 12 © 2013 SOASTA CONFIDENTIAL - All rights reserved.
  • 13. 13 © 2013 SOASTA CONFIDENTIAL - All rights reserved. “Website performance really affects your brand image when you have problems… The inherent savings to our brand by using CloudTest is priceless.” —Cheryl Davis, digital technical operations manager Hallmark Cards, Inc. Tested to 120% of expected load Saved 50% on testing expenses Reduced staff costs by 66% Eliminated test software and hardware costs
  • 14. 14 © 2013 SOASTA CONFIDENTIAL - All rights reserved. 2008 2013
  • 15. 15 © 2013 SOASTA CONFIDENTIAL - All rights reserved.
  • 16. 16 © 2013 SOASTA CONFIDENTIAL - All rights reserved. Meaningful scale & appropriate concurrency Most accurate models of real user behavior Control to start, stop & modify tests without delays Test on your schedule
  • 17. 17 © 2013 SOASTA CONFIDENTIAL - All rights reserved. Failover Scenarios Success Disasters End to End
  • 18. 18 © 2013 SOASTA CONFIDENTIAL - All rights reserved.
  • 19. 19 © 2013 SOASTA CONFIDENTIAL - All rights reserved.
  • 20. 20 © 2013 SOASTA CONFIDENTIAL - All rights reserved. s Memory leaks Test Scale Stage/Team Dev & Test Release & Deploy Network & Ops CDN file placement Load Balancer configuration Network bandwidth Network configuration DNS routing Inadequate server resources Default configuration settings Unbalanced web serversAuto-scaling failures Latency between systems Slow third-party plug-ins Garbage collection Database thread counts Inefficient database queriesSlow pages Conflict with other apps Test Lab Staging Production (100% +++) Search technology limits Method-level tuning Max sockets exceeded Firewall max capacity Global latency variance Security bottlenecks Continuous tuning for full-scale…and well beyond
  • 21. 21 © 2013 SOASTA CONFIDENTIAL - All rights reserved. ü Build and modify your tests based on real usage ü Test for spikes with the cloud ü Test what you don’t think you need to ü Measure, test and act in real time ü Test more (compress processes) ü Test in production
  • 22. 22 © 2013 SOASTA CONFIDENTIAL - All rights reserved. Mike Ostenberg
  • 23. Thanks Contact SOASTA: www.soasta.com/cloudtest/ info@soasta.com 866.344.8766 Follow us: twitter.com/cloudtest facebook.com/cloudtest RESOURCES Retail Readiness Page: http://www.soasta.com/retail-readiness/ Case studies: http://www.soasta.com/customers/ecommerce/ Webinars future and past: http://www.soasta.com/knowledge-center/webinars/ Need a test, fast? CloudTest On-Demand. - email info@soasta.com - You’ll hear from us by tomorrow