SlideShare a Scribd company logo

3 tips to increase mobile test coverage

•Download as PPTX, PDF•
•
SOASTA
SOASTA

“Mobile Test Coverage: It’s not just about the devices!" With the proliferation of mobile devices, there is a renewed discussion on Test Coverage as it relates to mobile functional testing. Many of our customers have taken a fresh look at their mobile strategy with a renewed focus on their test coverage strategy. What do they discover? That test coverage is not just about devices. Join us for an hour and we will walk you through the key areas that you need to focus on to keep your mobile strategy covered.

Related slideshows

Sd times-june-24-2015
Sd times-june-24-2015Sd times-june-24-2015
Sd times-june-24-2015
 
Four best practices for performance testing mobile apps soasta and utopia
Four best practices for performance testing mobile apps soasta and utopiaFour best practices for performance testing mobile apps soasta and utopia
Four best practices for performance testing mobile apps soasta and utopia
 
Four Best Practices for Modern Performance Testing
Four Best Practices for Modern Performance TestingFour Best Practices for Modern Performance Testing
Four Best Practices for Modern Performance Testing
 
1 of 25
May 13, 2015 Three Tips to Increase your mobile test coverage Dan Boutin – Senior Product Evangelist dboutin@soasta.com Mobile (404) 304-9529 @DanBoutinSOASTA
© 2014 SOASTA. All rights reserved. March 3, 2015 2 Agenda o What is Test Coverage, anyway? o Functional Testing Overview o How did the game, and the focus, change o So how do I determine the optimal test coverage for my app? o Step outside the box with me! o Let’s show you how you can do it! o Questions & Final Trivia for #swag
© 2014 SOASTA. All rights reserved. March 3, 2015 3 So what is test coverage anyway? o Answers: • A. The degree in which tests exercise the software features • B. Code Coverage • C. Device Coverage • D. How many tests are automated
© 2014 SOASTA. All rights reserved. March 3, 2015 4 Functional Testing Overview
What is Functional Testing? Functional Testing is testing the functionality of an app to meet a business requirement. Ex: Sears testing their app that a user can log in their account, search for shoes, select the size, go to the checkout and purchase.
What is Test Coverage? A measure of the proportion of an application exercised by a test suite, usually expressed as a percentage. Many different ways to measure. Here are some: Feature/function Scenario Transition Path (e.g. “click thru”.)
© 2014 SOASTA. All rights reserved. May 13, 2015 7 How did the game, and the focus, change?
© 2014 SOASTA. All rights reserved. Mobile is a customer acquisition strategy Mobile market is Frantic ○ How many business apps do you have on your phone? ○ How many applications do you use on your desktop?
© 2014 SOASTA. All rights reserved. May 13, 2015 9 Complex Ecosystem o Fragmentation and Plethora of Devices, OS/Platforms, & Screen Footprint o New capabilities to test o Regression Testing Spikes o More Network Considerations o More complicated Apps What was the fallout?
© 2014 SOASTA. All rights reserved. Testing cycles and development methodology Agile decreases cycle time but increases regression suites Agile or Waterfall? Agile regression tests increase as you progress the project
Functional Testing Optimal Test Coverage When is it “done”?
12 Automating the test fabrication process is Fundamental for functional aka “The Assembly Line” I tested 60% (141 out of 236) of the paths (picked by risk, as the most critical paths) and 80% (4 out of 5) of the Data Sets in the software, for a total of 53% (624 out of 1180) overall coverage. “I tested it.” Program Manager’s Translation:
© 2014 SOASTA. All rights reserved. Mobile has caused an explosion! Testing matrix has exploded Desktop App 300 test cases x 2 browsers = 600 test cases 600 tc * 30 mins per test case = 300 hrs 300 hrs / 6 hrs per day = 50 person days 50 pd / 5 people = 10 days of testing Mobile App 300 test cases x 26 devices = 7800 test cases 7800 tc * 30 mins per test case = 3900 hrs 3900 hrs / 6 hrs per day = 650 person days 650 pd / 65 people = 10 days of testing – 2 weeks 650 pd / 5 people = 130 days of testing – 26 weeks
Functional Testing Background
© 2014 SOASTA. All rights reserved. Functional Testing History is manual Manual Testing Automation Testing 100%100%
Functional Testing Needs Innovation
CONFIDENTIAL – Not for Distribution © 2015 SOASTA. All rights reserved. January 13, 2015 Build functional tests that are based on user behavior on your application First you need to determine the coverage model for the application.
© 2014 SOASTA. All rights reserved. Let your customers be your guide…
© 2014 SOASTA. All rights reserved. Mobile Testing The Road to Faster Testing Manual Testing Mobile Device Clouds Automation TouchTest Analytics mPulse
CONFIDENTIAL – Not for Distribution © 2015 SOASTA. All rights reserved. January 13, 2015 Analyze the most common devices of real users mPulse
CONFIDENTIAL – Not for Distribution © 2015 SOASTA. All rights reserved. January 13, 2015 Analyze the most common session paths of real users : Data Science Workbench
TouchTest DEMO – The path to automation 22
© 2014 SOASTA. All rights reserved. We will close with some Mobile Trivia @DanBoutinSOASTA
© 2014 SOASTA. All rights reserved. Step out of the Box…. o SOASTA TouchTest •FREE DownLoad: http://goo.gl/6CErMN o Visit our Blog: http://goo.gl/TA2HKP o Questions? @DanBoutinSOASTA
May 13, 2015 Three Tips to Increase your mobile test coverage Dan Boutin – Senior Product Evangelist dboutin@soasta.com Mobile (404) 304-9529 @DanBoutinSOASTA

More Related Content

3 tips to increase mobile test coverage

  • 1. May 13, 2015 Three Tips to Increase your mobile test coverage Dan Boutin – Senior Product Evangelist dboutin@soasta.com Mobile (404) 304-9529 @DanBoutinSOASTA
  • 2. © 2014 SOASTA. All rights reserved. March 3, 2015 2 Agenda o What is Test Coverage, anyway? o Functional Testing Overview o How did the game, and the focus, change o So how do I determine the optimal test coverage for my app? o Step outside the box with me! o Let’s show you how you can do it! o Questions & Final Trivia for #swag
  • 3. © 2014 SOASTA. All rights reserved. March 3, 2015 3 So what is test coverage anyway? o Answers: • A. The degree in which tests exercise the software features • B. Code Coverage • C. Device Coverage • D. How many tests are automated
  • 4. © 2014 SOASTA. All rights reserved. March 3, 2015 4 Functional Testing Overview
  • 5. What is Functional Testing? Functional Testing is testing the functionality of an app to meet a business requirement. Ex: Sears testing their app that a user can log in their account, search for shoes, select the size, go to the checkout and purchase.
  • 6. What is Test Coverage? A measure of the proportion of an application exercised by a test suite, usually expressed as a percentage. Many different ways to measure. Here are some: Feature/function Scenario Transition Path (e.g. “click thru”.)
  • 7. © 2014 SOASTA. All rights reserved. May 13, 2015 7 How did the game, and the focus, change?
  • 8. © 2014 SOASTA. All rights reserved. Mobile is a customer acquisition strategy Mobile market is Frantic ○ How many business apps do you have on your phone? ○ How many applications do you use on your desktop?
  • 9. © 2014 SOASTA. All rights reserved. May 13, 2015 9 Complex Ecosystem o Fragmentation and Plethora of Devices, OS/Platforms, & Screen Footprint o New capabilities to test o Regression Testing Spikes o More Network Considerations o More complicated Apps What was the fallout?
  • 10. © 2014 SOASTA. All rights reserved. Testing cycles and development methodology Agile decreases cycle time but increases regression suites Agile or Waterfall? Agile regression tests increase as you progress the project
  • 11. Functional Testing Optimal Test Coverage When is it “done”?
  • 12. 12 Automating the test fabrication process is Fundamental for functional aka “The Assembly Line” I tested 60% (141 out of 236) of the paths (picked by risk, as the most critical paths) and 80% (4 out of 5) of the Data Sets in the software, for a total of 53% (624 out of 1180) overall coverage. “I tested it.” Program Manager’s Translation:
  • 13. © 2014 SOASTA. All rights reserved. Mobile has caused an explosion! Testing matrix has exploded Desktop App 300 test cases x 2 browsers = 600 test cases 600 tc * 30 mins per test case = 300 hrs 300 hrs / 6 hrs per day = 50 person days 50 pd / 5 people = 10 days of testing Mobile App 300 test cases x 26 devices = 7800 test cases 7800 tc * 30 mins per test case = 3900 hrs 3900 hrs / 6 hrs per day = 650 person days 650 pd / 65 people = 10 days of testing – 2 weeks 650 pd / 5 people = 130 days of testing – 26 weeks
  • 15. © 2014 SOASTA. All rights reserved. Functional Testing History is manual Manual Testing Automation Testing 100%100%
  • 17. CONFIDENTIAL – Not for Distribution © 2015 SOASTA. All rights reserved. January 13, 2015 Build functional tests that are based on user behavior on your application First you need to determine the coverage model for the application.
  • 18. © 2014 SOASTA. All rights reserved. Let your customers be your guide…
  • 19. © 2014 SOASTA. All rights reserved. Mobile Testing The Road to Faster Testing Manual Testing Mobile Device Clouds Automation TouchTest Analytics mPulse
  • 20. CONFIDENTIAL – Not for Distribution © 2015 SOASTA. All rights reserved. January 13, 2015 Analyze the most common devices of real users mPulse
  • 21. CONFIDENTIAL – Not for Distribution © 2015 SOASTA. All rights reserved. January 13, 2015 Analyze the most common session paths of real users : Data Science Workbench
  • 22. TouchTest DEMO – The path to automation 22
  • 23. © 2014 SOASTA. All rights reserved. We will close with some Mobile Trivia @DanBoutinSOASTA
  • 24. © 2014 SOASTA. All rights reserved. Step out of the Box…. o SOASTA TouchTest •FREE DownLoad: http://goo.gl/6CErMN o Visit our Blog: http://goo.gl/TA2HKP o Questions? @DanBoutinSOASTA
  • 25. May 13, 2015 Three Tips to Increase your mobile test coverage Dan Boutin – Senior Product Evangelist dboutin@soasta.com Mobile (404) 304-9529 @DanBoutinSOASTA