Webinar effective mobile performance testing using real devices
•
0 likes•1,061 views
This document summarizes a webinar about effective mobile performance testing. It discusses: 1. The importance of testing mobile app performance given increasing user expectations and the risks of losing customers due to poor performance. 2. Challenges in mobile performance testing including supporting many different devices, carriers, and simulating real-world network conditions. 3. How network virtualization solutions can help by replicating production mobile networks in test environments to accurately test performance before deployment. 4. A demonstration of a mobile performance testing solution that monitors network conditions and analyzes app performance across various mobile networks and devices.
Report
Share
Webinar effective mobile performance testing using real devices
- 1. Effective Mobile Performance Testing Using Real Devices Enabling Enterprise mobility Extending Software Quality to Mobile Perfectomobile Mobile Testing Center of Excellence Group
- 2. Presenters Lydia Casillas Vice Presidents of Product Sales & Marketing Dave Berg Vice President of Product Strategy Amir Rozenberg Director of Product Management, Mobile Monitoring & Performance Testing Uzi Eilon Pre-Sales Director
- 3. Webinar Agenda 1 • Intro 2 3 • Mobile Performance Keys to success Shunra • Mobile Performance Considerations pains and solutions- SkyIT • MobileCloud Performance Live Demo 4 • Q&A
- 4. Mobile App Performance Keys to Success © Copyright 2012 Shunra Software, LLC. Information contained herein is subject to change without notice.
- 5. Shunra overview Company Overview Solution: Network virtualization for software testing to ensure application performance early in the dev / ops lifecycle Capture production network conditions Test applications on a virtualized network in the test environment Analyze and optimize application performance Value Prop: Average customer ROI < 3 months Average customer saves $1 million per year Customers: Over 2,500 customers 75% of the Fortune 500 Market Success: 5 5 of the Fortune 10 companies 7 of the top 10 banks in the US 5 of the top 10 global financial institutions 6 of the top 10 global retailers 7 of the top 12 global auto manufacturers 6 of the top 10 aerospace/defense companies 5 of the top 7 global pharmaceuticals 4 of the top 5 global telecoms © Copyright 2012 Shunra Software, LLC. Information contained herein is subject to change without notice. Representative Customers
- 6. State of the testing market Testing Remains Challenging (World Quality Report, Sept. 2012) Organizations struggle with the challenges of mobile – only 31% formally test mobile apps Lack of testing tools and knowledge persists – 59% characterize QA teams ability to effectively test as merely “average”; two-thirds do not have the right tools and over one-third lack appropriate testing methodologies Performance Awareness (Computerworld UK, May 2012) Performance problems are highly prevalent – 74% cite performance issues with critical business applications Performance problems are getting worse – 82% report speed and responsiveness problems in the past year and nearly half report issues occurring at greater frequency 6 © Copyright 2012 Shunra Software, LLC. Information contained herein is subject to change without notice.
- 7. Application complexity increases failure The average application connects to 38 discreet services 3rd Party Private or Public Cloud Services Customers Cloud Services Load Major ISP Web Balancers Servers App DB Servers Servers Mainframe Employees Storage Browsers & Devices 7 Mobile Carriers & ISPs Content Delivery Networks © Copyright 2012 Shunra Software, LLC. Information contained herein is subject to change without notice. Web Services Mobile Components
- 8. When customers “struggle” with an app… In the time it takes to blink an eye, 400 milliseconds to be exact, online and mobile users will have begun to abandon an application. App Provider Competitor 43% 16% 14% 12% Abandon the transaction, try later on computer Become more likely to prefer a competitor Log a complaint with customer service Go directly to a competitor Source: Harris Interactive Mobile Transactions Survey for TeaLeaf, March 2011 8 © Copyright 2012 Shunra Software, LLC. Information contained herein is subject to change without notice.
- 9. Requirements for accurate testing “Real-world” virtualization improves the reliability of test environments and results Test Platform Software defects cost the U.S. economy $59.5 billion annually. A full one-third of this cost could be avoided if better software testing was performed. Services The Economic Impacts of Inadequate Infrastructure for Software Testing, NIST 9 © Copyright 2012 Shunra Software, LLC. Information contained herein is subject to change without notice.
- 10. Network virtualization for software testing Comprehensive application performance testing • Discovery of production network conditions across Mobile, Cloud, WAN and Internet networks • Testing of applications in an environment with virtualized production network conditions • Analysis and optimization of performance before deployment and before poor performance affects users 10 © Copyright 2012 Shunra Software, LLC. Information contained herein is subject to change without notice.
- 11. Shunra Analytics Analysis, remediation and optimization • Automated optimization/remediation recommendations, proven to improve HTTP application performance by up to 40% • Automated Performance Scorecard highlights problem transactions • SLO/SLA validation • Secure analysis with HTTP/HTTPS decode • Waterfall diagram for resource analysis (resource size and load time) 11 © Copyright 2012 Shunra Software, LLC. Information contained herein is subject to change without notice.
- 12. Suggested Optimization Tips 7 techniques you can use right now 1. Put stylesheets at the bottom 2. Don’t put scripts in the HEAD 3. Use HTML5 storage 4. Cache your redirects 5. Design for progressive rendering 6. Reference all images in the HTML 7. Increase your server’s keep-alive timeout 12 © Copyright 2012 Shunra Software, LLC. Information contained herein is subject to change without notice.
- 13. Mobile Performance Considerations, Pains and Solutions Lydia Casillas Sky IT Group
- 14. About Sky I.T. Group • Established in 1997 • Headquartered in NYC, Office in: ATL, Raleigh, Boston, Baltimore, Ft. Lauderdale & San Jose • 60+ Employees • Focus: Leverage Technology to solve Business Challenges. People, Process & Technology • Perfecto, HP & Shunra Software Reseller • Perfecto & HP Solution Provider – Professional Services • Perfecto & HP Education Provider • Specialty Focus: • Managed “Testing-as-a-Service” • Application Testing Center-of-Excellence • Service Delivery – Throughout the USA Customers-at-a-Glance
- 15. Performance Testing Mobile Web & Native Applications are a NECESSITY!!! Why are more companies making Mobile Performance Testing a Higher priority & initiative? – Customers, Customers, Customers – More users are moving to smartphone & tablets to access business applications, purchase consumer goods or manage their daily lives, than ever before. – User expectations are at a high…no one wants to wait anymore – Businesses need to keep up with these trends or they will lose customers faster than acquiring them. – 57% of online consumers will leave a site after waiting more than 3 seconds. 80% of those people will never return. Satisfied customers tell 3 friends Angry customers tell 3,000
- 16. Common Customer Pain Points in Mobile Performance Testing In speaking with Financial & Commercial Entertainment Customers… Devices: – Increasing number of devices w/different OS & Screen sizes – Users: – 100M+ users belong to the Gigabyte Club (Over 1GB/month). • 3. How can you afford to test all these different devices, multiple providers when it changes daily? Can our infrastructure support that load? Network: – – Limited & Variable bandwidth on Mobile devices when data connectivity is spotty. Network Performance Impact with shared mobile and web users. • Are my customers getting the same service experience on their mobile device as they are on a desktop? Features Availability of devices from multiple providers to simulate real network traffic & performance usage, to test against. • 2. Cost: Apple, Android, Blackberry, Windows Mobile , etc. Mobile • Wireless 1. Location Carriers Manufacturers OS & Browsers
- 17. Considerations: Mobile Performance Testing Agile vs. Waterfall Manual vs. Automation In-house vs. Outsourced Satisfied customers tell 3 friends Angry customers tell 3,000
- 18. Customer Profile: Travel, Recreation & Leisure Customer Type of Mobile Application & Service Mobile Service: Reservation Service Mobile App, Customer profile, Travel Details, Travel Specials Mobile Device Support: iOS, Android (recently releases) Challenges: • • • • Various Android devices to test against Simulating user load with Mobile & standard user Traffic Analyze Network Traffic at different peak travel times No visibility into Mobile App Transactions Current Technology Gaps: • • • • • Supporting a wide Variety of Protocols or Custom Protocols Stress Testing – High user count simulations Deep Diagnostics to pinpoint cause of performance issues Limited Network Visibility Performance Analysis
- 19. How do we support these Pain Points & Gaps Functional UFT Performance Mobile Network Characteristics + Availability BSM PC LR Backend Monitoring: SiteScope/Diagnostics Mobile Protocols: End-to-End Mobile Application Lifecycle Management PEOPLE + PROCESS + TECHNOLOGY
- 21. Come Meet Us At HP Discover
- 22. Mobile Performance White paper In this white paper, you will be able to gain insights into the following mobile performance aspects: Key building blocks for a mobile performance solution Best practices and process to build a mobile performance solution Best practice solution design overview LINK to download (Registration required): http://help.perfectomobile.com/article/AA-01594/
- 23. Thank You! If you have any further questions, please contact: Perfecto Mobile info@perfectomobile.com Erank@perfectomobile.com Amirr@perfectomobile.com Lcasillas@skyitgroup.com d.berg@shunra.com Join us on LinkedIn Mobile Testing Center of Excellence