Mobile performance metrics and performance monitoring meetup 2017 05 10
•Download as PPTX, PDF•
0 likes•562 views
Bitbar hosted a local meetup in Helsinki where we discussed mobile performance metrics, performance testing/monitoring and using production data to optimize testing.
Report
Share
More Related Content
What's hot
What's hot (20)
Similar to Mobile performance metrics and performance monitoring meetup 2017 05 10
Similar to Mobile performance metrics and performance monitoring meetup 2017 05 10 (20)
More from Bitbar
More from Bitbar (13)
Recently uploaded
Recently uploaded (20)
Mobile performance metrics and performance monitoring meetup 2017 05 10
- 3. Bitbar Technologies • Bitbar Technologies is the company behind Testdroid • Operations: Finland (Helsinki & Oulu), Unites States (San Jose, CA) & Poland (Wroclaw) • Founded in 2009 – 52 Employees – Customers in 30+ Countries • Private Company – VC Backed – Qualcomm, DFJ, Creathor, Inventure • Provides Technology and Services For High Performance Mobile SW Development and Testing Solutions Based On Open Standards Customers include:
- 4. Mobile Testing Pain Points • HW Fragmentation – Hundreds of Permutations • Device Acquisition & Management • Inefficiency with Distributed Teams, App Platforms, Tooling Preferences • Slow Manual Testing • Detailed Reporting
- 5. Why Cloud? • Home brew setup vs pre-existing infrastructure • Device maintenance is a pain • We specialize in this area, so you don’t have to – Streamlined procurement – Scale won’t be an issue • There’s also the boss asking for reports and data
- 6. Mobile Game Testing • Has to be done no questions about it • Particularly difficult vs. other functional mobile app testing – No native UI controls • Resource intensive applications – Great variation on different devices – Crashes and slowness – Network connection dependent – Real mobile devices needed, simulators and emulators don’t represent the real thing
- 7. Mobile Game Testing in Bitbar Cloud • We don’t really mind which technique is used, our environment supports any. • Appium + image recognition – Can be laborious for early stage games where graphics keep changing a lot – Allows testing the released app – Take screenshots and find matches during tests – Can be slow as needs multiple tools (Appium + OpenCV) – Black box testing – can be done by anyone • Alternative sources of data - game logs guide testing – Special build for testing – Gray box testing – done by dev teams • Self playing games – Special build for testing – Gray box testing – done by dev teams
- 8. Demo! • Appium + image recognition – https://github.com/bitbar/testdroid-samples/tree/master/image- recognition
- 9. Gamebench Results • Frame rate: 30fps for slower games is ok. Games with rapid reflexes or simulated physics or collisions (like Angry Birds) target 60fps – If your game drops below 24fps (against a 30fps median) or 50fps (against a 60fps median), GameBench will give you a low score for frame rate stability. Consistently smooth games usually get a stability score of 80 percent or higher. • CPU Usage: 15 percent is a healthy limit for average normalised CPU usage. Unnormalised CPU usage threshold could be at 30 percent for a dual-core device. – Sustained excessive CPU usage increases battery drain and temperature -> the device throttles performance. • GPU Usage: many games push GPUs to their limits, this increases the likelihood of janks (i.e. micro-stutter) and frame rate drops. Aim for a peak of 80 or 90 percent GPU usage. Using over 70 percent of GPU on current high-end devices very often max out the GPU on older or cheaper devices. • Memory usage: RAM hogging is bad. Leaking memory is unacceptable. Casual games should aim for under 200MB on iOS and 500MB on Android. If you go above, blacklist older and low end devices to avoid bad reviews.