My 10 Mobile Automation Questions
•
2 likes•1,499 views
Once a proud Front End developer, now a total Mobile Automation n00b. I was given the task of rethinking mobile automation at the startup EverythingMe and got familiarized with this exciting new field. In this talk, I present the questions and answers I needed as a newcomer in order to begin the journey.
Report
Share
More Related Content
What's hot
What's hot (20)
Viewers also liked
Similar to My 10 Mobile Automation Questions
Similar to My 10 Mobile Automation Questions (20)
Recently uploaded
Recently uploaded (20)
My 10 Mobile Automation Questions
- 2. Hello! I’m Ran Ben Aharon Automation & Tools lead @ EverythingMe http://ranbena.com @ranbena
- 3. Track Record ▣ Netcraft ▣ Mobile Web dev ▣ Firefox OS ▣ HTML5IL + FEDs
- 4. I LUV ▣ Coding (JS + Python) ▣ Refactoring ▣ Pull Requesting ▣ Solving problems with Tools
- 5. <3
- 6. FIXED A TYPO NOW APP CRASHES IN PRODUCTION
- 8. DREAMED OF MAKING MOBILE APPS NOW I CAN’T SLEEP
- 9. ‘’ In order to move fast, you need good brakes. Joey Simhon EverythingMe
- 11. Community Share, Discuss, Support, Network
- 12. Doing Mobile Automation? 55% Attempted Yep 17%Not yet 28% Your answers
- 13. 10 Questions I needed to answer in order to begin
- 14. 1. Why is it so hard to automate mobile?
- 15. Y u so hard ▣ No widely used solution ▣ Hard to get advice ▣ Not everything fits ▣ Requires lots of attention ▣ Devices fragmentation
- 16. 24 devices Distinct devices 8 sizes Distinct screen dimensions 70% users Latest OS adoption rate
- 17. 18,796 devices Distinct devices (Aug ‘14) 150 sizes Distinct screen dimensions 2.5% users Latest OS adoption rate via OpenSignal
- 18. 2. How is it so different from desktop software automation?
- 19. Different from desktop ▣ Not yet mature ▣ No common open source solution ▣ Device fragmentation ▣ Users interact differently ▣ Device resources are limited ▣ App store limitations ▣ Rapid deployment
- 20. 3. Is there no solid open source solution out there? Why did we have to go and build our own?
- 21. No solid open source solution ▣ Maturity ▣ Single platform* ▣ Single dev language* ▣ No one-stop-shop CI > Testing fw > UI simulation fw > Visibility ▣ Each app has it’s unique needs □ None have ALL features □ Concentrate on UI testing * Excluding Appium
- 22. 4. How do the big boys do it?
- 23. The big boys Ebay Calabash Mobile automation at Ebay Slides Mozilla Robocop for Android Marionette for Firefox OS Automated testing for Firefox OS SoundCloud Robotium for Android Frank for iOS Facebook Selendroid for Android Yelp KIF for iOS Espresso for Android Zappos KIF for iOS
- 24. 5. Must test on real devices? Are emulators and simulators ok too?
- 25. Emulator ▣ Cheap ▣ Fast ▣ Multiple on 1 comp ▣ Sharable ▣ Mock battery, sensors, network ▣ No hw fatigue ▣ Low maintenance Real or Emulator? Real device ▣ Reliable results ▣ Any model available ▣ No need for resource emulation
- 26. 6. Device labs on premise or cloud service?
- 27. Pros ▣ Focus on most relevant devices ▣ No dependency ▣ No queue ▣ Real time ▣ Fast results* On premise
- 28. Pros ▣ Wide range of devices ▣ Scale ▣ Zero maintenance ▣ Real locations* ▣ Real cell networks* ▣ Your test fw + CI ▣ Built-in stress testing* ▣ Manual testing* Device cloud To name a few Appthwack Appurify Experitest Keynote Perfectomobile SauceLabs TestDroid ...
- 29. LessPainful
- 30. LessPainful
- 31. Google I/O ‘14
- 36. Pros ▣ Free ▣ Community ▣ Your code Open source To name a few ▣ Appium ▣ Calabash ▣ Espresso ▣ Frank ▣ KIF ▣ Robotium ▣ Selendroid ▣ MonkeyTalk ▣ ...
- 37. Android ▣ UiAutomator API 16+ (Appium) ▣ UiAutomation API 18+ (Robotium) ▣ Instrumentation API 1+ (Espresso, Selendroid, Calabash) User Action Simulation & Introspection iOS ▣ UiAutomation (Appium, Zucchini) ▣ UiSpec (Kif, Frank)
- 38. Magneto - Command your Droids Magneto
- 39. Pros ▣ Working solution ▣ Robust reports ▣ Script recording ▣ Device cloud integration* Proprietary solutions To name a few ▣ Experitest ▣ Jamo ▣ Keynote ▣ Ranorex ▣ SOASTA ▣ TenKod ▣ ...
- 40. 8. Who writes test scenarios? Should you?
- 41. ▣ Automation engineer? ▣ QA team? ▣ Developer? ▣ Product? Who writes tests?
- 42. 9. Where and when should tests run? Is there a widely adopted methodology?
- 44. Release Flow Stages Dev Review Merge Release Spec RC Build Build Build Build
- 45. 10. How do you maintain result trust?
- 46. ▣ Unreliable results ▣ Not part of release flow ▣ No visibility ▣ Cumbersome tools Distrust and disengagement
- 47. ▣ Reliable results ▣ Integral part of release flow ▣ Full visibility ▣ Delightful tools Trust and cooperation
- 48. UI Testing. Got it. Anything else?
- 49. Much much more. UI testing is just one aspect out of many. This is where it gets even more interesting!
- 50. Much more than UI testing ▣ Broken connectivity simulation ▣ Offline mode ▣ Excessive resource usage ▣ AB testing / feature switches ▣ Non visual events assertions ▣ Jank detection ▣ API mocks ▣ Stats testing ▣ OTA updates ▣ Stress testing ▣ Image recognition ▣ Crash detection ▣ Purchase/advert flow ▣ Sensors ▣ OS event simulation
- 51. More info ▣ GTAC - 2014 vids ▣ SmashingMagazine article ▣ TestDroid blog
- 52. Thanks! Any questions? You can find me at http://ranbena.com ran@ranbena.com @ranbena