LDNSE: Testdroid for Mobile App and Web Testing (London Selenium Meetup)
•
5 likes•2,682 views
This slide deck presents how to use Appium and test automation for mobile web testing on real devices at Testdroid Cloud.
Report
Share
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to LDNSE: Testdroid for Mobile App and Web Testing (London Selenium Meetup)
Similar to LDNSE: Testdroid for Mobile App and Web Testing (London Selenium Meetup) (20)
More from Bitbar
More from Bitbar (8)
Recently uploaded
Recently uploaded (20)
LDNSE: Testdroid for Mobile App and Web Testing (London Selenium Meetup)
- 1. We make mobile developers’ lives easier. Ville-Veikko Helppi Technical Product Manager ville-veikko.helppi@bitbar.com +358400949001 16 April 2015
- 2. TESTDROID AND SOME STATISTICS
- 3. Public Device Cloud on-demand devices (multitenant) Mobile app testing on thousands of real Android and iOS devices hosted by Bitbar Private Device Cloud Reserved Devices Hosted by Bitbar in the US and/or Europe Devices chosen by and reserved only for the Customer On-Premise Device Cloud Automated mobile app testing devices hosted by the customer, usually 30-500 devices 1 Product – 3 Deployment Options Testdroid Cloud Testdroid Enterprise Testdroid PrivateCloud
- 4. Q1'2015 – Test Run Stats
- 6. iOS 8.0 iOS 8.0.2 iOS 8.1 iOS 8.1.1 iOS 8.2 iOS 8 ALL KitKat 4.4 KitKat 4.4.2 KitKat 4.4.3 KitKat 4.4.4 KitKat ALL Lollipop 5.0 Lollipop 5.0.1 Lollipop 5.0.2 Lollipop ALL 0 5 10 15 20 25 30 35 Failed test runs. Percentage (%).
- 10. Android Fragmentation 2014 by Open Signal report July 2014
- 11. How Many Devices is Enough? ~90%market coverage can be achieved with 128 devices ~20%market coverage can be achieved with 12 devices US Market 25 Android devices = ~2/3 market Global Market 60 Android devices = ~1/2 market
- 13. OS versions Chipsets CPU + GPU Tens of OEMs Memory Displays (resolutions, physical hw) OEM mods Other hardware (connectivity calibration) Relation to other software Where Test Automation Can Help
- 14. Correct behaviour across platforms and browsers Integration with web back-ends Typically need to fully utilize HW (CPU+GPU) Resource (e.g. battery) consumption OpenGL ES 2/3 Functionality and usability Screen orientations, connectivity, user profiles Robustness Robustness and security! Brand Compliances, verification with back-ends and data Different Mobile 'App Verticals'
- 15. Manual Testing is not Agile
- 16. Test Automation is Agile
- 17. Why Real Devices Are Must-to-Have • Emulators cannot help you to test... • User Experience • Usability • Hardware • Software • Infrastructure 0 % = the percentage of your app users that use emulator to run your app!
- 18. Manual vs. Automation Manual Automation
- 19. Manual vs. Automation Smaller coverage, More money burnt & time wasted, Error-prone Manual Automation Large coverage, quickly completed, Less money & time wasted, Exact results.
- 20. Different Ways to Automate Testing Automatic test exercisersRecord and PlaybackHand written test scripts Benefits: Accurate, specific to your testing needs, plenty of options with frameworks, tools Fast to create, accurate, not as sensitive to human-errors as hand-written tests, tools avail’ty Fastest & extremely automated, excellent for smoke testing/quick testing, availability Tradeoffs: Takes a lot of time, ties resources to write test cases/scripts, error- prone (humans) Compelling Recorder+Playback tools available for only few test automation frameworks Not accurate as real test cases
- 21. Categorizing the Testing • Functionality of the "game-play", featuresFunctional Testing • Done always when new features / regressions are includedRegression Testing • How game runs on different configurationsCompatibility Testing • Different languages, geo-focused materialsLocalization Testing • Endurance test to determine if system can handle the loadSoak Testing • Measures the capacity of the systemStress Testing • Simplest form of performance testing, measures how system handles loadsLoad Testing • Isolation of the environment (e.g. from network) to see how game worksHermetic Testing Feature- based testing Performance testing End-user testing
- 23. Family Tree of Android Test Automation Frameworks JUnit Android Instrumentation Framework Robotium Espresso and Espresso v2 uiautomator Appium ExtSolo Calabash
- 24. What Framework Works You The Best? Robotium uiautomator Espresso Appium Calabash Android Yes Yes Yes Yes Yes iOS No No No Yes Yes Mobile web Yes (Android) Limited to x.y clicks No Yes (Android & iOS) Yes (Android) Scripting Language Java Java Java Almost any Ruby Test creation tools Testdroid Recorder UI Automator viewer Hierarchy Viewer Appium.app CLI Supported API levels All 16 => 8, 10, 15- All All Community Contributors Google Google Active Pretty quiet
- 25. Client Side Appium at Testdroid CloudTest Script Test Case Desired Capabilities { “device”: “Android”, “app”: “/Users/user/ApiDemos.apk” “app-package”: “com.example.android.apis” “app-activity”: “.ApiDemos” }
- 26. Test Script Test Case Desired Capabilities WebDriver http://localhost:4723/wd/hub Appium Server 4723 Device Localhost (DesiredCaps)
- 27. http://localhost:4723/wd/hub *Testdroid Caps http://appium.testdroid.com/wd/hub (DesiredCaps) Test Script Test Case Desired Capabilities WebDriver From Localhost to Testdroid
- 28. Client Side Execution Go to cloud.testdroid.com
- 29. Client Side Execution Get a Device Name Go to cloud.testdroid.com
- 30. Client Side Execution Add Testdroid Desired Caps to test script { “testdroid_username”: “user@domain.com”, “testdroid_password”: “p4s$w0rd”, “testdroid_project”: “My First Project”, “testdroid_testrun”: “Test 1”, “testdroid_device”: “iPad Mini 7.0.4 A1432”, “testdroid_app”: “http://domain.com/app_v1.ipa” . . “app”: “com.bitbar.testdroid.BitbarIOSSample” } Get a Device Name Go to cloud.testdroid.com
- 31. Client Side Execution driver = webdriver.Remote("http://appium.testdroid.com/wd/hub", desired_caps); Point the Webdriver to http://appium.testdroid.com/wd/hub Add Testdroid Desired Caps to test script Get a Device Name Go to cloud.testdroid.com
- 32. Client Side Execution Run the Test ScriptGet Results from Testdroid Cloud Point the Webdriver to http://appium.testdroid.com/wd/hub Add Testdroid Desired Caps to test script Get a Device Name Go to cloud.testdroid.com
- 33. Client Side Execution Pull the Results from the Result URL driver = webdriver.Remote("http://appium.testdroid.com/wd/hub", desired_caps); Run the Test ScriptGet Results from Testdroid Cloud Point the Webdriver to http://appium.testdroid.com/wd/hub Add Testdroid Desired Caps to test script Get a Device Name Go to cloud.testdroid.com
- 34. Multiple Devices – Client Side python testscript.py –device <devicename>1 python testscript.py –device <devicename>2 python testscript.py –device <devicename>n Test Script Test Cases Instigator Script deviceArray= [ “iPad 4 6.0.1 A1458”, “iPad mini 7.0.4 A1432”, . . . “iPhone 4S 6.1.3 A1387”, ]
- 35. 21 3 4 5 6 7 WebDriver Session Request @http://appium.testdroid.com/wd/hub WebDriver Session Response Testrun Configure Project Appium Ready Wait for Device to Become Available Device 1 Device 2 Device 3 Session Map Proxy Device Cluster Start Appium Desired Capabilities, .apk / .ipa SessionID SessionID Sessionid Test Script Appium Broker Client Side – Behind the Scenes
- 36. Introducing "Server Side Appium"
- 37. Demonstration Appium with Real Devices using Image Recognition for Hill Climb Racing
- 38. Setup • Using real Android devices at Testdroid Cloud • Parallel test runs without a need to configure desired capabilities • Device groups (= set of devices used for runs) can be manually created and configured
- 39. File Structure • pom.xml (maven) • testdroid.properties (overwritten after submitted to Cloud) • run-test.sh (shell script for execution) • image files
- 44. 15 seconds
- 46. Image Recognition • Resolution agnostic implementation • Identifies stretched and rotated images as well
- 47. Let's See How it Works!
- 48. THANK YOU! More information about frameworks, devices and testing at www.testdroid.com