SlideShare a Scribd company logo
© Utopia Solutions
Extending Test Automation To Mobile Applications
Lee Barnes, CTO
Utopia Solutions
© Utopia Solutions
Agenda
•What is Successful MTA?
•How Do We Achieve It?
•Getting Started
•Visual Based Tools
•Object Based Tools
•Evaluating Your Apps
•Tool Evaluation
•Summary & Questions
Mobile Test Automation (MTA)
© Utopia Solutions
SECTION 1
What Is Successful Mobile Test Automation?
© Utopia Solutions4
WIFI / Carrier Networks
Mobile Test Automation – Goal
Automation
Framework
Test Cases
Backend System
Single set of test
cases executing
across a diverse set
of real devices

Recommended for you

Selenium training
Selenium trainingSelenium training
Selenium training

Selenium is an open source tool used for automating web application testing. It was created in 2004 by Jason Huggins and supports recording and playback of test cases in browsers like Firefox. Selenium has four main components - Selenium IDE for recording and playback of tests, Selenium Remote Control for running tests on multiple browsers, Selenium WebDriver for direct browser control, and Selenium Grid for parallel testing on different machines. The latest version of Selenium is 3.0.

software developmentsoftware testingselenium
How to Reliably Measure and Optimize Graphics Performance of Your Android Games
How to Reliably Measure and Optimize Graphics Performance of Your Android GamesHow to Reliably Measure and Optimize Graphics Performance of Your Android Games
How to Reliably Measure and Optimize Graphics Performance of Your Android Games

Watch a live presentation at http://offer.bitbar.com/measure-and-optimize-graphics-performance-of-android-games Good graphics performance is closely related to fantastic user experience. That’s why we’ve integrated the unique GameBench profiling tool into Testdroid Cloud. This is a powerful resource for any developer who wants to spot performance bottlenecks and optimize their games across an array of Android devices with different silicon, screen sizes, memory, and many other variances in software and hardware. Stay tuned and join our upcoming webinars at http://bitbar.com/testing/webinars/

testdroidandroid testinggraphics optimization
Automation Testing With Appium
Automation Testing With AppiumAutomation Testing With Appium
Automation Testing With Appium

The slides cover the introduction to Appium, why is Appium used, Basic concepts, and terminologies used in Appium, Drivers supported by Appium.

appiumautomation with appiumknolx
© Utopia Solutions5
Test Automation Success Criteria
• Issue detection and recovery
• Accurate verification
• Unattended execution
Reliable
• Minimum sensitivity to application and test
case changes
• Test case definition separate from
automation code
Maintainable
• Test coverage expanded efficiently
• Automated test cases created by non-
automation resources
Scalable
© Utopia Solutions
SECTION 2
How Do We Achieve These Goals?
© Utopia Solutions7
Test Automation Approach
Mobile Test Automation Framework
Automated Test Cases Test Results
Test Definition
Test Execution
Non-technical / Test Analysts & Subject Matter Experts
Technical / Automation Engineers
© Utopia Solutions
Mobile Test Automation Framework
Business / Application Testing Components
Login
GoTo Stocks
GoTo News
Add Stock
Edit Stock
Remove
Stock
Logout
Get Position
Verify
Position
…
…
Object Interaction Methods
Object Exists
Set Value
Get Value
Select Item
Touch
Touch and
Hold
Get Image
Get Text
Send Key
Test Case
Login GoTo Stocks Add Stock
Verify
Position
Logout
MobileAutomationFramework

Recommended for you

Cross platform test automation using Appium
Cross platform test automation using AppiumCross platform test automation using Appium
Cross platform test automation using Appium

We develop almost identical apps for both Android and iOS. Maintaining separate test suites becomes an overhead over a period of time as the test suites begin to grow. We hare now gradually moving our test infrastructure to Appium so that we can have a single test repo which is easy to maintain.

cross-platformmobilebdd
Continuous testing on emulators using the espresso test automation framework
Continuous testing on emulators using the espresso test automation frameworkContinuous testing on emulators using the espresso test automation framework
Continuous testing on emulators using the espresso test automation framework

Continuous testing for mobile requires that dev teams have an environment that can easily scale. Although real device testing is a must, using emulators early in the development cycle gives teams a cost-effective and quick alternative for test automation in the cloud. This webinar will explore how to implement the Espresso, a widely used framework for testing mobile apps for the Android OS, on emulators. Espresso provides APIs for writing UI tests to simulate user interactions within a single target app.

android app testingespressomobile testing with emulators
Mobile Application Testing
Mobile Application Testing Mobile Application Testing
Mobile Application Testing

This document discusses mobile application testing and automation. It covers: - The types of mobile application testing including functionality, usability, and consistency testing as well as testing on emulators and different devices. - The challenges of mobile app testing like different devices, operating systems, and networks. - Popular mobile platforms and types of mobile apps. - The approach to mobile app testing including installation/uninstallation, workflows, performance, localization, and device interactions. - Types of mobile app testing like UI, compatibility, interruptions, and security testing. - The importance and benefits of automation for mobile app testing. - Popular mobile automation tools like MonkeyRunner, UIAutomator,

software testingsoftware development
© Utopia Solutions9
Test Automation Challenges
• Defining appropriate scope
• Integrating with SDLC
• Addressing technical issues
• Reliability, maintainability, scalability
General
• Platform diversity
• Device diversity
• Tool diversity
• Tool immaturity
• Lack of skilled resources
• Rate of technology change
• Rapid development cycles
Mobile
Specific
© Utopia Solutions
SECTION 3
Let’s Get Started
© Utopia Solutions11
Mobile Test Automation Tool Categories
Native Platform
Frameworks
Multi-Platform Tools
Visual Based Object Based
• Provided by mobile platform
vendor as part of SDK
• Typically interact with the
application at the UI object
level
• Interacts with devices
using visual methods (text
and image recognition)
• Interacts directly with
application UI objects using
native methods and
properties
• Object based interaction
with app UI controls
• Greatest degree of support
for native UI objects
• Control of device settings
• Supports multiple
platforms
• Ability to execute tests
across multiple devices
• Interact with entire device
• Supports multiple platforms
• Ability to execute tests
across multiple platforms
• Interacts with apps at the
object level for greater
reliability
• Limited to a single platform
• Requires code level access
to the app
• May be limited to simulator
(e.g. Blackberry)
• Generally less reliable
• No access to object
properties
• May require jail-breaking /
rooting
• Interaction is limited to app
under test
• Requires app to be
instrumented
• Android UI Automator
• iOS UI Automation
• SeeTest (Experitest)
• eggPlant (TestPlant)
• Perfecto Mobile
• Zap-Fix (Zap Tech.)
• Trust (Mobile Labs)
• TouchTest (SOASTA)
• M-eux Test (Jamo Solutions)
DescriptionProsConsExample
© Utopia Solutions12
Start With a Plan
Scope
OS/Device
Selection
Test Case
Selection
Non-Testing
Tasks
Technical
Challenges
Platform/Device
Diversity
Object
Identification
Process
Integration
Test Suite
Maintenance
Continuous
Integration

Recommended for you

Using Selenium To Test Mobile? Meet Appium!
Using Selenium To Test Mobile? Meet Appium!Using Selenium To Test Mobile? Meet Appium!
Using Selenium To Test Mobile? Meet Appium!

Selenium is an industry standard for web testing. Most of test automation engineers are well versed with this tool, and even use its broad capabilities to execute some mobile tests. Inspired by Selenium and built based on testing approach Appium was created. It is an open-source tool for automating native, mobile web, and hybrid applications on iOS and Android platforms. In this how-to webinar, we will show you how you can utilize your Selenium skills to scale your mobile testing with Appium. Join Giovanni Rago, Customer Success Manager at Sauce Labs, in this event that will give you new insights into the way you approach mobile testing. Here are the main takeaways: -- Difference between Selenium and Appium -- Which Selenium skills can be used when testing in Appium -- What’s the best testing strategy for mobile -- How much more there is to learn about Appium

appiumseleniumsauce labs
Everything You Need to Know About Testing Foldable Phones
Everything You Need to Know About Testing Foldable PhonesEverything You Need to Know About Testing Foldable Phones
Everything You Need to Know About Testing Foldable Phones

Foldable phones are here —and they’ll soon be in the hands of consumers. In fact, three of the leading Android vendors have already launched such devices — including Samsung, LG, and Huawei. And with this new generation of devices, DevOps teams will need to ensure that their apps are ready. Learn how foldable phones will change the testing process, as well as: - What the foldable phone landscape looks like. - Key features and pitfalls of testing apps on foldable phones. - Recommended test scenarios for foldable phones. - The architectural changes needed to make your apps ready for foldable smartphones.

devopsmobile testingmobile software testing
The ultimate guide to mobile app testing with appium
The ultimate guide to mobile app testing with appiumThe ultimate guide to mobile app testing with appium
The ultimate guide to mobile app testing with appium

Appium is an open-source test automation tool that allows developers to test mobile apps across platforms. It works by communicating with the mobile device and app to run automated tests scripts written in various programming languages. Appium tests can check that the user experience is seamless across different devices. The tool was initially created to test iOS apps but now supports Android as well. It has become popular for testing apps with user interfaces like games and banking apps across both iOS and Android devices.

best appium testing
© Utopia Solutions13
Automation Scope Selection
• Frequently tested
• Predictable results
• Easy to automate
• Manually tedious
“Traditional” rules still apply
• Tests executed across all target devices
• Avoid apps and tests that face many mobile
automation challenges (to be discussed)
Mobile specific
© Utopia Solutions
SECTION 4
Visual-based Tool Approach
© Utopia Solutions15
Visual Based Identification
Label
Label + Boundary
Image Recognition
Label Offset
Multi-Label Offset
© Utopia Solutions16
Cross Platform Issues – Inconsistent UI/UX
Button
Image/Position
Screen
Titles
Button
Labels

Recommended for you

Mobile test automation perfecto star east
Mobile test automation perfecto star eastMobile test automation perfecto star east
Mobile test automation perfecto star east

This document provides an overview of how to deliver winning mobile apps through continuous quality. It discusses establishing a mobile QA plan based on continuous quality lab (CQL) principles, including functional and non-functional testing, regression testing, and pillars of a CQL. It also outlines mobile QA best practices such as building a proper test plan, establishing a continuous integration workflow, and implementing a continuous automation regression model. The key is leveraging a hybrid cloud approach using real devices to ensure high quality, velocity, and market responsiveness for mobile apps.

perfecto mobilemobile
My 10 Mobile Automation Questions
My 10 Mobile Automation QuestionsMy 10 Mobile Automation Questions
My 10 Mobile Automation Questions

Once a proud Front End developer, now a total Mobile Automation n00b. I was given the task of rethinking mobile automation at the startup EverythingMe and got familiarized with this exciting new field. In this talk, I present the questions and answers I needed as a newcomer in order to begin the journey.

mobileengineeringtest
TechTalk: Wind Tunnel, Personas, and Testing Real UX
TechTalk: Wind Tunnel, Personas, and Testing Real UXTechTalk: Wind Tunnel, Personas, and Testing Real UX
TechTalk: Wind Tunnel, Personas, and Testing Real UX

Persona-based testing has never been easier Covered in this webinar: - Intro to Perfecto - Digital Challenges - Perfecto CQ Lab and Wind Tunnel™ Overview - Wind Tunnel Deep Dive - How to Add Wind Tunnel to your Test - Demo - Q&A By the end of this webinar, you'll be a master at adding UX to each of your tests!

© Utopia Solutions17
Visual Based Techniques – Complex Interaction
iOS On/Off Switch
Function iosSwitchGetState (Label)
Define OCR parameter ranges
Search for Label on device screen
If Label not found
status = switch not found error
Else
Loop through OCR parameter range
Search for switch state(ON/OFF)
If state found
status = switch state
Else
status = state not found error
End If
End Loop
End If
Return status
End Function
Function iosSwitchSetState (Label, State)
CurrentState = iosSwitchGetState (Label)
If CurrentState = error
status = error
Else
If CurrentState = desired state
status = success
Else
If desired state = ON
Touch right side of switch
Else
Touch left side of switch
End if
End if
status = success
End if
Return status
End Function
Hidden
Complexities
© Utopia Solutions18
Visual Interaction – Reusability?
Maybe…
• Label / object offset
• On / Off reversed
not
• Label offset right
• No On / Off label
© Utopia Solutions19
Object Identification - Defining Master Object Properties
Environment Object Type Attribute Value
iPhone Screen Method Unique Text
iPhone Screen Label Screen name
iPhone Screen Location 20,25:40,200
iPhone Edit Label Object name
iPhone Edit Offset 0,-20
…
• Defines general rules for identifying objects by platform
• Avoids adding all application objects to Object Repository
• One master file per application (typically)
Master Property File
© Utopia Solutions20
Object Identification - Define Object Map
ObjectScreenPlatform
iOS
Login
User
Name
Password
Add
Stock
Symbol
• Contains any screens and objects that don’t meet criteria
defined in Master Property File
• Objects organized hierarchically by platform and screen
UI Object Map

Recommended for you

SaaS Digital Assurance Lab
SaaS Digital Assurance LabSaaS Digital Assurance Lab
SaaS Digital Assurance Lab

This document summarizes SeeTestCloud, a quality assurance platform for mobile apps that provides access to hundreds of remote mobile devices hosted in data centers. Key capabilities include: hosting mobile devices for manual, automation and performance testing; integration with CI/CD tools; and capabilities for test automation, management, debugging and non-functional testing. The platform allows testing on any device or OS and offers enterprise-level security for dedicated, remote device access.

device cloudmobile device cloudmobile qa
iOS Application Testing
iOS Application TestingiOS Application Testing
iOS Application Testing

Mobile Application Testing, iOS Application Testing,iPhone Application Testing,iPad Application Testing,iPod Application Testing,

ios application testingmobile application testingipod application testing
Android & iPhone App Testing
 Android & iPhone App Testing Android & iPhone App Testing
Android & iPhone App Testing

SwaamTech, is an independent QA and Software Testing company helping clients to bring quality in there products. Contact us for testing of your SmartPhone App testing: support@swaam.com

swaamtechsoftware testingandroid
© Utopia Solutions
SECTION 5
Object-based Tool Approach
© Utopia Solutions© Utopia Solutions
Object Based Interaction
Tool Libraries Included
in Project Build
Object Properties
Visible to Tool
Class: UIButton
Label: Save
Class: UITextField
Label: Name
© Utopia Solutions23
Object Based Interaction – Cross Platform Differences
Simple Interaction – “Tap” the Save/Done Button
Class: avwButton
Method: Push
Implementation:
avwWindow(“MyStock”).avwButton(“Save”).Push
Class: iosButton
Method: Tap
Implementation:
iosView(“EditStock”).iosButton(“Done”).Tap
© Utopia Solutions24
Object Identification – Handling Cross Platform
Properties
Class: Button
Name: Save
Create Mapping of Universal Objects to Platform
Specific Objects
Properties
Class: avwButton
Label: Save
Method Name
Touch: Push
Object Path
avwWindow(“MyStock”)
Properties
Class: iosButton
Label: Done
Method Name
Touch: Tap
Object Path
iosView(“EditStock”)
Step 1

Recommended for you

7 use cases of real device testing
7 use cases of real device testing7 use cases of real device testing
7 use cases of real device testing

Real device testing is important for app development as it allows testing across different devices, operating systems, and real-world conditions that emulators cannot simulate. Some benefits of real device testing include effectively identifying bugs specific to device configurations, validating the app's layout and compatibility across various screens, checking network compatibility under different conditions, understanding user experience better than emulators, and stimulating real-world usage. Real device testing helps improve app quality by finding issues users may face.

test on real devices online
[Binh nguyen] Mobile Application Automation Testing iOS and Android
[Binh nguyen] Mobile Application Automation Testing iOS and Android [Binh nguyen] Mobile Application Automation Testing iOS and Android
[Binh nguyen] Mobile Application Automation Testing iOS and Android

Mobile application testing on iOS or Android can be quite a challenge for a Quality Assurance team. One reason is that mobile applications are often required to function on a variety of devices featuring different hardware capabilities, configurations, and specifications. Many different types of testing including automation are required to cover different areas of the application. Many people believe that automated testing may replace manual testing almost entirely. In reality, however, automated testing is still not able to cover all the types of testing that manual testing does. Automation Mobile Testing: framework, tools, some suggested solutions.

mobile testing
Addressing Mobile App Testing Challenges
Addressing Mobile App Testing ChallengesAddressing Mobile App Testing Challenges
Addressing Mobile App Testing Challenges

If the mobile technology train hasn’t arrived at your organization yet, it soon will. Are you ready to jump onboard and face the unique testing challenges presented by mobile applications? In this session, Lee will lead a journey to help you understand where mobile quality is, where it’s going, why it matters to you, and what you can do to help ensure mobile quality in your organization. Lee’s presentation will highlight testing challenges specific to mobile apps and present mobile testing best practices including techniques for leveraging test automation on mobile platforms. You will understand why testing in a mobile environment is different from traditional software testing and learn how to address the unique testing challenges presented by mobile applications. Attend this talk and walk away with a solid mobile testing baseline and best practices for addressing the challenges that lie ahead.

mobile testingmobile app testing
© Utopia Solutions25
Function TouchButton (Name)
Get execution platform
Retrieve platform specific object info
from Object Map
Construct statement to perform operation
Execute operation
Return operation status
End Function
Object Identification – Handling Cross Platform
Create Universal Method for All PlatformsStep 2
Properties
Class: Button
Name: Save
Properties
Class: iosButton
Label: Done
Method Name
Touch: Tap
Object Path
iosView(“EditStock”)
iosView(“EditStock”).iosButton(“Done”).Tap
© Utopia Solutions
SECTION 6
Evaluating Your Apps for Potential Challenges
© Utopia Solutions27
Analyze Business Processes for Differences
Differences
• Menu always visible
on iOS
• Different edit buttons
• Different column
headers
© Utopia Solutions28
Analyze Business Processes for Differences
Differences
• Different screen titles
• Different “add”
buttons
• Different “save”
buttons
• Different background
color on list

Recommended for you

Testing the UI of Mobile Applications
Testing the UI of Mobile ApplicationsTesting the UI of Mobile Applications
Testing the UI of Mobile Applications

Mobile applications and specifically their UI exhibit a set of features that make their design, implementation and testing particularly challenging: the variety of devices with their different characteristics and the quick pace of evolution. As with other types of E2E testing, the fragility of test cases represents an important hurdle to a wider adoption. This lecture will analyze the mobile-specific features of E2E testing; it will provide an overview of the testing technology approaches and the main tools available. A focus on the main causes of test fragility will provide an overview of the main pitfalls and provide a set of guidelines to avoid them. Finally we will focus on the prospective techniques to combine different approach to achieve higher test resilience and reusability. The lecture will provide evidence gathered from large empirical studies of OSS as well as experiment conducted on specific use cases.

mobileandroidsoftware testing
Mobile Automation Meetup Presentation [English]
Mobile Automation Meetup Presentation [English]Mobile Automation Meetup Presentation [English]
Mobile Automation Meetup Presentation [English]

This document summarizes a meetup about mobile automation testing using MonkeyTalk. It discusses the benefits of automation testing, how MonkeyTalk works for both iOS and Android applications, and provides a demo of using MonkeyTalk to automate tests on both platforms. The key points are that MonkeyTalk is a free tool that works across platforms, integrates with continuous integration, and allows scripting and recording of tests directly on devices through an agent.

automationsoftwarequality assurance
Mobile Application Testing
Mobile Application TestingMobile Application Testing
Mobile Application Testing

This document discusses mobile application testing and automation. It covers: - The types of mobile application testing including functionality, usability, and consistency testing. - Challenges of mobile app testing like different devices, platforms, and input methods. - Popular mobile platforms and types of mobile apps and devices. - Approaches to mobile app testing like installation/uninstallation, workflows, performance, localization, and device interactions. - Types of mobile app testing including UI, compatibility, interruptions, usability, and security testing. - The need for automation in mobile testing due to a wide variety of factors. - Popular mobile automation tools like MonkeyRunner, UIAutomator,

© Utopia Solutions29
Analyze Business Processes for Differences
Differences
• Cancel button
embedded in search
field on iOS
• Different “expand”
icons
© Utopia Solutions
SECTION 7
Handling Workflow Differences
© Utopia Solutions31
Handling Workflow Differences – Go To Stocks
Android
1. Tap “MENU”
2. Tap “Stocks”
3. Sync on “My
Stocks” Screen
iOS
1. Tap “My Stocks”
2. Sync on “My
Stocks” Screen
Implement at the Business Process
Component Level
Public Function GoToStocks
Platform = GetPlatform()
Select Case Platform
Case “iOS”
Tap (“MENU”)
Tap (“Stocks”)
status = WaitScreen (“My Stocks”)
Case “Android”
Tap (“My Stocks”)
status = WaitScreen (“My Stocks”)
Case Else
status = INVALID_PLATFORM
End Select
GoToStocks = status
End Function
© Utopia Solutions32
Mobile Test Automation Tool Evaluation
Top Ten Questions to Ask Tool Vendors
• Is your tool visual or object based?
• Does your tool require jailbreaking or rooting?
• How quickly do you support new mobile OS releases?
• How strong is the company (e.g. sales, customers, years, etc.)?
• How do I execute a single test across multiple platforms with
your tool?
• Can (or must) your tool integrate with other scripting engines?
• Do your support our mobile development platform?
• Can your tool interact with devices remotely?
• Does your tool integrate with our test management solution?
• Will you assist me with a proof of concept?

Recommended for you

Mobile Application testing
Mobile Application testingMobile Application testing
Mobile Application testing

This presentation represents the Mobile Application testing fundamentals. This presentation will describe How Mobile testing is different from Web testing. It will give you brief information about different mobile application techniques which needs to be taken care while testing Mobile Applications.

mobile appsmobile application developmentsoftware testing
LDNSE: Testdroid for Mobile App and Web Testing (London Selenium Meetup)
LDNSE: Testdroid for Mobile App and Web Testing (London Selenium Meetup)LDNSE: Testdroid for Mobile App and Web Testing (London Selenium Meetup)
LDNSE: Testdroid for Mobile App and Web Testing (London Selenium Meetup)

This slide deck presents how to use Appium and test automation for mobile web testing on real devices at Testdroid Cloud.

androidtest automationappium
Mobile applications testing (challenges, tools & techniques)
Mobile applications testing (challenges, tools & techniques)Mobile applications testing (challenges, tools & techniques)
Mobile applications testing (challenges, tools & techniques)

Device Fragmentation is a Big Challenge Devices Vary in Screen Size, Memory, Processing Power, Hardware Features etc. Apple iPhone is Least Fragmented among All Mobile Platforms Testing on All Target Handset/Devices Almost Impractical if Number of Target Handsets is Large Testing on All Target Operator Networks Almost Impractical if Number of Target Operators is Large Network Operator may Impose Certain Constraints

qatestingandroid
© Utopia Solutions33
Key Summary Points
• Set appropriate automation goals
• Start with a plan
• Scope
• Technical Challenges
• Process Integration
• Understand how visual/object based tools work
• Analyze your app for platform differences
• Address differences at appropriate level in your
framework
• Evaluate tools carefully
• Start small and expand over time
© Utopia Solutions34
Questions…
…. and Answers!
Direct future questions to:
Lee Barnes
Founder and CTO
Utopia Solutions, Inc.
Email: lee.barnes@utopiasolutions.com
Twitter: twitter.com/USI_LeeBarnes
LinkedIn: linkedin.com/in/leebarnes
Blog: www.RaiseYourMQ.com

More Related Content

What's hot

Everything You Need To Know about Appium and Selenium
Everything You Need To Know about Appium and SeleniumEverything You Need To Know about Appium and Selenium
Everything You Need To Know about Appium and Selenium
Lizzy Guido (she/her)
 
Meetup kickoff slides - Perfecto Mobile, September 2014
Meetup kickoff slides - Perfecto Mobile, September 2014Meetup kickoff slides - Perfecto Mobile, September 2014
Meetup kickoff slides - Perfecto Mobile, September 2014
Perfecto Mobile
 
Parallel Test Runs with Appium on Real Mobile Devices – Hands-on Webinar
Parallel Test Runs with Appium on Real Mobile Devices – Hands-on WebinarParallel Test Runs with Appium on Real Mobile Devices – Hands-on Webinar
Parallel Test Runs with Appium on Real Mobile Devices – Hands-on Webinar
Bitbar
 
Selenium training
Selenium trainingSelenium training
Selenium training
Shivaraj R
 
How to Reliably Measure and Optimize Graphics Performance of Your Android Games
How to Reliably Measure and Optimize Graphics Performance of Your Android GamesHow to Reliably Measure and Optimize Graphics Performance of Your Android Games
How to Reliably Measure and Optimize Graphics Performance of Your Android Games
Bitbar
 
Automation Testing With Appium
Automation Testing With AppiumAutomation Testing With Appium
Automation Testing With Appium
Knoldus Inc.
 
Cross platform test automation using Appium
Cross platform test automation using AppiumCross platform test automation using Appium
Cross platform test automation using Appium
Jatin Bhasin
 
Continuous testing on emulators using the espresso test automation framework
Continuous testing on emulators using the espresso test automation frameworkContinuous testing on emulators using the espresso test automation framework
Continuous testing on emulators using the espresso test automation framework
Sauce Labs
 
Mobile Application Testing
Mobile Application Testing Mobile Application Testing
Mobile Application Testing
Shivaraj R
 
Using Selenium To Test Mobile? Meet Appium!
Using Selenium To Test Mobile? Meet Appium!Using Selenium To Test Mobile? Meet Appium!
Using Selenium To Test Mobile? Meet Appium!
Sauce Labs
 
Everything You Need to Know About Testing Foldable Phones
Everything You Need to Know About Testing Foldable PhonesEverything You Need to Know About Testing Foldable Phones
Everything You Need to Know About Testing Foldable Phones
Perfecto by Perforce
 
The ultimate guide to mobile app testing with appium
The ultimate guide to mobile app testing with appiumThe ultimate guide to mobile app testing with appium
The ultimate guide to mobile app testing with appium
headspin2
 
Mobile test automation perfecto star east
Mobile test automation perfecto star eastMobile test automation perfecto star east
Mobile test automation perfecto star east
Perfecto Mobile
 
My 10 Mobile Automation Questions
My 10 Mobile Automation QuestionsMy 10 Mobile Automation Questions
My 10 Mobile Automation Questions
Ran Byron
 
TechTalk: Wind Tunnel, Personas, and Testing Real UX
TechTalk: Wind Tunnel, Personas, and Testing Real UXTechTalk: Wind Tunnel, Personas, and Testing Real UX
TechTalk: Wind Tunnel, Personas, and Testing Real UX
Lizzy Guido (she/her)
 
SaaS Digital Assurance Lab
SaaS Digital Assurance LabSaaS Digital Assurance Lab
SaaS Digital Assurance Lab
Experitest
 
iOS Application Testing
iOS Application TestingiOS Application Testing
iOS Application Testing
Mreetyunjaya Daas
 
Android & iPhone App Testing
 Android & iPhone App Testing Android & iPhone App Testing
Android & iPhone App Testing
SWAAM Tech
 
7 use cases of real device testing
7 use cases of real device testing7 use cases of real device testing
7 use cases of real device testing
headspin2
 
[Binh nguyen] Mobile Application Automation Testing iOS and Android
[Binh nguyen] Mobile Application Automation Testing iOS and Android [Binh nguyen] Mobile Application Automation Testing iOS and Android
[Binh nguyen] Mobile Application Automation Testing iOS and Android
Ho Chi Minh City Software Testing Club
 

What's hot (20)

Everything You Need To Know about Appium and Selenium
Everything You Need To Know about Appium and SeleniumEverything You Need To Know about Appium and Selenium
Everything You Need To Know about Appium and Selenium
 
Meetup kickoff slides - Perfecto Mobile, September 2014
Meetup kickoff slides - Perfecto Mobile, September 2014Meetup kickoff slides - Perfecto Mobile, September 2014
Meetup kickoff slides - Perfecto Mobile, September 2014
 
Parallel Test Runs with Appium on Real Mobile Devices – Hands-on Webinar
Parallel Test Runs with Appium on Real Mobile Devices – Hands-on WebinarParallel Test Runs with Appium on Real Mobile Devices – Hands-on Webinar
Parallel Test Runs with Appium on Real Mobile Devices – Hands-on Webinar
 
Selenium training
Selenium trainingSelenium training
Selenium training
 
How to Reliably Measure and Optimize Graphics Performance of Your Android Games
How to Reliably Measure and Optimize Graphics Performance of Your Android GamesHow to Reliably Measure and Optimize Graphics Performance of Your Android Games
How to Reliably Measure and Optimize Graphics Performance of Your Android Games
 
Automation Testing With Appium
Automation Testing With AppiumAutomation Testing With Appium
Automation Testing With Appium
 
Cross platform test automation using Appium
Cross platform test automation using AppiumCross platform test automation using Appium
Cross platform test automation using Appium
 
Continuous testing on emulators using the espresso test automation framework
Continuous testing on emulators using the espresso test automation frameworkContinuous testing on emulators using the espresso test automation framework
Continuous testing on emulators using the espresso test automation framework
 
Mobile Application Testing
Mobile Application Testing Mobile Application Testing
Mobile Application Testing
 
Using Selenium To Test Mobile? Meet Appium!
Using Selenium To Test Mobile? Meet Appium!Using Selenium To Test Mobile? Meet Appium!
Using Selenium To Test Mobile? Meet Appium!
 
Everything You Need to Know About Testing Foldable Phones
Everything You Need to Know About Testing Foldable PhonesEverything You Need to Know About Testing Foldable Phones
Everything You Need to Know About Testing Foldable Phones
 
The ultimate guide to mobile app testing with appium
The ultimate guide to mobile app testing with appiumThe ultimate guide to mobile app testing with appium
The ultimate guide to mobile app testing with appium
 
Mobile test automation perfecto star east
Mobile test automation perfecto star eastMobile test automation perfecto star east
Mobile test automation perfecto star east
 
My 10 Mobile Automation Questions
My 10 Mobile Automation QuestionsMy 10 Mobile Automation Questions
My 10 Mobile Automation Questions
 
TechTalk: Wind Tunnel, Personas, and Testing Real UX
TechTalk: Wind Tunnel, Personas, and Testing Real UXTechTalk: Wind Tunnel, Personas, and Testing Real UX
TechTalk: Wind Tunnel, Personas, and Testing Real UX
 
SaaS Digital Assurance Lab
SaaS Digital Assurance LabSaaS Digital Assurance Lab
SaaS Digital Assurance Lab
 
iOS Application Testing
iOS Application TestingiOS Application Testing
iOS Application Testing
 
Android & iPhone App Testing
 Android & iPhone App Testing Android & iPhone App Testing
Android & iPhone App Testing
 
7 use cases of real device testing
7 use cases of real device testing7 use cases of real device testing
7 use cases of real device testing
 
[Binh nguyen] Mobile Application Automation Testing iOS and Android
[Binh nguyen] Mobile Application Automation Testing iOS and Android [Binh nguyen] Mobile Application Automation Testing iOS and Android
[Binh nguyen] Mobile Application Automation Testing iOS and Android
 

Similar to Mobile Test Automation

Addressing Mobile App Testing Challenges
Addressing Mobile App Testing ChallengesAddressing Mobile App Testing Challenges
Addressing Mobile App Testing Challenges
Lee Barnes
 
Testing the UI of Mobile Applications
Testing the UI of Mobile ApplicationsTesting the UI of Mobile Applications
Testing the UI of Mobile Applications
Marco Torchiano
 
Mobile Automation Meetup Presentation [English]
Mobile Automation Meetup Presentation [English]Mobile Automation Meetup Presentation [English]
Mobile Automation Meetup Presentation [English]
Maria Carcar
 
Mobile Application Testing
Mobile Application TestingMobile Application Testing
Mobile Application Testing
Sun Technlogies
 
Mobile Application testing
Mobile Application testingMobile Application testing
Mobile Application testing
Mukta Gupta
 
LDNSE: Testdroid for Mobile App and Web Testing (London Selenium Meetup)
LDNSE: Testdroid for Mobile App and Web Testing (London Selenium Meetup)LDNSE: Testdroid for Mobile App and Web Testing (London Selenium Meetup)
LDNSE: Testdroid for Mobile App and Web Testing (London Selenium Meetup)
Bitbar
 
Mobile applications testing (challenges, tools & techniques)
Mobile applications testing (challenges, tools & techniques)Mobile applications testing (challenges, tools & techniques)
Mobile applications testing (challenges, tools & techniques)
Rakesh Jha
 
Raleigh User Forum Presentation
Raleigh User Forum PresentationRaleigh User Forum Presentation
Raleigh User Forum Presentation
Perfecto Mobile
 
Introducing the Applitools Self Healing Execution Cloud.pdf
Introducing the Applitools Self Healing Execution Cloud.pdfIntroducing the Applitools Self Healing Execution Cloud.pdf
Introducing the Applitools Self Healing Execution Cloud.pdf
Applitools
 
Shift Left Mobile Application Testing
Shift Left Mobile Application TestingShift Left Mobile Application Testing
Shift Left Mobile Application Testing
Josiah Renaudin
 
Mobile app testing
Mobile app testingMobile app testing
Mobile app testing
BugRaptors
 
Appium vs. Appium with Perfecto
Appium vs. Appium with PerfectoAppium vs. Appium with Perfecto
Appium vs. Appium with Perfecto
Lizzy Guido (she/her)
 
Appium & Selenium Alone vs Appium & Selenium with Perfecto
Appium & Selenium Alone vs Appium & Selenium with PerfectoAppium & Selenium Alone vs Appium & Selenium with Perfecto
Appium & Selenium Alone vs Appium & Selenium with Perfecto
Lizzy Guido (she/her)
 
Velocity Conference: Increasing Speed To Market In Mobile Development Through...
Velocity Conference: Increasing Speed To Market In Mobile Development Through...Velocity Conference: Increasing Speed To Market In Mobile Development Through...
Velocity Conference: Increasing Speed To Market In Mobile Development Through...
Intuit Inc.
 
Automation Proposal_V1.0
Automation Proposal_V1.0Automation Proposal_V1.0
Automation Proposal_V1.0
Dao Nhỏ
 
Mobile testing and its part in testing tools
Mobile testing and its part in testing toolsMobile testing and its part in testing tools
Mobile testing and its part in testing tools
Kari Kakkonen
 
Appium vs Appium with Perfecto
Appium vs Appium with PerfectoAppium vs Appium with Perfecto
Appium vs Appium with Perfecto
Lizzy Guido (she/her)
 
Velocity2013 mobile ci_intuit
Velocity2013 mobile ci_intuitVelocity2013 mobile ci_intuit
Velocity2013 mobile ci_intuit
Tina Su
 
Velocity2013 mobile ci_intuit
Velocity2013 mobile ci_intuitVelocity2013 mobile ci_intuit
Velocity2013 mobile ci_intuit
Tina Su
 
Speed to Market in Mobile Development
Speed to Market in Mobile DevelopmentSpeed to Market in Mobile Development
Speed to Market in Mobile Development
Tina Su
 

Similar to Mobile Test Automation (20)

Addressing Mobile App Testing Challenges
Addressing Mobile App Testing ChallengesAddressing Mobile App Testing Challenges
Addressing Mobile App Testing Challenges
 
Testing the UI of Mobile Applications
Testing the UI of Mobile ApplicationsTesting the UI of Mobile Applications
Testing the UI of Mobile Applications
 
Mobile Automation Meetup Presentation [English]
Mobile Automation Meetup Presentation [English]Mobile Automation Meetup Presentation [English]
Mobile Automation Meetup Presentation [English]
 
Mobile Application Testing
Mobile Application TestingMobile Application Testing
Mobile Application Testing
 
Mobile Application testing
Mobile Application testingMobile Application testing
Mobile Application testing
 
LDNSE: Testdroid for Mobile App and Web Testing (London Selenium Meetup)
LDNSE: Testdroid for Mobile App and Web Testing (London Selenium Meetup)LDNSE: Testdroid for Mobile App and Web Testing (London Selenium Meetup)
LDNSE: Testdroid for Mobile App and Web Testing (London Selenium Meetup)
 
Mobile applications testing (challenges, tools & techniques)
Mobile applications testing (challenges, tools & techniques)Mobile applications testing (challenges, tools & techniques)
Mobile applications testing (challenges, tools & techniques)
 
Raleigh User Forum Presentation
Raleigh User Forum PresentationRaleigh User Forum Presentation
Raleigh User Forum Presentation
 
Introducing the Applitools Self Healing Execution Cloud.pdf
Introducing the Applitools Self Healing Execution Cloud.pdfIntroducing the Applitools Self Healing Execution Cloud.pdf
Introducing the Applitools Self Healing Execution Cloud.pdf
 
Shift Left Mobile Application Testing
Shift Left Mobile Application TestingShift Left Mobile Application Testing
Shift Left Mobile Application Testing
 
Mobile app testing
Mobile app testingMobile app testing
Mobile app testing
 
Appium vs. Appium with Perfecto
Appium vs. Appium with PerfectoAppium vs. Appium with Perfecto
Appium vs. Appium with Perfecto
 
Appium & Selenium Alone vs Appium & Selenium with Perfecto
Appium & Selenium Alone vs Appium & Selenium with PerfectoAppium & Selenium Alone vs Appium & Selenium with Perfecto
Appium & Selenium Alone vs Appium & Selenium with Perfecto
 
Velocity Conference: Increasing Speed To Market In Mobile Development Through...
Velocity Conference: Increasing Speed To Market In Mobile Development Through...Velocity Conference: Increasing Speed To Market In Mobile Development Through...
Velocity Conference: Increasing Speed To Market In Mobile Development Through...
 
Automation Proposal_V1.0
Automation Proposal_V1.0Automation Proposal_V1.0
Automation Proposal_V1.0
 
Mobile testing and its part in testing tools
Mobile testing and its part in testing toolsMobile testing and its part in testing tools
Mobile testing and its part in testing tools
 
Appium vs Appium with Perfecto
Appium vs Appium with PerfectoAppium vs Appium with Perfecto
Appium vs Appium with Perfecto
 
Velocity2013 mobile ci_intuit
Velocity2013 mobile ci_intuitVelocity2013 mobile ci_intuit
Velocity2013 mobile ci_intuit
 
Velocity2013 mobile ci_intuit
Velocity2013 mobile ci_intuitVelocity2013 mobile ci_intuit
Velocity2013 mobile ci_intuit
 
Speed to Market in Mobile Development
Speed to Market in Mobile DevelopmentSpeed to Market in Mobile Development
Speed to Market in Mobile Development
 

Mobile Test Automation

  • 1. © Utopia Solutions Extending Test Automation To Mobile Applications Lee Barnes, CTO Utopia Solutions
  • 2. © Utopia Solutions Agenda •What is Successful MTA? •How Do We Achieve It? •Getting Started •Visual Based Tools •Object Based Tools •Evaluating Your Apps •Tool Evaluation •Summary & Questions Mobile Test Automation (MTA)
  • 3. © Utopia Solutions SECTION 1 What Is Successful Mobile Test Automation?
  • 4. © Utopia Solutions4 WIFI / Carrier Networks Mobile Test Automation – Goal Automation Framework Test Cases Backend System Single set of test cases executing across a diverse set of real devices
  • 5. © Utopia Solutions5 Test Automation Success Criteria • Issue detection and recovery • Accurate verification • Unattended execution Reliable • Minimum sensitivity to application and test case changes • Test case definition separate from automation code Maintainable • Test coverage expanded efficiently • Automated test cases created by non- automation resources Scalable
  • 6. © Utopia Solutions SECTION 2 How Do We Achieve These Goals?
  • 7. © Utopia Solutions7 Test Automation Approach Mobile Test Automation Framework Automated Test Cases Test Results Test Definition Test Execution Non-technical / Test Analysts & Subject Matter Experts Technical / Automation Engineers
  • 8. © Utopia Solutions Mobile Test Automation Framework Business / Application Testing Components Login GoTo Stocks GoTo News Add Stock Edit Stock Remove Stock Logout Get Position Verify Position … … Object Interaction Methods Object Exists Set Value Get Value Select Item Touch Touch and Hold Get Image Get Text Send Key Test Case Login GoTo Stocks Add Stock Verify Position Logout MobileAutomationFramework
  • 9. © Utopia Solutions9 Test Automation Challenges • Defining appropriate scope • Integrating with SDLC • Addressing technical issues • Reliability, maintainability, scalability General • Platform diversity • Device diversity • Tool diversity • Tool immaturity • Lack of skilled resources • Rate of technology change • Rapid development cycles Mobile Specific
  • 10. © Utopia Solutions SECTION 3 Let’s Get Started
  • 11. © Utopia Solutions11 Mobile Test Automation Tool Categories Native Platform Frameworks Multi-Platform Tools Visual Based Object Based • Provided by mobile platform vendor as part of SDK • Typically interact with the application at the UI object level • Interacts with devices using visual methods (text and image recognition) • Interacts directly with application UI objects using native methods and properties • Object based interaction with app UI controls • Greatest degree of support for native UI objects • Control of device settings • Supports multiple platforms • Ability to execute tests across multiple devices • Interact with entire device • Supports multiple platforms • Ability to execute tests across multiple platforms • Interacts with apps at the object level for greater reliability • Limited to a single platform • Requires code level access to the app • May be limited to simulator (e.g. Blackberry) • Generally less reliable • No access to object properties • May require jail-breaking / rooting • Interaction is limited to app under test • Requires app to be instrumented • Android UI Automator • iOS UI Automation • SeeTest (Experitest) • eggPlant (TestPlant) • Perfecto Mobile • Zap-Fix (Zap Tech.) • Trust (Mobile Labs) • TouchTest (SOASTA) • M-eux Test (Jamo Solutions) DescriptionProsConsExample
  • 12. © Utopia Solutions12 Start With a Plan Scope OS/Device Selection Test Case Selection Non-Testing Tasks Technical Challenges Platform/Device Diversity Object Identification Process Integration Test Suite Maintenance Continuous Integration
  • 13. © Utopia Solutions13 Automation Scope Selection • Frequently tested • Predictable results • Easy to automate • Manually tedious “Traditional” rules still apply • Tests executed across all target devices • Avoid apps and tests that face many mobile automation challenges (to be discussed) Mobile specific
  • 14. © Utopia Solutions SECTION 4 Visual-based Tool Approach
  • 15. © Utopia Solutions15 Visual Based Identification Label Label + Boundary Image Recognition Label Offset Multi-Label Offset
  • 16. © Utopia Solutions16 Cross Platform Issues – Inconsistent UI/UX Button Image/Position Screen Titles Button Labels
  • 17. © Utopia Solutions17 Visual Based Techniques – Complex Interaction iOS On/Off Switch Function iosSwitchGetState (Label) Define OCR parameter ranges Search for Label on device screen If Label not found status = switch not found error Else Loop through OCR parameter range Search for switch state(ON/OFF) If state found status = switch state Else status = state not found error End If End Loop End If Return status End Function Function iosSwitchSetState (Label, State) CurrentState = iosSwitchGetState (Label) If CurrentState = error status = error Else If CurrentState = desired state status = success Else If desired state = ON Touch right side of switch Else Touch left side of switch End if End if status = success End if Return status End Function Hidden Complexities
  • 18. © Utopia Solutions18 Visual Interaction – Reusability? Maybe… • Label / object offset • On / Off reversed not • Label offset right • No On / Off label
  • 19. © Utopia Solutions19 Object Identification - Defining Master Object Properties Environment Object Type Attribute Value iPhone Screen Method Unique Text iPhone Screen Label Screen name iPhone Screen Location 20,25:40,200 iPhone Edit Label Object name iPhone Edit Offset 0,-20 … • Defines general rules for identifying objects by platform • Avoids adding all application objects to Object Repository • One master file per application (typically) Master Property File
  • 20. © Utopia Solutions20 Object Identification - Define Object Map ObjectScreenPlatform iOS Login User Name Password Add Stock Symbol • Contains any screens and objects that don’t meet criteria defined in Master Property File • Objects organized hierarchically by platform and screen UI Object Map
  • 21. © Utopia Solutions SECTION 5 Object-based Tool Approach
  • 22. © Utopia Solutions© Utopia Solutions Object Based Interaction Tool Libraries Included in Project Build Object Properties Visible to Tool Class: UIButton Label: Save Class: UITextField Label: Name
  • 23. © Utopia Solutions23 Object Based Interaction – Cross Platform Differences Simple Interaction – “Tap” the Save/Done Button Class: avwButton Method: Push Implementation: avwWindow(“MyStock”).avwButton(“Save”).Push Class: iosButton Method: Tap Implementation: iosView(“EditStock”).iosButton(“Done”).Tap
  • 24. © Utopia Solutions24 Object Identification – Handling Cross Platform Properties Class: Button Name: Save Create Mapping of Universal Objects to Platform Specific Objects Properties Class: avwButton Label: Save Method Name Touch: Push Object Path avwWindow(“MyStock”) Properties Class: iosButton Label: Done Method Name Touch: Tap Object Path iosView(“EditStock”) Step 1
  • 25. © Utopia Solutions25 Function TouchButton (Name) Get execution platform Retrieve platform specific object info from Object Map Construct statement to perform operation Execute operation Return operation status End Function Object Identification – Handling Cross Platform Create Universal Method for All PlatformsStep 2 Properties Class: Button Name: Save Properties Class: iosButton Label: Done Method Name Touch: Tap Object Path iosView(“EditStock”) iosView(“EditStock”).iosButton(“Done”).Tap
  • 26. © Utopia Solutions SECTION 6 Evaluating Your Apps for Potential Challenges
  • 27. © Utopia Solutions27 Analyze Business Processes for Differences Differences • Menu always visible on iOS • Different edit buttons • Different column headers
  • 28. © Utopia Solutions28 Analyze Business Processes for Differences Differences • Different screen titles • Different “add” buttons • Different “save” buttons • Different background color on list
  • 29. © Utopia Solutions29 Analyze Business Processes for Differences Differences • Cancel button embedded in search field on iOS • Different “expand” icons
  • 30. © Utopia Solutions SECTION 7 Handling Workflow Differences
  • 31. © Utopia Solutions31 Handling Workflow Differences – Go To Stocks Android 1. Tap “MENU” 2. Tap “Stocks” 3. Sync on “My Stocks” Screen iOS 1. Tap “My Stocks” 2. Sync on “My Stocks” Screen Implement at the Business Process Component Level Public Function GoToStocks Platform = GetPlatform() Select Case Platform Case “iOS” Tap (“MENU”) Tap (“Stocks”) status = WaitScreen (“My Stocks”) Case “Android” Tap (“My Stocks”) status = WaitScreen (“My Stocks”) Case Else status = INVALID_PLATFORM End Select GoToStocks = status End Function
  • 32. © Utopia Solutions32 Mobile Test Automation Tool Evaluation Top Ten Questions to Ask Tool Vendors • Is your tool visual or object based? • Does your tool require jailbreaking or rooting? • How quickly do you support new mobile OS releases? • How strong is the company (e.g. sales, customers, years, etc.)? • How do I execute a single test across multiple platforms with your tool? • Can (or must) your tool integrate with other scripting engines? • Do your support our mobile development platform? • Can your tool interact with devices remotely? • Does your tool integrate with our test management solution? • Will you assist me with a proof of concept?
  • 33. © Utopia Solutions33 Key Summary Points • Set appropriate automation goals • Start with a plan • Scope • Technical Challenges • Process Integration • Understand how visual/object based tools work • Analyze your app for platform differences • Address differences at appropriate level in your framework • Evaluate tools carefully • Start small and expand over time
  • 34. © Utopia Solutions34 Questions… …. and Answers! Direct future questions to: Lee Barnes Founder and CTO Utopia Solutions, Inc. Email: lee.barnes@utopiasolutions.com Twitter: twitter.com/USI_LeeBarnes LinkedIn: linkedin.com/in/leebarnes Blog: www.RaiseYourMQ.com