Continuous Testing for Optimal Mobile Peroformance - STPCon Spring 2014

Editor's Notes

1. TouchTest has been integrated with popular CI frameworks like Jenkins and Bamboo, Making 100% autmomated regression testing using real devices a reality. We start with 3 core components: First: TouchTest. The SOASTA platform is deployed wherever needed. In a lab or hosted in an MSP or the cloud. Tests are captured and replayed on real devices, and all the work is done via any browser in a visual test envorinment Second, there is the Continous Integration system. We labeled Jenkins here, since it’s open source and we have excellent plugins, but this can be done with others like Bamboo. One point to make here is that if you a part of a separate QA team and Dev doesn’t use CI, this is still applicable. However you pick up builds to test, you plug in this CI solution. Third, you need some devices. Our solution uses off the shelf, unaltered iOS or Android devices over carrier or WiFi networks. This mean you can build labs inexpensively and quickly, and with TouchTest, any registered device, wherever it is, is part of your private device cloud. We can also ship a lab ready to run or host them for you, too. To finalize setting the stage, a CI server polls the source control system, looking for new builds. That’s just part of what they do. With TouchTest, our “agent” – just a web app – listens for the signal to run any automated tests. Here’s were the magic happens! Task 1: A new build hits the VCS signaling the CI server to compile and run the task that injects our library to “Make App Touch Testable” Task 2: This is one of the really hard parts. Some vendors have stated that mobile testing isn’t a good candidate for CI – this is one of the reasons…physically getting the Apps to the devices to test. We’ve automated this task, and every time the app is compiled, it’s pushed to any devices designated to receive it. There’s some addition magic here, to, we’ve built additional tasks to wake up sleeping devices and reboot them. This solves several other issues voices by customers who have used hosted device labs in the past. Task 3: Now the CI system calls TouchTest to kick of our tests – what we call compositions. Tastk 4: TouchTest contacts all the devices and runs the tests on the devices Task 5: Results including from the devices, including performance information like battery, CPU and memory, are sent to TouchTest. In the Jenkins UI – users see all the pass/fail results, and if the engineer wants to dig into the issue, we’ve embedded the TouchTest dashboard into Jenkins. Now lets take a quick look a TouchTest in action.