SlideShare a Scribd company logo
1Webinar
and
Present
Introducing
performance and
functional
automation early
with Continuous
Integration
2
CI Gets More Testing
TODAY’S PRESENTERS
Andy Pemberton: CloudBees, Sr. Dir. Solution Architecture
Mike Ostenberg: SOASTA, Dir. Solutions Engineering
Brad Johnson: SOASTA, VP Product
TWO COMPANIES, A SINGLE SOLUTION
- the leader in Mobile and Cloud Testing
- the Jenkins Enterprise company
Together, we offer a complete cloud-based service:
Continuous Delivery for Web & Mobile
@apemberton
@bradjohnsonsv
3
• Achieving (even) more test coverage with Jenkins
• Utilizing cloud in development and test
• CloudBees and Jenkins
• From Continuous Integration to Continuous Delivery
• CI with SOASTA CloudTest (load) and TouchTest (mobile)
Questions:
Submit in the question box during event
4
Testing
The Legacy of….WAIT
Ut
Unit Testing
Ft
Functional
Testing
Development Testing Operations
Pt
Performance
Testing
Mo
Monitoring
Legacy Approach: Silos and waiting
TIME
5
Eliminating Delays Compresses Time
Enabling more testing to be done earlier and more often
Ut
Unit Testing
Ft
Functional
Testing
Testing
Pt
Performance
Testing
Mo
Monitoring
Legacy Approach: Silos and waiting
Agile Approach: Automation and Continuous Testing
OperationsDevelopment
Dev Ops
Testing
TIME
6
* Source: Gartner, “Blueprint for Adding Continuous Integration as a Start to Your Continuous Delivery Process” Sean Kenefick, Aug 25, 2014
Traditionally
later-stage
testing!
Perf & mobile
testing plugins
enhance value
across delivery
cycle
7
8
Andy Pemberton
@apemberton
o #1 OSS CI server
o Written in Java
o Highly Extensible
o Founded ~2005
o Easy to install / use
o Extensible via 1000+ plugins
o Widely adopted
o 80K+ installations
o CloudBees knows Jenkins
o On-premise, cloud and hybrid
o Expert support
o Enterprise plugins
Have you met Jenkins?
o Jenkins founder, Kohsuke Kawaguchi, CloudBees CTO
o We optimize Jenkins for the enterprise
o Jenkins Operations Center
o Jenkins OSS Support
o Enterprise-class plugins
o Commit code and support the Jenkins Community
o Jenkins training and consulting
o Jenkins User Conferences (JUC) and newsletters
o …we are the Jenkins Enterprise experts!
CloudBees
CloudBees Jenkins Products
On-Premise In the Cloud Hybrid
No matter how you use Jenkins
Jenkins Enterprise
by CloudBees
Jenkins Operations Center
by CloudBees
DEV@cloud DEV@cloud
Hybrid
Jenkins: the Hub of Continuous Delivery
Source Code
Binaries
Quality Analysis Functional Test
Performance / Load Test
Production
Jenkins: the Hub of Continuous Delivery
Source Code
Binaries
Quality Analysis Functional Test
Performance / Load Test
Production
o “Survival is not mandatory
(nor is adopting DevOps)”
o Gene Kim, Wall Street Journal
- CIO Journal DevOps piece
o Seek out opportunities for
automation - testing is an
ideal first step
o Load, functional, and
mobile testing have been
challenging…
DevOps is a Journey
http://blogs.wsj.com/cio/2014/05/22/enterprise-devops-adoption-isnt-mandatory-but-neither-is-
survival/
8x more
frequent
releases
8000x faster
release process
2x higher
releases
success
12x faster
defect fixes
DevOps
15
Mike Ostenberg
16
o Demo:
• Creating a quick functional test in TouchTest
• Creating a quick performance test in CloudTest
• Running Load and Performance Tests from Jenkins:
• Play TouchTest/CloudTest tests from Jenkins
• Setting Thresholds to automatically alert on poor performance
• Plotting performance trending data directly in Jenkins
• Reviewing detailed results directly in Jenkins
• Jenkins Enterprise by CloudBees enhancements
• Add templates
• Add folders
• Add roles and permissions
17
Expertise
18
TouchTest
o Run MATT utility to create
testable application
o Install Testable application
on device
CloudTest
o Install conductor on laptop
o Modify mobile device proxy
settings
Jenkins
o Install SOASTA PlugIn
o Install Plot PlugIn
o Install Jenkins Enterprise by
CloudBees
• Open Source: online at jenkins-ci.org
• Automatically available on CloudBees DEV@cloud
• Complete control over CloudTest/TouchTest Automation from within your
Jenkins builds:
• Totally integrated into Jenkins environment
SOASTA Jenkins Plug-in
Jenkins Enterprise by CloudBees – Folders Plus Plugin
Jenkins Enterprise by CloudBees – Role-based Access
Control
Jenkins Enterprise by CloudBees – Templates Plugin
23
Thanks & Q&A
Contact SOASTA:
www.soasta.com/cloudtest/
info@soasta.com
866.344.8766
Follow us:
twitter.com/cloudtest
facebook.com/cloudtest
Knowledge Center
• White Papers
• Webinar Recordings
• Case Studies
CloudLink Community
• Support
• Tutorials
• Video
RESOURCES
Contact CloudBees
www.cloudbees.com
info@cloudbees.com
1.781.404.5100
Follow us:
twitter.com/cloudbees
facebook.com/CloudBees
www.SOASTA.com www.CloudBees.com/jenkins
blog.cloudbees.com
developer.cloudbees.com
• Platform Services
• Frameworks

More Related Content

How To Use Jenkins for Continuous Load and Mobile Testing with SOASTA & CloudBees

  • 2. 2 CI Gets More Testing TODAY’S PRESENTERS Andy Pemberton: CloudBees, Sr. Dir. Solution Architecture Mike Ostenberg: SOASTA, Dir. Solutions Engineering Brad Johnson: SOASTA, VP Product TWO COMPANIES, A SINGLE SOLUTION - the leader in Mobile and Cloud Testing - the Jenkins Enterprise company Together, we offer a complete cloud-based service: Continuous Delivery for Web & Mobile @apemberton @bradjohnsonsv
  • 3. 3 • Achieving (even) more test coverage with Jenkins • Utilizing cloud in development and test • CloudBees and Jenkins • From Continuous Integration to Continuous Delivery • CI with SOASTA CloudTest (load) and TouchTest (mobile) Questions: Submit in the question box during event
  • 4. 4 Testing The Legacy of….WAIT Ut Unit Testing Ft Functional Testing Development Testing Operations Pt Performance Testing Mo Monitoring Legacy Approach: Silos and waiting TIME
  • 5. 5 Eliminating Delays Compresses Time Enabling more testing to be done earlier and more often Ut Unit Testing Ft Functional Testing Testing Pt Performance Testing Mo Monitoring Legacy Approach: Silos and waiting Agile Approach: Automation and Continuous Testing OperationsDevelopment Dev Ops Testing TIME
  • 6. 6 * Source: Gartner, “Blueprint for Adding Continuous Integration as a Start to Your Continuous Delivery Process” Sean Kenefick, Aug 25, 2014 Traditionally later-stage testing! Perf & mobile testing plugins enhance value across delivery cycle
  • 7. 7
  • 9. o #1 OSS CI server o Written in Java o Highly Extensible o Founded ~2005 o Easy to install / use o Extensible via 1000+ plugins o Widely adopted o 80K+ installations o CloudBees knows Jenkins o On-premise, cloud and hybrid o Expert support o Enterprise plugins Have you met Jenkins?
  • 10. o Jenkins founder, Kohsuke Kawaguchi, CloudBees CTO o We optimize Jenkins for the enterprise o Jenkins Operations Center o Jenkins OSS Support o Enterprise-class plugins o Commit code and support the Jenkins Community o Jenkins training and consulting o Jenkins User Conferences (JUC) and newsletters o …we are the Jenkins Enterprise experts! CloudBees
  • 11. CloudBees Jenkins Products On-Premise In the Cloud Hybrid No matter how you use Jenkins Jenkins Enterprise by CloudBees Jenkins Operations Center by CloudBees DEV@cloud DEV@cloud Hybrid
  • 12. Jenkins: the Hub of Continuous Delivery Source Code Binaries Quality Analysis Functional Test Performance / Load Test Production
  • 13. Jenkins: the Hub of Continuous Delivery Source Code Binaries Quality Analysis Functional Test Performance / Load Test Production
  • 14. o “Survival is not mandatory (nor is adopting DevOps)” o Gene Kim, Wall Street Journal - CIO Journal DevOps piece o Seek out opportunities for automation - testing is an ideal first step o Load, functional, and mobile testing have been challenging… DevOps is a Journey http://blogs.wsj.com/cio/2014/05/22/enterprise-devops-adoption-isnt-mandatory-but-neither-is- survival/ 8x more frequent releases 8000x faster release process 2x higher releases success 12x faster defect fixes DevOps
  • 16. 16 o Demo: • Creating a quick functional test in TouchTest • Creating a quick performance test in CloudTest • Running Load and Performance Tests from Jenkins: • Play TouchTest/CloudTest tests from Jenkins • Setting Thresholds to automatically alert on poor performance • Plotting performance trending data directly in Jenkins • Reviewing detailed results directly in Jenkins • Jenkins Enterprise by CloudBees enhancements • Add templates • Add folders • Add roles and permissions
  • 18. 18 TouchTest o Run MATT utility to create testable application o Install Testable application on device CloudTest o Install conductor on laptop o Modify mobile device proxy settings Jenkins o Install SOASTA PlugIn o Install Plot PlugIn o Install Jenkins Enterprise by CloudBees
  • 19. • Open Source: online at jenkins-ci.org • Automatically available on CloudBees DEV@cloud • Complete control over CloudTest/TouchTest Automation from within your Jenkins builds: • Totally integrated into Jenkins environment SOASTA Jenkins Plug-in
  • 20. Jenkins Enterprise by CloudBees – Folders Plus Plugin
  • 21. Jenkins Enterprise by CloudBees – Role-based Access Control
  • 22. Jenkins Enterprise by CloudBees – Templates Plugin
  • 23. 23 Thanks & Q&A Contact SOASTA: www.soasta.com/cloudtest/ info@soasta.com 866.344.8766 Follow us: twitter.com/cloudtest facebook.com/cloudtest Knowledge Center • White Papers • Webinar Recordings • Case Studies CloudLink Community • Support • Tutorials • Video RESOURCES Contact CloudBees www.cloudbees.com info@cloudbees.com 1.781.404.5100 Follow us: twitter.com/cloudbees facebook.com/CloudBees www.SOASTA.com www.CloudBees.com/jenkins blog.cloudbees.com developer.cloudbees.com • Platform Services • Frameworks

Editor's Notes

  1. So, based on the poll results…. (there is a mix of users that test in development, staging, and production environment) … In today’s webinar, we will be focusing on performance testing as a continuous process, testing across your software product lifecycle - devops and perf eng is a team effort! Load testing has evolved along with continuous delivery and now requires that developers eliminate performance issues on their desktop while test and operations teams validate from the test lab to live production systems. Your test platform must be adaptable and accessible testing behind and beyond firewalls.  Generally speaking, traditionally, testing has begun at the component level and works outward toward the integration of the entire system. At each point of this process, there are different techniques to testing; however, with the market being so competitive and your users/customers expecting so much out of your product, the time to get this important function done may be compromised. Unfortunately, testing is often the bottleneck that causes teams to wait as manual testing completes or redundant testing tasks are executed by siloed test teams. The goal is not to introduce delays but instead to continually test throughout the lifecycle.
  2. While enabling more testing at all stages of the product lifecycle, we want to eliminate delays and compress the time to release. This approach requires communication and collaboration between various stakeholders - dev folks, QA team, the product owner and the Ops team. Whether your web app is for external or internal users, the performance of the app is critical – they want a fast and responsive site to get their job done. Most of us have been through this … but do we have the patience to sit and watch the spinning circle and see the pages render slowly down the browser? Perceived wait time can causes positive or negative reactions to your product. In an agile approach, automation and continuous testing compress time. Developers test in short iterations on their local machines and conduct unit tests. A big change we see in more agile shops using SOASTA is that the Dev team is empowered to build automated functional and performance tests on their code. Performance has traditionally been the domain of specialized teams, and still is, but if Developers eliminate code-level performance bugs, the Perf Team can build better and more complex tests to isolate issues outside of the code. Test and Ops team can then focus on tests with different objectives behind existing firewalls … The ops team can also do the same approach, working in conjunction with the developers. But the difference here is that the testing gets expanded focusing on areas of the code that involve your infrastructure – database, APIs / web services, smoke tests, custom code, critical single page apps. This bottom-up testing scenario gives tests low level components combined in clusters to validate the performance of a specific software function with increasingly complex and user-focused testing as time goes on.
  3. Work very closely with the community – lead sponsor of Jenkins User Conferences Focus is the plugin architecture
  4. High availability, scalability
  5. Jenkins is emerging as the hub of continuous delivery
  6. Now let’s dig a little deeper into SOASTA Solutions. Running on our single platform, each set of capabilities are packaged to provide a complete set of quality tools for your teams to meet the needs of an increasingly continuous delivery cycle.