Webinar: DevOps challenges facing QA

Webinar Presenters:
Garry Heon Brian Le Suer

Agenda
What is DevOps? Challenges
MetricsToolsResources & Skillsets
Introduction

| World’s 2nd Largest Independent Testing Company
|QualiTest recognized for the past 3 years as the only
visionary pure play QA and Testing Services Company
in the “ ” Magic Quadrant”About
QualiTest
Group

About
Zeenyx
| Creators of AscentialTest, the award-winning
testing solution for Windows, Web, dotNet,
Java, PowerBuilder, Terminal and PDF
| Committed to reducing the cost of software
testing by making testers more productive

What is DevOps?
| Born from Agile software development and the need to keep up with the new software
velocity
| It’s combined with the understanding of value of collaboration between organizations
throughout all stages of development
| The increased importance of operations in the SAAS world has driven the need for
innovation on the systems side
| As its name suggests, DevOps is a combination of software development, QA and
operations disciplines
| DevOps recognizes the interdependence of the groups

What DevOps is not?
| DevOps is not Development taking over Operations
| DevOps is not just culture
| DevOps is not just tools
| DevOps is not just Development and Operations

Challenges
| Alignment of development, QA and operations perspectives
| Learning new skills
| Frequent code changes must be tested quickly
| Automation suites must be resilient to application changes
| Having the ability to automate in sprint
| Non-automatable tests must be integrated into the test cycle (exploratory, Ux, usability
testing)
| Maintaining a developer to QA ratio that delivers consistently

Addressing the QA Challenges
| User stories are available and clearly defined before the sprint open/close meeting
| Mock-ups/HTML of the UI design are made available to Dev and QA team at the same time
| Testers must design their automation test case using mockups (pseudo code and/or the
layout of the keyword)
| When Development completes UI integration, DOM elements are available for QA to
complete the test case

Addressing the QA Challenges
| Testers work closely with Development
| When the Xpath or the identifier of a page object is available, QA can make changes immediately
| Page object identifiers/Xpaths must be implemented so that all page objects are defined uniquely
| An appropriate automation test suite should be identified and plugged into the continuous
integration pipeline

Resources
| Need to maintain a good developer to QA ratio
| “QualiTesters” are finding the most success
| Test resources that are well rounded who can analyze requirements, automate test cases and execute
them manually when needed
| SDETs find the transition easiest
| QA engineers are well-versed with technology, tools and process
| CI/CD, Agile
| Bamboo, Jenkins, Git/TFS, Automation frameworks, AWS
| Keep up-to-date on trending technologies related to DevOps

Resources
| QA engineers need to be empowered
| QA engineers need to communicate clearly during scrum meetings
| QA engineers need to hold other team members (Dev, Ops) accountable to commitments
| QA engineers need to facilitate collaboration within the teams
| QA engineers must address issues immediately

Requirements for
an automated
testing solution in
an Agile/DevOps
environment
|Frequent code/component changes means that:
| Tests must be executed frequently so that defects can be traced
directly to most recent changes
| Tests must be resilient to application changes
| Tests must be built using tools and approaches that makes it
possible to update tests quickly and easily

AscentialTest
Integration with
Build Systems
|AscentialTest provides integration with build systems so
that tests can be executed automatically on demand:
| AscentialTest’s command line provides a mechanism to checkout the
latest version of tests to target test machines
| AscentialTest’s execution engine is accessible from the command line
to allow tests to be run directly by the build system
| AscentialTest report generator is accessible from the command line
so that reports can be generated at the conclusion of a test run
| Zeenyx Portal makes test results and reports available to the entire
organization from any web browser

Rapid Test
Development and
Maintenance
|AscentialTest ‘AppObjects’ insulate tests from changes
in the application’s GUI
| AppObject definitions provide a single point of change
| If an application object is modified, only the AppObject definition
needs to be updated. No changes to tests are required.
|AscentialTest’s Step-based Tests
| Tests are built with reusable components called ‘steps’ that speed
up the test creation process and reduce the maintenance effort
when the target application changes.
| If application logic changes, only impacted ‘steps’ need to be
modified. Tests that implement those ‘steps’ do not require any
maintenance.

Webinar: DevOps challenges facing QA

DevOps Metrics
Metric Description
Change Volume How many user stories/fixes are being deployed.
Change Lead Time When commitment is made to business -> production.
QA Lead Time The time from when the code is committed to when it’s deployed
Change Failure Rate How many defects does a change introduce into the system
Production Defects Number of defects found in production
Hotfix/Rollback Production Defects Number of defects found in production that require a hotfix
MTTR (Rollbacks) When a failure occurs how much time does it take to roll back
MTTR When a failure occurs how much time does it take to recover (Hotfix)
Number of Hotfixes/Rollbacks Number of hotfixes deployed over time
Deployment Frequency How often is new code being deployed
Time to deploy Time to deploy new code
Percentage of Failed Deployments Percentage of deployments had issues
Availability Overall uptime of the system

gheon@qualitestgroup.com brian@zeenyx.com

www.QualiTestGroup.com
Thank You!

Webinar: DevOps challenges facing QA

Related slideshows

More Related Content

Webinar: DevOps challenges facing QA

Editor's Notes