SlideShare a Scribd company logo
DevOps Testing: Delivering
quality product faster !!!
AMIT KULKARNI
24 AUG 2018 @amtoya
1. DevOps Testing
2. Functional Test Automation
3. Security And Performance Testing
4. Other Testing – Visual, Accessibility, A/B
5. Manual/Exploratory Testing
6. Continuous Testing
7. Final Thoughts
24 AUG 2018 @amtoya
DevOps Testing
DevOps – How ?
1. Automated Testing
2. Continuous Integration
3. Continuous Delivery
4. Continuous Testing
 QUALITY MINDSET
24 AUG 2018 @amtoya
Test Automation Pyramid
24 AUG 2018 @amtoya
Source: https://twitter.com/noahsussman/status/836612175707930625
Source: https://martinfowler.com/bliki/TestPyramid.html
24 AUG 2018 @amtoya
Test Pyramid - Healthcare
24 AUG 2018 @amtoya
Functional Test Automation
Any
Other
Testing
Security
Testing
Performance
Testing
Functional Test Automation
UI – Service – Unit
Functional Test Automation
 Tools
 Frameworks
24 AUG 2018 @amtoya
Functional Test Automation
 Keyword Driven Framework
 Early Implementation
 Simple and Easy to build
 Tool Independent
 Reusability
24 AUG 2018 @amtoya
Non-Functional Testing
Any
Other
Testing
Performance
Testing
Security
Testing
Why Non-functional Testing ?
 Can cause serious damages – privacy violation,
financial, reputation
 Slow response time, handle concurrent users,
Application crashes if stressed
 Quality
 Not only Security and Performance Testers
responsibility
 Early Implementation – CI/CD
24 AUG 2018 @amtoya
Security Testing
Any
Other
Testing
Performance
Testing
Functional Test Automation
UI – Service – Unit
Security
Testing
What is Security testing ?
 Process intended to identify weakness in the
security mechanisms of an application
 Immune to SQL Injections, Cross-Site
Scripting, Sensitive Data Exposure
 OWASP-ZAP
24 AUG 2018 @amtoya
ZAP UI
Uses the concept of attack proxy to scan for vulnerabilities
24 AUG 2018 @amtoya
ZAP – Spider & Scan
24 AUG 2018 @amtoya
What is Automated Security
testing ?
Security Tool Enabled (ZAP)
+ Automated Functional (Regression) Tests
24 AUG 2018 @amtoya
How to do this ?
1. Enable ZAP
2. Explore the Application
3. Scan and report using ZAP-API
24 AUG 2018 @amtoya
Security - Report
24 AUG 2018 @amtoya
Performance Testing
Any
Other
Testing
Security
Testing
Functional Test Automation
UI – Service – Unit
Performance
Testing
Performance Testing
 Server Side
 Client Side
24 AUG 2018 @amtoya
2
8
AVERAGE LOADING TIME OF A
WEBSITE
Server Side 10-20 %
Client Side 80-90 %
Source: https://developer.yahoo.com/performance/rules.html
Client-Side Performance Tools
 WebPageTest.org
 SiteSpeed.io
 Google PageSpeed
 Lighthouse
24 AUG 2018 @amtoya
Manual Performance Testing
 Analyses web apps and web pages
 Modern performance metrics and insights
 Can run in Chrome DevTools
24 AUG 2018 @amtoya
Automated Performance Testing
 Using the Node CLI
 Installation: npm install -g lighthouse
 Command
 Lighthouse can produce a report as JSON or HTML
24 AUG 2018 @amtoya
Performance - Report
24 AUG 2018 @amtoya
Functional Test Automation
UI – Service – Unit
Security
Testing
Performance
Testing
Other Testing – Visual, Accessibility, A/B
Any
Other
Testing
Visual UI Testing
 Automate the testing of how software looks
 Verify - Icon Size, Padding, Color, Layout,
Text, Element position etc.
24 AUG 2018 @amtoya
Visual UI Testing - Tools
 PhantomCSS-CasperJS
 Selenium/Sahi
 BackstopJS
 Blink-diff
 Screener.io
 Applitools
24 AUG 2018 @amtoya
Visual UI Testing - Result
24 AUG 2018 @amtoya
Source: https://github.com/yahoo/blink-diff
Accessibility Testing
 What ?
 Subset of Usability Testing
 Usable by people with disabilities
 Why ?
 Provide everyone with equal access
 Meeting WCAG 2.0 at level AA
24 AUG 2018 @amtoya
Accessibility Testing - Tools
 AChecker
 Wave
 NVDA
 Lighthouse
24 AUG 2018 @amtoya
Accessibility Testing - Report
24 AUG 2018 @amtoya
A/B Testing
 What ?
A - Control and B – Challenger
 Why ?
 User feedback to improve
 Capturing analytics
24 AUG 2018 @amtoya
A/B Testing - Tools
 Adobe Target Analytics
 Google Analytics
 VWO – Visual Website
Optimizer
24 AUG 2018 @amtoya
Analytics for Testing
 User-flow, Page Traffic
 Modify Test Cases
 Optimize Testing
 Efficient Regression Test-Suite
24 AUG 2018 @amtoya
Manual/Exploratory Testing
Manual/Exploratory Testing
Manual/Exploratory Testing
 Key to high quality software
 Highly intellectual
 Time consuming
 Investigative – Discovering New Information
24 AUG 2018 @amtoya
Continuous Testing
24 AUG 2018 @amtoya
Final Thoughts
 Risk focussed
 Automated testing
 Take responsibility of Non-functional testing
 CI/CD
 Continuous testing
 Tool awareness
 Don’t push exploratory testing off to end-
users
 Quality Mindset
24 AUG 2018 @amtoya
Source: https://danashby.co.uk/2016/10/19/continuous-testing-in-devops
https://www.linkedin.com/in/amtoya
@amtoya
References
 https://github.com/AmitKulkarni9
 https://github.com/zaproxy/zap-api-python
 https://github.com/GoogleChrome/lighthouse
 https://github.com/yahoo/blink-diff
 https://achecker.ca/checker
24 AUG 2018 @amtoya

More Related Content

AmitKulkarni-DevOpsTesting-AustralianTestingDays