SlideShare a Scribd company logo
@lisacrispin
With material from Abby Bangser, Ashley
Hunsberger, Lisi Hocke, Janet Gregory, & more
Copyright 2023 Lisa Crispin
A Holistic Approach to Testing in
Continuous Delivery
Lisa Crispin
@lisacrispin
A little about me
Co-founder, Agile Testing Fellowship
Testing consultant & trainer
lisa@lisacrispin.com
https://lisacrispin.com
Holistic Testing: Strategies
for Agile Teams
Holistic Testing for
Continuous Delivery
AgileTestingFellow.com
Test Automation U
@lisacrispin
Today I’m talking about:
● Building confidence for continuous delivery/deployment (CD)
● Guiding conversations about risk & test coverage
● Quality – a whole team responsibility
Image from meet.google.com
@lisacrispin
Some journeys go wrong
● Slow feedback loops
● Regression failures
● Unexpected impacts
● Technical & testing debt

Recommended for you

Scrum, Kanban, or Scrumban: Which Is Right for You?
Scrum, Kanban, or Scrumban: Which Is Right for You?Scrum, Kanban, or Scrumban: Which Is Right for You?
Scrum, Kanban, or Scrumban: Which Is Right for You?

Agile is on everyone’s minds today, as more and more organizations are eager to reap the benefits of rapid iterations using customer-centric approaches. Organizations tend to run to Scrum first because it is the most recognized agile framework. But is Scrum always the right answer for a team and a business? Heidi Araya discusses the types of scenarios and projects in which Scrum may not be a good fit. She shares other frameworks—including Kanban and Scrumban—as potential alternatives to consider to ensure teams and projects select the right fit and can deliver great software efficiently. Some considerations include organizational culture, size of teams, team composition, types of work, industry requirements, overall project size, and type of project. Go back to your organizations and confidently select the right frameworks for your current and future roles and projects—and explain to management why the framework chosen is appropriate.

scrum
Agile isd by_lisa_cooney
Agile isd by_lisa_cooneyAgile isd by_lisa_cooney
Agile isd by_lisa_cooney

Lisa Cooney has successfully used Agile Instructional Systems Design (Agile ISD) for 3 years. She developed a two-week course on Agile ISD that was well-received. Now she manages a curriculum of six courses using Agile online tools. Agile ISD is iterative rather than sequential, with early and frequent releases of working software or courseware. It values individuals, collaboration, and responding to change over documentation and plans.

agileinstructional designscrum
Product Agility: 3 fundamentals from the trenches
Product Agility: 3 fundamentals from the trenchesProduct Agility: 3 fundamentals from the trenches
Product Agility: 3 fundamentals from the trenches

There is no silver bullet for Product and Business Agility. On this talk you will know which are the fundamentals and some of the initiatives in place in the OutSystems Engineering Journey to better responding rapidly and flexibly to ours customers demands.

agilityproductivityproduct
@lisacrispin
What gets in your team’s way?
Stop and think for a minute. What’s the
biggest obstacle for your team being
successful with continuous delivery?
Photo by Matthew Hamilton on Unsplash
@lisacrispin
One example from my experience
● Team embraced XP practices – TDD, refactoring, pairing, continuous
integration, …
● Thousands of automated regression tests at every level
● Reliable deployment pipelines, app in cloud, blue/green deploy
But…
● Too few testers
● Still had manual release regression checklist
● No time for sufficient exploratory testing
This Photo by Unknown
Author is licensed under CC
BY-NC-ND
@lisacrispin
Making frequent small changes confidently takes:
● Collaboration
● Continuous improvement
● Continuous learning
● A holistic approach
@lisacrispin
8
Get everyone engaged
Operations
Specialist
Designer
????
Programmer Domain
expert
Tester

Recommended for you

Practical Scrum - day 1
Practical Scrum - day 1Practical Scrum - day 1
Practical Scrum - day 1

This document provides an overview of practical scrum. It discusses the three scrum roles of product owner, scrum master, and team. It also describes the four scrum ceremonies and three artifacts. Key principles of scrum include self-organizing teams, empirical process, and delivering working software frequently. The document contrasts command-and-control with self-management and explains how the manager's role changes in an agile environment.

agile software developmentagilescrum master
The Business of Agile: Better, Faster, Cheaper
The Business of Agile: Better, Faster, CheaperThe Business of Agile: Better, Faster, Cheaper
The Business of Agile: Better, Faster, Cheaper

Ryan Ripley relates that during his last agile transformation project, a key stakeholder asked, “Why are we adopting agile?” Ryan talked about increasing quality, delivering software sooner, and fostering a more collaborative relationship with business partners. After a few moments, the stakeholder raised his hand and said, “I get all that. But how is all of this agile stuff any better, faster, or cheaper than what we do today?” Ryan says that leaders must answer the better, faster, cheaper question if they want their agile transformation and associated projects to move forward. To prepare for this critical question, Ryan explores how better, faster, cheaper translates in an agile organization. Take away experiments to use with your agile teams to define better in your organization, an understanding of how agile helps teams deliver value faster to their stakeholders, and how working in an agile way can make business costs cheaper by reducing team turnover, enhancing learning through pairing, and reducing overall costs of product ownership.

agile
6- Agile.engine - Asad safari
6- Agile.engine - Asad safari6- Agile.engine - Asad safari
6- Agile.engine - Asad safari

This document discusses how to effectively adopt an agile mindset and practices. It begins by looking at the original goals of going agile but finds that in reality, many agile adoptions face problems and challenges. It discusses the "broken windows theory" - how small issues can lead to bigger ones if not addressed. However, instead of focusing on problems, it recommends taking an Appreciative Inquiry approach through retrospective meetings. This focuses the team on successes and strengths, envisioning future improvements through positive dialogue to act as a continuous engine for agile transformation.

iranagile2014
@lisacrispin
Guiding conversations with visuals
@lisacrispin
Visualize your pipeline, optimize feedback
Faster feedback
More confidence
@lisacrispin
Do your automated test suites give you confidence?
Confidence
Flaky tests?
Poor
coverage? Hard to
maintain?
Hard to
diagnose?
https://github.com/ahunsberger/TestSuiteDesign - Ashley Hunsberger

Recommended for you

Agile Engine | By Asad Safari
Agile Engine | By Asad SafariAgile Engine | By Asad Safari
Agile Engine | By Asad Safari

This document discusses how to effectively adopt an agile mindset and practices. It begins by noting that while many organizations want to go agile, the reality is that adoption often faces problems and challenges. Drawing on the "broken windows" theory, it suggests that small issues that go unaddressed can undermine agile transformation over time. However, it advocates taking an appreciative inquiry approach to retrospectives, focusing on strengths and successes rather than problems. This positive mindset can act as a "continuous improvement engine" to foster ongoing enhancement of agile ways of working.

conferenceiranagile2014agile
"DevOps > CI+CD "
"DevOps > CI+CD ""DevOps > CI+CD "
"DevOps > CI+CD "

This document discusses DevOps and continuous delivery. It begins by introducing the speaker and defining DevOps as applying kanban principles and focusing on reducing lead time from idea to production. It then discusses how continuous integration, continuous delivery, test automation, and breaking work into small batches can help reduce lead time. The document emphasizes that DevOps is not just about tools but also culture and processes.

devopscontinuous delivery
Bosnia Agile Tuzla Meetup
Bosnia Agile Tuzla MeetupBosnia Agile Tuzla Meetup
Bosnia Agile Tuzla Meetup

Bosnia Agile slides from Bosnia Agile Tuzla meetup where attendees had a chance to learn about basics of Scrum, by certified Professional Scrum Product Owner Enis Zeherović, and then to participate in a great "Team Work" training that explains all the soft skills Scrum team or any other team needs to have to work smoothly.

agile meetup scrum tuzla bosnia agile ba
@lisacrispin
Some of my favorite Test Canvas questions
● What information should each suite provide? To
whom? How?
● How will the team know about test failures? Who is
responsible for looking into those?
● Do you pair on test automation, or do test code
reviews?
This Photo by Unknown Author is
licensed under CC BY-NC
@lisacrispin
Mitigating risks with automated regression tests
@lisacrispin
Talking about risk
Risk Storming
(https://riskstormingonline.com)
Traditional risk
analysis
Mind maps
@lisacrispin
Does your team know all the possible risks?
● Customers behave in unexpected ways
● Infrastructure components may fail
● External systems can impact ours
● …
What
do we
do?

Recommended for you

Refactoring Test Collaboration
Refactoring Test CollaborationRefactoring Test Collaboration
Refactoring Test Collaboration

Presented at Agile Testing Days US 2018 https://agiletestingdays.us/session/refactoring-test-collaboration/ Collective ownership for testing starts with understanding testing. Rework your team dynamics to evolve past duplication and improve performance through whole team testing. Take home practical patterns for improving your team's collaboration on testing. Because teams who own testing have more confidence in the customer value of their results. As the Pragmatic Programmers say, "refactoring is an activity that needs to be undertaken slowly, deliberately, and carefully," so how do we begin? In this session, we will experience the complex interactions of an agile team focused on demonstrating customer value by answering a series a questions: Where do testers get their ideas? How are you planning to accomplish this proposed testing, tester? Why not automate all the things? Who is going to do this manual testing and how does it work? How do we know whether we're testing the right things? Build your own list of TODOs from these various practical collaboration approaches and begin deduping your team's testing for a better first day back at the office.

software testingagilecollaboration
Stc chandan patary_ua_testing _ stc
Stc chandan patary_ua_testing _ stcStc chandan patary_ua_testing _ stc
Stc chandan patary_ua_testing _ stc

This document discusses ways to improve product development cycles through effective user acceptance testing and engagement. It describes how one organization achieved targets like reducing production to release cycles from 15 to 10 months by having product owners actively participate in validation testing. Stories from the trenches are shared about having product engineers directly involved in validation testing to better represent customer needs. Exploratory "monkey" testing of system boundaries is also discussed. The key takeaway is how design thinking principles were applied, like creating focus groups, prototyping solutions, and getting faster customer feedback through various iterative testing approaches.

Manual Testing in Scrum is Hard (But Not Impossible)
Manual Testing in Scrum is Hard (But Not Impossible)Manual Testing in Scrum is Hard (But Not Impossible)
Manual Testing in Scrum is Hard (But Not Impossible)

Lesley Wallace presents on challenges with testing within a Scrum sprint and provides solutions. She asks the audience questions about their testing practices. Wallace explains that everyone on the team is responsible for quality and testing. To test within a sprint, stories must have decreased scope and testing must only focus on the story's scope. If more time is needed for regression testing, a "hardening sprint" can be added before release. Automation helps with testing and there are tools for all budgets.

software testingtestingagile
@lisacrispin
Observability
● Ask the questions you didn’t know you’d need to ask - unknowns
● Complex systems fail in complex ways
● With enough information, we can respond quickly
Cindy Sridharan,
https://medium.com/@copy
construct/testing-in-
production-the-safe-way-
18ca102d0ef1
@lisacrispin
Holistic Testing
Janet Gregory
https://janetgregory.ca
/testing-from-a-
holistic-point-of-view/
@lisacrispin
In which stages do you currently get involved?
@lisacrispin
What makes it work?
● Commitment to a level of quality
○ Identifying what’s valuable to customers
○ Bug prevention over bug detection
○ Fast response to prod issues
● Diverse perspectives, skill sets, biases
● Competencies > roles

Recommended for you

Agile for Me- CodeStock 2009
Agile for Me- CodeStock 2009Agile for Me- CodeStock 2009
Agile for Me- CodeStock 2009

This document provides an overview of Agile software development. It begins by defining Agile as a project management process that encourages frequent inspection and adaptation. It then discusses some common Agile practices like Scrum and eXtreme Programming. The Agile Manifesto values individuals and interactions, working software, customer collaboration, and responding to change. Finally, it provides advice for different roles on how Agile can benefit them and their work.

Lean agile meets design thinking
Lean agile meets design thinkingLean agile meets design thinking
Lean agile meets design thinking

As companies evolve to adopt, integrate and leverage software as the defining element of their success in the 21st century, a rash of processes and methodologies are vying for their product teams' attention. This Session will give you guidelines on how to start an innovative business lean and fast by using design thinking, lean and agile approaches and how to build high-performing digital product teams. The session will finish with discussing Lean Agile meets Design Thinking to give a meaningful conclusion.

#dubailifestyle #agility #performance #digitalprod#agileschool#designthinking
Good agile / Bad agile: Proving the value of Agile to a skeptical organization
Good agile / Bad agile: Proving the value of Agile to a skeptical organizationGood agile / Bad agile: Proving the value of Agile to a skeptical organization
Good agile / Bad agile: Proving the value of Agile to a skeptical organization

Is Agile worth it? What value can being Agile bring to your organization? Done right, Agile software development methodologies can help your organization deliver greater value to customers and other stakeholders more efficiently and with reduced risk. Done wrong, Agile methodologies become an endlessly iterating feature factory, facing an ever-growing backlog. In this interactive session, attendees discussed: - How to identify what’s most valuable to build next - How to ensure that the features you build are not just functional, but used and valued - How to measure and effectively communicate the value that you create Led by Alan Albert of MarketFit, this session at Agile Vancouver explored theory, examples, and exercises showing how to unlock the power of discovering, creating, and communicating value.

agileagile software developmentproduct management
@lisacrispin
Cross-discipline pairing, ensemble testing
Picture from Ensemble Programming Guidebook, Maaret Pyhäjärvi
@lisacrispin
We’re humans! (or possibly dragons, donkeys, unicorns…)
Build relationships
● Friendly conversations
● Do food
● Share something useful
● Ask for help
Katrina Clokie has excellent tips in her book, A
Practical Guide to Testing in DevOps
@lisacrispin
Building a quality culture
• Transformative leaders
• Trust and psychological safety
• Whole team ”owns” product
• “You build it, you run it”
• Focus on quality, not speed
@lisacrispin
Principles of CD – Jez Humble & David Farley
● Build quality in
● Work in small batches
● Computers perform repetitive tasks, people solve
problems
● Relentlessly pursue continuous improvement
● Everyone is responsible

Recommended for you

Agile antipatterns webinar august17
Agile antipatterns webinar august17Agile antipatterns webinar august17
Agile antipatterns webinar august17

This document provides a summary of Raj Kasturi's background and qualifications. It lists that Raj has over 25 years of IT experience including eight years of enterprise Agile experience. It also notes that he has been an adjunct faculty member at Penn State, has 18+ years of teaching experience in areas like Scrum and project management, and has experience in roles like Agile Coach, Scrum Trainer, and Scrum Master. The document provides Raj's contact information and identifies that he is a speaker who volunteers at agile conferences and user groups.

antipatterns of agile/scruminformation technologyagile software development
Path to Agility - Adoption Patterns to Overcome Transformation Pitfalls
Path to Agility - Adoption Patterns to Overcome Transformation PitfallsPath to Agility - Adoption Patterns to Overcome Transformation Pitfalls
Path to Agility - Adoption Patterns to Overcome Transformation Pitfalls

Has your organization's Agile adoption stalled or hit a ceiling? Using his experience working with a diverse set of organizations, David Hawks will share patterns he has discovered that avoid common pitfalls. In this hands-on session you will learn a proven path to agility for many organizations and understand where you fit. Participants will apply this knowledge to create their own customized action plan to make further progress on their Agile journey.

agile transformationagileagile adoption
Building the Ideal CI-CD Pipeline_ Achieving Visual Perfection
Building the Ideal CI-CD Pipeline_ Achieving Visual PerfectionBuilding the Ideal CI-CD Pipeline_ Achieving Visual Perfection
Building the Ideal CI-CD Pipeline_ Achieving Visual Perfection

Explore the advantages of integrating AI-powered testing into the CI/CD pipeline in this session from Applitools engineer Brandon Murray. More information and session materials at applitools.com Discover how shift-left strategies and advanced testing in CI/CD pipelines can enhance customer satisfaction and streamline development processes, including: • Significantly reduced time and effort needed for test creation and maintenance compared to traditional testing methods. • Enhanced UI coverage that eliminates the necessity for manual testing, leading to quicker and more effective testing processes. • Effortless integration with the development workflow, offering instant feedback on pull requests and facilitating swifter product releases.

applitoolsapplitools eyesapplitools ai
@lisacrispin
Get your team together and talk
What are our quality goals? Are we
committed to building quality in?
How can we apply the Holistic
Testing Model to fit testing activities
in throughout the software
development life cycle?
@lisacrispin
A few resources
● ”Agile Testing for the Whole Team” training course, https://agiletestingfellow.com
● Agile Testing Condensed, Agile Testing and More Agile Testing, Lisa Crispin and Janet
Gregory, https://agiletester.ca
● “Test Automation in DevOps”, A Test Automation U course by Lisa Crispin
https://testautomationu.applitools.com/test-automation-in-devops/
● Continuous Delivery by Jez Humble and David Farley, https://continuousdelivery.com
● A Practical Guide to Testing in DevOps by Katrina Clokie
https://leanpub.com/testingindevops
● Accelerate by Dr. Nicole Forsgren, Jez Humble, Gene Kim
● Ashley Hunsberger’s Test Suite Canvas
https://github.com/ahunsberger/TestSuiteDesign
● https://lisacrispin.com/observability-continuous-delivery-devops-related-resources/
● ExploreIt! Elizabeth Hendrickson

More Related Content

Similar to A Holistic Approach to Testing in Continuous Delivery

Testing is a team problem
Testing is a team problemTesting is a team problem
Testing is a team problem
Janet Gregory
 
Nasty Impediments: Unclog the Pipe for Business Agility
Nasty Impediments: Unclog the Pipe for Business AgilityNasty Impediments: Unclog the Pipe for Business Agility
Nasty Impediments: Unclog the Pipe for Business Agility
Stacia Heimgartner Viscardi, CST, CEO
 
Agile Testing - What, why and how.
Agile Testing - What, why and how.Agile Testing - What, why and how.
Agile Testing - What, why and how.
Asim Kazmi
 
Scrum, Kanban, or Scrumban: Which Is Right for You?
Scrum, Kanban, or Scrumban: Which Is Right for You?Scrum, Kanban, or Scrumban: Which Is Right for You?
Scrum, Kanban, or Scrumban: Which Is Right for You?
TechWell
 
Agile isd by_lisa_cooney
Agile isd by_lisa_cooneyAgile isd by_lisa_cooney
Agile isd by_lisa_cooney
Lisa Boyer Cooney (she/her)
 
Product Agility: 3 fundamentals from the trenches
Product Agility: 3 fundamentals from the trenchesProduct Agility: 3 fundamentals from the trenches
Product Agility: 3 fundamentals from the trenches
Pedro Teixeira
 
Practical Scrum - day 1
Practical Scrum - day 1Practical Scrum - day 1
Practical Scrum - day 1
Anat (Alon) Salhov
 
The Business of Agile: Better, Faster, Cheaper
The Business of Agile: Better, Faster, CheaperThe Business of Agile: Better, Faster, Cheaper
The Business of Agile: Better, Faster, Cheaper
TechWell
 
6- Agile.engine - Asad safari
6- Agile.engine - Asad safari6- Agile.engine - Asad safari
6- Agile.engine - Asad safari
Ali Moghadam
 
Agile Engine | By Asad Safari
Agile Engine | By Asad SafariAgile Engine | By Asad Safari
Agile Engine | By Asad Safari
Iran Agile Community
 
"DevOps > CI+CD "
"DevOps > CI+CD ""DevOps > CI+CD "
"DevOps > CI+CD "
Innovation Roots
 
Bosnia Agile Tuzla Meetup
Bosnia Agile Tuzla MeetupBosnia Agile Tuzla Meetup
Bosnia Agile Tuzla Meetup
Bosnia Agile
 
Refactoring Test Collaboration
Refactoring Test CollaborationRefactoring Test Collaboration
Refactoring Test Collaboration
Claire Moss
 
Stc chandan patary_ua_testing _ stc
Stc chandan patary_ua_testing _ stcStc chandan patary_ua_testing _ stc
Stc chandan patary_ua_testing _ stc
Chandan Patary
 
Manual Testing in Scrum is Hard (But Not Impossible)
Manual Testing in Scrum is Hard (But Not Impossible)Manual Testing in Scrum is Hard (But Not Impossible)
Manual Testing in Scrum is Hard (But Not Impossible)
Lesley Wallace, CSP, CSM, ICP-ACC, SA
 
Agile for Me- CodeStock 2009
Agile for Me- CodeStock 2009Agile for Me- CodeStock 2009
Agile for Me- CodeStock 2009
Adrian Carr
 
Lean agile meets design thinking
Lean agile meets design thinkingLean agile meets design thinking
Lean agile meets design thinking
Ravneet Kaur
 
Good agile / Bad agile: Proving the value of Agile to a skeptical organization
Good agile / Bad agile: Proving the value of Agile to a skeptical organizationGood agile / Bad agile: Proving the value of Agile to a skeptical organization
Good agile / Bad agile: Proving the value of Agile to a skeptical organization
Alan Albert
 
Agile antipatterns webinar august17
Agile antipatterns webinar august17Agile antipatterns webinar august17
Agile antipatterns webinar august17
Raj Kasturi, CSP-SM®, CSP-PO®, CSM®, CSPO®, MBA
 
Path to Agility - Adoption Patterns to Overcome Transformation Pitfalls
Path to Agility - Adoption Patterns to Overcome Transformation PitfallsPath to Agility - Adoption Patterns to Overcome Transformation Pitfalls
Path to Agility - Adoption Patterns to Overcome Transformation Pitfalls
Agile Velocity
 

Similar to A Holistic Approach to Testing in Continuous Delivery (20)

Testing is a team problem
Testing is a team problemTesting is a team problem
Testing is a team problem
 
Nasty Impediments: Unclog the Pipe for Business Agility
Nasty Impediments: Unclog the Pipe for Business AgilityNasty Impediments: Unclog the Pipe for Business Agility
Nasty Impediments: Unclog the Pipe for Business Agility
 
Agile Testing - What, why and how.
Agile Testing - What, why and how.Agile Testing - What, why and how.
Agile Testing - What, why and how.
 
Scrum, Kanban, or Scrumban: Which Is Right for You?
Scrum, Kanban, or Scrumban: Which Is Right for You?Scrum, Kanban, or Scrumban: Which Is Right for You?
Scrum, Kanban, or Scrumban: Which Is Right for You?
 
Agile isd by_lisa_cooney
Agile isd by_lisa_cooneyAgile isd by_lisa_cooney
Agile isd by_lisa_cooney
 
Product Agility: 3 fundamentals from the trenches
Product Agility: 3 fundamentals from the trenchesProduct Agility: 3 fundamentals from the trenches
Product Agility: 3 fundamentals from the trenches
 
Practical Scrum - day 1
Practical Scrum - day 1Practical Scrum - day 1
Practical Scrum - day 1
 
The Business of Agile: Better, Faster, Cheaper
The Business of Agile: Better, Faster, CheaperThe Business of Agile: Better, Faster, Cheaper
The Business of Agile: Better, Faster, Cheaper
 
6- Agile.engine - Asad safari
6- Agile.engine - Asad safari6- Agile.engine - Asad safari
6- Agile.engine - Asad safari
 
Agile Engine | By Asad Safari
Agile Engine | By Asad SafariAgile Engine | By Asad Safari
Agile Engine | By Asad Safari
 
"DevOps > CI+CD "
"DevOps > CI+CD ""DevOps > CI+CD "
"DevOps > CI+CD "
 
Bosnia Agile Tuzla Meetup
Bosnia Agile Tuzla MeetupBosnia Agile Tuzla Meetup
Bosnia Agile Tuzla Meetup
 
Refactoring Test Collaboration
Refactoring Test CollaborationRefactoring Test Collaboration
Refactoring Test Collaboration
 
Stc chandan patary_ua_testing _ stc
Stc chandan patary_ua_testing _ stcStc chandan patary_ua_testing _ stc
Stc chandan patary_ua_testing _ stc
 
Manual Testing in Scrum is Hard (But Not Impossible)
Manual Testing in Scrum is Hard (But Not Impossible)Manual Testing in Scrum is Hard (But Not Impossible)
Manual Testing in Scrum is Hard (But Not Impossible)
 
Agile for Me- CodeStock 2009
Agile for Me- CodeStock 2009Agile for Me- CodeStock 2009
Agile for Me- CodeStock 2009
 
Lean agile meets design thinking
Lean agile meets design thinkingLean agile meets design thinking
Lean agile meets design thinking
 
Good agile / Bad agile: Proving the value of Agile to a skeptical organization
Good agile / Bad agile: Proving the value of Agile to a skeptical organizationGood agile / Bad agile: Proving the value of Agile to a skeptical organization
Good agile / Bad agile: Proving the value of Agile to a skeptical organization
 
Agile antipatterns webinar august17
Agile antipatterns webinar august17Agile antipatterns webinar august17
Agile antipatterns webinar august17
 
Path to Agility - Adoption Patterns to Overcome Transformation Pitfalls
Path to Agility - Adoption Patterns to Overcome Transformation PitfallsPath to Agility - Adoption Patterns to Overcome Transformation Pitfalls
Path to Agility - Adoption Patterns to Overcome Transformation Pitfalls
 

More from Applitools

Building the Ideal CI-CD Pipeline_ Achieving Visual Perfection
Building the Ideal CI-CD Pipeline_ Achieving Visual PerfectionBuilding the Ideal CI-CD Pipeline_ Achieving Visual Perfection
Building the Ideal CI-CD Pipeline_ Achieving Visual Perfection
Applitools
 
Leveraging AI for Mobile App Testing on Real Devices | Applitools + Kobiton
Leveraging AI for Mobile App Testing on Real Devices | Applitools + KobitonLeveraging AI for Mobile App Testing on Real Devices | Applitools + Kobiton
Leveraging AI for Mobile App Testing on Real Devices | Applitools + Kobiton
Applitools
 
Streamlining Your Tech Stack: A Blueprint for Enhanced Efficiency and Coverag...
Streamlining Your Tech Stack: A Blueprint for Enhanced Efficiency and Coverag...Streamlining Your Tech Stack: A Blueprint for Enhanced Efficiency and Coverag...
Streamlining Your Tech Stack: A Blueprint for Enhanced Efficiency and Coverag...
Applitools
 
Visual AI for eCommerce: Improving Conversions with a Flawless UI
Visual AI for eCommerce: Improving Conversions with a Flawless UIVisual AI for eCommerce: Improving Conversions with a Flawless UI
Visual AI for eCommerce: Improving Conversions with a Flawless UI
Applitools
 
A Test Automation Platform Designed for the Future
A Test Automation Platform Designed for the FutureA Test Automation Platform Designed for the Future
A Test Automation Platform Designed for the Future
Applitools
 
Add AI to Your SDLC, presented by Applitools and Curiosity
Add AI to Your SDLC, presented by Applitools and CuriosityAdd AI to Your SDLC, presented by Applitools and Curiosity
Add AI to Your SDLC, presented by Applitools and Curiosity
Applitools
 
The Future of AI-Based Test Automation
The Future of AI-Based Test AutomationThe Future of AI-Based Test Automation
The Future of AI-Based Test Automation
Applitools
 
Test Automation at Scale: Lessons from Top-Performing Distributed Teams
Test Automation at Scale: Lessons from Top-Performing Distributed TeamsTest Automation at Scale: Lessons from Top-Performing Distributed Teams
Test Automation at Scale: Lessons from Top-Performing Distributed Teams
Applitools
 
Can AI Autogenerate and Run Automated Tests?
Can AI Autogenerate and Run Automated Tests?Can AI Autogenerate and Run Automated Tests?
Can AI Autogenerate and Run Automated Tests?
Applitools
 
Triple Assurance: AI-Powered Test Automation in UI Design and Functionality
Triple Assurance: AI-Powered Test Automation in UI Design and FunctionalityTriple Assurance: AI-Powered Test Automation in UI Design and Functionality
Triple Assurance: AI-Powered Test Automation in UI Design and Functionality
Applitools
 
Navigating the Challenges of Testing at Scale: Lessons from Top-Performing Teams
Navigating the Challenges of Testing at Scale: Lessons from Top-Performing TeamsNavigating the Challenges of Testing at Scale: Lessons from Top-Performing Teams
Navigating the Challenges of Testing at Scale: Lessons from Top-Performing Teams
Applitools
 
Introducing the Applitools Self Healing Execution Cloud.pdf
Introducing the Applitools Self Healing Execution Cloud.pdfIntroducing the Applitools Self Healing Execution Cloud.pdf
Introducing the Applitools Self Healing Execution Cloud.pdf
Applitools
 
Unlocking the Power of ChatGPT and AI in Testing - NextSteps, presented by Ap...
Unlocking the Power of ChatGPT and AI in Testing - NextSteps, presented by Ap...Unlocking the Power of ChatGPT and AI in Testing - NextSteps, presented by Ap...
Unlocking the Power of ChatGPT and AI in Testing - NextSteps, presented by Ap...
Applitools
 
Collaborating From Design To Experience: Introducing Centra
Collaborating From Design To Experience: Introducing CentraCollaborating From Design To Experience: Introducing Centra
Collaborating From Design To Experience: Introducing Centra
Applitools
 
What the QA Position Will Look Like in the Future
What the QA Position Will Look Like in the FutureWhat the QA Position Will Look Like in the Future
What the QA Position Will Look Like in the Future
Applitools
 
Getting Started with Visual Testing
Getting Started with Visual TestingGetting Started with Visual Testing
Getting Started with Visual Testing
Applitools
 
Workshop: Head-to-Head Web Testing: Part 1 with Cypress
Workshop: Head-to-Head Web Testing: Part 1 with CypressWorkshop: Head-to-Head Web Testing: Part 1 with Cypress
Workshop: Head-to-Head Web Testing: Part 1 with Cypress
Applitools
 
From Washing Cars To Automating Test Applications
From Washing Cars To Automating Test ApplicationsFrom Washing Cars To Automating Test Applications
From Washing Cars To Automating Test Applications
Applitools
 
AI-Powered-Cross-Browser Testing
AI-Powered-Cross-Browser TestingAI-Powered-Cross-Browser Testing
AI-Powered-Cross-Browser Testing
Applitools
 
Workshop: An Introduction to API Automation with Javascript
Workshop: An Introduction to API Automation with JavascriptWorkshop: An Introduction to API Automation with Javascript
Workshop: An Introduction to API Automation with Javascript
Applitools
 

More from Applitools (20)

Building the Ideal CI-CD Pipeline_ Achieving Visual Perfection
Building the Ideal CI-CD Pipeline_ Achieving Visual PerfectionBuilding the Ideal CI-CD Pipeline_ Achieving Visual Perfection
Building the Ideal CI-CD Pipeline_ Achieving Visual Perfection
 
Leveraging AI for Mobile App Testing on Real Devices | Applitools + Kobiton
Leveraging AI for Mobile App Testing on Real Devices | Applitools + KobitonLeveraging AI for Mobile App Testing on Real Devices | Applitools + Kobiton
Leveraging AI for Mobile App Testing on Real Devices | Applitools + Kobiton
 
Streamlining Your Tech Stack: A Blueprint for Enhanced Efficiency and Coverag...
Streamlining Your Tech Stack: A Blueprint for Enhanced Efficiency and Coverag...Streamlining Your Tech Stack: A Blueprint for Enhanced Efficiency and Coverag...
Streamlining Your Tech Stack: A Blueprint for Enhanced Efficiency and Coverag...
 
Visual AI for eCommerce: Improving Conversions with a Flawless UI
Visual AI for eCommerce: Improving Conversions with a Flawless UIVisual AI for eCommerce: Improving Conversions with a Flawless UI
Visual AI for eCommerce: Improving Conversions with a Flawless UI
 
A Test Automation Platform Designed for the Future
A Test Automation Platform Designed for the FutureA Test Automation Platform Designed for the Future
A Test Automation Platform Designed for the Future
 
Add AI to Your SDLC, presented by Applitools and Curiosity
Add AI to Your SDLC, presented by Applitools and CuriosityAdd AI to Your SDLC, presented by Applitools and Curiosity
Add AI to Your SDLC, presented by Applitools and Curiosity
 
The Future of AI-Based Test Automation
The Future of AI-Based Test AutomationThe Future of AI-Based Test Automation
The Future of AI-Based Test Automation
 
Test Automation at Scale: Lessons from Top-Performing Distributed Teams
Test Automation at Scale: Lessons from Top-Performing Distributed TeamsTest Automation at Scale: Lessons from Top-Performing Distributed Teams
Test Automation at Scale: Lessons from Top-Performing Distributed Teams
 
Can AI Autogenerate and Run Automated Tests?
Can AI Autogenerate and Run Automated Tests?Can AI Autogenerate and Run Automated Tests?
Can AI Autogenerate and Run Automated Tests?
 
Triple Assurance: AI-Powered Test Automation in UI Design and Functionality
Triple Assurance: AI-Powered Test Automation in UI Design and FunctionalityTriple Assurance: AI-Powered Test Automation in UI Design and Functionality
Triple Assurance: AI-Powered Test Automation in UI Design and Functionality
 
Navigating the Challenges of Testing at Scale: Lessons from Top-Performing Teams
Navigating the Challenges of Testing at Scale: Lessons from Top-Performing TeamsNavigating the Challenges of Testing at Scale: Lessons from Top-Performing Teams
Navigating the Challenges of Testing at Scale: Lessons from Top-Performing Teams
 
Introducing the Applitools Self Healing Execution Cloud.pdf
Introducing the Applitools Self Healing Execution Cloud.pdfIntroducing the Applitools Self Healing Execution Cloud.pdf
Introducing the Applitools Self Healing Execution Cloud.pdf
 
Unlocking the Power of ChatGPT and AI in Testing - NextSteps, presented by Ap...
Unlocking the Power of ChatGPT and AI in Testing - NextSteps, presented by Ap...Unlocking the Power of ChatGPT and AI in Testing - NextSteps, presented by Ap...
Unlocking the Power of ChatGPT and AI in Testing - NextSteps, presented by Ap...
 
Collaborating From Design To Experience: Introducing Centra
Collaborating From Design To Experience: Introducing CentraCollaborating From Design To Experience: Introducing Centra
Collaborating From Design To Experience: Introducing Centra
 
What the QA Position Will Look Like in the Future
What the QA Position Will Look Like in the FutureWhat the QA Position Will Look Like in the Future
What the QA Position Will Look Like in the Future
 
Getting Started with Visual Testing
Getting Started with Visual TestingGetting Started with Visual Testing
Getting Started with Visual Testing
 
Workshop: Head-to-Head Web Testing: Part 1 with Cypress
Workshop: Head-to-Head Web Testing: Part 1 with CypressWorkshop: Head-to-Head Web Testing: Part 1 with Cypress
Workshop: Head-to-Head Web Testing: Part 1 with Cypress
 
From Washing Cars To Automating Test Applications
From Washing Cars To Automating Test ApplicationsFrom Washing Cars To Automating Test Applications
From Washing Cars To Automating Test Applications
 
AI-Powered-Cross-Browser Testing
AI-Powered-Cross-Browser TestingAI-Powered-Cross-Browser Testing
AI-Powered-Cross-Browser Testing
 
Workshop: An Introduction to API Automation with Javascript
Workshop: An Introduction to API Automation with JavascriptWorkshop: An Introduction to API Automation with Javascript
Workshop: An Introduction to API Automation with Javascript
 

Recently uploaded

Migrate your Infrastructure to the AWS Cloud
Migrate your Infrastructure to the AWS CloudMigrate your Infrastructure to the AWS Cloud
Migrate your Infrastructure to the AWS Cloud
Ortus Solutions, Corp
 
Shivam Pandit working on Php Web Developer.
Shivam Pandit working on Php Web Developer.Shivam Pandit working on Php Web Developer.
Shivam Pandit working on Php Web Developer.
shivamt017
 
WEBINAR SLIDES: CCX for Cloud Service Providers
WEBINAR SLIDES: CCX for Cloud Service ProvidersWEBINAR SLIDES: CCX for Cloud Service Providers
WEBINAR SLIDES: CCX for Cloud Service Providers
Severalnines
 
React Native vs Flutter - SSTech System
React Native vs Flutter  - SSTech SystemReact Native vs Flutter  - SSTech System
React Native vs Flutter - SSTech System
SSTech System
 
CViewSurvey Digitech Pvt Ltd that works on a proven C.A.A.G. model.
CViewSurvey Digitech Pvt Ltd that  works on a proven C.A.A.G. model.CViewSurvey Digitech Pvt Ltd that  works on a proven C.A.A.G. model.
CViewSurvey Digitech Pvt Ltd that works on a proven C.A.A.G. model.
bhatinidhi2001
 
ThaiPy meetup - Indexes and Django
ThaiPy meetup - Indexes and DjangoThaiPy meetup - Indexes and Django
ThaiPy meetup - Indexes and Django
akshesh doshi
 
NYC 26-Jun-2024 Combined Presentations.pdf
NYC 26-Jun-2024 Combined Presentations.pdfNYC 26-Jun-2024 Combined Presentations.pdf
NYC 26-Jun-2024 Combined Presentations.pdf
AUGNYC
 
How we built TryBoxLang in under 48 hours
How we built TryBoxLang in under 48 hoursHow we built TryBoxLang in under 48 hours
How we built TryBoxLang in under 48 hours
Ortus Solutions, Corp
 
COMPSAC 2024 D&I Panel: Charting a Course for Equity: Strategies for Overcomi...
COMPSAC 2024 D&I Panel: Charting a Course for Equity: Strategies for Overcomi...COMPSAC 2024 D&I Panel: Charting a Course for Equity: Strategies for Overcomi...
COMPSAC 2024 D&I Panel: Charting a Course for Equity: Strategies for Overcomi...
Hironori Washizaki
 
MVP Mobile Application - Codearrest.pptx
MVP Mobile Application - Codearrest.pptxMVP Mobile Application - Codearrest.pptx
MVP Mobile Application - Codearrest.pptx
Mitchell Marsh
 
Development of Chatbot Using AI\ML Technologies
Development of Chatbot Using AI\ML TechnologiesDevelopment of Chatbot Using AI\ML Technologies
Development of Chatbot Using AI\ML Technologies
MaisnamLuwangPibarel
 
dachnug51 - HCL Sametime 12 as a Software Appliance.pdf
dachnug51 - HCL Sametime 12 as a Software Appliance.pdfdachnug51 - HCL Sametime 12 as a Software Appliance.pdf
dachnug51 - HCL Sametime 12 as a Software Appliance.pdf
DNUG e.V.
 
Addressing the Top 9 User Pain Points with Visual Design Elements.pptx
Addressing the Top 9 User Pain Points with Visual Design Elements.pptxAddressing the Top 9 User Pain Points with Visual Design Elements.pptx
Addressing the Top 9 User Pain Points with Visual Design Elements.pptx
Sparity1
 
dachnug51 - All you ever wanted to know about domino licensing.pdf
dachnug51 - All you ever wanted to know about domino licensing.pdfdachnug51 - All you ever wanted to know about domino licensing.pdf
dachnug51 - All you ever wanted to know about domino licensing.pdf
DNUG e.V.
 
active-directory-auditing-solution (2).pptx
active-directory-auditing-solution (2).pptxactive-directory-auditing-solution (2).pptx
active-directory-auditing-solution (2).pptx
sudsdeep
 
Leading Project Management Tool Taskruop.pptx
Leading Project Management Tool Taskruop.pptxLeading Project Management Tool Taskruop.pptx
Leading Project Management Tool Taskruop.pptx
taskroupseo
 
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...
karim wahed
 
Intro to Amazon Web Services (AWS) and Gen AI
Intro to Amazon Web Services (AWS) and Gen AIIntro to Amazon Web Services (AWS) and Gen AI
Intro to Amazon Web Services (AWS) and Gen AI
Ortus Solutions, Corp
 
Cultural Shifts: Embracing DevOps for Organizational Transformation
Cultural Shifts: Embracing DevOps for Organizational TransformationCultural Shifts: Embracing DevOps for Organizational Transformation
Cultural Shifts: Embracing DevOps for Organizational Transformation
Mindfire Solution
 
BITCOIN HEIST RANSOMEWARE ATTACK PREDICTION
BITCOIN HEIST RANSOMEWARE ATTACK PREDICTIONBITCOIN HEIST RANSOMEWARE ATTACK PREDICTION
BITCOIN HEIST RANSOMEWARE ATTACK PREDICTION
ssuser2b426d1
 

Recently uploaded (20)

Migrate your Infrastructure to the AWS Cloud
Migrate your Infrastructure to the AWS CloudMigrate your Infrastructure to the AWS Cloud
Migrate your Infrastructure to the AWS Cloud
 
Shivam Pandit working on Php Web Developer.
Shivam Pandit working on Php Web Developer.Shivam Pandit working on Php Web Developer.
Shivam Pandit working on Php Web Developer.
 
WEBINAR SLIDES: CCX for Cloud Service Providers
WEBINAR SLIDES: CCX for Cloud Service ProvidersWEBINAR SLIDES: CCX for Cloud Service Providers
WEBINAR SLIDES: CCX for Cloud Service Providers
 
React Native vs Flutter - SSTech System
React Native vs Flutter  - SSTech SystemReact Native vs Flutter  - SSTech System
React Native vs Flutter - SSTech System
 
CViewSurvey Digitech Pvt Ltd that works on a proven C.A.A.G. model.
CViewSurvey Digitech Pvt Ltd that  works on a proven C.A.A.G. model.CViewSurvey Digitech Pvt Ltd that  works on a proven C.A.A.G. model.
CViewSurvey Digitech Pvt Ltd that works on a proven C.A.A.G. model.
 
ThaiPy meetup - Indexes and Django
ThaiPy meetup - Indexes and DjangoThaiPy meetup - Indexes and Django
ThaiPy meetup - Indexes and Django
 
NYC 26-Jun-2024 Combined Presentations.pdf
NYC 26-Jun-2024 Combined Presentations.pdfNYC 26-Jun-2024 Combined Presentations.pdf
NYC 26-Jun-2024 Combined Presentations.pdf
 
How we built TryBoxLang in under 48 hours
How we built TryBoxLang in under 48 hoursHow we built TryBoxLang in under 48 hours
How we built TryBoxLang in under 48 hours
 
COMPSAC 2024 D&I Panel: Charting a Course for Equity: Strategies for Overcomi...
COMPSAC 2024 D&I Panel: Charting a Course for Equity: Strategies for Overcomi...COMPSAC 2024 D&I Panel: Charting a Course for Equity: Strategies for Overcomi...
COMPSAC 2024 D&I Panel: Charting a Course for Equity: Strategies for Overcomi...
 
MVP Mobile Application - Codearrest.pptx
MVP Mobile Application - Codearrest.pptxMVP Mobile Application - Codearrest.pptx
MVP Mobile Application - Codearrest.pptx
 
Development of Chatbot Using AI\ML Technologies
Development of Chatbot Using AI\ML TechnologiesDevelopment of Chatbot Using AI\ML Technologies
Development of Chatbot Using AI\ML Technologies
 
dachnug51 - HCL Sametime 12 as a Software Appliance.pdf
dachnug51 - HCL Sametime 12 as a Software Appliance.pdfdachnug51 - HCL Sametime 12 as a Software Appliance.pdf
dachnug51 - HCL Sametime 12 as a Software Appliance.pdf
 
Addressing the Top 9 User Pain Points with Visual Design Elements.pptx
Addressing the Top 9 User Pain Points with Visual Design Elements.pptxAddressing the Top 9 User Pain Points with Visual Design Elements.pptx
Addressing the Top 9 User Pain Points with Visual Design Elements.pptx
 
dachnug51 - All you ever wanted to know about domino licensing.pdf
dachnug51 - All you ever wanted to know about domino licensing.pdfdachnug51 - All you ever wanted to know about domino licensing.pdf
dachnug51 - All you ever wanted to know about domino licensing.pdf
 
active-directory-auditing-solution (2).pptx
active-directory-auditing-solution (2).pptxactive-directory-auditing-solution (2).pptx
active-directory-auditing-solution (2).pptx
 
Leading Project Management Tool Taskruop.pptx
Leading Project Management Tool Taskruop.pptxLeading Project Management Tool Taskruop.pptx
Leading Project Management Tool Taskruop.pptx
 
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...
 
Intro to Amazon Web Services (AWS) and Gen AI
Intro to Amazon Web Services (AWS) and Gen AIIntro to Amazon Web Services (AWS) and Gen AI
Intro to Amazon Web Services (AWS) and Gen AI
 
Cultural Shifts: Embracing DevOps for Organizational Transformation
Cultural Shifts: Embracing DevOps for Organizational TransformationCultural Shifts: Embracing DevOps for Organizational Transformation
Cultural Shifts: Embracing DevOps for Organizational Transformation
 
BITCOIN HEIST RANSOMEWARE ATTACK PREDICTION
BITCOIN HEIST RANSOMEWARE ATTACK PREDICTIONBITCOIN HEIST RANSOMEWARE ATTACK PREDICTION
BITCOIN HEIST RANSOMEWARE ATTACK PREDICTION
 

A Holistic Approach to Testing in Continuous Delivery

  • 1. @lisacrispin With material from Abby Bangser, Ashley Hunsberger, Lisi Hocke, Janet Gregory, & more Copyright 2023 Lisa Crispin A Holistic Approach to Testing in Continuous Delivery Lisa Crispin
  • 2. @lisacrispin A little about me Co-founder, Agile Testing Fellowship Testing consultant & trainer lisa@lisacrispin.com https://lisacrispin.com Holistic Testing: Strategies for Agile Teams Holistic Testing for Continuous Delivery AgileTestingFellow.com Test Automation U
  • 3. @lisacrispin Today I’m talking about: ● Building confidence for continuous delivery/deployment (CD) ● Guiding conversations about risk & test coverage ● Quality – a whole team responsibility Image from meet.google.com
  • 4. @lisacrispin Some journeys go wrong ● Slow feedback loops ● Regression failures ● Unexpected impacts ● Technical & testing debt
  • 5. @lisacrispin What gets in your team’s way? Stop and think for a minute. What’s the biggest obstacle for your team being successful with continuous delivery? Photo by Matthew Hamilton on Unsplash
  • 6. @lisacrispin One example from my experience ● Team embraced XP practices – TDD, refactoring, pairing, continuous integration, … ● Thousands of automated regression tests at every level ● Reliable deployment pipelines, app in cloud, blue/green deploy But… ● Too few testers ● Still had manual release regression checklist ● No time for sufficient exploratory testing This Photo by Unknown Author is licensed under CC BY-NC-ND
  • 7. @lisacrispin Making frequent small changes confidently takes: ● Collaboration ● Continuous improvement ● Continuous learning ● A holistic approach
  • 10. @lisacrispin Visualize your pipeline, optimize feedback Faster feedback More confidence
  • 11. @lisacrispin Do your automated test suites give you confidence? Confidence Flaky tests? Poor coverage? Hard to maintain? Hard to diagnose?
  • 13. @lisacrispin Some of my favorite Test Canvas questions ● What information should each suite provide? To whom? How? ● How will the team know about test failures? Who is responsible for looking into those? ● Do you pair on test automation, or do test code reviews? This Photo by Unknown Author is licensed under CC BY-NC
  • 14. @lisacrispin Mitigating risks with automated regression tests
  • 15. @lisacrispin Talking about risk Risk Storming (https://riskstormingonline.com) Traditional risk analysis Mind maps
  • 16. @lisacrispin Does your team know all the possible risks? ● Customers behave in unexpected ways ● Infrastructure components may fail ● External systems can impact ours ● … What do we do?
  • 17. @lisacrispin Observability ● Ask the questions you didn’t know you’d need to ask - unknowns ● Complex systems fail in complex ways ● With enough information, we can respond quickly Cindy Sridharan, https://medium.com/@copy construct/testing-in- production-the-safe-way- 18ca102d0ef1
  • 19. @lisacrispin In which stages do you currently get involved?
  • 20. @lisacrispin What makes it work? ● Commitment to a level of quality ○ Identifying what’s valuable to customers ○ Bug prevention over bug detection ○ Fast response to prod issues ● Diverse perspectives, skill sets, biases ● Competencies > roles
  • 21. @lisacrispin Cross-discipline pairing, ensemble testing Picture from Ensemble Programming Guidebook, Maaret Pyhäjärvi
  • 22. @lisacrispin We’re humans! (or possibly dragons, donkeys, unicorns…) Build relationships ● Friendly conversations ● Do food ● Share something useful ● Ask for help Katrina Clokie has excellent tips in her book, A Practical Guide to Testing in DevOps
  • 23. @lisacrispin Building a quality culture • Transformative leaders • Trust and psychological safety • Whole team ”owns” product • “You build it, you run it” • Focus on quality, not speed
  • 24. @lisacrispin Principles of CD – Jez Humble & David Farley ● Build quality in ● Work in small batches ● Computers perform repetitive tasks, people solve problems ● Relentlessly pursue continuous improvement ● Everyone is responsible
  • 25. @lisacrispin Get your team together and talk What are our quality goals? Are we committed to building quality in? How can we apply the Holistic Testing Model to fit testing activities in throughout the software development life cycle?
  • 26. @lisacrispin A few resources ● ”Agile Testing for the Whole Team” training course, https://agiletestingfellow.com ● Agile Testing Condensed, Agile Testing and More Agile Testing, Lisa Crispin and Janet Gregory, https://agiletester.ca ● “Test Automation in DevOps”, A Test Automation U course by Lisa Crispin https://testautomationu.applitools.com/test-automation-in-devops/ ● Continuous Delivery by Jez Humble and David Farley, https://continuousdelivery.com ● A Practical Guide to Testing in DevOps by Katrina Clokie https://leanpub.com/testingindevops ● Accelerate by Dr. Nicole Forsgren, Jez Humble, Gene Kim ● Ashley Hunsberger’s Test Suite Canvas https://github.com/ahunsberger/TestSuiteDesign ● https://lisacrispin.com/observability-continuous-delivery-devops-related-resources/ ● ExploreIt! Elizabeth Hendrickson