SlideShare a Scribd company logo
Practical Test Process
Improvement using ISTQB
Anton Muzhailo
Speaker info
Anton Muzhailo, Ukraine
Senior Python Automation Engineer, GlobalLogic
• ISTQB Certified Test Manager
• ISTQB Certified Trainings Coach
• 3 years of mentoring experience, 200+ students
in/muzhailo/
General Question
What is the value of ISTQB knowledge and
certifications?
- Common vocabulary of testing terms
- Test process consistency improving
- Test effectiveness improving
- Common & best practices
- Recognition as expert
- Be on the same level as many others
True Question
What is the TRUE value of ISTQB knowledge and
certifications?
- Clear and visible ROI is a great argument for management
- Calculated real amount of time and resources saved
- Wider view-angle always leads to better risk-management
- Starting “from scratch” always require some classical basis to start
- Adapting classical approaches to real cases using your expertise
- Lots of approaches and tools cannot become obsolete

Recommended for you

Kristian Fischer - Put Test in the Driver's Seat
Kristian Fischer - Put Test in the Driver's SeatKristian Fischer - Put Test in the Driver's Seat
Kristian Fischer - Put Test in the Driver's Seat

The document discusses how test managers are often seen as "black sheep" who raise issues without solutions and cause delays. It argues that test managers need to shift from a reactive to proactive role by getting involved early in projects, changing attitudes, and applying a test management dashboard to provide transparency and value. The dashboard would use KPIs and metrics to track testing progress, quality, risks, and deliver early warnings so test managers are seen as project victors rather than victims.

software testing
Test process improvement – how hard can it be?
Test process improvement – how hard can it be?Test process improvement – how hard can it be?
Test process improvement – how hard can it be?

This presentation looks at how to make any improvement project a success, and it does this by using some fun case study examples looking at how Geoff has seen things go wrong, and from the depth of despair we will look at how things can be made to be successful. View webinar recording - https://testhuddle.com/resource/test-process-improvement-hard-can/

test huddleesconfstest process
Oana Feidi - Debugging - Root cause analysis - CodeCamp-10-may-2014
Oana Feidi - Debugging - Root cause analysis - CodeCamp-10-may-2014Oana Feidi - Debugging - Root cause analysis - CodeCamp-10-may-2014
Oana Feidi - Debugging - Root cause analysis - CodeCamp-10-may-2014

1) The document discusses root cause analysis techniques like the 5 Whys approach to identify underlying causes of problems that are not obvious from symptoms. It provides an example of using 5 Whys to determine why a customer was unhappy. 2) Lessons learned processes are described, including organizing workshops at the end of projects to identify positive and negative technical and management lessons to derive solutions and prevent repeat mistakes. 3) Applying lessons learned from previous similar projects is advised by selecting applicable lessons upfront and defining specific actions to address them over the course of new projects.

Quality Assessment
Problem: Your are a Test Manager of a team on a
new project moved from another country. There
are 500 test cases written. No formal specification
available.
• How to see “the big picture”?
• What is the ~ test coverage?
• How to start?
• Where to focus the testing effort?
Quality Metrics
Metrics that are not valid are dangerous!
Metrics
Qualitative Quantitative
- Expensive
- Risky
- Hard to do well
- Suffer from own
qualitative problems
- Could give a false
impression
- Can’t be used everywhere
- Could be invalid
- Hard to choose valid
metrics
Is this metric valid?
𝑇𝑒𝑠𝑡 𝑐𝑎𝑠𝑒 𝑝𝑎𝑠𝑠 𝑟𝑎𝑡𝑖𝑜 =
# 𝑜𝑓 𝑡𝑒𝑠𝑡𝑠 𝑝𝑎𝑠𝑠𝑒𝑑
# 𝑜𝑓 𝑡𝑒𝑠𝑡𝑠 𝑟𝑎𝑛 Pretty popular, huh?
Is this metric valid?
𝑇𝑒𝑠𝑡 𝑐𝑎𝑠𝑒 𝑝𝑎𝑠𝑠 𝑟𝑎𝑡𝑖𝑜 =
# 𝑜𝑓 𝑡𝑒𝑠𝑡𝑠 𝑝𝑎𝑠𝑠𝑒𝑑
# 𝑜𝑓 𝑡𝑒𝑠𝑡𝑠 𝑟𝑎𝑛
- How about other 238459293465013475
tests, which were not run or even defined?
Pretty popular, huh?

Recommended for you

Isabel Evans - Quality In Use - EuroSTAR 2011
Isabel Evans - Quality In Use - EuroSTAR 2011Isabel Evans - Quality In Use - EuroSTAR 2011
Isabel Evans - Quality In Use - EuroSTAR 2011

EuroSTAR Software Testing Conference 2011 presentation on Quality In Use by Isabel Evans. See more at: http://conference.eurostarsoftwaretesting.com/past-presentations/

eurostar conferencesoftware testing
New Model Testing: A New Test Process and Tool
New Model Testing:  A New Test Process and ToolNew Model Testing:  A New Test Process and Tool
New Model Testing: A New Test Process and Tool

Paul Gerrard presented a new test process and tool called Cervaya that combines elements of structured and exploratory testing. The process involves testers surveying features using Cervaya to iteratively build system models and test plans. This shifts testing earlier in the development process. Cervaya logs tester activity, supports real-time collaboration, and could generate documentation. The goal is to make testing more aligned with agile and continuous delivery approaches. Gerrard invited collaboration on further developing Cervaya.

robotstest managementeurostar conferences
Controlling Project during Development with a Defect Model, Ben Linders, Euro...
Controlling Project during Development with a Defect Model, Ben Linders, Euro...Controlling Project during Development with a Defect Model, Ben Linders, Euro...
Controlling Project during Development with a Defect Model, Ben Linders, Euro...

The document describes a project defect model used at Ericsson to control project quality and performance. It tracks defects inserted during phases like specification and design, and detects defects during testing phases. This provides early signals about quality risks. On a pilot project, the model helped clarify requirements, enforce design rules, and focus testing. It accurately predicted defects at release, allowing better planning. The model has now been adopted for several projects to improve quality tracking, risk management, and decision making.

defectsbusinesspdm
Is this metric valid?
𝑇𝑒𝑠𝑡 𝑐𝑎𝑠𝑒 𝑝𝑎𝑠𝑠 𝑟𝑎𝑡𝑖𝑜 =
# 𝑜𝑓 𝑡𝑒𝑠𝑡𝑠 𝑝𝑎𝑠𝑠𝑒𝑑
# 𝑜𝑓 𝑡𝑒𝑠𝑡𝑠 𝑟𝑎𝑛
- How about other 238459293465013475
tests, which were not run or even defined?
Pretty popular, huh?
- But we measure test coverage… Seems sufficient!
Is this metric valid?
𝑇𝑒𝑠𝑡 𝑐𝑎𝑠𝑒 𝑝𝑎𝑠𝑠 𝑟𝑎𝑡𝑖𝑜 =
# 𝑜𝑓 𝑡𝑒𝑠𝑡𝑠 𝑝𝑎𝑠𝑠𝑒𝑑
# 𝑜𝑓 𝑡𝑒𝑠𝑡𝑠 𝑟𝑎𝑛
- How about other 238459293465013475
tests, which were not run or even defined?
Pretty popular, huh?
- But we measure test coverage… Seems sufficient!
- Using which metrics?
Good metrics
Once you have two choices with its own pros and cons –
you need a hybrid to have both pros’ and none cons
© Unicorn from fluffy world
Good metrics
Once you have two choices with its own pros and cons –
you need a hybrid to have both pros’ and none cons
The popular way of risk mitigation is to split the risk areas to the
small parts and manage risks within these parts.
© Unicorn from fluffy world
Our world require risk assessments

Recommended for you

Making a Project a Complete Success with Post-Implementation Strategies | Jul...
Making a Project a Complete Success with Post-Implementation Strategies | Jul...Making a Project a Complete Success with Post-Implementation Strategies | Jul...
Making a Project a Complete Success with Post-Implementation Strategies | Jul...

Going live is a huge accomplishment, but the journey has only just begun. Now that you’re operating in your new system, what are your plans to get the most of it and minimize disruptions? Hindsight is 20-20. Gain strategies from someone who lives in the post-implementation world. She not only knows the common pitfalls and setbacks, but will share tools and techniques of how to resolve and prevent them! Join Julie, NAV Support Lead to gain strategies, tools and techniques to succeed at making your vision a reality after go-live. Julie Bilodeau BDO Canada LLP BDO Connections 2016 | Knowledge Pod

microsoft dynamicsbusiness solutionsoftware testing
Ruud Teunissen - Test Process Improvement on a Shoestring
Ruud Teunissen -  Test Process Improvement on a Shoestring Ruud Teunissen -  Test Process Improvement on a Shoestring
Ruud Teunissen - Test Process Improvement on a Shoestring

EuroSTAR Software Testing Conference 2009 presentation on Test Process Improvement on a Shoestring by Ruud Teunissen. See more at conferences.eurostarsoftwaretesting.com/past-presentations/

software testing
How to use selenium successfully
How to use selenium successfullyHow to use selenium successfully
How to use selenium successfully

In this webinar, Dave Haeffner (Elemental Selenium, USA) discusses how to: - Build an integrated feedback loop to automate test runs and find issues fast - Setup your own infrastructure or connect to a cloud provider -Dramatically improve test times with parallelization https://huddle.eurostarsoftwaretesting.com/resource/webinar/use-selenium-successfully/

weninaresconfssoftware testing
Such a deep term “Quality”
• Must be measured every iteration
• Have to be defined in %
• Bugs should be represented in Quality’s terms
• Such metrics gives more clear vision about product readiness
© ISTQB
Glossary
Quality. The degree to which a component, system or process meets
specified requirements and/or user/customer needs and expectations.
Test Coverage Measurement
Test cases fully covers the area 1 point
Test cases almost covers the area 0.9 pts
Test cases covers some of functionality 0.5 pts
There are some basic test cases written 0.1 pts
There are no test cases written 0 pts
𝑇𝑒𝑠𝑡 𝑐𝑜𝑣𝑒𝑟𝑎𝑔𝑒 =
𝑇𝑜𝑡𝑎𝑙 𝑃𝑜𝑖𝑛𝑡𝑠
𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑎𝑟𝑒𝑎𝑠
𝑇𝑜𝑡𝑎𝑙 𝑝𝑜𝑖𝑛𝑡𝑠 = 𝑎𝑟𝑒𝑎1_𝑝𝑡𝑠 + … + 𝑎𝑟𝑒𝑎𝑁_𝑝𝑡𝑠
• Area size can vary depending on your expectations and resources
• Each area can be divided and measured using the same approach
Business workflow map
Name Q
Search 11
Product Page 10
Main Page 9
Cart 9
Login 8
Discounts 8
Supported Countries 5
FAQ 5
More Details 5
Shipping Terms 4
Product Category 4
Checkout 4
Registration 4
Product Subcategory 3
Comment & rate 3
Filters 2
Business map: https://ufile.io/19o89
Test Coverage Measurement
𝑇𝑒𝑠𝑡 𝑐𝑜𝑣𝑒𝑟𝑎𝑔𝑒 =
9.6
16
= 0.6 ⇒ 60%
𝑇𝑜𝑡𝑎𝑙 𝑝𝑜𝑖𝑛𝑡𝑠 = 9.6
# Name Points
1 Search 0.5
2 Product Page 0.9
3 Main Page 1
4 Cart 1
5 Login 1
6 Discounts 0.1
7 Supported Countries 0.5
8 FAQ 0
9 More Details 0.5
10 Shipping Terms 1
11 Product Category 0.5
12 Checkout 0.9
13 Registration 1
14 Product Subcategory 0.1
15 Comment & rate 0.5
16 Filters 0.1
𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑎𝑟𝑒𝑎𝑠 = 16
• This approach can be used for progress measurement on every iteration
• This approach can show the most and least covered areas
• Estimates should be done using review with multiple participants
• Correlation between most/least covered areas and areas priority/risk can
be established and measured
• If each area is still too big to estimate – it can be divided to sub-areas and
estimated the same way

Recommended for you

Controlling Project during Development with a Defect Model, Ben Linders, ICST...
Controlling Project during Development with a Defect Model, Ben Linders, ICST...Controlling Project during Development with a Defect Model, Ben Linders, ICST...
Controlling Project during Development with a Defect Model, Ben Linders, ICST...

Wouldn’t it be nice if you could have more insight into the quality of a product, while it is developed, and not afterwards? Would you like to be able to estimate how many defects are inserted in the product in a certain phase, and how effective a (test) phase is in capturing these defects? To optimize your test phases regarding focus and effort in relation to how many defects they will find? This presentation will show a simple but very effective model that makes it possible: The Project Defect Model. The aim of the Project Defect Model is to track product quality, take corrective actions and reduce quality risks. To get more insight into the quality of the product during development, it is needed to measure the software development processes with two views: Introduction and detection of defects. Introduction is done during the specification, design and coding phases; defects are either introduced into documents or into the actual product. Detection of defects is done via inspections and test during all the phases of the project. A tool was developed using a spreadsheet. The purpose of the tool was to estimate the number of defects per phase, and to track all defects discovered in inspections and tests against these estimates. The tool supported analysis of the data with both calculated values and graphs comparing actuals to estimates in terms of current status and trends over time. The Project Defect Model has been beneficial to projects. It has helped estimating, planning, and tracking quality during the project, including an estimate of the the number of defects left in the released product. The quality data has been used in the project together with time and cost data, to take better decisions on test, review and inspections, and design. Also it has identified quality risks at an early stage, helping the project take corrective actions and decisions on product release and maintenance capacity planning. Finally the model provided insight into the effectiveness of the verification activities, supporting effective process improvement. Paper: The presentation is on a defect planning/tracking tool and approach. Focus will be upon: • Goals: What was the purpose of the model, why developed, what did we want to reach? • How: Show the definition of the model and its implementation and application. • Tools: The tool that was developed to implement the model, how it works, strengths. • Results: How did the model and tool help the project? Did it live up to its purpose? • Success factors: What were key issues that we have dealt successfully with? • Future: How is this model used in future projects, what could further increase its benefits?

seidefectsbusiness
New model
New modelNew model
New model

I believe that our existing models of testing are not fit for purpose – they are inconsistent, controversial, partial, proprietary and stuck in the past. They are not going to support us in the rapidly emerging technologies and approaches. The certification schemes that should represent the interests and integrity of our profession don’t, and we are left with schemes that are popular, but have low value, lower esteem and attract harsh criticism. My goal in proposing the New Model is to stimulate new thinking in this area. eurostarconferences.com testhuddle.com

One size does not fit all
One size does not fit allOne size does not fit all
One size does not fit all

This document discusses choosing software development processes and methods that fit an organization's needs. It argues that one size does not fit all and that the development leader should understand their work and mix of projects in order to choose appropriate methods. Projects can involve varying levels of innovation and uncertainty, and the document provides examples of analytics and practices that fit different types of projects, from routine low-innovation work to high-innovation projects with more uncertainty. It emphasizes the need to measure progress and continuously adapt methods based on outcomes.

sgce
Regression amount estimation
 Change in Registration
shows the regression area
immediately
 Easier to prioritize the tests
 Faster impact and
regression estimation in
case of customer issue
Issue-based metrics
# Area Number of
Blockers
Number of
Majors
Number of
Minors
1 Search 0 8 14
2 Product Page 2 5 11
3 Main Page 0 0 5
4 Cart 2 3 1
5 Login 0 0 7
6 Discounts 0 3 4
7 Supported Countries 0 0 0
… … … … …
- So, what’s the quality?
- Where we are?
- Are we ready to release?
- When we will be ready?
Accepted definition of ready:
• No blockers
• All majors have workarounds
• No severe Performance or Usability impact
- Not yet.
- 4 blockers to go
- No, 13 majors doesn’t have workarounds
- Dunno
Impact-based metrics
# Area Blockers
impact
Majors
impact
Minors
impact
Total
impact
1 Search * * * 0.5
2 Product Page * * * 0
3 Main Page * * * 0.9
4 Cart * * * 0.5
5 Login * * * 0.9
6 Discounts * * * 0.9
7 Supported Countries * * * 1
… … … … …
No impact (can be released) 1 point
Little impact (can be released) 0.9 pts
Available for regular testing (can’t be released) 0.5 pts
Severe impact (not available for release/testing) 0.1 pts
Doesn’t exist or completely blocked 0 pts
* - optional
𝑄𝑢𝑎𝑙𝑖𝑡𝑦 =
4.7
7
= 0.67 ⇒ 67%
𝑇𝑜𝑡𝑎𝑙 𝑝𝑜𝑖𝑛𝑡𝑠 = 3.9 𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑎𝑟𝑒𝑎𝑠 = 7
Quality can also me measured roughly:
𝑄𝑢𝑎𝑙𝑖𝑡𝑦 =
# 𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛𝑎𝑙𝑖𝑡𝑖𝑒𝑠 𝑟𝑒𝑎𝑑𝑦 𝑓𝑜𝑟 𝑟𝑒𝑙𝑒𝑎𝑠𝑒
𝑡𝑜𝑡𝑎𝑙
𝑄𝑢𝑎𝑙𝑖𝑡𝑦 =
4
7
= 0.57 = 57%
Test Plan
Test Plan. A document describing the scope, approach, resources and schedule of intended
test activities. It identifies amongst others test items, the features to be tested, the testing
tasks, who will do each task, degree of tester independence, the test environment, the test
design techniques and entry and exit criteria to be used, and the rationale for their choice, and
any risks requiring contingency planning. It is a record of the test planning process
© ISTQB
Glossary
Problem: Do we really need test plans in Agile? We have JIRA with stories
and epics. Why we need an additional test artifact?
- Too
complex…
- Obsolete
artifact
- Too much additional
paperwork
- We are still fine
without it, don’t care
- Nah..my grandma
used that in 50s’
- That’s only for mature
organizations

Recommended for you

Growing a Company Test Community: Roles and Paths for Testers
Growing a Company Test Community: Roles and Paths for TestersGrowing a Company Test Community: Roles and Paths for Testers
Growing a Company Test Community: Roles and Paths for Testers

Over the past three years, our company’s test team has grown from three lonesome testers to a community of nine – with more planned. Since we don’t see testers as “click monkeys”, but as valuable and integrated project members who bring a specific skill set to the table, it’s important for us to choose testers well and to train them in various areas so that they can contribute, grow and see their own career path within testing. To structure to our internal tester training program, we have been developing role descriptions, education paths and career options for our testers, which I’d like to share with you in this webinar. View webinar - https://huddle.eurostarsoftwaretesting.com/resource/webinar/growing-company-test-community-roles-paths-testers/

pathswebinareurostar conferences
Lean pilots by Mariya Breyter from Dun & Bradstreet
Lean pilots by Mariya Breyter from Dun & BradstreetLean pilots by Mariya Breyter from Dun & Bradstreet
Lean pilots by Mariya Breyter from Dun & Bradstreet

Lean pilots provide an innovative framework for solving enterprise challenges through agile and lean methodologies. Case studies highlight successes in automating contracts, streamlining third party onboarding, improving the DUNS research process, and ensuring Privacy Shield compliance. Lessons learned include welcoming early failure, using cross-functional self-organizing teams, making decisions with data, and ensuring enterprise collaboration. The framework establishes a repeatable process for running lean experiments to minimize risk and waste.

lean it summit 2017devopsagile
Dr. Tafline Murnane & Dr. Stuart Reid - Practical Approaches to Motivating Te...
Dr. Tafline Murnane & Dr. Stuart Reid - Practical Approaches to Motivating Te...Dr. Tafline Murnane & Dr. Stuart Reid - Practical Approaches to Motivating Te...
Dr. Tafline Murnane & Dr. Stuart Reid - Practical Approaches to Motivating Te...

EuroSTAR Software Testing Conference 2011 presentation on Practical Approaches to Motivating Testers by Dr. Tafline Murnane & Dr. Stuart Reid . See more at conferences.eurostarsoftwaretesting.com/past-presentations/

motivation
Test Plan artifacts
1. Test Plan Identifier
2. References
3. Introduction
4. Test Items
5. Software Risk Issues
6. Features to be Tested
7. Features not to be Tested
8. Approach
9. Item Pass/Fail Criteria
10. Suspension Criteria and Resumption Requirements
11. Test Deliverables
12. Remaining Test Tasks
13. Environmental Needs
14. Staffing and Training Needs
15. Responsibilities
16. Schedule
17. Planning Risks and Contingencies
18. Approvals
19. Glossary
Define what you need.
That’s a basis long enough to
cover everything’s needed
Focusing on ‘big picture’
1 2 3 4 5 60
• 6 new features
• New environment
• 20 known issues fix
Full Regression
RELEASE 1 RELEASE 2
Full Regression
New features
Testing
New features
Testing
Validation sprint Validation sprint
Test closure
• New major version
• Mac support
TC design
TC update
TC update
 Provides an “expected result” for test monitoring and control activities
 Shows the planning problems as early as possible
 Allows complex planning in case of several test teams using Master Test Plan
 Shows the “big picture”
Test Control & Monitoring
• Such activities can be
performed best using the
metrics and actions,
defined within test plan.
• For each metric there
should be the action
point predicted.
• The evaluate method also
should be clearly defined
• Defect metrics
• Pass rate
• Employees
• Equipment
• Licenses
• Difference
between
Planned vs
Actual
• Time
• Money
Cost Schedule
QualityResources
Test Control & Monitoring
# Metrics Metrics
Type
Collected
Period
Evaluate Method Criteria Conclusion
1 Milestone
achieving time
Schedule Weekly Compare the actual time
spent vs planned
+/- 20% If deviation is above acceptable
values – need an action point
2 Nightly CI set
of tests exec.
time
Resource Daily Check if previous run
took less than 10 hrs
< 10
hrs
12 hours between each workday. If
set of tests reaches 10 hrs – we need
another environment set to order
3 Cloud
environment
cost
Cost Weekly Compare the estimated
budget draw rate with
actual
<120% Cost rate depends on traffic and
number of users. If deviation is more
then 10% we need an action point
4 Duplicate /
Not a bug
issues
Quality Monthly Comparing the actual
number with total bugs
<10% If more - a time loss. Possibly some
engineers need the testing skills
improvement or product training
5 Defect
clustering
Quality Monthly Determine modules,
which has many issues in
> 5% If more – this modules require
attention and effort allocation

Recommended for you

How to Juggle Multiple Beta Tests at Once
How to Juggle Multiple Beta Tests at OnceHow to Juggle Multiple Beta Tests at Once
How to Juggle Multiple Beta Tests at Once

Running a single beta test is hard. Running multiple beta tests can make you want to tear your hair out! In this webinar, our beta managers as they share their advice for handling the demands of multiple beta tests at once. Click this link to access the on-demand webinar: https://www.centercode.com/webinar/2016/july/

centercodebeta testingcustomer validation
DevOps Test Engineering: Putting the ‘Continuous’ in Testing, an ITSM Academy...
DevOps Test Engineering: Putting the ‘Continuous’ in Testing, an ITSM Academy...DevOps Test Engineering: Putting the ‘Continuous’ in Testing, an ITSM Academy...
DevOps Test Engineering: Putting the ‘Continuous’ in Testing, an ITSM Academy...

Presenter: Anne Hungate President, Daring Systems You’ve heard about Continuous Integration and Continuous Delivery but what’s common as code makes its way through those processes? Testing. With DevOps Testing (also known as Continuous Testing), testing tasks are engineered to be continuously completed end-to-end across the entire development to deployment pipeline. Developers, QA analysts, security professionals, IT Operations analysts…everyone is a tester in a DevOps environment. Join us to learn more about DevOps Testing and the emerging role of DevOps Test Engineer.

operationsanalyzedevops
Gerlof Hoekstra - OMG What Have We Done - EuroSTAR 2013
Gerlof Hoekstra - OMG What Have We Done - EuroSTAR 2013Gerlof Hoekstra - OMG What Have We Done - EuroSTAR 2013
Gerlof Hoekstra - OMG What Have We Done - EuroSTAR 2013

EuroSTAR Software Testing Conference 2013 presentation on "OMG What Have We Done" by Gerlof Hoekstra. See more at: http://conference.eurostarsoftwaretesting.com/past-presentations/

eurostar conferencesoftware testing
Defects severity estimation
Problem: How to choose the good and precise technique for severity
estimation no matter how many different testers with different points of
view involved?
Requirement: Application should support launching in UI
mode both from command-line and desktop.
Issue: Dropbox doesn’t start using command-line.
What is the Severity?
Defects severity estimation
Problem: How to choose the good and precise technique for severity
estimation no matter how many different testers with different points of
view involved?
Requirement: Application should support launching in UI
mode both from command-line and desktop.
Issue: Dropbox doesn’t start using command-line.
What is the Severity?
Defects severity estimation
Severity. The degree of impact that a defect has on the development or
operation of a component or system. © ISTQB
Glossary
Critical Leads to termination. Failed function is unusable and NO acceptable workaround
Major Leads to termination. Failed function is unusable and acceptable workaround EXISTS
Moderate Doesn’t result in termination.
Failed function produces incorrect/incomplete results and NO acceptable workaround
Minor Doesn’t result in termination.
Failed function produces incorrect/incomplete results and acceptable workaround EXISTS
Cosmetic Impacting look and feel of the application or related to enhancement
Define your own severity estimation guidelines!
Root cause analysis
Test
objective
Test
condition 1
Test case
1.1
Test case
1.2
Defect 1
Test
condition 2
Test case
2.1
Test
condition 3
Test case
3.1
Test case
3.2
??? ??? Defect 2
o What is the root cause
of Defect 1?
o Maybe several defects
have the same root
cause?
o Why Defect 2 wasn’t
covered by any of test
cases?
o How to improve the test
process?

Recommended for you

10 signs your testing is not enough
10 signs your testing is not enough10 signs your testing is not enough
10 signs your testing is not enough

The document discusses 10 signs that an organization's software testing may not be enough. These include having excessive production bugs, bugs found during user acceptance testing, growing bug counts over test cycles, not investing in testing compared to competitors, lacking clear criteria for what constitutes "enough" testing, testers advising against releasing software, weak prevention efforts like code reviews, lack of developer unit testing, frequently reduced testing periods causing deadline problems, and high tester turnover. The document advocates treating testing as risk management, increasing test reuse and automation, and addresses common challenges and questions around software testing.

sqa days 20
2 anton muzhailo - formal test process improvement. how to invest to the te...
2   anton muzhailo - formal test process improvement. how to invest to the te...2   anton muzhailo - formal test process improvement. how to invest to the te...
2 anton muzhailo - formal test process improvement. how to invest to the te...

Anton muzhailo - formal test process improvement. how to invest to the test process improvement and win

Is Test Planning a lost art in Agile? by Michelle Williams
Is Test Planning a lost art in Agile? by Michelle WilliamsIs Test Planning a lost art in Agile? by Michelle Williams
Is Test Planning a lost art in Agile? by Michelle Williams

This document provides an overview of a presentation on agile test planning. It discusses the challenges of agile requirements and how test strategies serve a purpose beyond a single sprint. It also examines how the agile manifesto relates to planning and the value of test plans in agile. The presentation outlines four testing phases in agile - requirements and design, story/feature verification, system verification, and acceptance. It provides examples of what should be included in a test plan for each phase such as scenarios, automation approach, dependencies, and acceptance criteria.

Questions
5 minutes.
You can also ask questions for me in the lounge zone

More Related Content

What's hot

Useful stepping stones in growth towards Agile testing door Kees Blokland.
Useful stepping stones in growth towards Agile testing door Kees Blokland.Useful stepping stones in growth towards Agile testing door Kees Blokland.
Useful stepping stones in growth towards Agile testing door Kees Blokland.
Kees Blokland
 
Jonathan Kohl - Is Agile Distracting You?
Jonathan Kohl - Is Agile Distracting You?Jonathan Kohl - Is Agile Distracting You?
Jonathan Kohl - Is Agile Distracting You?
TEST Huddle
 
Dorothy Graham - Can The Past Tell Us The Future
Dorothy Graham -  Can The Past Tell Us The FutureDorothy Graham -  Can The Past Tell Us The Future
Dorothy Graham - Can The Past Tell Us The Future
TEST Huddle
 
Kristian Fischer - Put Test in the Driver's Seat
Kristian Fischer - Put Test in the Driver's SeatKristian Fischer - Put Test in the Driver's Seat
Kristian Fischer - Put Test in the Driver's Seat
TEST Huddle
 
Test process improvement – how hard can it be?
Test process improvement – how hard can it be?Test process improvement – how hard can it be?
Test process improvement – how hard can it be?
TEST Huddle
 
Oana Feidi - Debugging - Root cause analysis - CodeCamp-10-may-2014
Oana Feidi - Debugging - Root cause analysis - CodeCamp-10-may-2014Oana Feidi - Debugging - Root cause analysis - CodeCamp-10-may-2014
Oana Feidi - Debugging - Root cause analysis - CodeCamp-10-may-2014
Codecamp Romania
 
Isabel Evans - Quality In Use - EuroSTAR 2011
Isabel Evans - Quality In Use - EuroSTAR 2011Isabel Evans - Quality In Use - EuroSTAR 2011
Isabel Evans - Quality In Use - EuroSTAR 2011
TEST Huddle
 
New Model Testing: A New Test Process and Tool
New Model Testing:  A New Test Process and ToolNew Model Testing:  A New Test Process and Tool
New Model Testing: A New Test Process and Tool
TEST Huddle
 
Controlling Project during Development with a Defect Model, Ben Linders, Euro...
Controlling Project during Development with a Defect Model, Ben Linders, Euro...Controlling Project during Development with a Defect Model, Ben Linders, Euro...
Controlling Project during Development with a Defect Model, Ben Linders, Euro...
Ben Linders
 
Making a Project a Complete Success with Post-Implementation Strategies | Jul...
Making a Project a Complete Success with Post-Implementation Strategies | Jul...Making a Project a Complete Success with Post-Implementation Strategies | Jul...
Making a Project a Complete Success with Post-Implementation Strategies | Jul...
Katie Elliott
 
Ruud Teunissen - Test Process Improvement on a Shoestring
Ruud Teunissen -  Test Process Improvement on a Shoestring Ruud Teunissen -  Test Process Improvement on a Shoestring
Ruud Teunissen - Test Process Improvement on a Shoestring
TEST Huddle
 
How to use selenium successfully
How to use selenium successfullyHow to use selenium successfully
How to use selenium successfully
TEST Huddle
 
Controlling Project during Development with a Defect Model, Ben Linders, ICST...
Controlling Project during Development with a Defect Model, Ben Linders, ICST...Controlling Project during Development with a Defect Model, Ben Linders, ICST...
Controlling Project during Development with a Defect Model, Ben Linders, ICST...
Ben Linders
 
New model
New modelNew model
New model
TEST Huddle
 
One size does not fit all
One size does not fit allOne size does not fit all
One size does not fit all
Software Guru
 
Growing a Company Test Community: Roles and Paths for Testers
Growing a Company Test Community: Roles and Paths for TestersGrowing a Company Test Community: Roles and Paths for Testers
Growing a Company Test Community: Roles and Paths for Testers
TEST Huddle
 
Lean pilots by Mariya Breyter from Dun & Bradstreet
Lean pilots by Mariya Breyter from Dun & BradstreetLean pilots by Mariya Breyter from Dun & Bradstreet
Lean pilots by Mariya Breyter from Dun & Bradstreet
Institut Lean France
 
Dr. Tafline Murnane & Dr. Stuart Reid - Practical Approaches to Motivating Te...
Dr. Tafline Murnane & Dr. Stuart Reid - Practical Approaches to Motivating Te...Dr. Tafline Murnane & Dr. Stuart Reid - Practical Approaches to Motivating Te...
Dr. Tafline Murnane & Dr. Stuart Reid - Practical Approaches to Motivating Te...
TEST Huddle
 
How to Juggle Multiple Beta Tests at Once
How to Juggle Multiple Beta Tests at OnceHow to Juggle Multiple Beta Tests at Once
How to Juggle Multiple Beta Tests at Once
Centercode
 
DevOps Test Engineering: Putting the ‘Continuous’ in Testing, an ITSM Academy...
DevOps Test Engineering: Putting the ‘Continuous’ in Testing, an ITSM Academy...DevOps Test Engineering: Putting the ‘Continuous’ in Testing, an ITSM Academy...
DevOps Test Engineering: Putting the ‘Continuous’ in Testing, an ITSM Academy...
ITSM Academy, Inc.
 

What's hot (20)

Useful stepping stones in growth towards Agile testing door Kees Blokland.
Useful stepping stones in growth towards Agile testing door Kees Blokland.Useful stepping stones in growth towards Agile testing door Kees Blokland.
Useful stepping stones in growth towards Agile testing door Kees Blokland.
 
Jonathan Kohl - Is Agile Distracting You?
Jonathan Kohl - Is Agile Distracting You?Jonathan Kohl - Is Agile Distracting You?
Jonathan Kohl - Is Agile Distracting You?
 
Dorothy Graham - Can The Past Tell Us The Future
Dorothy Graham -  Can The Past Tell Us The FutureDorothy Graham -  Can The Past Tell Us The Future
Dorothy Graham - Can The Past Tell Us The Future
 
Kristian Fischer - Put Test in the Driver's Seat
Kristian Fischer - Put Test in the Driver's SeatKristian Fischer - Put Test in the Driver's Seat
Kristian Fischer - Put Test in the Driver's Seat
 
Test process improvement – how hard can it be?
Test process improvement – how hard can it be?Test process improvement – how hard can it be?
Test process improvement – how hard can it be?
 
Oana Feidi - Debugging - Root cause analysis - CodeCamp-10-may-2014
Oana Feidi - Debugging - Root cause analysis - CodeCamp-10-may-2014Oana Feidi - Debugging - Root cause analysis - CodeCamp-10-may-2014
Oana Feidi - Debugging - Root cause analysis - CodeCamp-10-may-2014
 
Isabel Evans - Quality In Use - EuroSTAR 2011
Isabel Evans - Quality In Use - EuroSTAR 2011Isabel Evans - Quality In Use - EuroSTAR 2011
Isabel Evans - Quality In Use - EuroSTAR 2011
 
New Model Testing: A New Test Process and Tool
New Model Testing:  A New Test Process and ToolNew Model Testing:  A New Test Process and Tool
New Model Testing: A New Test Process and Tool
 
Controlling Project during Development with a Defect Model, Ben Linders, Euro...
Controlling Project during Development with a Defect Model, Ben Linders, Euro...Controlling Project during Development with a Defect Model, Ben Linders, Euro...
Controlling Project during Development with a Defect Model, Ben Linders, Euro...
 
Making a Project a Complete Success with Post-Implementation Strategies | Jul...
Making a Project a Complete Success with Post-Implementation Strategies | Jul...Making a Project a Complete Success with Post-Implementation Strategies | Jul...
Making a Project a Complete Success with Post-Implementation Strategies | Jul...
 
Ruud Teunissen - Test Process Improvement on a Shoestring
Ruud Teunissen -  Test Process Improvement on a Shoestring Ruud Teunissen -  Test Process Improvement on a Shoestring
Ruud Teunissen - Test Process Improvement on a Shoestring
 
How to use selenium successfully
How to use selenium successfullyHow to use selenium successfully
How to use selenium successfully
 
Controlling Project during Development with a Defect Model, Ben Linders, ICST...
Controlling Project during Development with a Defect Model, Ben Linders, ICST...Controlling Project during Development with a Defect Model, Ben Linders, ICST...
Controlling Project during Development with a Defect Model, Ben Linders, ICST...
 
New model
New modelNew model
New model
 
One size does not fit all
One size does not fit allOne size does not fit all
One size does not fit all
 
Growing a Company Test Community: Roles and Paths for Testers
Growing a Company Test Community: Roles and Paths for TestersGrowing a Company Test Community: Roles and Paths for Testers
Growing a Company Test Community: Roles and Paths for Testers
 
Lean pilots by Mariya Breyter from Dun & Bradstreet
Lean pilots by Mariya Breyter from Dun & BradstreetLean pilots by Mariya Breyter from Dun & Bradstreet
Lean pilots by Mariya Breyter from Dun & Bradstreet
 
Dr. Tafline Murnane & Dr. Stuart Reid - Practical Approaches to Motivating Te...
Dr. Tafline Murnane & Dr. Stuart Reid - Practical Approaches to Motivating Te...Dr. Tafline Murnane & Dr. Stuart Reid - Practical Approaches to Motivating Te...
Dr. Tafline Murnane & Dr. Stuart Reid - Practical Approaches to Motivating Te...
 
How to Juggle Multiple Beta Tests at Once
How to Juggle Multiple Beta Tests at OnceHow to Juggle Multiple Beta Tests at Once
How to Juggle Multiple Beta Tests at Once
 
DevOps Test Engineering: Putting the ‘Continuous’ in Testing, an ITSM Academy...
DevOps Test Engineering: Putting the ‘Continuous’ in Testing, an ITSM Academy...DevOps Test Engineering: Putting the ‘Continuous’ in Testing, an ITSM Academy...
DevOps Test Engineering: Putting the ‘Continuous’ in Testing, an ITSM Academy...
 

Similar to Anton Muzhailo - Practical Test Process Improvement using ISTQB

Gerlof Hoekstra - OMG What Have We Done - EuroSTAR 2013
Gerlof Hoekstra - OMG What Have We Done - EuroSTAR 2013Gerlof Hoekstra - OMG What Have We Done - EuroSTAR 2013
Gerlof Hoekstra - OMG What Have We Done - EuroSTAR 2013
TEST Huddle
 
10 signs your testing is not enough
10 signs your testing is not enough10 signs your testing is not enough
10 signs your testing is not enough
SQALab
 
2 anton muzhailo - formal test process improvement. how to invest to the te...
2   anton muzhailo - formal test process improvement. how to invest to the te...2   anton muzhailo - formal test process improvement. how to invest to the te...
2 anton muzhailo - formal test process improvement. how to invest to the te...
Ievgenii Katsan
 
Is Test Planning a lost art in Agile? by Michelle Williams
Is Test Planning a lost art in Agile? by Michelle WilliamsIs Test Planning a lost art in Agile? by Michelle Williams
Is Test Planning a lost art in Agile? by Michelle Williams
QA or the Highway
 
Predictive Analytics in Software Testing
Predictive Analytics in Software TestingPredictive Analytics in Software Testing
Predictive Analytics in Software Testing
Pavan Kumar Kodedela
 
Benchmarking Mini-series Part #2: Conducting Quick, Cost-Effective UX Benchma...
Benchmarking Mini-series Part #2: Conducting Quick, Cost-Effective UX Benchma...Benchmarking Mini-series Part #2: Conducting Quick, Cost-Effective UX Benchma...
Benchmarking Mini-series Part #2: Conducting Quick, Cost-Effective UX Benchma...
UserZoom
 
Develop a Defect Prevention Strategy—or Else!
Develop a Defect Prevention Strategy—or Else!Develop a Defect Prevention Strategy—or Else!
Develop a Defect Prevention Strategy—or Else!
TechWell
 
A confused tester in agile world finalversion
A confused tester in agile world finalversionA confused tester in agile world finalversion
A confused tester in agile world finalversion
Ashish Kumar
 
Software Quality Metrics Do's and Don'ts - XBOSoft-QAI Webinar
Software Quality Metrics Do's and Don'ts - XBOSoft-QAI WebinarSoftware Quality Metrics Do's and Don'ts - XBOSoft-QAI Webinar
Software Quality Metrics Do's and Don'ts - XBOSoft-QAI Webinar
XBOSoft
 
software testing metrics do's - don'ts-XBOSoft-QAI Webinar
software testing metrics do's - don'ts-XBOSoft-QAI Webinarsoftware testing metrics do's - don'ts-XBOSoft-QAI Webinar
software testing metrics do's - don'ts-XBOSoft-QAI Webinar
XBOSoft
 
AgileTestStrategy.pptx
AgileTestStrategy.pptxAgileTestStrategy.pptx
AgileTestStrategy.pptx
EdisonTobon3
 
Sanitized tb swstmppp1516july
Sanitized tb swstmppp1516julySanitized tb swstmppp1516july
Sanitized tb swstmppp1516july
Agile Testing alliance
 
Use the Windshield, Not the Mirror Predictive Metrics that Drive Successful ...
 Use the Windshield, Not the Mirror Predictive Metrics that Drive Successful ... Use the Windshield, Not the Mirror Predictive Metrics that Drive Successful ...
Use the Windshield, Not the Mirror Predictive Metrics that Drive Successful ...
Seapine Software
 
Manual Testing real time questions .pdf
Manual Testing real time questions .pdfManual Testing real time questions .pdf
Manual Testing real time questions .pdf
TiktokIndia2
 
Notes on usability testing
Notes on usability testingNotes on usability testing
Notes on usability testing
Sean Donnelly BA MSc QFA
 
Gap assessment Continuous Testing
Gap assessment   Continuous TestingGap assessment   Continuous Testing
Gap assessment Continuous Testing
Marc Hornbeek
 
The Good, The Bad, and The Metrics
 The Good, The Bad, and The Metrics The Good, The Bad, and The Metrics
The Good, The Bad, and The Metrics
TeamQualityPro
 
Best Practices for Benchmarking the Website User Experience featuring Measuri...
Best Practices for Benchmarking the Website User Experience featuring Measuri...Best Practices for Benchmarking the Website User Experience featuring Measuri...
Best Practices for Benchmarking the Website User Experience featuring Measuri...
UserZoom
 
Beyond "Quality Assurance"
Beyond "Quality Assurance"Beyond "Quality Assurance"
Beyond "Quality Assurance"
Jason Benton
 
Life of a pragmatic tester
Life of a pragmatic testerLife of a pragmatic tester
Life of a pragmatic tester
Gitte Ottosen
 

Similar to Anton Muzhailo - Practical Test Process Improvement using ISTQB (20)

Gerlof Hoekstra - OMG What Have We Done - EuroSTAR 2013
Gerlof Hoekstra - OMG What Have We Done - EuroSTAR 2013Gerlof Hoekstra - OMG What Have We Done - EuroSTAR 2013
Gerlof Hoekstra - OMG What Have We Done - EuroSTAR 2013
 
10 signs your testing is not enough
10 signs your testing is not enough10 signs your testing is not enough
10 signs your testing is not enough
 
2 anton muzhailo - formal test process improvement. how to invest to the te...
2   anton muzhailo - formal test process improvement. how to invest to the te...2   anton muzhailo - formal test process improvement. how to invest to the te...
2 anton muzhailo - formal test process improvement. how to invest to the te...
 
Is Test Planning a lost art in Agile? by Michelle Williams
Is Test Planning a lost art in Agile? by Michelle WilliamsIs Test Planning a lost art in Agile? by Michelle Williams
Is Test Planning a lost art in Agile? by Michelle Williams
 
Predictive Analytics in Software Testing
Predictive Analytics in Software TestingPredictive Analytics in Software Testing
Predictive Analytics in Software Testing
 
Benchmarking Mini-series Part #2: Conducting Quick, Cost-Effective UX Benchma...
Benchmarking Mini-series Part #2: Conducting Quick, Cost-Effective UX Benchma...Benchmarking Mini-series Part #2: Conducting Quick, Cost-Effective UX Benchma...
Benchmarking Mini-series Part #2: Conducting Quick, Cost-Effective UX Benchma...
 
Develop a Defect Prevention Strategy—or Else!
Develop a Defect Prevention Strategy—or Else!Develop a Defect Prevention Strategy—or Else!
Develop a Defect Prevention Strategy—or Else!
 
A confused tester in agile world finalversion
A confused tester in agile world finalversionA confused tester in agile world finalversion
A confused tester in agile world finalversion
 
Software Quality Metrics Do's and Don'ts - XBOSoft-QAI Webinar
Software Quality Metrics Do's and Don'ts - XBOSoft-QAI WebinarSoftware Quality Metrics Do's and Don'ts - XBOSoft-QAI Webinar
Software Quality Metrics Do's and Don'ts - XBOSoft-QAI Webinar
 
software testing metrics do's - don'ts-XBOSoft-QAI Webinar
software testing metrics do's - don'ts-XBOSoft-QAI Webinarsoftware testing metrics do's - don'ts-XBOSoft-QAI Webinar
software testing metrics do's - don'ts-XBOSoft-QAI Webinar
 
AgileTestStrategy.pptx
AgileTestStrategy.pptxAgileTestStrategy.pptx
AgileTestStrategy.pptx
 
Sanitized tb swstmppp1516july
Sanitized tb swstmppp1516julySanitized tb swstmppp1516july
Sanitized tb swstmppp1516july
 
Use the Windshield, Not the Mirror Predictive Metrics that Drive Successful ...
 Use the Windshield, Not the Mirror Predictive Metrics that Drive Successful ... Use the Windshield, Not the Mirror Predictive Metrics that Drive Successful ...
Use the Windshield, Not the Mirror Predictive Metrics that Drive Successful ...
 
Manual Testing real time questions .pdf
Manual Testing real time questions .pdfManual Testing real time questions .pdf
Manual Testing real time questions .pdf
 
Notes on usability testing
Notes on usability testingNotes on usability testing
Notes on usability testing
 
Gap assessment Continuous Testing
Gap assessment   Continuous TestingGap assessment   Continuous Testing
Gap assessment Continuous Testing
 
The Good, The Bad, and The Metrics
 The Good, The Bad, and The Metrics The Good, The Bad, and The Metrics
The Good, The Bad, and The Metrics
 
Best Practices for Benchmarking the Website User Experience featuring Measuri...
Best Practices for Benchmarking the Website User Experience featuring Measuri...Best Practices for Benchmarking the Website User Experience featuring Measuri...
Best Practices for Benchmarking the Website User Experience featuring Measuri...
 
Beyond "Quality Assurance"
Beyond "Quality Assurance"Beyond "Quality Assurance"
Beyond "Quality Assurance"
 
Life of a pragmatic tester
Life of a pragmatic testerLife of a pragmatic tester
Life of a pragmatic tester
 

More from Ievgenii Katsan

8 andrew kalyuzhin - 30 ux-advices, that will make users love you
8   andrew kalyuzhin - 30 ux-advices, that will make users love you8   andrew kalyuzhin - 30 ux-advices, that will make users love you
8 andrew kalyuzhin - 30 ux-advices, that will make users love you
Ievgenii Katsan
 
5 hans van loenhoud - master-class the 7 skills of highly successful teams
5   hans van loenhoud - master-class the 7 skills of highly successful teams5   hans van loenhoud - master-class the 7 skills of highly successful teams
5 hans van loenhoud - master-class the 7 skills of highly successful teams
Ievgenii Katsan
 
4 alexey orlov - life of product in startup and enterprise
4   alexey orlov - life of product in startup and enterprise4   alexey orlov - life of product in startup and enterprise
4 alexey orlov - life of product in startup and enterprise
Ievgenii Katsan
 
3 dmitry gomeniuk - how to make data-driven decisions in saa s products
3   dmitry gomeniuk - how to make data-driven decisions in saa s products3   dmitry gomeniuk - how to make data-driven decisions in saa s products
3 dmitry gomeniuk - how to make data-driven decisions in saa s products
Ievgenii Katsan
 
7 hans van loenhoud - the problem-goal-solution trinity
7   hans van loenhoud - the problem-goal-solution trinity7   hans van loenhoud - the problem-goal-solution trinity
7 hans van loenhoud - the problem-goal-solution trinity
Ievgenii Katsan
 
1 hans van loenhoud -
1   hans van loenhoud - 1   hans van loenhoud -
1 hans van loenhoud -
Ievgenii Katsan
 
3 denys gobov - change request specification the knowledge base or the task...
3   denys gobov - change request specification the knowledge base or the task...3   denys gobov - change request specification the knowledge base or the task...
3 denys gobov - change request specification the knowledge base or the task...
Ievgenii Katsan
 
5 victoria cupet - learn to play business analysis
5   victoria cupet - learn to play business analysis5   victoria cupet - learn to play business analysis
5 victoria cupet - learn to play business analysis
Ievgenii Katsan
 
5 alina petrenko - key requirements elicitation during the first contact wi...
5   alina petrenko - key requirements elicitation during the first contact wi...5   alina petrenko - key requirements elicitation during the first contact wi...
5 alina petrenko - key requirements elicitation during the first contact wi...
Ievgenii Katsan
 
3 karabak kuyavets transformation of business analyst to product owner
3   karabak kuyavets transformation of business analyst to product owner3   karabak kuyavets transformation of business analyst to product owner
3 karabak kuyavets transformation of business analyst to product owner
Ievgenii Katsan
 
4 andrii melnykov - stakeholder management for pd ms and b-as and why it is...
4   andrii melnykov - stakeholder management for pd ms and b-as and why it is...4   andrii melnykov - stakeholder management for pd ms and b-as and why it is...
4 andrii melnykov - stakeholder management for pd ms and b-as and why it is...
Ievgenii Katsan
 
3 zornitsa nikolova - the product manager between decision making and facil...
3   zornitsa nikolova - the product manager between decision making and facil...3   zornitsa nikolova - the product manager between decision making and facil...
3 zornitsa nikolova - the product manager between decision making and facil...
Ievgenii Katsan
 
4 viktoriya gudym - how to effectively manage remote employees
4   viktoriya gudym - how to effectively manage remote employees4   viktoriya gudym - how to effectively manage remote employees
4 viktoriya gudym - how to effectively manage remote employees
Ievgenii Katsan
 
9 natali renska - product and outsource development, how to cook 2 meals in...
9   natali renska - product and outsource development, how to cook 2 meals in...9   natali renska - product and outsource development, how to cook 2 meals in...
9 natali renska - product and outsource development, how to cook 2 meals in...
Ievgenii Katsan
 
7 denis parkhomenko - from idea to execution how to make a product that cus...
7   denis parkhomenko - from idea to execution how to make a product that cus...7   denis parkhomenko - from idea to execution how to make a product that cus...
7 denis parkhomenko - from idea to execution how to make a product that cus...
Ievgenii Katsan
 
6 anton vitiaz - inside the mvp in 3 days
6   anton vitiaz - inside the mvp in 3 days6   anton vitiaz - inside the mvp in 3 days
6 anton vitiaz - inside the mvp in 3 days
Ievgenii Katsan
 
5 mariya popova - ideal product management. unicorns in our reality
5   mariya popova - ideal product management. unicorns in our reality5   mariya popova - ideal product management. unicorns in our reality
5 mariya popova - ideal product management. unicorns in our reality
Ievgenii Katsan
 
2 victor podzubanov - design thinking game
2   victor podzubanov - design thinking game2   victor podzubanov - design thinking game
2 victor podzubanov - design thinking game
Ievgenii Katsan
 
3 sergiy potapov - analyst to product owner
3   sergiy potapov - analyst to product owner3   sergiy potapov - analyst to product owner
3 sergiy potapov - analyst to product owner
Ievgenii Katsan
 
4 anton parkhomenko - how to make effective user research with no budget at...
4   anton parkhomenko - how to make effective user research with no budget at...4   anton parkhomenko - how to make effective user research with no budget at...
4 anton parkhomenko - how to make effective user research with no budget at...
Ievgenii Katsan
 

More from Ievgenii Katsan (20)

8 andrew kalyuzhin - 30 ux-advices, that will make users love you
8   andrew kalyuzhin - 30 ux-advices, that will make users love you8   andrew kalyuzhin - 30 ux-advices, that will make users love you
8 andrew kalyuzhin - 30 ux-advices, that will make users love you
 
5 hans van loenhoud - master-class the 7 skills of highly successful teams
5   hans van loenhoud - master-class the 7 skills of highly successful teams5   hans van loenhoud - master-class the 7 skills of highly successful teams
5 hans van loenhoud - master-class the 7 skills of highly successful teams
 
4 alexey orlov - life of product in startup and enterprise
4   alexey orlov - life of product in startup and enterprise4   alexey orlov - life of product in startup and enterprise
4 alexey orlov - life of product in startup and enterprise
 
3 dmitry gomeniuk - how to make data-driven decisions in saa s products
3   dmitry gomeniuk - how to make data-driven decisions in saa s products3   dmitry gomeniuk - how to make data-driven decisions in saa s products
3 dmitry gomeniuk - how to make data-driven decisions in saa s products
 
7 hans van loenhoud - the problem-goal-solution trinity
7   hans van loenhoud - the problem-goal-solution trinity7   hans van loenhoud - the problem-goal-solution trinity
7 hans van loenhoud - the problem-goal-solution trinity
 
1 hans van loenhoud -
1   hans van loenhoud - 1   hans van loenhoud -
1 hans van loenhoud -
 
3 denys gobov - change request specification the knowledge base or the task...
3   denys gobov - change request specification the knowledge base or the task...3   denys gobov - change request specification the knowledge base or the task...
3 denys gobov - change request specification the knowledge base or the task...
 
5 victoria cupet - learn to play business analysis
5   victoria cupet - learn to play business analysis5   victoria cupet - learn to play business analysis
5 victoria cupet - learn to play business analysis
 
5 alina petrenko - key requirements elicitation during the first contact wi...
5   alina petrenko - key requirements elicitation during the first contact wi...5   alina petrenko - key requirements elicitation during the first contact wi...
5 alina petrenko - key requirements elicitation during the first contact wi...
 
3 karabak kuyavets transformation of business analyst to product owner
3   karabak kuyavets transformation of business analyst to product owner3   karabak kuyavets transformation of business analyst to product owner
3 karabak kuyavets transformation of business analyst to product owner
 
4 andrii melnykov - stakeholder management for pd ms and b-as and why it is...
4   andrii melnykov - stakeholder management for pd ms and b-as and why it is...4   andrii melnykov - stakeholder management for pd ms and b-as and why it is...
4 andrii melnykov - stakeholder management for pd ms and b-as and why it is...
 
3 zornitsa nikolova - the product manager between decision making and facil...
3   zornitsa nikolova - the product manager between decision making and facil...3   zornitsa nikolova - the product manager between decision making and facil...
3 zornitsa nikolova - the product manager between decision making and facil...
 
4 viktoriya gudym - how to effectively manage remote employees
4   viktoriya gudym - how to effectively manage remote employees4   viktoriya gudym - how to effectively manage remote employees
4 viktoriya gudym - how to effectively manage remote employees
 
9 natali renska - product and outsource development, how to cook 2 meals in...
9   natali renska - product and outsource development, how to cook 2 meals in...9   natali renska - product and outsource development, how to cook 2 meals in...
9 natali renska - product and outsource development, how to cook 2 meals in...
 
7 denis parkhomenko - from idea to execution how to make a product that cus...
7   denis parkhomenko - from idea to execution how to make a product that cus...7   denis parkhomenko - from idea to execution how to make a product that cus...
7 denis parkhomenko - from idea to execution how to make a product that cus...
 
6 anton vitiaz - inside the mvp in 3 days
6   anton vitiaz - inside the mvp in 3 days6   anton vitiaz - inside the mvp in 3 days
6 anton vitiaz - inside the mvp in 3 days
 
5 mariya popova - ideal product management. unicorns in our reality
5   mariya popova - ideal product management. unicorns in our reality5   mariya popova - ideal product management. unicorns in our reality
5 mariya popova - ideal product management. unicorns in our reality
 
2 victor podzubanov - design thinking game
2   victor podzubanov - design thinking game2   victor podzubanov - design thinking game
2 victor podzubanov - design thinking game
 
3 sergiy potapov - analyst to product owner
3   sergiy potapov - analyst to product owner3   sergiy potapov - analyst to product owner
3 sergiy potapov - analyst to product owner
 
4 anton parkhomenko - how to make effective user research with no budget at...
4   anton parkhomenko - how to make effective user research with no budget at...4   anton parkhomenko - how to make effective user research with no budget at...
4 anton parkhomenko - how to make effective user research with no budget at...
 

Recently uploaded

22519 - Client-Side Scripting Language (CSS) chapter 1 notes .pdf
22519 - Client-Side Scripting Language (CSS) chapter 1 notes .pdf22519 - Client-Side Scripting Language (CSS) chapter 1 notes .pdf
22519 - Client-Side Scripting Language (CSS) chapter 1 notes .pdf
sharvaridhokte
 
Introduction to IP address concept - Computer Networking
Introduction to IP address concept - Computer NetworkingIntroduction to IP address concept - Computer Networking
Introduction to IP address concept - Computer Networking
Md.Shohel Rana ( M.Sc in CSE Khulna University of Engineering & Technology (KUET))
 
Development of Chatbot Using AI/ML Technologies
Development of  Chatbot Using AI/ML TechnologiesDevelopment of  Chatbot Using AI/ML Technologies
Development of Chatbot Using AI/ML Technologies
maisnampibarel
 
CONVEGNO DA IRETI 18 giugno 2024 | PASQUALE Donato
CONVEGNO DA IRETI 18 giugno 2024 | PASQUALE DonatoCONVEGNO DA IRETI 18 giugno 2024 | PASQUALE Donato
CONVEGNO DA IRETI 18 giugno 2024 | PASQUALE Donato
Servizi a rete
 
Paharganj @ℂall @Girls ꧁❤ 9873777170 ❤꧂VIP Arti Singh Top Model Safe
Paharganj @ℂall @Girls ꧁❤ 9873777170 ❤꧂VIP Arti Singh Top Model SafePaharganj @ℂall @Girls ꧁❤ 9873777170 ❤꧂VIP Arti Singh Top Model Safe
Paharganj @ℂall @Girls ꧁❤ 9873777170 ❤꧂VIP Arti Singh Top Model Safe
aarusi sexy model
 
21CV61- Module 3 (CONSTRUCTION MANAGEMENT AND ENTREPRENEURSHIP.pptx
21CV61- Module 3 (CONSTRUCTION MANAGEMENT AND ENTREPRENEURSHIP.pptx21CV61- Module 3 (CONSTRUCTION MANAGEMENT AND ENTREPRENEURSHIP.pptx
21CV61- Module 3 (CONSTRUCTION MANAGEMENT AND ENTREPRENEURSHIP.pptx
sanabts249
 
Germany Offshore Wind 010724 RE (1) 2 test.pptx
Germany Offshore Wind 010724 RE (1) 2 test.pptxGermany Offshore Wind 010724 RE (1) 2 test.pptx
Germany Offshore Wind 010724 RE (1) 2 test.pptx
rebecca841358
 
Response & Safe AI at Summer School of AI at IIITH
Response & Safe AI at Summer School of AI at IIITHResponse & Safe AI at Summer School of AI at IIITH
Response & Safe AI at Summer School of AI at IIITH
IIIT Hyderabad
 
How to Manage Internal Notes in Odoo 17 POS
How to Manage Internal Notes in Odoo 17 POSHow to Manage Internal Notes in Odoo 17 POS
How to Manage Internal Notes in Odoo 17 POS
Celine George
 
LeetCode Database problems solved using PySpark.pdf
LeetCode Database problems solved using PySpark.pdfLeetCode Database problems solved using PySpark.pdf
LeetCode Database problems solved using PySpark.pdf
pavanaroshni1977
 
IS Code SP 23: Handbook on concrete mixes
IS Code SP 23: Handbook  on concrete mixesIS Code SP 23: Handbook  on concrete mixes
IS Code SP 23: Handbook on concrete mixes
Mani Krishna Sarkar
 
Bangalore @ℂall @Girls ꧁❤ 0000000000 ❤꧂@ℂall @Girls Service Vip Top Model Safe
Bangalore @ℂall @Girls ꧁❤ 0000000000 ❤꧂@ℂall @Girls Service Vip Top Model SafeBangalore @ℂall @Girls ꧁❤ 0000000000 ❤꧂@ℂall @Girls Service Vip Top Model Safe
Bangalore @ℂall @Girls ꧁❤ 0000000000 ❤꧂@ℂall @Girls Service Vip Top Model Safe
bookhotbebes1
 
Social media management system project report.pdf
Social media management system project report.pdfSocial media management system project report.pdf
Social media management system project report.pdf
Kamal Acharya
 
Trends in Computer Aided Design and MFG.
Trends in Computer Aided Design and MFG.Trends in Computer Aided Design and MFG.
Trends in Computer Aided Design and MFG.
Tool and Die Tech
 
L-3536-Cost Benifit Analysis in ESIA.pptx
L-3536-Cost Benifit Analysis in ESIA.pptxL-3536-Cost Benifit Analysis in ESIA.pptx
L-3536-Cost Benifit Analysis in ESIA.pptx
naseki5964
 
Vernier Caliper and How to use Vernier Caliper.ppsx
Vernier Caliper and How to use Vernier Caliper.ppsxVernier Caliper and How to use Vernier Caliper.ppsx
Vernier Caliper and How to use Vernier Caliper.ppsx
Tool and Die Tech
 
GUIA_LEGAL_CHAPTER-9_COLOMBIAN ELECTRICITY (1).pdf
GUIA_LEGAL_CHAPTER-9_COLOMBIAN ELECTRICITY (1).pdfGUIA_LEGAL_CHAPTER-9_COLOMBIAN ELECTRICITY (1).pdf
GUIA_LEGAL_CHAPTER-9_COLOMBIAN ELECTRICITY (1).pdf
ProexportColombia1
 
1239_2.pdf IS CODE FOR GI PIPE FOR PROCUREMENT
1239_2.pdf IS CODE FOR GI PIPE FOR PROCUREMENT1239_2.pdf IS CODE FOR GI PIPE FOR PROCUREMENT
1239_2.pdf IS CODE FOR GI PIPE FOR PROCUREMENT
Mani Krishna Sarkar
 
Quadcopter Dynamics, Stability and Control
Quadcopter Dynamics, Stability and ControlQuadcopter Dynamics, Stability and Control
Quadcopter Dynamics, Stability and Control
Blesson Easo Varghese
 
OCS Training - Rig Equipment Inspection - Advanced 5 Days_IADC.pdf
OCS Training - Rig Equipment Inspection - Advanced 5 Days_IADC.pdfOCS Training - Rig Equipment Inspection - Advanced 5 Days_IADC.pdf
OCS Training - Rig Equipment Inspection - Advanced 5 Days_IADC.pdf
Muanisa Waras
 

Recently uploaded (20)

22519 - Client-Side Scripting Language (CSS) chapter 1 notes .pdf
22519 - Client-Side Scripting Language (CSS) chapter 1 notes .pdf22519 - Client-Side Scripting Language (CSS) chapter 1 notes .pdf
22519 - Client-Side Scripting Language (CSS) chapter 1 notes .pdf
 
Introduction to IP address concept - Computer Networking
Introduction to IP address concept - Computer NetworkingIntroduction to IP address concept - Computer Networking
Introduction to IP address concept - Computer Networking
 
Development of Chatbot Using AI/ML Technologies
Development of  Chatbot Using AI/ML TechnologiesDevelopment of  Chatbot Using AI/ML Technologies
Development of Chatbot Using AI/ML Technologies
 
CONVEGNO DA IRETI 18 giugno 2024 | PASQUALE Donato
CONVEGNO DA IRETI 18 giugno 2024 | PASQUALE DonatoCONVEGNO DA IRETI 18 giugno 2024 | PASQUALE Donato
CONVEGNO DA IRETI 18 giugno 2024 | PASQUALE Donato
 
Paharganj @ℂall @Girls ꧁❤ 9873777170 ❤꧂VIP Arti Singh Top Model Safe
Paharganj @ℂall @Girls ꧁❤ 9873777170 ❤꧂VIP Arti Singh Top Model SafePaharganj @ℂall @Girls ꧁❤ 9873777170 ❤꧂VIP Arti Singh Top Model Safe
Paharganj @ℂall @Girls ꧁❤ 9873777170 ❤꧂VIP Arti Singh Top Model Safe
 
21CV61- Module 3 (CONSTRUCTION MANAGEMENT AND ENTREPRENEURSHIP.pptx
21CV61- Module 3 (CONSTRUCTION MANAGEMENT AND ENTREPRENEURSHIP.pptx21CV61- Module 3 (CONSTRUCTION MANAGEMENT AND ENTREPRENEURSHIP.pptx
21CV61- Module 3 (CONSTRUCTION MANAGEMENT AND ENTREPRENEURSHIP.pptx
 
Germany Offshore Wind 010724 RE (1) 2 test.pptx
Germany Offshore Wind 010724 RE (1) 2 test.pptxGermany Offshore Wind 010724 RE (1) 2 test.pptx
Germany Offshore Wind 010724 RE (1) 2 test.pptx
 
Response & Safe AI at Summer School of AI at IIITH
Response & Safe AI at Summer School of AI at IIITHResponse & Safe AI at Summer School of AI at IIITH
Response & Safe AI at Summer School of AI at IIITH
 
How to Manage Internal Notes in Odoo 17 POS
How to Manage Internal Notes in Odoo 17 POSHow to Manage Internal Notes in Odoo 17 POS
How to Manage Internal Notes in Odoo 17 POS
 
LeetCode Database problems solved using PySpark.pdf
LeetCode Database problems solved using PySpark.pdfLeetCode Database problems solved using PySpark.pdf
LeetCode Database problems solved using PySpark.pdf
 
IS Code SP 23: Handbook on concrete mixes
IS Code SP 23: Handbook  on concrete mixesIS Code SP 23: Handbook  on concrete mixes
IS Code SP 23: Handbook on concrete mixes
 
Bangalore @ℂall @Girls ꧁❤ 0000000000 ❤꧂@ℂall @Girls Service Vip Top Model Safe
Bangalore @ℂall @Girls ꧁❤ 0000000000 ❤꧂@ℂall @Girls Service Vip Top Model SafeBangalore @ℂall @Girls ꧁❤ 0000000000 ❤꧂@ℂall @Girls Service Vip Top Model Safe
Bangalore @ℂall @Girls ꧁❤ 0000000000 ❤꧂@ℂall @Girls Service Vip Top Model Safe
 
Social media management system project report.pdf
Social media management system project report.pdfSocial media management system project report.pdf
Social media management system project report.pdf
 
Trends in Computer Aided Design and MFG.
Trends in Computer Aided Design and MFG.Trends in Computer Aided Design and MFG.
Trends in Computer Aided Design and MFG.
 
L-3536-Cost Benifit Analysis in ESIA.pptx
L-3536-Cost Benifit Analysis in ESIA.pptxL-3536-Cost Benifit Analysis in ESIA.pptx
L-3536-Cost Benifit Analysis in ESIA.pptx
 
Vernier Caliper and How to use Vernier Caliper.ppsx
Vernier Caliper and How to use Vernier Caliper.ppsxVernier Caliper and How to use Vernier Caliper.ppsx
Vernier Caliper and How to use Vernier Caliper.ppsx
 
GUIA_LEGAL_CHAPTER-9_COLOMBIAN ELECTRICITY (1).pdf
GUIA_LEGAL_CHAPTER-9_COLOMBIAN ELECTRICITY (1).pdfGUIA_LEGAL_CHAPTER-9_COLOMBIAN ELECTRICITY (1).pdf
GUIA_LEGAL_CHAPTER-9_COLOMBIAN ELECTRICITY (1).pdf
 
1239_2.pdf IS CODE FOR GI PIPE FOR PROCUREMENT
1239_2.pdf IS CODE FOR GI PIPE FOR PROCUREMENT1239_2.pdf IS CODE FOR GI PIPE FOR PROCUREMENT
1239_2.pdf IS CODE FOR GI PIPE FOR PROCUREMENT
 
Quadcopter Dynamics, Stability and Control
Quadcopter Dynamics, Stability and ControlQuadcopter Dynamics, Stability and Control
Quadcopter Dynamics, Stability and Control
 
OCS Training - Rig Equipment Inspection - Advanced 5 Days_IADC.pdf
OCS Training - Rig Equipment Inspection - Advanced 5 Days_IADC.pdfOCS Training - Rig Equipment Inspection - Advanced 5 Days_IADC.pdf
OCS Training - Rig Equipment Inspection - Advanced 5 Days_IADC.pdf
 

Anton Muzhailo - Practical Test Process Improvement using ISTQB

  • 1. Practical Test Process Improvement using ISTQB Anton Muzhailo
  • 2. Speaker info Anton Muzhailo, Ukraine Senior Python Automation Engineer, GlobalLogic • ISTQB Certified Test Manager • ISTQB Certified Trainings Coach • 3 years of mentoring experience, 200+ students in/muzhailo/
  • 3. General Question What is the value of ISTQB knowledge and certifications? - Common vocabulary of testing terms - Test process consistency improving - Test effectiveness improving - Common & best practices - Recognition as expert - Be on the same level as many others
  • 4. True Question What is the TRUE value of ISTQB knowledge and certifications? - Clear and visible ROI is a great argument for management - Calculated real amount of time and resources saved - Wider view-angle always leads to better risk-management - Starting “from scratch” always require some classical basis to start - Adapting classical approaches to real cases using your expertise - Lots of approaches and tools cannot become obsolete
  • 5. Quality Assessment Problem: Your are a Test Manager of a team on a new project moved from another country. There are 500 test cases written. No formal specification available. • How to see “the big picture”? • What is the ~ test coverage? • How to start? • Where to focus the testing effort?
  • 6. Quality Metrics Metrics that are not valid are dangerous! Metrics Qualitative Quantitative - Expensive - Risky - Hard to do well - Suffer from own qualitative problems - Could give a false impression - Can’t be used everywhere - Could be invalid - Hard to choose valid metrics
  • 7. Is this metric valid? 𝑇𝑒𝑠𝑡 𝑐𝑎𝑠𝑒 𝑝𝑎𝑠𝑠 𝑟𝑎𝑡𝑖𝑜 = # 𝑜𝑓 𝑡𝑒𝑠𝑡𝑠 𝑝𝑎𝑠𝑠𝑒𝑑 # 𝑜𝑓 𝑡𝑒𝑠𝑡𝑠 𝑟𝑎𝑛 Pretty popular, huh?
  • 8. Is this metric valid? 𝑇𝑒𝑠𝑡 𝑐𝑎𝑠𝑒 𝑝𝑎𝑠𝑠 𝑟𝑎𝑡𝑖𝑜 = # 𝑜𝑓 𝑡𝑒𝑠𝑡𝑠 𝑝𝑎𝑠𝑠𝑒𝑑 # 𝑜𝑓 𝑡𝑒𝑠𝑡𝑠 𝑟𝑎𝑛 - How about other 238459293465013475 tests, which were not run or even defined? Pretty popular, huh?
  • 9. Is this metric valid? 𝑇𝑒𝑠𝑡 𝑐𝑎𝑠𝑒 𝑝𝑎𝑠𝑠 𝑟𝑎𝑡𝑖𝑜 = # 𝑜𝑓 𝑡𝑒𝑠𝑡𝑠 𝑝𝑎𝑠𝑠𝑒𝑑 # 𝑜𝑓 𝑡𝑒𝑠𝑡𝑠 𝑟𝑎𝑛 - How about other 238459293465013475 tests, which were not run or even defined? Pretty popular, huh? - But we measure test coverage… Seems sufficient!
  • 10. Is this metric valid? 𝑇𝑒𝑠𝑡 𝑐𝑎𝑠𝑒 𝑝𝑎𝑠𝑠 𝑟𝑎𝑡𝑖𝑜 = # 𝑜𝑓 𝑡𝑒𝑠𝑡𝑠 𝑝𝑎𝑠𝑠𝑒𝑑 # 𝑜𝑓 𝑡𝑒𝑠𝑡𝑠 𝑟𝑎𝑛 - How about other 238459293465013475 tests, which were not run or even defined? Pretty popular, huh? - But we measure test coverage… Seems sufficient! - Using which metrics?
  • 11. Good metrics Once you have two choices with its own pros and cons – you need a hybrid to have both pros’ and none cons © Unicorn from fluffy world
  • 12. Good metrics Once you have two choices with its own pros and cons – you need a hybrid to have both pros’ and none cons The popular way of risk mitigation is to split the risk areas to the small parts and manage risks within these parts. © Unicorn from fluffy world Our world require risk assessments
  • 13. Such a deep term “Quality” • Must be measured every iteration • Have to be defined in % • Bugs should be represented in Quality’s terms • Such metrics gives more clear vision about product readiness © ISTQB Glossary Quality. The degree to which a component, system or process meets specified requirements and/or user/customer needs and expectations.
  • 14. Test Coverage Measurement Test cases fully covers the area 1 point Test cases almost covers the area 0.9 pts Test cases covers some of functionality 0.5 pts There are some basic test cases written 0.1 pts There are no test cases written 0 pts 𝑇𝑒𝑠𝑡 𝑐𝑜𝑣𝑒𝑟𝑎𝑔𝑒 = 𝑇𝑜𝑡𝑎𝑙 𝑃𝑜𝑖𝑛𝑡𝑠 𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑎𝑟𝑒𝑎𝑠 𝑇𝑜𝑡𝑎𝑙 𝑝𝑜𝑖𝑛𝑡𝑠 = 𝑎𝑟𝑒𝑎1_𝑝𝑡𝑠 + … + 𝑎𝑟𝑒𝑎𝑁_𝑝𝑡𝑠 • Area size can vary depending on your expectations and resources • Each area can be divided and measured using the same approach
  • 15. Business workflow map Name Q Search 11 Product Page 10 Main Page 9 Cart 9 Login 8 Discounts 8 Supported Countries 5 FAQ 5 More Details 5 Shipping Terms 4 Product Category 4 Checkout 4 Registration 4 Product Subcategory 3 Comment & rate 3 Filters 2 Business map: https://ufile.io/19o89
  • 16. Test Coverage Measurement 𝑇𝑒𝑠𝑡 𝑐𝑜𝑣𝑒𝑟𝑎𝑔𝑒 = 9.6 16 = 0.6 ⇒ 60% 𝑇𝑜𝑡𝑎𝑙 𝑝𝑜𝑖𝑛𝑡𝑠 = 9.6 # Name Points 1 Search 0.5 2 Product Page 0.9 3 Main Page 1 4 Cart 1 5 Login 1 6 Discounts 0.1 7 Supported Countries 0.5 8 FAQ 0 9 More Details 0.5 10 Shipping Terms 1 11 Product Category 0.5 12 Checkout 0.9 13 Registration 1 14 Product Subcategory 0.1 15 Comment & rate 0.5 16 Filters 0.1 𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑎𝑟𝑒𝑎𝑠 = 16 • This approach can be used for progress measurement on every iteration • This approach can show the most and least covered areas • Estimates should be done using review with multiple participants • Correlation between most/least covered areas and areas priority/risk can be established and measured • If each area is still too big to estimate – it can be divided to sub-areas and estimated the same way
  • 17. Regression amount estimation  Change in Registration shows the regression area immediately  Easier to prioritize the tests  Faster impact and regression estimation in case of customer issue
  • 18. Issue-based metrics # Area Number of Blockers Number of Majors Number of Minors 1 Search 0 8 14 2 Product Page 2 5 11 3 Main Page 0 0 5 4 Cart 2 3 1 5 Login 0 0 7 6 Discounts 0 3 4 7 Supported Countries 0 0 0 … … … … … - So, what’s the quality? - Where we are? - Are we ready to release? - When we will be ready? Accepted definition of ready: • No blockers • All majors have workarounds • No severe Performance or Usability impact - Not yet. - 4 blockers to go - No, 13 majors doesn’t have workarounds - Dunno
  • 19. Impact-based metrics # Area Blockers impact Majors impact Minors impact Total impact 1 Search * * * 0.5 2 Product Page * * * 0 3 Main Page * * * 0.9 4 Cart * * * 0.5 5 Login * * * 0.9 6 Discounts * * * 0.9 7 Supported Countries * * * 1 … … … … … No impact (can be released) 1 point Little impact (can be released) 0.9 pts Available for regular testing (can’t be released) 0.5 pts Severe impact (not available for release/testing) 0.1 pts Doesn’t exist or completely blocked 0 pts * - optional 𝑄𝑢𝑎𝑙𝑖𝑡𝑦 = 4.7 7 = 0.67 ⇒ 67% 𝑇𝑜𝑡𝑎𝑙 𝑝𝑜𝑖𝑛𝑡𝑠 = 3.9 𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑎𝑟𝑒𝑎𝑠 = 7 Quality can also me measured roughly: 𝑄𝑢𝑎𝑙𝑖𝑡𝑦 = # 𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛𝑎𝑙𝑖𝑡𝑖𝑒𝑠 𝑟𝑒𝑎𝑑𝑦 𝑓𝑜𝑟 𝑟𝑒𝑙𝑒𝑎𝑠𝑒 𝑡𝑜𝑡𝑎𝑙 𝑄𝑢𝑎𝑙𝑖𝑡𝑦 = 4 7 = 0.57 = 57%
  • 20. Test Plan Test Plan. A document describing the scope, approach, resources and schedule of intended test activities. It identifies amongst others test items, the features to be tested, the testing tasks, who will do each task, degree of tester independence, the test environment, the test design techniques and entry and exit criteria to be used, and the rationale for their choice, and any risks requiring contingency planning. It is a record of the test planning process © ISTQB Glossary Problem: Do we really need test plans in Agile? We have JIRA with stories and epics. Why we need an additional test artifact? - Too complex… - Obsolete artifact - Too much additional paperwork - We are still fine without it, don’t care - Nah..my grandma used that in 50s’ - That’s only for mature organizations
  • 21. Test Plan artifacts 1. Test Plan Identifier 2. References 3. Introduction 4. Test Items 5. Software Risk Issues 6. Features to be Tested 7. Features not to be Tested 8. Approach 9. Item Pass/Fail Criteria 10. Suspension Criteria and Resumption Requirements 11. Test Deliverables 12. Remaining Test Tasks 13. Environmental Needs 14. Staffing and Training Needs 15. Responsibilities 16. Schedule 17. Planning Risks and Contingencies 18. Approvals 19. Glossary Define what you need. That’s a basis long enough to cover everything’s needed
  • 22. Focusing on ‘big picture’ 1 2 3 4 5 60 • 6 new features • New environment • 20 known issues fix Full Regression RELEASE 1 RELEASE 2 Full Regression New features Testing New features Testing Validation sprint Validation sprint Test closure • New major version • Mac support TC design TC update TC update  Provides an “expected result” for test monitoring and control activities  Shows the planning problems as early as possible  Allows complex planning in case of several test teams using Master Test Plan  Shows the “big picture”
  • 23. Test Control & Monitoring • Such activities can be performed best using the metrics and actions, defined within test plan. • For each metric there should be the action point predicted. • The evaluate method also should be clearly defined • Defect metrics • Pass rate • Employees • Equipment • Licenses • Difference between Planned vs Actual • Time • Money Cost Schedule QualityResources
  • 24. Test Control & Monitoring # Metrics Metrics Type Collected Period Evaluate Method Criteria Conclusion 1 Milestone achieving time Schedule Weekly Compare the actual time spent vs planned +/- 20% If deviation is above acceptable values – need an action point 2 Nightly CI set of tests exec. time Resource Daily Check if previous run took less than 10 hrs < 10 hrs 12 hours between each workday. If set of tests reaches 10 hrs – we need another environment set to order 3 Cloud environment cost Cost Weekly Compare the estimated budget draw rate with actual <120% Cost rate depends on traffic and number of users. If deviation is more then 10% we need an action point 4 Duplicate / Not a bug issues Quality Monthly Comparing the actual number with total bugs <10% If more - a time loss. Possibly some engineers need the testing skills improvement or product training 5 Defect clustering Quality Monthly Determine modules, which has many issues in > 5% If more – this modules require attention and effort allocation
  • 25. Defects severity estimation Problem: How to choose the good and precise technique for severity estimation no matter how many different testers with different points of view involved? Requirement: Application should support launching in UI mode both from command-line and desktop. Issue: Dropbox doesn’t start using command-line. What is the Severity?
  • 26. Defects severity estimation Problem: How to choose the good and precise technique for severity estimation no matter how many different testers with different points of view involved? Requirement: Application should support launching in UI mode both from command-line and desktop. Issue: Dropbox doesn’t start using command-line. What is the Severity?
  • 27. Defects severity estimation Severity. The degree of impact that a defect has on the development or operation of a component or system. © ISTQB Glossary Critical Leads to termination. Failed function is unusable and NO acceptable workaround Major Leads to termination. Failed function is unusable and acceptable workaround EXISTS Moderate Doesn’t result in termination. Failed function produces incorrect/incomplete results and NO acceptable workaround Minor Doesn’t result in termination. Failed function produces incorrect/incomplete results and acceptable workaround EXISTS Cosmetic Impacting look and feel of the application or related to enhancement Define your own severity estimation guidelines!
  • 28. Root cause analysis Test objective Test condition 1 Test case 1.1 Test case 1.2 Defect 1 Test condition 2 Test case 2.1 Test condition 3 Test case 3.1 Test case 3.2 ??? ??? Defect 2 o What is the root cause of Defect 1? o Maybe several defects have the same root cause? o Why Defect 2 wasn’t covered by any of test cases? o How to improve the test process?
  • 29. Questions 5 minutes. You can also ask questions for me in the lounge zone