Chapter 3 - Reviews
•
0 likes•412 views
This is chapter 3 of ISTQB Advance Test Manager certification. This presentation helps aspirants understand and prepare the content of the certification.
Report
Share
Recommended for you
Recommended for you
Recommended for you
Recommended for you
Recommended for you
More Related Content
What's hot
What's hot (20)
Similar to Chapter 3 - Reviews
Similar to Chapter 3 - Reviews (20)
More from Neeraj Kumar Singh
More from Neeraj Kumar Singh (18)
Recently uploaded
Recently uploaded (20)
Chapter 3 - Reviews
- 1. Reviews 1 Testing Process 2 Test Management 3 Reviews Software Testing - ISTQB Advance Test Manager Exam Preparation Chapter 3 Neeraj Kumar Singh 4 Defect Management 5 Improving Process 6 Test Tools 7 People Skills
- 2. Reviews Contents 3.1 Introduction 3.2 Management Reviews and Audits 3.3 Managing Reviews 3.4 Metrics for Reviews 3.5 Managing Formal Reviews Neeraj Kumar Singh
- 3. Since reviews are a form of static testing, Test Managers may be responsible for their overall success, particularly with respect to testware products. In the wider context of software projects, however, this responsibility should be a matter of organizational policy. Given the possible widespread application of formal reviews across many disciplines, both before and within software projects, the responsible party may be a Test Manager, or a Quality Assurance Manager, or a trained Review Coordinator. In this syllabus, the responsible party (whoever that is) is referred to as the review leader. The review leader should ensure that an environment exists that is conducive to the implementation of the success factors. In addition, the review leader should devise a measurement plan to ensure that the reviews provide effective value. Participants in reviews should have review training to better understand their respective roles in any review process. All review participants must be committed to the benefits of a well-conducted review. When done properly, reviews are the single biggest, and most cost-effective, contributor to overall delivered quality. It is thus of paramount importance that review leaders are able to implement efficient reviews in their projects and demonstrate the benefits of these reviews. Neeraj Kumar Singh Reviews Introduction
- 4. Possible reviews within a project include: Contractual reviews, initiated at project inception and at major project milestones Requirements reviews, initiated when the requirements are available for review, which ideally cover both functional and non-functional requirements Top level design reviews, initiated when the overall architectural design is available for review Detailed design reviews, initiated when the detailed design is available for review Code reviews, carried out as individual modules of software are created, which may include the unit tests and their results as well as the code itself Test work product reviews, which may cover the test plan(s), test conditions, quality risk analysis results, tests, test data, test environments, and test results Test entry (test readiness) reviews and test exit reviews for each test level, which respectively check the test entry criteria prior to starting test execution and the test exit criteria prior to concluding testing Acceptance reviews, used to obtain customer or stakeholder approval for a system In addition to these, Test Managers may also be involved in: Management reviews Audits Neeraj Kumar Singh Reviews Introduction
- 5. Reviews Contents 3.1 Introduction 3.2 Management Reviews and Audits 3.3 Managing Reviews 3.4 Metrics for Reviews 3.5 Managing Formal Reviews Neeraj Kumar Singh
- 6. Management reviews are used to monitor progress, assess status, and make decisions about future actions. These reviews support decisions about the future of the project, such as adapting the level of resources, implementing corrective actions or changing the scope of the project. The following are key characteristics of management reviews: Conducted by or for managers having direct responsibility for the project or system Conducted by or for a stakeholder or decision maker, e.g., a higher level manager or director Check consistency with and deviations from plans Check adequacy of management procedures Assess project risks Evaluate impact of actions and ways to measure these impacts Produce lists of action items, issues to be resolved and decisions made Management reviews of processes, such as project retrospectives (i.e., lessons learned), are an integral part of process improvement activities. Test Managers should participate in and may initiate management reviews of testing progress. Neeraj Kumar Singh Reviews Management Reviews
- 7. Audits are usually performed to demonstrate conformance to a defined set of criteria, most likely an applicable standard, regulatory constraint, or a contractual obligation. As such, audits are intended to provide independent evaluation of compliance to processes, regulations, standards, etc. The following are key characteristics of audits: Conducted and moderated by a lead auditor Evidence of compliance collected through interviews, witnessing and examining documents Documented results include observations, recommendations, corrective actions and a pass/fail assessment Neeraj Kumar Singh Reviews Audits
- 8. Reviews Contents 3.1 Introduction 3.2 Management Reviews and Audits 3.3 Managing Reviews 3.4 Metrics for Reviews 3.5 Managing Formal Reviews Neeraj Kumar Singh
- 9. Reviews should be planned to take place at natural break points or milestones within the software project. Typically, reviews should be held after requirements and design definitions, with associated reviews starting with the business objectives and working down to the lowest level of design. Management reviews should take place at major project milestones, often as part of a verification activity before, during, and after test execution and other significant project phases. The review strategy must be coordinated with the test policy and the overall test strategy. Before formulating an overall review plan at the project level, the review leader (who may be a Test Manager) should take into account: What should be reviewed (product and processes) Who should be involved in specific reviews Which relevant risk factors to cover Early in the project planning phase, the review leader should identify the items to be reviewed and select the appropriate review type (informal review, walkthrough, technical review or inspection, or a mixture of two or three types) and level of formality. Neeraj Kumar Singh Reviews Managing Reviews
- 10. The return on investment for reviews is the difference between the cost of conducting the review, and the cost of dealing with the same defects at a later stage (or missing them altogether) if the review had not been done. The Cost of Quality calculation, can be used to help determine this number. Determining the optimal time to perform reviews depends on the following: The availability of the items to review in a sufficiently final format The availability of the right personnel for the review The time when the final version of the item should be available The time required for the review process of that specific item Adequate metrics for review evaluation should be defined by the review leader during test planning. If inspections are used, then brief inspections should be conducted at the author's request, as document fragments are completed (e.g., individual requirements or sections). The objectives of the review process must be defined during test planning. This includes conducting effective and efficient reviews and reaching consensus decisions regarding review feedback. Neeraj Kumar Singh Reviews Managing Reviews
- 11. Review planning should address the risks associated with technical factors, organizational factors and people issues when performing reviews. The availability of reviewers with sufficient technical knowledge is critical to a successful review. All teams within the project should be involved in planning for the reviews, which should ensure that each team is committed to the success of the review process. Planning must ensure that each organization is allocating sufficient time for required reviewers to prepare for, and participate in, the reviews at appropriate points in the project schedule. Time should also be planned for any required technical or process training for the reviewers. Backup reviewers should be identified in case key reviewers become unavailable due to changes in personal or business plans. During the actual execution of formal reviews, the review leader must ensure that: Adequate measurements are provided by the participants in the reviews to allow evaluation of review efficiency Checklists are created, and maintained to improve future reviews Defect severity and priority evaluation are defined for use in defect management of issues found during reviews Neeraj Kumar Singh Reviews Managing Reviews
- 12. After each review, the review leader should: Collect the review metrics and ensure that the issues identified are sufficiently resolved to meet the specific test objectives for the review Use the review metrics as input when determining the return on investment (ROI) for reviews Provide feedback information to the relevant stakeholders Provide feedback to review participants To evaluate the effectiveness of reviews, Test Managers can compare actual results found in subsequent testing (i.e., after the reviews) with the results from the review reports. In the case where a work product is reviewed, approved based on the review, but later found defective, the review leader should consider ways in which the review process might have allowed the defects to escape. Likely causes include problems with the review process (e.g., poor entry/exit criteria), improper composition of the review team, inadequate review tools (checklists, etc.), insufficient reviewer training and experience, and too little preparation and review meeting time. A pattern of escaped defects (especially major defects), repeated across several projects, indicates that there are significant problems with the conduct of reviews. Neeraj Kumar Singh Reviews Managing Reviews
- 13. Reviews Contents 3.1 Introduction 3.2 Management Reviews and Audits 3.3 Managing Reviews 3.4 Metrics for Reviews 3.5 Managing Formal Reviews Neeraj Kumar Singh
- 14. Review leaders (who, as mentioned in the previous sections, may be Test Managers) must ensure that metrics are available to: Evaluate the quality of the reviewed item Evaluate the cost of conducting the review Evaluate the downstream benefit of having conducted the review Review leaders can use the measurements to determine the return on investment and efficiency of the reviews. These metrics can also be used for reporting and for process improvement activities. Neeraj Kumar Singh Reviews Metrics for Reviews
- 15. For each work product reviewed, the following metrics can be measured and reported for product evaluation: Work-product size (pages, lines of code, etc.) Preparation time (prior to the review) Time to conduct the review Rework time to fix defects Duration of the review process Number of defects found and their severity Identification of defect clusters within the work product (i.e., areas that have a higher defect density) Type of review (informal review, walkthrough, technical review or inspection) Average defect density (e.g., defects per page or per thousand lines of code) Estimated residual defects (or residual defect density) Neeraj Kumar Singh Reviews Metrics for Reviews
- 16. For each review the following metrics can be measured and reported for process evaluation: Defect detection effectiveness (taking into account defects found later in the lifecycle) Improvement of review process effort and timing Percent coverage of planned work products Types of defects found and their severity Participant surveys about effectiveness and efficiency of the review process Cost of quality metrics for review defects versus dynamic test defects and production defects Correlation of review effectiveness (review type versus defect detection effectiveness) Number of reviewers Defects found per work-hour expended Estimated project time saved Average defect effort (i.e., the total detection and fix time divided by the number of defects) Neeraj Kumar Singh Reviews Metrics for Reviews
- 17. Reviews Contents 3.1 Introduction 3.2 Management Reviews and Audits 3.3 Managing Reviews 3.4 Metrics for Reviews 3.5 Managing Formal Reviews Neeraj Kumar Singh
- 18. The ISTQB Foundation Level syllabus describes the different phases of a formal review: planning, kick-off, individual preparation, review meeting, rework and follow-up. To correctly implement formal reviews, review leaders need to ensure that all steps in the review process are followed. Formal reviews have a number of characteristics such as: Defined entry and exit criteria Checklists to be used by the reviewers Deliverables such as reports, evaluation sheets or other review summary sheets Metrics for reporting on the review effectiveness, efficiency, and progress Prior to initiating a formal review, fulfillment of the review prerequisites (as defined in the procedure or in the list of entry criteria) should be confirmed by the review leader. Neeraj Kumar Singh Reviews Managing Formal Reviews
- 19. If the prerequisite conditions for the formal review are not fulfilled, the review leader may propose one of the following to the review authority for final decision: Redefinition of the review with revised objectives Corrective actions necessary for the review to proceed Postponement of the review As part of controlling a formal review, these reviews are monitored in the context of the overall (higher level) program, and are associated with project quality assurance activities. Control of formal reviews includes feedback information using product and process metrics. Neeraj Kumar Singh Reviews Managing Formal Reviews
- 20. Reviews 1 Testing Process 2 Test Management 3 Reviews Software Testing - ISTQB Advance Test Manager Exam Preparation Chapter 3 Neeraj Kumar Singh 4 Defect Management 5 Improving Process 6 Test Tools 7 People Skills
- 21. Neeraj Kumar Singh Reviews Sample Questions Pattern Source : istqb.org
- 22. 1. As a test manager, you are participating in a meeting with other members of the project management team. The agenda of the meeting is to discuss whether the project can begin acceptance testing based on the system test exit criteria, the acceptance test entry criteria, and other business considerations. Which of the following statements is true? Select ONE option. Answer Set a. The meeting is a management review, because the project management team is evaluating the situation and determining the next steps. b. The meeting is an audit, because the project management team is checking compliance against defined criteria by checking evidence. c. The meeting is a management review, because the project management team is checking to ensure that progress is being made on the project. d. The meeting is an audit, because the project management team is going to issue a pass/fail assessment against the criteria. . Reviews Sample Question
- 23. 2. You are the test manager working on an agile project developing information apps. Due to missing and incorrect functionality reported by users, formal reviews of all user stories are planned. The reviews are led by you, the test manager. The main objective of the reviews is the agreement of all stakeholders on the user story format, granularity, completeness, and preciseness. The following persons play the role of reviewers: chief developer (CD), test analyst (TA), product manager (PM), and domain expert (DE). During the kickoff-meeting, the CD complains about being pulled off his real duty. After the individual reviews, the following table depicts the number of defects found by each of the three reviewers: You have to decide how to proceed with the review. Which of the following options should be chosen by the test manager? Select ONE option. Answer Set a. Ask for a discussion between the TM and the CD moderated by a third person to get the CD involved. b. Postpone the review session and redefine the review with the objectives of the CD. c. Proceed timely with the review session, with the role “scribe” assigned to the CD. d. Cancel the review, write a report to upper management emphasizing the missing involvement of the CD. Reviews Sample Question