Ch14

2. Your Instructor Rex Black President RBCS, CTO RBCSNZ, CTO PureTesting: worldwide testing services companies President ASTQB and ISTQB: non-profit professional organizations Quarter century of software engineering experience Author of 5 books and dozens of articles Married, father of two

3. The Materials The slides follow the ISTQB Foundation Syllabus 2007 The terms come from the latest ISTQB Glossary Two mock exams are included to assess your readiness for the ISTQB Foundation exam Each section includes sample exam questions to check your knowledge as you proceed Exercises are included for main techniques

4. ISTQB Foundation Syllabus 2007 Developed by a team of nine authors spanning seven countries: Thomas M ü ller (chair), Rex Black, Sigrid Eldh, Debra Friedenberg, Dorothy Graham, Kalus Olsen, Maaret Pyh äjärvi, Geoff Thompson and Erik van Veenendaal Forty primary reviewers spanning nine countries Final review and approval by 33 National Boards Distills over 1,000 person-years of experience The ISTQB and the authors are the source of the syllabus (copyright © 2005 and 2007 by the authors and ISTQB) which is used by permission as the basis for this course

5. The Main Topics Covered Fundamentals of testing Testing throughout the software lifecycle Static techniques Test design techniques Test management Tool support for testing

6. What You’ll Gain from This Course Understand the foundational best practices of software testing Adapt testing efforts for the context in which they will occur Design tests using black-box, white-box, and experienced-based techniques Build confidence, reduce risk, and find defects in the software you test Explain software testing objectives and results to non-testers in terms they’ll understand Identify and successfully introduce appropriate testing tools and processes

7. Preparing for the ISTQB Foundation Exam Download the Foundation Syllabus and Glossary from www.astqb.org Read the Syllabus and study Glossary definitions Take the first mock exam (end of this section), noting carefully areas you need to give special attention Work through each chapter and section of this course, trying all sample exam questions and exercises, reviewing topics as needed Take the second mock exam at the end of the course If you pass, you should be ready for the real exam If you fail, go back and review the materials you missed Contact your National Board about exam options (e.g., www.astqb.org in the United States)

8. Test Engineering Foundation Essential Knowledge for Test Professionals Mock Exam

9. Let’s See Where You Stand Starting on the next slide, you’ll find the first mock exam for this course This exam will measure where you stand on the material covered in the syllabus Don’t worry if you don’t pass this exam, just pay attention to the questions you miss You’ll want to pay special attention to these topics

10. Articulate Quizmaker Quiz Place holder - Mock Exam1

11. Chapter 1 : Fundamentals of Testing

12. Test Engineering Foundation Essential Knowledge for Test Professionals Chapter 1: Foundation of Testing

13. 1.Fundamentals of Testing Why is testing necessary? What is testing? General testing principles Fundamental test process The psychology of testing

14. Test Engineering Foundation Essential Knowledge for Test Professionals Chapter 1: Fundamentals of Testing Section 1: Why is testing necessary? Part 1

15. Why is Testing Necessary? Key concepts How bugs can cause harm Bugs and their effects The necessity of testing The role of testing in quality assurance Terms to remember

16. The Threat of Bugs Company Damaged reputation for quality High or unpredictable maintenance costs Unexpected delays in release cycles Lack of confidence in system Lawsuits Environment Pollution Waste People, societies, and states Lost jobs Lost lives Lost rights Lost missions Lost wars

17. Where Bugs Come From and What Bugs Do People make errors which put bugs (defects) into the system Requirements and design specifications Code (business logic and user interface) Documentation (electronic and hard copy) When defective code is executed, failures occur If there failures are visible to customers, users, or other stakeholders, dissatisfaction with system quality may result

18. Test Engineering Foundation Essential Knowledge for Test Professionals Chapter 1: Foundation of Testing Section 1: Why is testing necessary? Part 1: Sample exam questions

19. Articulate Quizmaker Quiz Placeholder – Ch1S1P1

20. Test Engineering Foundation Essential Knowledge for Test Professionals Chapter 1: Foundation of Testing Section 1: Why is testing necessary? Part 2

21. What Does “Quality” Mean to You? “ Fitness for use” vs. “Conformance to requirements” Testing and quality Tests give confidence where they find few bugs Passing tests reduce the level of quality risk Failing tests provide a chance to improve quality The test set gives an assessment of quality What are the important quality characteristics for your system? Are you testing them (enough)? Testing, quality assurance, and quality improvement Ideally, testing is part of a larger quality assurance strategy for a project For future projects, analyze the root causes of defects found on current projects and take steps to reduce their incidence

22. Test Engineering Foundation Essential Knowledge for Test Professionals Chapter 1: Fundamentals of Testing Section 1: Why is testing necessary? Part 2: Sample exam questions

24. Test Engineering Foundation Essential Knowledge for Test Professionals Chapter 1: Fundamentals of Testing Section 2: What is testing? Part 1

25. What is Testing? Key concepts Common objectives of testing The purpose of testing… in software development, maintenance, and operations… … to find defects, provide confidence and information, and prevent defects Terms to remember

26. Testing Objectives Typical general testing objectives Find bugs and provide programmers with the information they need to fix important bugs Gain confidence about the level of quality of the system Prevent defects (through early involvement in reviews and advanced test design) Provide information about the most important aspects of the quality of the system under test Help management understand system quality Can you think of others for your projects? Do you tent to align your plans and actions with the testing objectives set by management?

27. Test Phases and Objectives Unit/Component Test: Find bugs in the individual pieces of the system under test before the pieces are fully integrated into the system Integration/String Test: Find bugs in the relationships and interfaces between pairs and groups of components in the system under test as the pieces come together System Test: Find bugs in the overall and particular behaviors, functions, and responses of the system under test as a whole Acceptance/Pilot Test: Demonstrate that the product is ready for deployment/release or to assess quality and give information on the risk of deployment/release Maintenance Test: Check for defects introduced during development of the changes Operational Test: Assess non-functional system characteristics such as reliability or availability usually in the operational environment

28. Effectiveness and Efficiency Effective: Producing a decided, decisive, or desired result; impressive To be effective testers, we must select the appropriate objective and desired results Efficient: Productive of desired effects; especially productive without waste To be efficient testers, we must allocate resources (time and money) These terms are most meaningful in the context of the entire development or maintenance process, not just the testing process

29. Test Engineering Foundation Essential Knowledge for Test Professionals Chapter 1: Fundamentals of Testing Section 2: What is testing? Part 1: Sample exam questions

31. Test Engineering Foundation Essential Knowledge for Test Professionals Chapter 1: Fundamentals of Testing Section 2: What is testing? Part 2

32. Testing vs. Debugging Testing finds failures that are caused by bugs Debugging… … identifies the root cause of a bug… … repairs the code… … and checks that the defect is fixed correctly Confirmation testing ensures the fix resolves the observed failure Different responsibilities: Testers test Programmers debug

33. Find-Debug-Confirm Testing Debugging Developer 7.Is the problem fixed? Does the system now pass the same test it failed before? Does the test of the system still behave properly? 1. Can I reproduce the failure? 2. Does the failure indicate a test bug or a system bug? 3. What factors influence the failure? 4. What is the root cause of the failure? 5. How can I repair the defect without introducing new problems? 6. Is my fix properly debugged? Tester Tester Test Team Bug Report Clear handoffs and Division of Labor T hrough Bug Reporting A nd Test Release Process Bug Fix Development Team

34. Beyond Test Execution Testing is not just running tests against a running system Other test activities, before and after execution, include… Planning and control Choosing test conditions Design test cases Checking test results Evaluating exit criteria Test result reporting Closure/ end-of-test tasks We will revisit this later

35. Test Engineering Foundation Essential Knowledge for Test Professionals Chapter 1: Fundamentals of Testing Section 2: What is testing? Part 2: Sample exam questions

37. Test Engineering Foundation Essential Knowledge for Test Professionals Chapter 1: Fundamentals of Testing Section 3: General testing principles Part 1

38. General Testing Principles Key concepts Testing reveals the presence of bugs Impossibility of exhaustive testing Benefits of early testing Lumpiness of bugs: defect clustering Pesticide paradox Testing should adapt to specific needs Absence-of-errors fallacy Terms to remember

39. Testing Reveals Presence of Bugs: A Parable You have a beautiful vegetable garden, but one day you see eaten leaves on the tomatoes “ Oh no,” you think, “I have hornworms!” You know you have bugs in your garden If you had not seen the symptoms, could you be sure you had no bugs? Some bugs are easy to spot, others aren’t Testing can reveal the presence of bugs, but cannot prove their absence

40. Mission Impossible: Exhaustive Testing “ Just make sure the software works before we ship it…” This charter is demonstrably impossible The execution paths in non-trivial software are almost infinite Large dataflows separated across space (features) and time (static data) Slight changes can cause regressions which are not linear to the size of the change Myriad usage profiles and field configurations, some unknown and some unknowable Bottom line: Exhaustive testing (all combinations of inputs and preconditions) is not possible

41. Defusing Exhaustive Testing Expectations Exhaustive testing as a way to prove the software works is a common (mis)expectation Bad expectations create problems for test professionals and test teams Unachievable high demands on test group Perception of incompetence when these demands aren’t met Testers must be ready to communicate (in words the project stakeholders will understand) how testing can contribute

42. Test Engineering Foundation Essential Knowledge for Test Professionals Chapter 1: Fundamentals of Testing Section 3: General testing principles Part 1: Sample exam questions

45. Defect Clustering Studies have long shown and continue to show the unequal distribution of bugs MVS: 38% of field bugs in 4% of modules IMS: 57% of field bugs in 7% of modules Capers Jones reports that the excessive presence of error-prone modules causes a 50% reduction of productivity in software maintenance

46. Five components with bug counts of less that 60 each.

47. Benefits of Early QA and Testing The cost of a bug tends to increase as the project continues Most of the costs associated with pre-release bugs tend to be associated with the effort required to remove them, so the higher cost means longer schedules The more bugs enter a quality assurance or test activity, the more bugs will escape from that activity

48. 그림

49. Test Engineering Foundation Essential Knowledge for Test Professionals Chapter 1: Fundamentals of Testing Section 3: General testing principles Part 2: Cost of quality questions

52. Pesticide Paradox Return to your vegetable garden Your spray pesticide on your garden, and the hornworms die, but the pesticide is not effective against all bugs Just as pesticides become less effective, so do tests Functional tests can’t find performance bugs Try new test techniques ― if the objective is to find bugs

53. These two pictures compare efficiencies of the top 11 bug-finding test suites (out of 27). The top graph shows the first time the set of test suites was run. The bottom graph shows the results for the fifth time that same set of test suites was run. T he average test suite efficiency in the first pass, 0.4, is used as the axis crossing point in both graphs. With one exception, the same test suites are in the top 11 test suites, which shows bug clustering again. However, all test suites are less effective after five executions.

54. Absence-of-Errors Fallacy Finding and fixing many bugs does not guarantee user, customer, and/or stakeholder satisfaction Many low-defect products have failed in the market place Successful projects balance competing forces in terms of features, schedule, budget, and qualtiy

55. Testing Should Adapt to Needs Different projects, organizations, and products have different testing needs Best testing practices exist (and are discussed in this course) but you need to tailor them to your project Failure to adapt the test team and its methods to these needs is a common result of dissolution of test teams

56. Test Engineering Foundation Essential Knowledge for Test Professionals Chapter 1: Fundamentals of Testing Section 3: General testing principles Part 3: Sample exam questions

58. Test Engineering Foundation Essential Knowledge for Test Professionals Chapter 1: Fundamentals of Testing Section 4: Fundamental test process Part 1

59. Fundamental Test Process Key concepts Planning and control Analysis and design Implementation and execution Evaluating test exit criteria and reporting Test closure activities Terms to remember

60. ISTQB Fundamental Test Process You can think of the test process in terms of the following steps Planning and control Analysis and design Implementation and execution Evaluating test exit criteria and reporting Test closure activities These steps may overlap, be concurrent, and/or iterate

61. Planning and Control Planning Determine test scope, risks, objectives, strategies Determine required test resources Implement the test strategies Schedule test analysis and design Schedule implementation, execution and evaluation of tests Determine the test exit criteria Control Measure and analyze results Monitor and document progress, coverage and test exit criteria Initiate corrective actions Make decisions Many planning and control activities involve obtaining project team and project management agreement, support, and consensus.

62. Test Engineering Foundation Essential Knowledge for Test Professionals Chapter 1: Fundamentals of Testing Section 4: Fundamental test process Part 1: Sample exam questions

64. Test Engineering Foundation Essential Knowledge for Test Professionals Chapter 1: Fundamentals of Testing Section 4: Fundamental test process Part 2

65. Analysis and Design Analysis Review the test basis (e.g., requirements or design specifications, network/system architecture, quality risks) Identify and prioritize test conditions, test requirements, or test objectives and required test data based on anlysis of test items (e.g., their behavior, specification, and structure) Evaluate testability of the requirements and system Design Design and prioritize combinations of test data, actions, and expected results Identify the test data needed for test conditions and cases Design the test environment Identify infrastructure, tools

66. Implementation and Execution Implementation Develop, implement, and prioritize test cases, create data, write procedures Create test harnesses, scripts Organize test suites and sequences of test procedures Verify the test environment Execution Execute test cases (manual or automated) Log test results, and the versions of the software under test, test tools and the testware Compare actual and expected results Report and analyze incidents Repeat corrected and/or updated tests Run confirmation and/or regression tests

67. Exit Criteria, Reporting, and Closure Exit and reporting Check test logs against the exit criteria in the test plan Assess if more tests are needed or if the exit criteria specified should be changed Write a test summary report for stakeholders Closure Confirm test deliverables, final resolution or deferral of bug reports, and the acceptance of the system Finalize and archive testware, test environment and test infrastructure Deliver testware to the maintenance organization Perform a retrospective to capture improvements for future releases, projects, and test processes

68. Test Engineering Foundation Essential Knowledge for Test Professionals Chapter 1: Fundamentals of Testing Section 4: Fundamental test process Part 2: Sample exam questions

70. Test Engineering Foundation Essential Knowledge for Test Professionals Chapter 1: Fundamentals of Testing Section 5: The psychology of testing Part 1

71. Psychology of Testing Key concepts Psychological factors for testing success Clear objectives Self-testing and independent testing Respectful communication Programmer and tester outlooks for success Terms to remember

72. Good Tester Attributes The attributes of a good tester include: Curiosity Professional pessimism A critical eye Attention to detail Good communication skills The proper mix of skills is needed, too

73. Defining Tester Skills Reading Specifications, e-mail, test cases, etc. Writing Test cases, bug reports, test documentation, etc. Technology, project, testing skills Technology: Programming languages and more, like operating systems, networking, HTML/Web, etc. Application domain: banking, human factors, office applications, etc. Testing: scripting, exploring and attacking the system, automation, performance modeling, etc.

74. Differing Mindsets Testing and reviewing are different activities than developing, and the mindsets are different, too Developers can test their code, but often have the wrong mindset to do it effectively Separation of duties (via independent testing) helps focus testing Professional testers are more effective at testing activities, especially finding failures, as they are more objective and don’t have the author’s bias

75. Degrees of Independence Though developers can (and should) test their own code, independent testers are typically more effective at finding failures Levels of test independence By test item’s author (low level of independence) By another person(s) within the same team By a person(s) from a different team or test specialists By a person(s) from a different organization or company (high level of independence)

76. Test Engineering Foundation Essential Knowledge for Test Professionals Chapter 1: Fundamentals of Testing Section 5: The psychology of testing Part 1: Sample exam questions

78. Test Engineering Foundation Essential Knowledge for Test Professionals Chapter 1: Fundamentals of Testing Section 5: The psychology of testing Part 2

79. Clear Objectives People and projects are driven by objectives People usually try to align themselves with objectives set by management and other stakeholders, but often confusion exists about the objectives of testing For example should testing find defects or confirm that software works? If it is to find defects, what percentage? If it is to build confidence, to what level? A test policy can help to clearly state the objectives of testing

80. Focus Focus problems Pursuing issues narrow-mindedly, losing sight of more important priorities Getting distracted from key tasks Balance and re-evaluate priorities every so often Stay focused on the goals and objectives of the test project

81. Constructor or Destructor? Some testers who find failures are perceived as criticizing the product or the author Sometimes, testing is seen as a destructive activity Done properly, testing is essential to managing product risks Project teams must recognize the difference between making a mess, pointing out a mess, and cleaning up a mess…and know who is responsible for each Testers must communicate about the messes they point out in a constructive way

82. Bad News or Bad Guy? Testers are sometimes on receiving end of emotions brought on by news of project problems Communication skills can help Collaborate for better quality Communicate neutrally, about facts, without criticism Understand your colleagues and how they’ll react to your findings Confirm your colleagues understood what you said Confirm you understand your colleagues Schedule pressure can exacerbate this problem

83. Test Engineering Foundation Essential Knowledge for Test Professionals Chapter 1: Fundamentals of Testing Section 5: The psychology of testing Part 2: Sample exam questions

85. Test Engineering Foundation Essential Knowledge for Test Professionals Chapter 1: Fundamentals of Testing Cross-section Sample exam questions

86. Articulate Quizmaker Quiz Placeholder – Ch1XS

Ch14

Related slideshows

Recommended for you

Recommended for you

Recommended for you

Recommended for you

Recommended for you

Recommended for you

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (19)

Similar to Ch14

Similar to Ch14 (20)

More from Khalid Rafiq

More from Khalid Rafiq (15)

Recently uploaded

Recently uploaded (20)

Ch14