Chapter 1 - Introduction and Objectives for Test Automation

Intro & Objectives for Test Automation
1 Introduction to
Test Automation
2 Preparation for
Test Automation
3 gTAA
Software Testing - ISTQB Advance
Test Automation Engineer Exam Preparation
Chapter 1
Neeraj Kumar Singh
5 Reporting
& Metrics
6 Transitioning
Manual to Automation
7 Verifying TAS
8 Continuous
Improvement
4 Risks &
Contingencies

Introduction & Objectives for Test Automation
Contents
1.1 Purpose of Test Automation
1.2 Success Factors in Test Automation
Neeraj Kumar Singh

Purpose of Test Automation
Test automation is expected to help run many test cases consistently and repeatedly on diﬀerent
versions of the SUT and/or environments. But test automation is more than a mechanism for running a
test suite without human interaction. It involves a process of designing the testware, including:
 Software
 Documentation
 Test cases
 Test environments
 Test data
Testware is necessary for the testing activities that include:
 Implementing automated test cases
 Monitoring and controlling the execution of automated tests
 Interpreting, reporting and logging the automated test results
Neeraj Kumar Singh

Test automation has different approaches for interacting with a SUT:
 Testing through the public interfaces to classes, modules or libraries of the SUT (API testing)
 Testing through the user interface of the SUT (e.g., GUI testing or CLI testing)
 Testing through a service or protocol
Objectives of test automation include:
 Improving test efficiency
 Providing wider function coverage
 Reducing the total test cost
 Performing tests that manual testers cannot
 Shortening the test execution period
 Increasing the test frequency/reducing the time required for test cycles
Neeraj Kumar Singh

Advantages of test automation include:
 More tests can be run per build
 The possibility to create tests that cannot be done manually (real-time, remote, parallel tests)
 Tests can be more complex
 Tests run faster
 Tests are less subject to operator error
 More effective and efficient use of testing resources
 Quicker feedback regarding software quality
 Improved system reliability(e.g., repeatability, consistency)
 Improved consistency of tests
Neeraj Kumar Singh

Disadvantages of test automation include:
 Additional costs are involved
 Initial investment to setup TAS
 Requires additional technologies
 Team needs to have development and automation skills
 On-going TAS maintenance requirement
 Can distract from testing objectives, e.g., focusing on automating tests cases at the expense of executing tests
 Tests can become more complex
 Additional errors may be introduced by automation
Limitations of test automation include:
 Not all manual tests can be automated
 The automation can only check machine-interpretable results
 The automation can only check actual results that can be verified by an automated test oracle
 Not a replacement for exploratory testing
Neeraj Kumar Singh

Success Factors in Test Automation
Test Automation Architecture (TAA)
 The Test Automation Architecture (TAA) is very closely aligned with the architecture of a software
product. It should be clear which functional and non-functional requirements the architecture is to
support. Typically this will be the most important requirements.
 Often TAA is designed for maintainability, performance and learnability. It is helpful to involve
software engineers who understand the architecture of the SUT.
SUT Testability
 The SUT needs to be designed for testability that supports automated testing. In the case of GUI
testing, this could mean that the SUT should decouple as much as possible the GUI interaction and
data from the appearance of the graphical interface.
 In the case of API testing, this could mean that more classes, modules or the command-line
interface need to be exposed as public so that they can be tested.
 The testable parts of the SUT should be targeted first. Generally, a key factor in the success of test
automation lies in the ease of implementing automated test scripts.
Neeraj Kumar Singh

Test Automation Strategy
 A practical and consistent test automation strategy that addresses maintainability and consistency of
the SUT.
 It may not be possible to apply the test automation strategy in the same way to both old and new
parts of the SUT. When creating the automation strategy, consider the costs, benefits and risks of
applying it to different parts of the code.
 Consideration should be given to testing both the user interface and the API with automated test
cases to check the consistency of the results.
Neeraj Kumar Singh

Test Automation Framework (TAF)
A test automation framework (TAF) that is easy to use, well documented and maintainable, supports a
consistent approach to automating tests.
In order to establish an easy to use and maintainable TAF, the following must be done:
 Implement reporting facilities
 Enable easy troubleshooting
 Address the test environment appropriately
 Document the automated test cases
 Trace the automated test
 Enable easy maintenance
 Keep the automated tests up-to-date
 Plan for deployment
 Retire tests as needed
 Monitor and restore the SUT
Neeraj Kumar Singh

 The test automation code can be complex to maintain. It is not unusual to have as much code for
testing as the code for the SUT. This is why it is of utmost importance that the test code be
maintainable. This is due to the different test tools being used, the different types of verification
that are used and the different testware artifacts that have to be maintained.
 With these maintenance considerations in mind, in addition to the important items that should be
done, there are a few that should not be done, as follows:
 Do not create code that is sensitive to the interface (i.e., it would be affected by changes in the graphical
interface or in non-essential parts of the API).
 Do not create test automation that is sensitive to data changes or has a high dependency on particular data
values (e.g., test input depending on other test outputs).
 Do not create an automation environment that is sensitive to the context (e.g., operating system date and
time, operating system localization parameters or the contents of another application). In this case, it is better
to use test stubs as necessary so the environment can be controlled.
Neeraj Kumar Singh

Sample Questions
1. Which of the following is considered to be an advantage of test automation over manual testing?
Answer Set
a. The time required for test execution is lengthened and the coverage is increased
b. The time required for test execution is shortened and the coverage is increased
c. The time required for test execution is lengthened and the coverage is reduced
d. The time required for test execution is shortened and the coverage is reduced
Neeraj Kumar Singh

Sample Questions
2. Which of the following is an important technical success factor for any significant automation project?
Answer Set
a. The TAA must be designed for learnability
b. The TAA must support the ability to automate all manual tests
c. The SUT must be self-documenting
d. The GUI interaction and data must be coupled with the graphical interface
Neeraj Kumar Singh

Chapter 1 - Introduction and Objectives for Test Automation

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Chapter 1 - Introduction and Objectives for Test Automation

Similar to Chapter 1 - Introduction and Objectives for Test Automation (20)

More from Neeraj Kumar Singh

More from Neeraj Kumar Singh (20)

Recently uploaded

Recently uploaded (20)

Chapter 1 - Introduction and Objectives for Test Automation

Editor's Notes