Randy Rice presented on lessons learned from user acceptance testing (UAT) on four different projects. The first project involved a new laboratory testing system that had severe performance issues and required three redeployments. The second project with the same company was more successful due to improved testing practices. The third project involved designing many tests based on business scenarios before knowing the user interface. The last project involved a complex legal system where system testing found most defects and UAT was limited due to user capabilities. Key lessons included not relying solely on UAT, having contingency plans, and flexibility in testing plans.
Success in user acceptance testing is all about orchestration - deciding and letting your team know who does what and when.
With the right practices and tools, running successful UAT projects can be effortless, efficient, and fast.
Kanoah is an innovative company that provides test management solutions integrated with the Atlassian JIRA platform. Kanoah Tests allows users to plan, author, execute, track and report on tests directly within JIRA for better collaboration. It offers features like test case management, test execution and reporting, and a REST API for test automation. Customers praise Kanoah Tests for its seamless JIRA integration, support for both agile and traditional testing, and responsive customer support.
Deliver Fast, Break Nothing Via Effective Building Developer and Tester Colla...
The document discusses problems with developer and tester collaboration such as trade-offs between speed, scope and quality. It proposes solutions like testers providing checklists to guide developers and collaborating more closely throughout the development process. Some obstacles to effective collaboration are changing mindsets, building trust between roles, and lack of shared understanding around quality responsibility. With improved collaboration over time through checklists and other means, teams can deliver high quality software faster.
Importance of a Test Management Tool for Your Project
The age we live in demands for RAD(Rapid Application Development) models where testing provides a sense of relief by delivering a sound quality check. From noting down the test scenarios and developing respective test cases, to collecting the test results and sharing them with the team. Turning a blind eye towards the tiniest aspect while testing may have a high impact on your project delivery, or in worst cases, it could even lead to postponement of your release date. Fortunately, we have test management tools available in the market to help orchestrate our release cycle.
Software testing involves investigating a product or service to provide information about its quality to stakeholders, while test automation uses special software to control test execution and compare actual and expected outcomes. There are different types of automation frameworks including linear, keyword driven, data driven, and hybrid approaches, with advantages like reliability and speed but also disadvantages like high upfront investment and potential to leave some testing areas uncovered.
In computer science, all-pairs testing or pairwise testing is a combinatorial method of software testing that, for each pair of input parameters to a system (typically, a software algorithm), tests all possible discrete combinations of those parameters.
Case study on Banking Software Testing - FINACLE : UAT
This is a case study of an user acceptace testing done for a large Global bank for Finacle 10.x; Finacle is a universal banking system from Infosys Technologies . Retail, corporate and other subsystems tested.
Negative testing is all about ensuring that a product or application under test does NOT fail when an unexpected input is being fed. The purpose of Negative testing is to break the system and to verify the application response during unintentional inputs.
The document discusses key concepts in software testing and quality analysis from the viewpoint of customers and producers. It defines bugs and errors, and outlines common causes like complexity, changing requirements, and time pressure. Testing aims to discover faults and weaknesses through execution with the intent of finding errors. The document also mentions software development lifecycles like waterfall model and V-model, as well as standards organizations. It stresses finding and fixing defects early to improve quality.
The document discusses software testing concepts including:
- Quality assurance ensures processes are established to produce products that meet specifications.
- Testing determines if a product meets requirements and identifies failures to meet requirements.
- A test plan is written by the lead tester and includes the testing strategy, resources, and plans. It outlines test cases and procedures to validate software meets specifications.
- Testing begins in the define system phase to ensure requirements are testable, and continues through subsequent phases including product testing, acceptance testing, and deployment. Documentation and repeatable processes are critical to quality assurance.
The document contains answers to 30 common QA interview questions. Some key points addressed include:
- The differences between QA, QC and software testing.
- When QA should start in a project and the role of QA.
- The differences between verification and validation, and smoke testing vs sanity testing.
- What testware, retesting, regression testing and data-driven testing are.
- Challenges of software testing like understanding requirements and time constraints.
- Factors for choosing automated vs manual testing and different SDLC models.
The document outlines seven principles of software testing: 1) Testing shows the presence of errors, not their absence; 2) Exhaustive testing of all possible test cases is impossible; 3) Testing early in the development cycle is important to more easily fix defects; 4) Defects tend to cluster together, following an 80-20 distribution; 5) Test effectiveness fades over time as software changes; 6) Testing methods depend on the type of application; 7) Finding no errors does not mean the system is usable - user requirements must still be met.
Ho Chi Minh City Software Testing Conference January 2015
Software Testing in the Agile World
Website: www.hcmc-stc.org
Author: Tam Bui
Do you have an understanding of software automation testing? Have you often faced challenges when implementing automation testing on your projects? Have successfully achieved the cost and time targets for your automation testing projects?
Joining the talk, you will a better understanding of automation testing practices and its benefits. In this presentation, I will share my experiences in applying test automation on my projects. My experiences are concentrated in areas such as team collaboration and selecting suitable test cases for automation.
Software testers, automation testers and test managers can get great benefits from the talk. Developers can join to understand how to collaborate with the automation testing team. Senior managers will get more understanding about automation testing principles and then help their automation test teams implement automation testing effectively.
When a test manager receives a project to work with, he would like to comprehend the scope of the project, the test objectives such as project timeline, project resources and budget. The Test Manager then needs to think about the test strategy. Selecting an appropriate test strategy is crucial for his project success. There are several test strategies for the Test Manager to select such as analytical, model-based, methodical, process or standard-compliant, dynamic, consultative or directed, and regression-averse. One of the most common and important test strategy is the analytical one that includes risk-based and specification-based testing. Comprehending analytical strategy and its methodologies will help the test manager guide software testing activities to reach the right targets to fulfill the testing objectives. That will make the customers happy and accept his company products. Then he and his company will get paid and great compensation from the customers. From there, his company business will continue to expand and everybody will be happy.
The talk will bring ideas about the analytical strategy and how to run risk-based and specification-based testing activities. Definitely the talk will bring good value to software testing audiences especially test managers. Testers, developers, project managers and higher management can benefit from the talk in the way that they understand and facilitate software testing methodologies in software development life cycle.
Tips and tricks for successful uat testing 2.1panayaofficial
Success in user acceptance testing is all about orchestration - deciding and letting your team know who does what and when.
With the right practices and tools, running successful UAT projects can be effortless, efficient, and fast.
Kanoah is an innovative company that provides test management solutions integrated with the Atlassian JIRA platform. Kanoah Tests allows users to plan, author, execute, track and report on tests directly within JIRA for better collaboration. It offers features like test case management, test execution and reporting, and a REST API for test automation. Customers praise Kanoah Tests for its seamless JIRA integration, support for both agile and traditional testing, and responsive customer support.
The document discusses problems with developer and tester collaboration such as trade-offs between speed, scope and quality. It proposes solutions like testers providing checklists to guide developers and collaborating more closely throughout the development process. Some obstacles to effective collaboration are changing mindsets, building trust between roles, and lack of shared understanding around quality responsibility. With improved collaboration over time through checklists and other means, teams can deliver high quality software faster.
Importance of a Test Management Tool for Your ProjectSarah Elson
The age we live in demands for RAD(Rapid Application Development) models where testing provides a sense of relief by delivering a sound quality check. From noting down the test scenarios and developing respective test cases, to collecting the test results and sharing them with the team. Turning a blind eye towards the tiniest aspect while testing may have a high impact on your project delivery, or in worst cases, it could even lead to postponement of your release date. Fortunately, we have test management tools available in the market to help orchestrate our release cycle.
Software testing involves investigating a product or service to provide information about its quality to stakeholders, while test automation uses special software to control test execution and compare actual and expected outcomes. There are different types of automation frameworks including linear, keyword driven, data driven, and hybrid approaches, with advantages like reliability and speed but also disadvantages like high upfront investment and potential to leave some testing areas uncovered.
In computer science, all-pairs testing or pairwise testing is a combinatorial method of software testing that, for each pair of input parameters to a system (typically, a software algorithm), tests all possible discrete combinations of those parameters.
This is a case study of an user acceptace testing done for a large Global bank for Finacle 10.x; Finacle is a universal banking system from Infosys Technologies . Retail, corporate and other subsystems tested.
Negative testing is all about ensuring that a product or application under test does NOT fail when an unexpected input is being fed. The purpose of Negative testing is to break the system and to verify the application response during unintentional inputs.
The document discusses key concepts in software testing and quality analysis from the viewpoint of customers and producers. It defines bugs and errors, and outlines common causes like complexity, changing requirements, and time pressure. Testing aims to discover faults and weaknesses through execution with the intent of finding errors. The document also mentions software development lifecycles like waterfall model and V-model, as well as standards organizations. It stresses finding and fixing defects early to improve quality.
The document discusses software testing concepts including:
- Quality assurance ensures processes are established to produce products that meet specifications.
- Testing determines if a product meets requirements and identifies failures to meet requirements.
- A test plan is written by the lead tester and includes the testing strategy, resources, and plans. It outlines test cases and procedures to validate software meets specifications.
- Testing begins in the define system phase to ensure requirements are testable, and continues through subsequent phases including product testing, acceptance testing, and deployment. Documentation and repeatable processes are critical to quality assurance.
The document contains answers to 30 common QA interview questions. Some key points addressed include:
- The differences between QA, QC and software testing.
- When QA should start in a project and the role of QA.
- The differences between verification and validation, and smoke testing vs sanity testing.
- What testware, retesting, regression testing and data-driven testing are.
- Challenges of software testing like understanding requirements and time constraints.
- Factors for choosing automated vs manual testing and different SDLC models.
YouTube Link: https://youtu.be/cv6GvRCIuTs
** Test Automation Masters Program: https://www.edureka.co/masters-progra... **
This Edureka PPT on "What is Software Testing?" will give you a brief introduction to what software testing and all the basics concept related to software testing.
The following are the topics covered in the session:
Software Risks
What is Software Testing?
Principles of Sofware Testing
Software Testing Life Cycle
Dynamic Software Testing
Future of Sofware Testing
Follow us to never miss an update in the future.
YouTube: https://www.youtube.com/user/edurekaIN
Instagram: https://www.instagram.com/edureka_learning/
Facebook: https://www.facebook.com/edurekaIN/
Twitter: https://twitter.com/edurekain
LinkedIn: https://www.linkedin.com/company/edureka
Castbox: https://castbox.fm/networks/505?country=in
The document discusses software inspections and defect management. It defines key terms like defects, defect classification, and causes of defects. It explains that rework makes up 44% of project costs and discusses how inspections can help reduce defects and rework. Formal inspections involve individual preparation, overview meetings, review planning, inspection meetings, and follow up action to identify and address defects early. Benefits of inspections include increased productivity, reduced defects, and preparation for subsequent phases.
The document provides answers to 31 questions related to software quality assurance (QA) and testing. It defines key QA terms like QA, QC, verification, validation, smoke testing, and sanity testing. It also discusses topics like the QA role in a project, bug lifecycles, priority and severity levels of bugs, regression testing, data-driven testing, alpha and beta testing, test stubs and drivers, monkey testing, and benefits of automated testing.
Automation test is an interesting research problem in recent years. There are many reasons why we use automation test in the software development. In the traditional approach, automation test has been used for regression test, functional test, performance test… in order to find or prevent bugs and software quality assurance. In this research, we have a novel approach using automation test to build software monitoring solution. The purpose of automation scripts use as monitoring software to capture images and write logs. The architecture pattern of automation for monitoring based on automation test tool, cloud service, and scheduler. The proposed architecture pattern has been applied for online advertisement monitoring. Instead of reporting passed/failed, automation scripts will monitor whether the advertisement is display or not, and how often it was display on multiple platforms. The proposed architecture pattern can also apply for video advertisement monitoring solution.
As a software tester, you may often face a situation in which your customer requires completing testing faster than you can handle given your effort and the amount of test. For example, in order to complete testing 2000 test cases for a build, you need at least 10 days to complete all testing. However, your customer needs to test and release the build within 5 days. You need to make a tough decision to handle this request. This presentation offers you one of the approaches that you can pursue. The presentation discusses an approach to prioritizing test cases using the principles of value-based software engineering. The approach is based on the principle that not every test case is equally importantly, e.g., not each of the 2000 test cases has the same value. A simple Excel tool will also be provided to allow you quickly prioritize test cases and select the ones that generate best value for your customer.
Strategy vs. Tactical Testing: Actions for Today, Plans for TomorrowEggplant
In his STAREAST Virtual+ presentation, Chuck Schneider from Cerner Corporation shared his 6 pillars for strategic planning in testing and offered guidance to navigate the necessary pivot towards tactical execution when faced with a survival situation. Chuck provided a clear, 4-step guide on how to quickly develop and implement a tactical testing plan to avoid the pitfalls of a delayed response. In this presentation you will discover how to harness your strengths, achieve focus, and deliver results in times of incredible change.
The document discusses various stages of testing in the software development lifecycle according to the V-model. It describes component testing as the lowest level of testing done in isolation on individual software modules. An overview of the component testing process is provided, including planning, specification, execution, recording, and completion checking stages. Black box and white box test design techniques for specifying test cases at the component level are also outlined.
- Automating performance tests through continuous integration can provide direct feedback on performance changes after code releases and infrastructure changes. It allows performance issues to be detected and addressed earlier.
- Key best practices include starting with a single important test scenario, focusing on robustness over realism, visualizing trend data over time, and analyzing results to update thresholds and catch regressions.
- The goal is to continuously monitor performance through the pipeline and in production to better understand impacts of changes and flag any performance issues for further investigation. Automated tests complement but do not replace thorough acceptance testing.
This document discusses acceptance testing, which is formal testing conducted by end users to determine if a system meets requirements and business processes before it is accepted. The document outlines what acceptance testing is, different types including user acceptance testing and operational acceptance testing, common application areas, how it fits into software development lifecycles, challenges, and guidelines for success. It also briefly discusses outsourcing acceptance testing.
DevOpsDays Houston 2019 - Lee Barnes - Effective Test Automation in DevOps - ...DevOpsDays Houston
High performing DevOps teams point to effective test automation as a key to their success. This talk delivers key automation practices required to assess the risk of moving their builds through the pipeline - including balancing test scope & risk, test env/data management and continuous improvement
The document discusses effective test automation in DevOps. It begins with an introduction of the speaker and an overview of the topics to be covered. These include test automation in DevOps, common obstacles to automation success, and the pillars of effective test automation regarding scope, approach, and test environment and data management. The document emphasizes that continuous testing requires reliable automated tests, stopping production when tests fail, and developing in small batches. It also outlines challenges around test environments and data availability hindering automation goals.
10-3 Clinical Informatics System Selection & ImplementationCorinn Pope
Section ten, module three of the clinical informatics course discusses the information system lifecycle. In this slide deck, we'll cover how to pick a clinical information system that works best for you. Also included are three free practice questions. If you would like more information or resources, be sure to check out our site at http://www.informaticspro.com.
The document discusses several approaches to system development including the waterfall model, prototyping model, incremental model, and spiral model. The waterfall model involves sequential phases from requirements analysis to maintenance. The prototyping model develops initial prototypes to refine user requirements, while the incremental model delivers software in iterations. The spiral model combines elements of waterfall and prototyping, with risk analysis and evaluation at each phase.
The document outlines the key steps in creating a functional testing strategy:
1. Understanding system requirements to identify business processes, data, and security needs.
2. Identifying test scenarios to describe specific business processes to test.
3. Defining test objectives to ensure the system's functionality, data accuracy, and security.
Quality Assurance in Modern Software DevelopmentZahra Sadeghi
This document discusses quality assurance in modern software development. It begins by providing resources on the topic and outlining the agenda. It then reviews basic concepts of software, quality, and the differences between quality assurance and quality control. It introduces several quality models including McCall's quality model and discusses important factors in software quality. Finally, it covers quality assurance methodology using PDCA, quality management tools including Ishikawa diagrams and Pareto charts, and software quality testing. The document provides a comprehensive overview of key aspects of quality assurance in software development.
User expectations have changed over the last decade. Customers today expect access to their applications and data from all devices (mobile, laptop, desktop, tablet, etc.) with similar performance from any of those devices at all times of the day. In a world of growing complexity where architects and application designers are dependent on 3rd party providers to delivering part (or at time entire) of the application how does one ensure consistent delivery of performance. This presentation provides a view of some of the challenges involved and how not to make costly mistakes.
The document discusses software quality assurance and defines quality as meeting customer requirements within agreed timescales and costs, and providing customer satisfaction. It discusses standard definitions of software quality, views of quality, and quality criteria. Large software projects often fail due to quality problems. Software quality engineering aims to meet quality expectations through validation and verification activities. Its main tasks include quality planning, execution of quality assurance activities like testing, and measurement and analysis. A quality engineering process manages these activities to achieve preset quality goals.
DevOps Workshop - Addressing Quality Challenges of Highly Complex and Integra...Andrew Williams
Is your delivery and testing approach able to keep pace with today’s business demands? Poor or low levels of collaboration between application owners, developers and testers using a variety of tools & practices and long cycles of provisioning of dev/test environments can impede quality, increase waste and cost. Join this workshop discussion and share your opinion. Collaborate with peers on best practices to eliminate testing bottlenecks through virtualized dependent services and how to stand up realistic, production-like test labs that can be easily deployed, shared, and updated as systems change. The workshop will conclude with Q&A among the participants.
How to Migrate Drug Safety and Pharmacovigilance Data Cost-Effectively and wi...Perficient
This document discusses challenges with data migration projects and provides an overview of a solution called Accel-Migrate. It notes that the success rate for the data migration portion of projects is typically between 16-60% due to issues like poorly defined scope and inadequate testing. Accel-Migrate aims to address these issues through an assessment, automated testing that verifies 100% of migrated data, validation inclusive of evidence for audits, and process reengineering support. The methodology employs pre-migration testing, configuration of migration software, and parallel process reengineering to allow for simultaneous technical and process work.
Learn how to establish a greater sense of confidence in your release cycle, along with the practices and processes to create a high-performing engineering culture within your team.
What slows down your mobile SDLC?
We analyzed the testing strategies from 350 enterprise app developers, testers and QA manager to find out what causes delays.
Learn how to accelerate the mobile app lifecycle from development to deployment and discover:
What factors slow down app testing
How these factors delay release cycles
Strategies to speed up app testing and delivery
- The document outlines Polarion's test management software capabilities including creating and managing test cases, defects, requirements and specifications with Polarion LiveDocs. It allows defining and running test runs with the Polarion Testing Framework.
- It discusses how Polarion can help integrate requirements, testing and defect management and manage activities with all stakeholders.
- The presentation then demonstrates Polarion's abilities like requirements and test traceability, test planning and execution, impact analysis and reporting across projects.
The document discusses the Unified Process (UP) methodology for software development. It describes the key aspects of UP including iterative development with timeboxed iterations, four phases (inception, elaboration, construction, transition), architecture-centric and risk-driven approach, and nine core workflows (business modeling, requirements, design, implementation, test, deployment, project management, configuration management, environment). The document provides details on each of these aspects of UP and best practices for its implementation on a software project.
Isabel Evans stopped drawing and painting after being told she was not very good at it, which led to a loss of confidence in her creative and professional abilities. However, she realized that attempting creative activities is important for cognitive and emotional development, and that making mistakes and learning from failures allows for growth. By reengaging with failure through art and with support from others, Isabel was able to regain confidence in her abilities and reboot her career. The document discusses different perspectives on failure and the importance of learning from mistakes.
Instill a DevOps Testing Culture in Your Team and Organization TechWell
The DevOps movement is here. Companies across many industries are breaking down siloed IT departments and federating them into product development teams. Testing and its practices are at the heart of these changes. Traditionally, IT organizations have been staffed with mostly manual testers and a limited number of automation and performance engineers. To keep pace with development in the new “you build it, you own it” environment, testing teams and individuals must develop new technical skills and even embrace coding to stay relevant and add greater value to the business. DevOps really starts with testing. Join Adam Auerbach as he explains what DevOps is and how it relates to testing. He describes how testing must change from top to bottom and how to access your own environment to identify improvement opportunities. Adam dives into practices like service virtualization, test data management, and continuous testing so you can understand where you are now and identify steps needed to instill a DevOps testing culture in your team and organization.
Test Design for Fully Automated Build ArchitectureTechWell
This document summarizes a half-day tutorial on test design for fully automated build architectures presented by Melissa Benua of mParticle at STAREAST 2018. The tutorial covered guiding principles for test design including prioritizing important and reliable tests, structuring automated pipelines around components, packages, and releases, and monitoring test results through code coverage, flaky test handling, and logging versus counters. It also included exercises mapping test cases to functional boundaries and categories of tests to pipeline stages.
System-Level Test Automation: Ensuring a Good StartTechWell
Many organizations invest a lot of effort in test automation at the system level but then have serious problems later on. As a leader, how can you ensure that your new automation efforts will get off to a good start? What can you do to ensure that your automation work provides continuing value? This tutorial covers both “theory” and “practice”. Dot Graham explains the critical issues for getting a good start, and Chris Loder describes his experiences in getting good automation started at a number of companies. The tutorial covers the most important management issues you must address for test automation success, particularly when you are new to automation, and how to choose the best approaches for your organization—no matter which automation tools you use. Focusing on system level testing, Dot and Chris explain how automation affects staffing, who should be responsible for which automation tasks, how managers can best support automation efforts to promote success, what you can realistically expect in benefits and how to report them. They explain—for non-techies—the key technical issues that can make or break your automation effort. Come away with your own clarified automation objectives, and a draft test automation strategy to use to plan your own system-level test automation.
Build Your Mobile App Quality and Test StrategyTechWell
Let’s build a mobile app quality and testing strategy together. Whether you have a web, hybrid, or native app, building a quality and testing strategy means (1) knowing what data and tools you have available to make agile decisions, (2) understanding your customers and your competitors, and (3) testing your app under real-world conditions. Jason Arbon guides you through the latest techniques, data, and tools to ensure the awesomeness of your mobile app quality and testing strategy. Leave this interactive session with a strategy for your very own app—or one you pretend to own. The information Jason shares is based on data from Appdiff’s next-gen mobile app testing platform, lessons from Applause/uTest’s crowd, text mining hundreds of millions of app store reviews, and in-depth discussions with top mobile app development teams.
Testing Transformation: The Art and Science for SuccessTechWell
Technologies, testing processes, and the role of the tester have evolved significantly in the past few years with the advent of agile, DevOps, and other new technologies. It is critical that we testing professionals evaluate ourselves and continue to add tangible value to our organizations. In your work, are you focused on the trivial or on real game changers? Jennifer Bonine describes critical elements that help you artfully blend people, process, and technology to create a synergistic relationship that adds value. Jennifer shares ideas on mastering politics, maneuvering core vs. context, and innovating your technology strategies and processes. She explores how new processes can be introduced in an organization, what the role of organizational culture is in determining the success of a project, and how you can know what tools will add value vs. simply adding overhead and complexity. Jennifer reviews critically needed tester skills and discusses a continual learning model to evolve your skills and stay relevant. This discussion can lead you to technologies, processes, and skills you can stake your career on.
We’ve all been there. We work incredibly hard to develop a feature and design tests based on written requirements. We build a detailed test plan that aligns the tests with the software and the documented business needs. And when we put the tests to the software, it all falls apart because the requirements were changed without informing everyone. Mary Thorn says help is at hand. Enter behavior-driven development (BDD), and Cucumber and SpecFlow, tools for running automated acceptance tests and facilitating BDD. Mary explores the nuances of Cucumber and SpecFlow, and shows you how to implement BDD and agile acceptance testing. By fostering collaboration for implementing active requirements via a common language and format, Cucumber and SpecFlow bridge the communication gap between business stakeholders and implementation teams. In this workshop, practice writing feature files with the best practices Mary has discovered over numerous implementations. If you experience developers not coding to requirements, testers not getting requirements updates, or customers who feel out of the loop and don’t get what they ask for, Mary has answers for you.
Develop WebDriver Automated Tests—and Keep Your SanityTechWell
Many teams go crazy because of brittle, high-maintenance automated test suites. Jim Holmes helps you understand how to create a flexible, maintainable, high-value suite of functional tests using Selenium WebDriver. Learn the basics of what to test, what not to test, and how to avoid overlapping with other types of testing. Jim includes both philosophical concepts and hands-on coding. Testers who haven't written code should not be intimidated! We'll pair you up to make sure you're successful. Learn to create practical tests dealing with advanced situations such as input validation, AJAX delays, and working with file downloads. Additionally, discover when you need to work together with developers to create a system that's more easily testable. This tutorial focuses primarily on automating web tests, but many of the same concepts can be applied to other UI environments. Demos and labs will be in C# and Java using WebDriver. Leave this tutorial having learned how to write high-value WebDriver tests—and stay sane while doing so.
DevOps is a cultural shift aimed at streamlining intergroup communication and improving operational efficiency for development and operations groups. Over time, inclusion of other IT groups under the DevOps umbrella has become the norm for many organizations. But even broadening the boundaries of DevOps, the conversation has been largely devoid of the business units’ place at the table. A common mistake organizations make while going through the DevOps transformation is drawing a line at the IT boundary. If that occurs, a larger, more inclusive silo within the organization is created, operating in an informational vacuum and causing operational inefficiency and goal misalignment. Sharing his experiences working on both sides of the fence, Leon Fayer describes the importance of including business units in order to align technology decisions with business goals. Leon discusses inclusion of business units in existing agile processes, benefits of cross-departmental monitoring, and a business-first approach to technology decisions.
Eliminate Cloud Waste with a Holistic DevOps StrategyTechWell
Chris Parlette maintains that renting infrastructure on demand is the most disruptive trend in IT in decades. In 2016, enterprises spent $23B on public cloud IaaS services. By 2020, that figure is expected to reach $65B. The public cloud is now used like a utility, and like any utility, there is waste. Who's responsible for optimizing the infrastructure and reducing wasted expenses? It’s DevOps. The excess expense, known as cloud waste, comprises several interrelated problems: services running when they don't need to be, improperly sized infrastructure, orphaned resources, and shadow IT. There are a few core tenets of DevOps—holistic thinking, no silos, rapid useful feedback, and automation—that can be applied to reducing your cloud waste. Join Chris to learn why you should include continuous cost optimization in your DevOps processes. Automate cost control, reduce your cloud expenses, and make your life easier.
Transform Test Organizations for the New World of DevOpsTechWell
With the recent emergence of DevOps across the industry, testing organizations are being challenged to transform themselves significantly within a short period of time to stay meaningful within their organizations. It’s not easy to plan and approach these changes considering the way testing organizations have remained structured for ages. These challenges start from foundational organizational structures and can cut across leadership influence, competencies, tools strategy, infrastructure, and other dimensions. Sumit Kumar shares his experience assisting various organizations to overcome these challenges using an organized DevOps enablement framework. The framework includes radical restructuring, turning the tools strategy upside down, a multidimensional workforce enablement supported by infrastructure changes, redeveloped collaborations models, and more. From his real world experiences Sumit shares tips for approaching this journey and explains the roadmap for testing organizations to transform themselves to lead the quality in DevOps.
The Fourth Constraint in Project Delivery—LeadershipTechWell
All too often, the triple constraints—time, cost, and quality—are bandied about as if they are the be-all, end-all. While they are important, leadership—the fourth and larger underpinning constraint—influences the first three. Statistics on project success and failure abound, and these measurements are usually taken against the triple constraints. According to the Project Management Institute, only 53 percent of projects are completed within budget, and only 49 percent are completed on time. If so many projects overrun budget and are late, we can’t really say, “Good, fast, or cheap—pick two.” Rob Burkett talks about leadership at every level of a team. He shares his insights and stories gleaned from his years of IT and project management experience. Rob speaks to some of the glaring difficulties in the workplace in general and some specifically related to IT delivery and project management. Leave with a clearer understanding of how to communicate with teams and team members, and gain a better understanding of how you can be a leader—up and down your organization.
Resolve the Contradiction of Specialists within Agile TeamsTechWell
As teams grow, organizations often draw a distinction between feature teams, which deliver the visible business value to the user, and component teams, which manage shared work. Steve Berczuk says that this distinction can help organizations be more productive and scale effectively, but he recognizes that not all shared work fits into this model. Some work is best handled by “specialists,” that is people with unique skills. Although teams composed entirely of T-shaped people is ideal, certain skills are hard to come by and are used irregularly across an organization. Since these specialists often need to work closely with teams, rather than working from their own backlog, they don’t fit into the component team model. The use of shared resources presents challenges to the agile planning model. Steve Berczuk shares how teams such as those providing infrastructure services and specialists can fit into a feature+component team model, and how variations such as embedding specialists in a scrum team can both present process challenges and add significant value to both the team and the larger organization.
Pin the Tail on the Metric: A Field-Tested Agile GameTechWell
Metrics don’t have to be a necessary evil. If done right, metrics can help guide us to make better forward-looking decisions, rather than being used for simply managing or monitoring. They can help us identify trade-offs between options for what to do next versus punitive or worse, purely managerial measures. Steve Martin won’t be giving the Top Ten List of field-tested metrics you should use. Instead, in this interactive mini-workshop, he leads you through the critical thinking necessary for you to determine what is right for you to measure. First, Steve explores why you want to measure something—whether it’s for a team, a portfolio, or even an agile transformation. Next, he provides multiple real-life metrics examples to help drive home concepts behind characteristics of good and bad metrics. Finally, Steve shows how to run his field-tested agile game—Pin the Tail on the Metric. Take back this activity to help you guide metrics conversations at your organization.
Agile Performance Holarchy (APH)—A Model for Scaling Agile TeamsTechWell
A hierarchy is an organizational network that has a top and a bottom, and where position is determined by rank, importance, and value. A holarchy is a network that has no top or bottom and where each person’s value derives from his ability, rather than position. As more companies seek the benefits of agile, leaders need to build and sustain delivery capability while scaling agile without introducing unnecessary process and overhead. The Agile Performance Holarchy (APH) is an empirical model for scaling and sustaining agility while continuing to deliver great products. Jeff Dalton designed the APH by drawing from lessons learned observing and assessing hundreds of agile companies and teams. The APH helps implement a holarchy—a system composed of interacting organizational units called holons—centered on a series of performance circles that embody the behaviors of high performing agile organizations. Jeff describes how APH provides guidelines in the areas of leadership, values, teaming, visioning, governing, building, supporting, and engaging within an all-agile organization. Join Jeff to see what the APH is all about and how you can use it in your team and organization.
A Business-First Approach to DevOps ImplementationTechWell
DevOps is a cultural shift aimed at streamlining intergroup communication and improving operational efficiency for development and operations groups. Over time, inclusion of other IT groups under the DevOps umbrella has become the norm for many organizations. But even broadening the boundaries of DevOps, the conversation has been largely devoid of the business units’ place at the table. A common mistake organizations make while going through the DevOps transformation is drawing a line at the IT boundary. If that occurs, a larger, more inclusive silo within the organization is created, operating in an informational vacuum and causing operational inefficiency and goal misalignment. Sharing his experiences working on both sides of the fence, Leon Fayer describes the importance of including business units in order to align technology decisions with business goals. Leon discusses inclusion of business units in existing agile processes, benefits of cross-departmental monitoring, and a business-first approach to technology decisions.
Databases in a Continuous Integration/Delivery ProcessTechWell
The document summarizes a presentation about including databases in a continuous integration/delivery process. It discusses treating database code like application code by placing it under version control and integrating databases into the DevOps software development pipeline. This allows databases to be built, tested, and released like other software through continuous integration, delivery, and deployment.
Mobile Testing: What—and What Not—to AutomateTechWell
Organizations are moving rapidly into mobile technology, which has significantly increased the demand for testing of mobile applications. David Dangs says testers naturally are turning to automation to help ease the workload, increase potential test coverage, and improve testing efficiency. But should you try to automate all things mobile? Unfortunately, the answer is not always clear. Mobile has its own set of complications, compounded by a wide variety of devices and OS platforms. Join David to learn what mobile testing activities are ripe for automation—and those items best left to manual efforts. He describes the various considerations for automating each type of mobile application: mobile web, native app, and hybrid applications. David also covers device-level testing, types of testing, available automation tools, and recommendations for automation effectiveness. Finally, based on his years of mobile testing experience, David provides some tips and tricks to approach mobile automation. Leave with a clear plan for automating your mobile applications.
Cultural Intelligence: A Key Skill for SuccessTechWell
Diversity is becoming the norm in everyday life. However, introducing global delivery models without a proper understanding of intercultural differences can lead to difficulty, frustration, and reduced productivity. Priyanka Sharma and Thena Barry say that in our diverse world, we need teams with people who can cross these boundaries, communicate effectively, and build the diverse networks necessary to avoid problems. We need to learn about cultural intelligence (CI) and cultural quotient (CQ). CI is the ability to relate and work effectively across cultures. CQ is the cognitive, motivational, and behavioral capacity to understand and respond to beliefs, values, attitudes, and behaviors of individuals and groups. Together, CI and CQ can help us build behavioral capacities that aid motivation, behavior, and productivity in teams as well as individuals. Priyanka and Thena show how to build a more culturally intelligent place with tools and techniques from Leading with Cultural Intelligence, as well as content from the Hofstede cultural model. In addition, they illustrate the model with real-life experiences and demonstrate how they adapted in similar circumstances.
Turn the Lights On: A Power Utility Company's Agile TransformationTechWell
Why would a century-old utility with no direct competitors take on the challenge of transforming its entire IT application organization to an agile methodology? In an increasingly interconnected world, the expectations of customers continue to evolve. From smart meters to smart phones, IoT is creating a crisis point for industries not accustomed to rapid change. Glen Morris explains that pizzas can be tracked by the minute and packages at every stop, and customers now expect this same customer service model should exist for all industries—including power. Glen examines how to create momentum and transform non-IT-focused industries to an agile model. If you are struggling with gaining traction in your pursuit of agile within your business, Glen gives you concrete, practical experiences to leverage in your pursuit. Finally, he communicates how to gain buy-in from business partners who have no idea or concern about agile or its methodologies. If your business partners look at you with amusement when you mention the need for a dedicated Product Owner, join Glen as he walks you through the approaches to overcoming agile skepticism.
Quality Patents: Patents That Stand the Test of TimeAurora Consulting
Is your patent a vanity piece of paper for your office wall? Or is it a reliable, defendable, assertable, property right? The difference is often quality.
Is your patent simply a transactional cost and a large pile of legal bills for your startup? Or is it a leverageable asset worthy of attracting precious investment dollars, worth its cost in multiples of valuation? The difference is often quality.
Is your patent application only good enough to get through the examination process? Or has it been crafted to stand the tests of time and varied audiences if you later need to assert that document against an infringer, find yourself litigating with it in an Article 3 Court at the hands of a judge and jury, God forbid, end up having to defend its validity at the PTAB, or even needing to use it to block pirated imports at the International Trade Commission? The difference is often quality.
Quality will be our focus for a good chunk of the remainder of this season. What goes into a quality patent, and where possible, how do you get it without breaking the bank?
** Episode Overview **
In this first episode of our quality series, Kristen Hansen and the panel discuss:
⦿ What do we mean when we say patent quality?
⦿ Why is patent quality important?
⦿ How to balance quality and budget
⦿ The importance of searching, continuations, and draftsperson domain expertise
⦿ Very practical tips, tricks, examples, and Kristen’s Musts for drafting quality applications
https://www.aurorapatents.com/patently-strategic-podcast.html
Mitigating the Impact of State Management in Cloud Stream Processing SystemsScyllaDB
Stream processing is a crucial component of modern data infrastructure, but constructing an efficient and scalable stream processing system can be challenging. Decoupling compute and storage architecture has emerged as an effective solution to these challenges, but it can introduce high latency issues, especially when dealing with complex continuous queries that necessitate managing extra-large internal states.
In this talk, we focus on addressing the high latency issues associated with S3 storage in stream processing systems that employ a decoupled compute and storage architecture. We delve into the root causes of latency in this context and explore various techniques to minimize the impact of S3 latency on stream processing performance. Our proposed approach is to implement a tiered storage mechanism that leverages a blend of high-performance and low-cost storage tiers to reduce data movement between the compute and storage layers while maintaining efficient processing.
Throughout the talk, we will present experimental results that demonstrate the effectiveness of our approach in mitigating the impact of S3 latency on stream processing. By the end of the talk, attendees will have gained insights into how to optimize their stream processing systems for reduced latency and improved cost-efficiency.
Advanced Techniques for Cyber Security Analysis and Anomaly DetectionBert Blevins
Cybersecurity is a major concern in today's connected digital world. Threats to organizations are constantly evolving and have the potential to compromise sensitive information, disrupt operations, and lead to significant financial losses. Traditional cybersecurity techniques often fall short against modern attackers. Therefore, advanced techniques for cyber security analysis and anomaly detection are essential for protecting digital assets. This blog explores these cutting-edge methods, providing a comprehensive overview of their application and importance.
Coordinate Systems in FME 101 - Webinar SlidesSafe Software
If you’ve ever had to analyze a map or GPS data, chances are you’ve encountered and even worked with coordinate systems. As historical data continually updates through GPS, understanding coordinate systems is increasingly crucial. However, not everyone knows why they exist or how to effectively use them for data-driven insights.
During this webinar, you’ll learn exactly what coordinate systems are and how you can use FME to maintain and transform your data’s coordinate systems in an easy-to-digest way, accurately representing the geographical space that it exists within. During this webinar, you will have the chance to:
- Enhance Your Understanding: Gain a clear overview of what coordinate systems are and their value
- Learn Practical Applications: Why we need datams and projections, plus units between coordinate systems
- Maximize with FME: Understand how FME handles coordinate systems, including a brief summary of the 3 main reprojectors
- Custom Coordinate Systems: Learn how to work with FME and coordinate systems beyond what is natively supported
- Look Ahead: Gain insights into where FME is headed with coordinate systems in the future
Don’t miss the opportunity to improve the value you receive from your coordinate system data, ultimately allowing you to streamline your data analysis and maximize your time. See you there!
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptxSynapseIndia
Your comprehensive guide to RPA in healthcare for 2024. Explore the benefits, use cases, and emerging trends of robotic process automation. Understand the challenges and prepare for the future of healthcare automation
Are you interested in dipping your toes in the cloud native observability waters, but as an engineer you are not sure where to get started with tracing problems through your microservices and application landscapes on Kubernetes? Then this is the session for you, where we take you on your first steps in an active open-source project that offers a buffet of languages, challenges, and opportunities for getting started with telemetry data.
The project is called openTelemetry, but before diving into the specifics, we’ll start with de-mystifying key concepts and terms such as observability, telemetry, instrumentation, cardinality, percentile to lay a foundation. After understanding the nuts and bolts of observability and distributed traces, we’ll explore the openTelemetry community; its Special Interest Groups (SIGs), repositories, and how to become not only an end-user, but possibly a contributor.We will wrap up with an overview of the components in this project, such as the Collector, the OpenTelemetry protocol (OTLP), its APIs, and its SDKs.
Attendees will leave with an understanding of key observability concepts, become grounded in distributed tracing terminology, be aware of the components of openTelemetry, and know how to take their first steps to an open-source contribution!
Key Takeaways: Open source, vendor neutral instrumentation is an exciting new reality as the industry standardizes on openTelemetry for observability. OpenTelemetry is on a mission to enable effective observability by making high-quality, portable telemetry ubiquitous. The world of observability and monitoring today has a steep learning curve and in order to achieve ubiquity, the project would benefit from growing our contributor community.
How Social Media Hackers Help You to See Your Wife's Message.pdfHackersList
In the modern digital era, social media platforms have become integral to our daily lives. These platforms, including Facebook, Instagram, WhatsApp, and Snapchat, offer countless ways to connect, share, and communicate.
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdfNeo4j
Presented at Gartner Data & Analytics, London Maty 2024. BT Group has used the Neo4j Graph Database to enable impressive digital transformation programs over the last 6 years. By re-imagining their operational support systems to adopt self-serve and data lead principles they have substantially reduced the number of applications and complexity of their operations. The result has been a substantial reduction in risk and costs while improving time to value, innovation, and process automation. Join this session to hear their story, the lessons they learned along the way and how their future innovation plans include the exploration of uses of EKG + Generative AI.
Sustainability requires ingenuity and stewardship. Did you know Pigging Solutions pigging systems help you achieve your sustainable manufacturing goals AND provide rapid return on investment.
How? Our systems recover over 99% of product in transfer piping. Recovering trapped product from transfer lines that would otherwise become flush-waste, means you can increase batch yields and eliminate flush waste. From raw materials to finished product, if you can pump it, we can pig it.
Measuring the Impact of Network Latency at TwitterScyllaDB
Widya Salim and Victor Ma will outline the causal impact analysis, framework, and key learnings used to quantify the impact of reducing Twitter's network latency.
The DealBook is our annual overview of the Ukrainian tech investment industry. This edition comprehensively covers the full year 2023 and the first deals of 2024.
INDIAN AIR FORCE FIGHTER PLANES LIST.pdfjackson110191
These fighter aircraft have uses outside of traditional combat situations. They are essential in defending India's territorial integrity, averting dangers, and delivering aid to those in need during natural calamities. Additionally, the IAF improves its interoperability and fortifies international military alliances by working together and conducting joint exercises with other air forces.
An invited talk given by Mark Billinghurst on Research Directions for Cross Reality Interfaces. This was given on July 2nd 2024 as part of the 2024 Summer School on Cross Reality in Hagenberg, Austria (July 1st - 7th)
The Rise of Supernetwork Data Intensive ComputingLarry Smarr
Invited Remote Lecture to SC21
The International Conference for High Performance Computing, Networking, Storage, and Analysis
St. Louis, Missouri
November 18, 2021
20240704 QFM023 Engineering Leadership Reading List June 2024
A Funny Thing Happened on the Way to User Acceptance Testing
1. T1
Test Management
5/8/2014 9:45:00 AM
A Funny Thing Happened on the
Way to User Acceptance Testing
Presented by:
Randy Rice
Rice Consulting Services, Inc.
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
2. Randy Rice
Rice Consulting Services, Inc.
A leading author, speaker, and consultant with more than thirty years of experience in the field
of software testing and software quality, Randy Rice has worked with organizations worldwide to
improve the quality of their information systems and optimize their testing processes. He is
coauthor (with William E. Perry) of Surviving the Top Ten Challenges of Software Testing and
Testing Dirty Systems. Randy is an officer of the American Software Testing Qualifications
Board (ASTQB). Founder, principal consultant, and trainer at Rice Consulting Services, Randy
can be contacted at riceconsulting.com where he publishes articles, newsletters, and other
content about software testing and software quality. Visit Randy’s blog.
3. 4/26/2014
1
A FUNNY THING
HAPPENED ON THE
WAY TO THE
ACCEPTANCE TEST
RANDALL W. RICE, CTAL
RICE CONSULTING SERVICES, INC.
WWW.RICECONSULTING.COM
2
THIS PRESENTATION
• The account of four different
acceptance tests, in three
organizations.
• The names have been withheld and
the data generalized to protect
privacy.
• One project was in-house developed
and the other three were vendor-
developed systems.
• So, a more traditional UAT approach
was taken.
4. 4/26/2014
2
3
A COMMON
PERCEPTION OF UAT
• UAT is often seen as that last golden moment or phase of
testing, where
• Users give feedback/acceptance
• Minor problems are identified and fixed
• The project is implemented on time
• High fives, all around
4
IN REALITY…
• UAT is one of the most risky and
explosive levels of testing.
• UAT is greatly needed, but happens at
the worst time to find major defects –
at the end of the project.
• Users may be unfriendly to the new
system
• They like the current one just fine,
thank you.
• Much of your UAT planning may be
ignored.
• People tend to underestimate how
many cycles of regression testing are
needed.
5. 4/26/2014
3
5
THERE ARE MANY
QUESTIONS ABOUT UAT
• Who plans it?
• Who performs it?
• Should it only be manual in nature?
• What is the basis for test design and
evaluation?
• When should it be performed?
• Where should it be performed?
• Who leads it?
• How much weight should be given to it?
6
PROJECT #1
• Medical laboratory testing business
that closely resembles a manufacturing
environment.
• New Technology for the company and
for the industry.
• The previous project had failed
• The company almost went out of
business because of it!
• Very high growth in terms of both
business and employees.
• Company at risk of failure.
• This project was truly mission-critical.
6. 4/26/2014
4
7
PROJECT #1 – CONT’D.
• Few functional requirements.
• 8 pages for over 400 modules
• Test team had little knowledge of:
• subject matter,
• test design,
• or testing at all.
• Very little unit or integration testing
being done by developers.
• Some system testing was done.
• UAT was the focus.
8
DEFECT DISCOVERY
AND BACKLOG
System Test UAT 1st Deploy
2nd Deploy
3rd Deploy
4 weeks 3 weeks
7. 4/26/2014
5
9
PROJECT #1 RESULTS
• Very high defect levels in testing.
• Many were resolved before implementation.
• Severe performance problems.
• Old system could process 8,000 units/day
• New system could process 400 units/day
• Many problems due to the new technology being used
• “bleeding edge” issues
• “Deadline or else” attitude
• The business was under extreme pressure to deploy due to
increased processing volumes.
• System was de-installed/re-installed 3 times before
performance was acceptable to deploy.
10
WHAT WE LEARNED
• Requirements are important.
• Even if you have to create some form of
them after the software has been
written.
• Early testing is important.
• That would have caught early
performance bottlenecks.
• Teamwork is critical.
• Things got so bad we had to have a “do
over.”
• The deadline is a bad criteria for
implementation.
• Always have a “Plan B”.
8. 4/26/2014
6
11
UAT LESSONS
• Build good relationships with subject
matter experts.
• They often determine acceptance
• Listen to the end-users.
• Understand what’s important
• Don’t rely on UAT for defect detection.
• Interesting factoid
• A similar project with the exact same
technology failed due to performance
errors two years later for a city water
utility. $1.8 million lawsuit lost by the
vendor.
12
PROJECT #2
• Same company as before, but two
years later
• Integration of a vendor-developed
and customized accounting
system
• Lots of defects in the vendor
system
• Implemented two months late with
practically zero defects.
9. 4/26/2014
7
13
WHAT MADE THE
DIFFERENCE?
• Same people – testers, IT manager, developer
• Different project manager who was a big
supporter of testing
• More experience with the technology
• Better understanding of testing and test design
• A repeatable process
• Less pressure to implement
• Having a contingency plan
• Having the courage to delay deployment in favor
of quality.
• The financials had to be right.
14
PROJECT #3
• New government-sponsored entity.
• Everything was new – building, people,
systems
• System was a vendor-developed
workers compensation system.
• Some customization
• Little documentation
• Designed all tests based on business
scenarios.
• We had no idea of the UI design.
10. 4/26/2014
8
15
KEY FACTORS
• No end-users in place at first to help with
any UAT planning.
• In fact, we had to train the end-users in
the system and the business.
• Lots of test planning was involved
• 50% or more effort in planning and
optimizing tests.
• This paid off big in test execution and
training
16
RESULTS
• Tested 700 modules with 250 business
scenario tests.
• We had designed over 300 tests
• The management and test team felt
confident after 250 tests we had covered
enough of the system.
• Found many defects in a system that
had been in use in other companies for
years.
• Reused a lot of the testware as training
aids.
• Successful launch of the organization
and system.
11. 4/26/2014
9
17
HARD LESSONS
LEARNED
• “You don’t know what you don’t know”
AND “You sometimes don’t know what you
think you know.”
• Newly hired SME with over 30 years
workers comp experience provided
information that was different (correct)
than what we had been told during test
design.
• We had to assign two people for two weeks
to create new tests.
• These were complex financial functions –
we couldn’t make it up on the fly.
18
HARD LESSONS
LEARNED (2)
• Real users are needed for UAT.
• Sometimes the heavy lifting of test
design may be done by other testers,
but users need heavy involvement.
12. 4/26/2014
10
19
PROJECT #4
• State government, Legal application
• Vendor-developed and customized
• Highly complex system purchased to replace two co-
existing systems.
• Half of the counties in the state used one system, the other
half used another.
• Usability factors were low on the new system
• Data conversion correctness was critical
20
THE GOOD SIDE
• Well-defined project processes
• Highly engaged management and
stakeholders
• Good project planning and tracking
• Incremental implementation strategy
• The entire system was implemented,
only one county at a time.
• Heavy system testing
• Good team attitude
13. 4/26/2014
11
21
THE CHALLENGES
• The system’s learning curve was very high.
• The key stakeholders set a high bar for acceptance.
• The actual users were few in number and were only able to
perform a few of the planned tests.
• Very high defect levels.
22
LEADING UP TO
VENDOR SELECTION
• Over 2 years of meeting with users and
stakeholders to determine business
needs.
• Included:
• JAD sessions
• Creation of “as-is” and “to-be” use
cases
• Set of acceptance criteria
(approximately 350 acceptance criteria
items)
14. 4/26/2014
12
23
THE STRATEGY
• Create test scenarios that described
the trail to follow in testing a task,
but not to the level of keystrokes.
• Based on use cases.
• The problem turned out to be that
even the BAs and trainers had
difficulty in performing the scenarios.
• System complexity was high.
• Training had not been conducted.
• Usability was low
24
DEFECT DISCOVERY
AND BACKLOG
System Test UAT 1st Deploy
10 weeks 4 weeks
750
250
15. 4/26/2014
13
25
WHAT WAS
VALIDATED
• The precise “point and click” scripts provided
by the vendor were long and difficult to
perform.
• Each one took days.
• Plus, there were errors in the scripts and
differences between what the script indicated
and what the system did.
26
THE BIG SURPRISES
• We planned the system test to be a practice run for UAT.
• It turned out to be the most productive phase of testing in
terms of finding defects.
• We planned for a 10 week UAT effort with 10 users
• It turned out to be a 2 week effort with 4 users.
• First sense of trouble: initial users were exhausted after 3
days of a pre-test effort.
16. 4/26/2014
14
27
THE BIG SURPRISES (2)
• We used none of the planned tests (around 350 scenarios)
in UAT.
• Instead, it was a guided “happy path” walkthrough, noting
problems along the way.
• Defects were found, but the majority of defects had been
found in system test.
28
LESSONS LEARNED
• The early system test was invaluable in
finding defects.
• Learning the system is critical for users in
new systems before they are able to test it.
• The test documentation is not enough to
provide context of how the system works.
• It took a lot of flexibility on the part of
everyone (client, vendor, testers, users,
stakeholders) to make it to the first
implementation.
• Sometimes actual users just aren’t able to
perform a rigorous test.
17. 4/26/2014
15
29
WHAT CAN WE LEARN FROM
ALL THESE PROJECTS?
• UAT is a much-needed test, but happens at the worst
possible time – just before implementation.
• You can take some of the late defect impact away with
system testing and reviews.
• You can lessen the risk of deployment by implementing to
a smaller and lower risk user base first.
• Actual end-users are good for performing UAT, but much
depends on what you are testing and the capabilities of
the users.
• The reality is the users are going to have to use the system
in real-life anyway.
• However, not all users are good testers!
30
WHAT CAN WE
LEARN? (2)
• Be careful how much time and effort
you invest in planning for UAT before
the capabilities are truly known.
• That is, senior management may want
actual users to test for 8 weeks, but if
the people aren’t available or can’t
handle the load, then it probably isn’t
going to happen.
• Don’t place all the weight of testing on
UAT.
• In project #4 our system testing found
a majority of the defects.
18. 4/26/2014
16
31
WHAT CAN WE
LEARN? (3)
• UAT test planning isn’t bad, just expect
changes.
• People, software, business, timelines –
they all change.
• Try to optimize and prioritize.
• Example: If you have 500 points of
acceptance criteria, can they be
validated with 200 tests?
• Which of the acceptance criteria are
critical, needed and “nice to have”?
32
19. 4/26/2014
17
33
BIO - RANDALL W. RICE
• Over 35 years experience in building and testing
information systems in a variety of industries
and technical environments
• ASTQB Certified Tester – Foundation level,
Advanced level (Full)
• Director, American Software Testing Qualification
Board (ASTQB)
• Chairperson, 1995 - 2000 QAI’’’’s annual software
testing conference
• Co-author with William E. Perry, Surviving the
Top Ten Challenges of Software Testing and
Testing Dirty Systems
• Principal Consultant and Trainer, Rice
Consulting Services, Inc.
34
CONTACT INFORMATION
Randall W. Rice, CTAL
Rice Consulting Services, Inc.
P.O. Box 892003
Oklahoma City, OK 73170
Ph: 405-691-8075
Fax: 405-691-1441
Web site: www.riceconsulting.com
e-mail: rrice@riceconsulting.com