Visual Validation - The missing tip of the automation pyramid @AgileIndia2020

The document discusses using Applitools for visual AI testing. It provides an overview of Applitools, which allows testing user interfaces through visual assertions rather than traditional code-based assertions. This avoids issues like managing locators and adjusting assertions. The document then demonstrates how to set up and implement visual AI testing using Applitools in Java, including configuring Eyes, setting up tests, and executing visual checks on elements and full pages. It concludes that while visual AI testing has advantages, a hybrid approach combining it with traditional testing may be best to avoid false positives.

I will share my experience of SDLC enablement on enterprise level. Uncover pitfalls and gotchas about building of developer friendly CI enabled service using industry standard static and dynamic scanning tools, CI platforms, ReportPortal, Carrier platform and Jira integration service.

Agile Methodology is not new. Many organisations / teams have already adopted Agile way of Software Development or are in the enablement journey for the same. What does this mean for Testing? There is no doubt that the Testing approach and mindset also needs to change to be in tune with the Agile Development methodology. Learn what does it mean to Test on Agile Projects. Also, learn how Test Automation approach needs to change for the team to be successful! Also, see the skills and capabilities required to make anyone in the Quality / Testing role add tremendous value to the team!

This document discusses the perils of using the page object pattern for test automation frameworks. It describes the page object pattern, how it can be used to model pages in code and reduce duplication. However, it also notes limitations like tests becoming too implementation focused rather than intent focused. It then introduces a business layer page object pattern that aims to address these limitations by validating business requirements and separating concerns with abstraction layers. This allows the test pyramid to remain balanced and makes maintenance and scaling easier.

Slides explaining the Agile process, and how Test Automation plays a key role in enabling teams to be successful in this fast moving process.

Slides from my talk in Selenium Conference London 2016 about "Sharing (Less) Pain with Protractor & Selenium WebDriver" See blog for more information - http://essenceoftesting.blogspot.com/2016/11/shared-relatively-less-pain-of-using.html My blog: https://essenceoftesting.blogspot.com

Slides from workshop conducted at ThoughtWorks, Pune in vodQA, on Sat, 27th August, 2016. Workshop Facilitators - Anand Bagmar, Smriti Tuteja, Pallipuspa Samal, Rohit Singhal, S Ramalingam, Shilpa G More information about vodQA, and this workshop can be found from the following links - https://essenceoftesting.blogspot.com/2016/08/vodqa-pune-less-talk-only-action-agenda.html https://essenceoftesting.blogspot.com/search/label/vodQA https://essenceoftesting.blogspot.com/search/label/waat

Here are the slides from my recent talks on "To Deploy or Not-To-Deploy - decide using TTA's Trend and Failure Analysis" More information available from my blog: http://essenceoftesting.blogspot.com/search/label/tta

Slides from my 4-hour workshop on Client-Side Performance Testing conducted at Phoenix, AZ in STPCon 2017 (March). Workshop Takeaways: Understand difference between is Performance Testing and Performance Engineering. Hand’s on experience of some open-source tools to monitor, measure and automate Client-side Performance Testing. Examples / code walk-through of some ways to automate Client-side Performance Testing. See blog for more details - https://essenceoftesting.blogspot.com/2017/03/workshop-client-side-performance.html

Learning Objectives: The most used and heard about buzz words in the Software Industry today are … IoT and Big Data! With IoT, with a creative mindset looking for opportunities and ways to add value, the possibilities are infinite. With each such opportunity, there is a huge volume of data being generated - which if analyzed and used correctly, can feed into creating more opportunities and increased value propositions. There are 2 types of analysis that one needs to think about. 1. How is the end-user interacting with the product? This will give some level of understanding into how to re-position and focus on the true value add features for the product. 2. With the huge volume of data being generated by the end-user interactions, and the data being captured by all devices in the food-chain of the offering, it is important to identify patterns from what has happened, and find out new product / value opportunities based on usage patterns. Learn what is Web Analytics, why is it important, and see some techniques how you can test it manually and and also automate that validation.

In agile software development world, we are dealing with many test tasks such as user story testing, exploratory testing, check-list based testing, regression testing, performance testing, security testing in each sprint. Besides these testing activities, one of the test types which is considerably getting crucial is visual regression testing. Visual regression testing focuses on to check visual contents and animations, page layout, and responsive design of a website/app. Because of the limits of human vision, human-based visual regression testing is generally error-prone and cumbersome. Hence, automation is inevitable. It enables us to run the tests much more precisely in a short time period. Also, it saves us a significant amount of time to deal with other manual test activities in each sprint. In this talk, we will walk through well-known open-source and commercial solutions for visual test automation. We will learn which technologies they use, what type of visual tests they are suitable for, and their major differences between each other. Besides this overview, we will also make a real-life visual test automation demo by using Selenium, ImageMagick, and AShot.

Slides from the workshop I conducted on "Client-side Performance Testing". Abstract of the workshop: In this workshop, we will see the different dimensions of Performance Testing and Performance Engineering, and focus on Client-side Performance Testing. Before we get to doing some Client-side Performance Testing activities, we will first understand how to look at client-side performance, and putting that in the context of the product under test. We will see, using a case study, the impact of caching on performance, the good & the bad! We will then experiment with some tools like WebPageTest and Page Speed to understand how to measure client-side performance. Lastly - just understanding the performance of the product is not sufficient. We will look at how to automate the testing for this activity - using WebPageTest (private instance setup), and experiment with yslow - as a low-cost, programatic alternative to WebPageTest.

Learn the why, what and how of testing with concurrency. In this talk, Mike will take attendees on an automation journey of in-house machine testing into cloud-based testing. He will show what worked, what didn’t and discuss things to improve as the machines are increasingly doing the work.

Building a Test Automation Framework is easy - there are so many resources / guides / blogs / etc. available to help you get started and help solve the issues you get along the journey. However, building a "good" Test Automation Framework is not very easy. There are a lot of principles and practices you need to use, in the right context, with a good set of skills required to make the Test Automation Framework maintainable, scalable and reusable. Design Patterns play a big role in helping achieve this goal of building a good and robust framework. In this talk, we will talk about, and see examples of various types of patterns you can use for: Build your Test Automation Framework Test Data Management Locators / IDs (for finding / interacting with elements in the browser / app) Using these patterns you will be able to build a good framework, that will help keep your tests running fast, and reliably in your CI / CD setup!

Slides from my talk in Agile India 2015, Discuss Agile 2015 on "Enabling Continuous Delivery (CD) in Enterprises with Testing"

The document discusses different strategies for testing mobile apps, including testing functionality, usability, performance, and more extensively than just functionality. It addresses testing on real devices versus emulators, the need for accessibility testing, and tools for testing areas like contrast, text-to-speech, location services and network bandwidth. The document also discusses visual testing strategies like using AI to detect visual differences and validating user interfaces.

Bhumika S, Anand Bagmar (Thoughtworks) How many times do we test the same things at multiple layers, multiple levels, adding time to the build process and testing cycle, delaying the feedback? We know what to test and how to test, but what is the right place to test it? In this workshop, we will demonstrate how we, as QA’s, can identify which tests can be classified as unit tests, integration tests, and functional tests. Using a case study, we will see how each component can be tested as part of unit testing; the integration of different parts and the functioning of a software system as a whole, and how functional tests fit into this big picture. We will then bring all these tests together to understand and build the testing pyramid and how it enables us to build the right testing framework with fewer Selenium, i.e., functional tests.

The document discusses visual validation testing as a missing piece of the automation testing pyramid. Visual testing is important but challenging as it is typically done manually, which is tedious, error-prone, and difficult to scale. Automating visual testing can help by taking screenshots of expected user interfaces and comparing them to actual screenshots. However, automating also poses challenges around false positives/negatives, maintaining baselines, and accounting for product changes. The document promotes using artificial intelligence in visual test automation to help address these challenges and advocates for including visual testing in an organization's overall quality and automation strategy.

Everyone wants to make quick releases, but the look-and-feel UX validation is a manual, slow, and error-prone activity. Any UX-related issues propping up cause huge brand-value and revenue loss, may lead to social-trolling, and, even worse, dilute your user base. This is an area where AI & ML can help. In this hands-on workshop, using examples, we will explore: the importance of automated visual validation how an AI-powered tool, Applitools Visual AI, can solve this problem. Integrate Applitools Visual AI in your Selenium-Java automation and learn by practice: The different AI algorithms various Applitools capabilities and features scale your automation using the Applitools Ultrafast Grid.

Slides from my talk on "Visual Validation - The missing tip of the automation pyramid", given at QA Symposium, Feb 2020

My thoughts presented in vodQA at ThoughtWorks Bangalore on 21st June 2014 on what is the "Future of Testing, Test Automation and the Quality Analyst" See my blog for more details: http://essenceoftesting.blogspot.in/2014/06/future-of-testing-test-automation-and.html

Slides from my talk in Selenium Conference 2016 about "Sharing the pain with Protractor & Selenium WebDriver" See blog for more information - http://essenceoftesting.blogspot.com/2016/06/sharing-pain-using-protractor.html My blog: http://essenceoftesting.blogspot.com

In a team which is developing mobile apps for multiple platforms, it becomes extremely difficult to maintain multiple Automation codebases and frameworks to test the app. This becomes a bigger problem when the same team is churning out multiple flavours of the same app. What this showcases is how we have built an automation framework that tests 2 different flavours of an app along with using the same framework to test these apps across 3 different platforms. The session will also include the challenges we faced while moving from a purely Functional approach to writing code to an Object-oriented approach to structure our automation code. This will help other testing practitioners attending this session in building a robust automation framework with a very reliable and scalable codebase that can be extended beyond just one application/platform.

This document discusses and recommends various UX design tools for improving productivity. It covers tools for usability testing and user research like Silverback App, Morae, and Solidify App. It also discusses mobile usability testing tools like UX Recorder and LookBack. Other sections cover accessibility testing tools like Wave and AChecker. Tools for interdepartmental collaboration that are highlighted include TargetProcess, Balsamiq, Asana, and InVisionApp. The document provides the purposes and benefits of each tool.

Manual functional testing is a slow, tedious, and error prone process. As we continue to incrementally build software, the corresponding regression test suite continues to grow. Rarely is time allotted to consolidate and keep these test cases in sync with the product under development. If these test cases are used as the basis for automation, the resulting suite is composed of very granular tests that are often quite brittle in nature. Using a case study, Anand Bagmar describes how behavior-driven testing (BDT) can be applied to identify the right type of test cases for manual and automated regression testing. Learn how the BDT technique can be applied in your context and domain, regardless of the tools and technologies used in your project and organization.

Slides from my talk on "Techniques to Eradicate Flaky Tests" at Testing Talks - The Reunion, Melbourne, Australia on 20th Oct 2022

During this webinar, Anand Bagmar demonstrates how AI tools such as ChatGPT can be applied to various stages of the software development life cycle (SDLC) using an eCommerce application case study. Find the on-demand recording and more info at https://applitools.info/b59 Key takeaways: • Learn how to use ChatGPT to add AI power to your testing and test automation • Understand the limitations of the technology and where human expertise is crucial • Gain insight into different AI-based tools • Adopt AI-based tools to stay relevant and optimize work for developers and testers * ChatGPT and OpenAI belong to OpenAI, L.L.C.

The document discusses expanding a test portfolio with Coded UI tests and cloud-based load testing in Visual Studio. It covers automating tests with Microsoft Test Manager, creating automated UI tests using Coded UI, and performing load testing on applications hosted in the cloud using Visual Studio Online. Conducting load testing early and customizing tests to measure key performance metrics can help ensure applications meet expectations before public release.

My talk at CodeFest 2017 in Novosibirsk, Russia. I talk about the benefits of adding a app crawler to your build process. In todays Agile world it's becoming difficult to keep up with the amount of manual and exploratory testing with shorter and shorter sprint iterations. It's time to put machines to work and help take some of the load off of us!

This document summarizes Vishal Banthia's presentation on achieving reliable mobile test automation. It discusses the difficulties of flaky tests and inconsistent test execution environments. It then presents solutions like capturing comprehensive test reports with screenshots, videos, and logs to better diagnose flaky tests. It also recommends leveraging cloud services for scalable and accessible test execution. The presentation demonstrates these concepts through tools like Bitrise, OpenSTF, and CircleCI used in Mercari's mobile test automation pipeline.

This document discusses how to implement automated performance testing using Maven and JMeter. It describes configuring Maven to run JMeter tests as part of a continuous integration build cycle. Running performance tests automatically allows issues to be detected quickly before code is deployed. The process involves packaging JMeter tests with Maven, configuring Maven profiles and plugins to run tests across different environments and datasets, and using tools like Chronos and Bamboo for reporting and comparing performance over time.

This document introduces and summarizes Django, an open-source web framework written in Python. It highlights key features of Django, including its automated administration interface, object-relational mapper (ORM), generic views, forms, URL configuration, templates, internationalization support, and built-in user authentication. The document also discusses how Django aims to minimize the time it takes to develop software through automation, reusable apps, and other features. Examples are provided of large sites built with Django to demonstrate its performance and scalability.

https://confengine.com/conferences/selenium-conf-2022/proposal/17141/changing-tyres-in-a-moving-car-making-functional-test-automation-effective In my experience, I have seen teams struggle with more than one, and in many cases, all of the above statements. On retrospection, I have realized, that most of these challenges are because of a combination of the following issues: Lack of holistic vision, understanding, skills, and capabilities for taking on this seemingly easy activity of functional test automation Lack of time for design, implementation, and maintenance of the automated test code In this session, I will share, with examples, the features & capabilities that are not used right in Test Automation implementation resulting in you answering “NO” to the above statements. These are anti-patterns of functional automation implementation and have a negative impact on the quality of feedback from your automated tests. More importantly, I will share approaches and solutions of how you can avoid these anti-patterns and evolve to answer “YES” to each of the above statements.

This document discusses automating visual software testing. It begins by explaining that visual testing is used to verify a graphical user interface appears correctly. It then discusses why visual testing should be automated, such as because the test matrix is too large to cover manually across different browsers, devices, resolutions, etc. It also notes that mobile app quality is critical so testing needs to be thorough. The document outlines how visual testing works by taking screenshots, comparing them to baselines, and reporting differences. It provides examples of tools that can be used for visual testing and discusses how tests can be integrated into development pipelines from the code level through production.

My talk is up: https://vimeo.com/179151225 Testing is not Dead => Confidently shipping to production Presented at Sydney Cococaheads, July 2016 1. Type Safety 2. Unit Tests 3. Snapshot 4. Stubbed UI Test 5. Integration Test 6. AAA Shakedown 7. Dogfood 8. Blitz

** Full webinar recording: ** It is no secret that many teams struggle with automated functional UI testing - some to the point where it is completely abandoned - even though the UI is the most significant interface of the system. In this session, Adam Carmi -- Applitools CTO and Co-founder -- reviewed the main weaknesses of traditional approaches to UI testing, and how they negatively affect test stability, maintainability, coverage, execution speed, and overall ROI. He also discussed how these weaknesses become even more severe when running tests across multiple devices and browsers. Adam demonstrated how Visual AI -- the innovative technology powering Applitools' testing engine -- can be applied on your existing pipeline to efficiently implement functional UI tests with a fraction of the effort while drastically increasing test coverage and reducing test execution time. Adam also showed a live coding session, where he converted a traditional UI test into a Visual AI-based test in minutes, and executed it across dozens of devices and browsers in seconds using the Applitools Ultrafast Grid.