This document provides an agenda and overview for a performance testing training with JMeter. It begins with an introduction to performance testing, including the purpose and types of performance testing. It then covers getting started with JMeter, including installation, setup, and running JMeter. The remainder of the document outlines the content to be covered, including building test plans with JMeter, load and performance testing of websites, parameterization, adding assertions, and best practices. The goal is to teach participants how to use JMeter to perform various types of performance testing of applications and websites.
Load testing involves systematically stressing a system or application to determine its behavior and stability under different load conditions. There are different types of load tests that can be run depending on the test goals. It is important to measure key metrics like response times, failures, and system resource usage during a load test to understand the system's performance limits and how it degrades as load increases. Load test results should indicate the maximum number of users the system can support while meeting performance requirements as well as insights into how the system will perform as usage grows over time.
This document discusses performance testing and tools for performance testing like JMeter. It provides details on how JMeter can be used to perform load testing and performance testing on different protocols. It outlines the steps to build test plans to test the performance of web applications and databases using JMeter including adding users, requests, listeners and more.
The document discusses gathering requirements for performance testing an application. It lists questions to ask about the application type and architecture, test environment, workload model, and performance goals. Key information needs include the application technology, database and server used, network details, protocols, user sessions and load over time, and goals for response times and system utilization under load. The requirements gathered will help determine the appropriate performance tests and pass/fail criteria.
The document discusses performance testing. It defines performance testing as determining how fast and stable a system is. It outlines why performance testing is important to identify problems early, prevent revenue and credibility loss from poor performance, and ensure systems meet expectations. It also discusses various performance testing terms, metrics, processes, limitations of manual testing, and benefits of automation using tools like LoadRunner and JMeter.
Load testing is performed using tools like JMeter to determine how a system performs under normal and peak load conditions. JMeter is an open source load testing tool that can simulate many users accessing a web application concurrently. It allows users to record tests from a browser, parameterize tests using variables and CSV files, add logic and functions, and analyze results. While it has limitations like not supporting all embedded content and being limited by a single computer's network connection, JMeter is a powerful free load testing option supported on many platforms.
Performance testing validates an application's responsiveness, stability, and other quality attributes under various workloads. It involves load testing, stress testing, endurance testing, spike testing, volume testing, availability testing, and scalability testing. The key parameters analyzed are response time, throughput, and memory utilization. Performance testing helps determine an application's speed, scalability, stability, and ability to handle changes in load and traffic over time.
The document discusses performance testing using Apache JMeter. It covers topics like an overview of performance testing, the purpose of performance testing, key types of performance testing like load testing and stress testing. It also discusses pre-requisites of performance testing, the performance testing life cycle, challenges of performance testing and how to record and playback tests using JMeter.
Performance Testing And Its Type | Benefits Of Performance Testing
Performance testing is in general, a testing practice performed to determine how a system performs in terms of responsiveness and stability under a particular workload. It can also serve to investigate, measure, validate or verify other quality attributes of the system, such as scalability, reliability and resource usage.
This document discusses performance testing with JMeter. It provides an introduction to performance testing and why it is important. It describes the tools needed for a performance testing system, including virtual user simulators, system monitoring tools, and application monitoring tools. It then focuses on using JMeter for performance testing, explaining why JMeter is chosen and some basic parameters to consider like response time and throughput.
The document summarizes the results of performance testing on a system. It provides throughput and scalability numbers from tests, graphs of metrics, and recommendations for developers to improve performance based on issues identified. The performance testing process and approach are also outlined. The resultant deliverable is a performance and scalability document containing the test results but not intended as a formal system sizing guide.
Load Testing Best Practices: Application complexity is increasing, yet the stringent requirements for web performance is increasing exponentially. Learn more about the three major types of load testing, determine which you need and how to conduct them.
This document outlines a performance test plan for Sakai 2.5.0. It describes the objectives, approach, test types, metrics, goals, tools, and data preparation. The objectives are to validate Sakai meets minimum performance standards and test any new or changed tools. Tests include capacity, consistent load, and single function stress tests. Metrics like response time, CPU utilization, and errors will be measured. Goals include average response time under 2.5s and max under 30s, CPU under 75%, and 500 concurrent users supported. Silk Performer will be used to run tests against a Sakai/Tomcat/Oracle environment. Over 92,000 students and 1,557 instructors of data will be preloaded
The document summarizes a training session on performance testing using LoadRunner. It discusses planning load tests, the components of LoadRunner, creating scripts and scenarios, and enhancing scripts. Key points covered include the purpose of different types of tests, goals for performance testing, the workflow of a load test using LoadRunner, and developing scripts using Virtual User Generator.
This document discusses performance testing and provides information on several related topics:
- It defines performance, load, and stress testing and explains their differences.
- It outlines why performance testing is important, when it should be conducted, and what aspects of a system should be tested.
- The performance testing process is described as involving planning, creating test scenarios and scripts, running tests, monitoring tests, and analyzing results.
- Automated performance testing is presented as more effective than manual testing due to issues with resources, coordination, and repeatability when using human testers.
JMeter is a free and open source desktop application used to load test and performance test web services, data bases, and other applications. It provides a GUI interface and can also be run in non-GUI mode via command line. A JMeter test plan contains thread groups, samplers, listeners, timers, and other elements to simulate load on a system and measure performance. JMeter scripts can be recorded by configuring JMeter as a proxy server or imported from other recording tools. Running JMeter tests helps identify bottlenecks and ensure systems can handle expected loads.
Performance testing with JMeter provides an introduction to key concepts and how to implement performance tests using JMeter. Some important steps include designing test plans, preparing the environment, determining metrics and goals, notifying stakeholders, and using JMeter elements like thread groups, samplers, listeners, assertions and configuration elements to simulate load and measure performance. JMeter is an open source tool that can run in GUI or non-GUI mode for load testing web applications and determining maximum operating capacity and bottlenecks under heavy loads.
July webinar l How to Handle the Holiday Retail Rush with Agile Performance T...
In this Q&A-style webinar, you'll learn:
1. How and why to load test at least three months prior to the holidays
2. How to integrate CI/CD into your holiday load testing
3. How to determine and evaluate load curves
This presentation includes:
- Why performance matters for digital businesses?
- Use Cases for performance / load testing
- Load Test Design Considerations
- Tools and Technologies
- Methodology and Approach
- Activities and Deliverables
- Load Testing Success Stories
Performance testing is done to determine a system's responsiveness under different loads. It aims to optimize user experience. Types of performance testing include load, stress, soak/endurance, volume, scalability, and spike testing. The goals are to assess production readiness, compare platforms, evaluate configurations, and check against criteria. Pre-requisites include a stable test environment similar to production. The testing process involves establishing baselines and benchmarks, running tests, and analyzing results to identify bottlenecks and decide on fixes. Common issues relate to servers, databases, networks, and applications. Optimization involves improvements, upgrades, and tuning. Challenges include setting up the test environment and analyzing large amounts of test data.
The document discusses performance testing and provides details about:
1) The objectives of performance testing including validating requirements, checking capacity, and identifying issues.
2) The differences between performance, load, and stress testing.
3) Why performance testing is important including checking scalability, stability, availability, and gaining confidence.
4) Parameters to consider in performance testing like throughput, latency, efficiency, and degradation.
5) Potential sources of performance bottlenecks like the network, web server, application server, and database server.
Applying a Methodical Approach to Website Performance
When addressing website performance issues, developers typically jump to conclusions, focusing on the perceived causes rather than uncovering the real causes through research.
Mitchel Sellers will show you how to approach website performance issues with a level of consistency that ensures they're properly identified and resolved so you'll avoid jumping to conclusions in the future.
You can watch the webinar recording here:
https://www.postsharp.net/documentation/video?id=190066128
Holiday Readiness: Best Practices for Successful Holiday Readiness Testing
Best Practices for Successful Holiday Readiness Testing: Are you already thinking of, and planning for Black Friday? Learn which load tests to use and why to load test early and often so that you are prepared for the holidays.
[Webinar] Getting Started with Performance Testing
The document discusses the importance of performance testing applications. It notes that 47% of consumers expect a page to load within 2 seconds and 40% will abandon a site that takes over 3 seconds to load. Performance testing is important when high loads are expected, large data volumes are involved, or applications feel sluggish. An example performance testing project is described that aims to support 4000 users with responses under 5 seconds and conduct stress testing. Key aspects of performance testing like load testing, tools, test planning, analysis and best practices are also covered.
1) Traditional load testing is limited in its ability to accurately measure end-user experience and identify issues with third-party components.
2) Load testing 2.0 uses real user testing from geographically distributed locations to more realistically drive large volumes of load and uncover regional response time discrepancies and external errors.
3) An online retailer used load testing 2.0 to identify that a third-party component was insufficient under load, affecting the performance of their overall application.
Are You Ready For More Visitors Cognizant Gomez Jan20
1) Traditional load testing is limited in its ability to accurately measure end-user experience and identify issues with third-party components.
2) Load testing 2.0 uses real user testing from geographically distributed locations to better understand regional response times and external factors that impact performance.
3) A case study showed that load testing 2.0 uncovered poor response times for key revenue regions that traditional load testing failed to detect.
Best Practices In Load And Stress Testing Cmg Seminar[1]
The document discusses best practices for performance testing. It provides an overview of the typical performance testing process, including defining goals, planning tests, scripting tests, executing tests, analyzing results, and delivering findings. It also discusses considerations for choosing testing tools and resources as well as common pitfalls to avoid, such as not testing, poor planning, relying on customers to find issues, using the wrong tools, and failing to properly isolate variables.
Architecting and Tuning IIB/eXtreme Scale for Maximum Performance and Reliabi...
Abstract: Recent projects have stressed the "need for speed" while handling large amounts of data, with near zero downtime. An analysis of multiple environments has identified optimizations and architectures that improve both performance and reliability. The session covers data gathering and analysis, discussing everything from the network (multiple NICs, nearby catalogs, high speed Ethernet), to the latest features of extreme scale. Performance analysis helps pinpoint where time is spent (bottlenecks) and we discuss optimization techniques (MQ tuning, IIB performance best practices) as well as helpful IBM support pacs. Log Analysis pinpoints system stress points (e.g. CPU starvation) and steps on the path to near zero downtime.
This document discusses the importance of performance testing cloud applications and outlines best practices for defining performance requirements, testing methodology, and identifying issues. It provides examples of performance problems found in databases, applications, operating systems, and networks. The key goals of performance testing are to understand system behavior under load, find bottlenecks and hidden bugs, and verify that requirements are met.
Performance testing is one of the kinds of Non-Functional Testing. Building any successful product hinges on its performance. User experience is the deciding unit of fruitful application and Performance testing helps to reach there. You will learn the key concept of performance testing, how the IT industry gets benefitted, what are the different types of Performance Testing, their lifecycle, and much more.
This document provides information about the SCQAA-SF organization and an upcoming event. SCQAA-SF is a chapter that sponsors sharing of information to promote quality practices in IT through networking, training and professional development. They meet every two months in San Fernando Valley. The upcoming event will feature presentations on technology advancements and methodology, networking opportunities, and opportunities to earn PDU and other credits. Recently, the organization revised their membership dues policy to better accommodate members' needs.
Break data silos with real-time connectivity using Confluent Cloud Connectors
Connectors integrate Apache Kafka® with external data systems, enabling you to move away from a brittle spaghetti architecture to one that is more streamlined, secure, and future-proof. However, if your team still spends multiple dev cycles building and managing connectors using just open source Kafka Connect, it’s time to consider a faster and cost-effective alternative.
Seamless PostgreSQL to Snowflake Data Transfer in 8 Simple Steps
Unlock the full potential of your data by effortlessly migrating from PostgreSQL to Snowflake, the leading cloud data warehouse. This comprehensive guide presents an easy-to-follow 8-step process using Estuary Flow, an open-source data operations platform designed to simplify data pipelines.
Discover how to seamlessly transfer your PostgreSQL data to Snowflake, leveraging Estuary Flow's intuitive interface and powerful real-time replication capabilities. Harness the power of both platforms to create a robust data ecosystem that drives business intelligence, analytics, and data-driven decision-making.
Key Takeaways:
1. Effortless Migration: Learn how to migrate your PostgreSQL data to Snowflake in 8 simple steps, even with limited technical expertise.
2. Real-Time Insights: Achieve near-instantaneous data syncing for up-to-the-minute analytics and reporting.
3. Cost-Effective Solution: Lower your total cost of ownership (TCO) with Estuary Flow's efficient and scalable architecture.
4. Seamless Integration: Combine the strengths of PostgreSQL's transactional power with Snowflake's cloud-native scalability and data warehousing features.
Don't miss out on this opportunity to unlock the full potential of your data. Read & Download this comprehensive guide now and embark on a seamless data journey from PostgreSQL to Snowflake with Estuary Flow!
Try it Free: https://dashboard.estuary.dev/register
A Comparative Analysis of Functional and Non-Functional Testing.pdf
A robust software testing strategy encompassing functional and non-functional testing is fundamental for development teams. These twin pillars are essential for ensuring the success of your applications. But why are they so critical?
Functional testing rigorously examines the application's processes against predefined requirements, ensuring they align seamlessly. Conversely, non-functional testing evaluates performance and reliability under load, enhancing the end-user experience.
NBFC Software: Optimize Your Non-Banking Financial Company
NBFC Software: Optimize Your Non-Banking Financial Company
Enhance Your Financial Services with Comprehensive NBFC Software
NBFC software provides a complete solution for non-banking financial companies, streamlining banking and accounting functions to reduce operational costs. Our software is designed to meet the diverse needs of NBFCs, including investment banks, insurance companies, and hedge funds.
Key Features of NBFC Software:
Centralized Database: Facilitates inter-branch collaboration and smooth operations with a unified platform.
Automation: Simplifies loan lifecycle management and account maintenance, ensuring efficient delivery of financial services.
Customization: Highly customizable to fit specific business needs, offering flexibility in managing various loan types such as home loans, mortgage loans, personal loans, and more.
Security: Ensures safe and secure handling of financial transactions and sensitive data.
User-Friendly Interface: Designed to be intuitive and easy to use, reducing the learning curve for employees.
Cost-Effective: Reduces the need for additional manpower by automating tasks, making it a budget-friendly solution. Benefits of NBFC Software:
Go Paperless: Transition to a fully digital operation, eliminating offline work.
Transparency: Enables managers and executives to monitor various points of the banking process easily.
Defaulter Tracking: Helps track loan defaulters, maintaining a healthy loan management system.
Increased Accessibility: Cutting-edge technology increases the accessibility and usability of NBFC operations. Request a Demo Now!
React vs Next js: Which is Better for Web Development? - Semiosis Software Pr...
React and Next.js are complementary tools in web development. React, a JavaScript library, specializes in building user interfaces with its component-based architecture and efficient state management. Next.js extends React by providing server-side rendering, routing, and other utilities, making it ideal for building SEO-friendly, high-performance web applications.
Ansys Mechanical enables you to solve complex structural engineering problems and make better, faster design decisions. With the finite element analysis (FEA) solvers available in the suite, you can customize and automate solutions for your structural mechanics problems and parameterize them to analyze multiple design scenarios. Ansys Mechanical is a dynamic tool that has a complete range of analysis tools.
COMPSAC 2024 D&I Panel: Charting a Course for Equity: Strategies for Overcomi...
Hironori Washizaki, "Charting a Course for Equity: Strategies for Overcoming Challenges and Promoting Inclusion in the Metaverse", IEEE COMPSAC 2024 D&I Panel, 2024.
Explore the rapid development journey of TryBoxLang, completed in just 48 hours. This session delves into the innovative process behind creating TryBoxLang, a platform designed to showcase the capabilities of BoxLang by Ortus Solutions. Discover the challenges, strategies, and outcomes of this accelerated development effort, highlighting how TryBoxLang provides a practical introduction to BoxLang's features and benefits.
What is OCR Technology and How to Extract Text from Any Image for Free
Discover the fascinating world of Optical Character Recognition (OCR) technology with our comprehensive presentation. Learn how OCR converts various types of documents, such as scanned paper documents, PDFs, or images captured by a digital camera, into editable and searchable data. Dive into the history, modern applications, and future trends of OCR technology. Get step-by-step instructions on how to extract text from any image online for free using a simple tool, along with best practices for OCR image preparation. Ideal for professionals, students, and tech enthusiasts looking to harness the power of OCR.
Software development... for all? (keynote at ICSOFT'2024)
Our world runs on software. It governs all major aspects of our life. It is an enabler for research and innovation, and is critical for business competitivity. Traditional software engineering techniques have achieved high effectiveness, but still may fall short on delivering software at the accelerated pace and with the increasing quality that future scenarios will require.
To attack this issue, some software paradigms raise the automation of software development via higher levels of abstraction through domain-specific languages (e.g., in model-driven engineering) and empowering non-professional developers with the possibility to build their own software (e.g., in low-code development approaches). In a software-demanding world, this is an attractive possibility, and perhaps -- paraphrasing Andy Warhol -- "in the future, everyone will be a developer for 15 minutes". However, to make this possible, methods are required to tweak languages to their context of use (crucial given the diversity of backgrounds and purposes), and the assistance to developers throughout the development process (especially critical for non-professionals).
In this keynote talk at ICSOFT'2024 I presented enabling techniques for this vision, supporting the creation of families of domain-specific languages, their adaptation to the usage context; and the augmentation of low-code environments with assistants and recommender systems to guide developers (professional or not) in the development process.
CViewSurvey Digitech Pvt Ltd that works on a proven C.A.A.G. model.
CViewSurvey is a SaaS-based Web & Mobile application that provides digital transformation to traditional paper surveys and feedback for customer & employee experience, field & market research that helps you evaluate your customer's as well as employee's loyalty.
With our unique C.A.A.G. Collect, Analysis, Act & Grow approach; business & industry’s can create customized surveys on web, publish on app to collect unlimited response & review AI backed real-time data analytics on mobile & tablets anytime, anywhere. Data collected when offline is securely stored in the device, which syncs to the cloud server when connected to any network.
- JMeter is an open source load testing tool that can test web applications and other services. It uses virtual users to simulate real user load on a system.
- JMeter tests are prepared by recording HTTP requests using a proxy server. Tests are organized into thread groups and loops to simulate different user behaviors and loads.
- Tests can be made generic by using variables and default values so the same tests can be run against different environments. Assertions are added to validate responses.
- Tests are run in non-GUI mode for load testing and can be distributed across multiple machines for high user loads. Test results are analyzed using aggregated graphs and result trees.
The document discusses performance testing, including its goals, importance, types, prerequisites, management approaches, testing cycle, activities, common issues, typical fixes, challenges, and best practices. The key types of performance testing are load, stress, soak/endurance, volume/spike, scalability, and configuration testing. Performance testing aims to assess production readiness, compare platforms/configurations, evaluate against criteria, and discover poor performance. It is important for meeting user expectations and avoiding lost revenue.
In this presentation which was delivered to testers in Manchester, I help would-be performance testers to get started in performance testing. Drawing on my experiences as a performance tester and test manager, I explain the principles of performance testing and highlight some of the pitfalls.
This document provides an agenda and overview for a performance testing training with JMeter. It begins with an introduction to performance testing, including the purpose and types of performance testing. It then covers getting started with JMeter, including installation, setup, and running JMeter. The remainder of the document outlines the content to be covered, including building test plans with JMeter, load and performance testing of websites, parameterization, adding assertions, and best practices. The goal is to teach participants how to use JMeter to perform various types of performance testing of applications and websites.
Load testing involves systematically stressing a system or application to determine its behavior and stability under different load conditions. There are different types of load tests that can be run depending on the test goals. It is important to measure key metrics like response times, failures, and system resource usage during a load test to understand the system's performance limits and how it degrades as load increases. Load test results should indicate the maximum number of users the system can support while meeting performance requirements as well as insights into how the system will perform as usage grows over time.
This document discusses performance testing and tools for performance testing like JMeter. It provides details on how JMeter can be used to perform load testing and performance testing on different protocols. It outlines the steps to build test plans to test the performance of web applications and databases using JMeter including adding users, requests, listeners and more.
The document discusses gathering requirements for performance testing an application. It lists questions to ask about the application type and architecture, test environment, workload model, and performance goals. Key information needs include the application technology, database and server used, network details, protocols, user sessions and load over time, and goals for response times and system utilization under load. The requirements gathered will help determine the appropriate performance tests and pass/fail criteria.
The document discusses performance testing. It defines performance testing as determining how fast and stable a system is. It outlines why performance testing is important to identify problems early, prevent revenue and credibility loss from poor performance, and ensure systems meet expectations. It also discusses various performance testing terms, metrics, processes, limitations of manual testing, and benefits of automation using tools like LoadRunner and JMeter.
Load testing is performed using tools like JMeter to determine how a system performs under normal and peak load conditions. JMeter is an open source load testing tool that can simulate many users accessing a web application concurrently. It allows users to record tests from a browser, parameterize tests using variables and CSV files, add logic and functions, and analyze results. While it has limitations like not supporting all embedded content and being limited by a single computer's network connection, JMeter is a powerful free load testing option supported on many platforms.
Performance testing validates an application's responsiveness, stability, and other quality attributes under various workloads. It involves load testing, stress testing, endurance testing, spike testing, volume testing, availability testing, and scalability testing. The key parameters analyzed are response time, throughput, and memory utilization. Performance testing helps determine an application's speed, scalability, stability, and ability to handle changes in load and traffic over time.
The document discusses performance testing using Apache JMeter. It covers topics like an overview of performance testing, the purpose of performance testing, key types of performance testing like load testing and stress testing. It also discusses pre-requisites of performance testing, the performance testing life cycle, challenges of performance testing and how to record and playback tests using JMeter.
Performance Testing And Its Type | Benefits Of Performance TestingKostCare
Performance testing is in general, a testing practice performed to determine how a system performs in terms of responsiveness and stability under a particular workload. It can also serve to investigate, measure, validate or verify other quality attributes of the system, such as scalability, reliability and resource usage.
Performance testing and reporting with JMeterjvSlideshare
This document discusses performance testing with JMeter. It provides an introduction to performance testing and why it is important. It describes the tools needed for a performance testing system, including virtual user simulators, system monitoring tools, and application monitoring tools. It then focuses on using JMeter for performance testing, explaining why JMeter is chosen and some basic parameters to consider like response time and throughput.
The document summarizes the results of performance testing on a system. It provides throughput and scalability numbers from tests, graphs of metrics, and recommendations for developers to improve performance based on issues identified. The performance testing process and approach are also outlined. The resultant deliverable is a performance and scalability document containing the test results but not intended as a formal system sizing guide.
Load Testing Best Practices: Application complexity is increasing, yet the stringent requirements for web performance is increasing exponentially. Learn more about the three major types of load testing, determine which you need and how to conduct them.
This document outlines a performance test plan for Sakai 2.5.0. It describes the objectives, approach, test types, metrics, goals, tools, and data preparation. The objectives are to validate Sakai meets minimum performance standards and test any new or changed tools. Tests include capacity, consistent load, and single function stress tests. Metrics like response time, CPU utilization, and errors will be measured. Goals include average response time under 2.5s and max under 30s, CPU under 75%, and 500 concurrent users supported. Silk Performer will be used to run tests against a Sakai/Tomcat/Oracle environment. Over 92,000 students and 1,557 instructors of data will be preloaded
The document summarizes a training session on performance testing using LoadRunner. It discusses planning load tests, the components of LoadRunner, creating scripts and scenarios, and enhancing scripts. Key points covered include the purpose of different types of tests, goals for performance testing, the workflow of a load test using LoadRunner, and developing scripts using Virtual User Generator.
This document discusses performance testing and provides information on several related topics:
- It defines performance, load, and stress testing and explains their differences.
- It outlines why performance testing is important, when it should be conducted, and what aspects of a system should be tested.
- The performance testing process is described as involving planning, creating test scenarios and scripts, running tests, monitoring tests, and analyzing results.
- Automated performance testing is presented as more effective than manual testing due to issues with resources, coordination, and repeatability when using human testers.
JMeter is a free and open source desktop application used to load test and performance test web services, data bases, and other applications. It provides a GUI interface and can also be run in non-GUI mode via command line. A JMeter test plan contains thread groups, samplers, listeners, timers, and other elements to simulate load on a system and measure performance. JMeter scripts can be recorded by configuring JMeter as a proxy server or imported from other recording tools. Running JMeter tests helps identify bottlenecks and ensure systems can handle expected loads.
Performance testing with JMeter provides an introduction to key concepts and how to implement performance tests using JMeter. Some important steps include designing test plans, preparing the environment, determining metrics and goals, notifying stakeholders, and using JMeter elements like thread groups, samplers, listeners, assertions and configuration elements to simulate load and measure performance. JMeter is an open source tool that can run in GUI or non-GUI mode for load testing web applications and determining maximum operating capacity and bottlenecks under heavy loads.
July webinar l How to Handle the Holiday Retail Rush with Agile Performance T...Apica
In this Q&A-style webinar, you'll learn:
1. How and why to load test at least three months prior to the holidays
2. How to integrate CI/CD into your holiday load testing
3. How to determine and evaluate load curves
This presentation includes:
- Why performance matters for digital businesses?
- Use Cases for performance / load testing
- Load Test Design Considerations
- Tools and Technologies
- Methodology and Approach
- Activities and Deliverables
- Load Testing Success Stories
Performance testing is done to determine a system's responsiveness under different loads. It aims to optimize user experience. Types of performance testing include load, stress, soak/endurance, volume, scalability, and spike testing. The goals are to assess production readiness, compare platforms, evaluate configurations, and check against criteria. Pre-requisites include a stable test environment similar to production. The testing process involves establishing baselines and benchmarks, running tests, and analyzing results to identify bottlenecks and decide on fixes. Common issues relate to servers, databases, networks, and applications. Optimization involves improvements, upgrades, and tuning. Challenges include setting up the test environment and analyzing large amounts of test data.
The document discusses performance testing and provides details about:
1) The objectives of performance testing including validating requirements, checking capacity, and identifying issues.
2) The differences between performance, load, and stress testing.
3) Why performance testing is important including checking scalability, stability, availability, and gaining confidence.
4) Parameters to consider in performance testing like throughput, latency, efficiency, and degradation.
5) Potential sources of performance bottlenecks like the network, web server, application server, and database server.
When addressing website performance issues, developers typically jump to conclusions, focusing on the perceived causes rather than uncovering the real causes through research.
Mitchel Sellers will show you how to approach website performance issues with a level of consistency that ensures they're properly identified and resolved so you'll avoid jumping to conclusions in the future.
You can watch the webinar recording here:
https://www.postsharp.net/documentation/video?id=190066128
Holiday Readiness: Best Practices for Successful Holiday Readiness TestingApica
Best Practices for Successful Holiday Readiness Testing: Are you already thinking of, and planning for Black Friday? Learn which load tests to use and why to load test early and often so that you are prepared for the holidays.
[Webinar] Getting Started with Performance TestingMaharshi Shah
The document discusses the importance of performance testing applications. It notes that 47% of consumers expect a page to load within 2 seconds and 40% will abandon a site that takes over 3 seconds to load. Performance testing is important when high loads are expected, large data volumes are involved, or applications feel sluggish. An example performance testing project is described that aims to support 4000 users with responses under 5 seconds and conduct stress testing. Key aspects of performance testing like load testing, tools, test planning, analysis and best practices are also covered.
1) Traditional load testing is limited in its ability to accurately measure end-user experience and identify issues with third-party components.
2) Load testing 2.0 uses real user testing from geographically distributed locations to more realistically drive large volumes of load and uncover regional response time discrepancies and external errors.
3) An online retailer used load testing 2.0 to identify that a third-party component was insufficient under load, affecting the performance of their overall application.
Are You Ready For More Visitors Cognizant Gomez Jan20Compuware APM
1) Traditional load testing is limited in its ability to accurately measure end-user experience and identify issues with third-party components.
2) Load testing 2.0 uses real user testing from geographically distributed locations to better understand regional response times and external factors that impact performance.
3) A case study showed that load testing 2.0 uncovered poor response times for key revenue regions that traditional load testing failed to detect.
Best Practices In Load And Stress Testing Cmg Seminar[1]Munirathnam Naidu
The document discusses best practices for performance testing. It provides an overview of the typical performance testing process, including defining goals, planning tests, scripting tests, executing tests, analyzing results, and delivering findings. It also discusses considerations for choosing testing tools and resources as well as common pitfalls to avoid, such as not testing, poor planning, relying on customers to find issues, using the wrong tools, and failing to properly isolate variables.
Architecting and Tuning IIB/eXtreme Scale for Maximum Performance and Reliabi...Prolifics
Abstract: Recent projects have stressed the "need for speed" while handling large amounts of data, with near zero downtime. An analysis of multiple environments has identified optimizations and architectures that improve both performance and reliability. The session covers data gathering and analysis, discussing everything from the network (multiple NICs, nearby catalogs, high speed Ethernet), to the latest features of extreme scale. Performance analysis helps pinpoint where time is spent (bottlenecks) and we discuss optimization techniques (MQ tuning, IIB performance best practices) as well as helpful IBM support pacs. Log Analysis pinpoints system stress points (e.g. CPU starvation) and steps on the path to near zero downtime.
Adding Value in the Cloud with Performance TestRodolfo Kohn
This document discusses the importance of performance testing cloud applications and outlines best practices for defining performance requirements, testing methodology, and identifying issues. It provides examples of performance problems found in databases, applications, operating systems, and networks. The key goals of performance testing are to understand system behavior under load, find bottlenecks and hidden bugs, and verify that requirements are met.
Performance testing is one of the kinds of Non-Functional Testing. Building any successful product hinges on its performance. User experience is the deciding unit of fruitful application and Performance testing helps to reach there. You will learn the key concept of performance testing, how the IT industry gets benefitted, what are the different types of Performance Testing, their lifecycle, and much more.
This document provides information about the SCQAA-SF organization and an upcoming event. SCQAA-SF is a chapter that sponsors sharing of information to promote quality practices in IT through networking, training and professional development. They meet every two months in San Fernando Valley. The upcoming event will feature presentations on technology advancements and methodology, networking opportunities, and opportunities to earn PDU and other credits. Recently, the organization revised their membership dues policy to better accommodate members' needs.
Similar to Performance Testing from Scratch + JMeter intro (20)
Break data silos with real-time connectivity using Confluent Cloud Connectorsconfluent
Connectors integrate Apache Kafka® with external data systems, enabling you to move away from a brittle spaghetti architecture to one that is more streamlined, secure, and future-proof. However, if your team still spends multiple dev cycles building and managing connectors using just open source Kafka Connect, it’s time to consider a faster and cost-effective alternative.
Seamless PostgreSQL to Snowflake Data Transfer in 8 Simple StepsEstuary Flow
Unlock the full potential of your data by effortlessly migrating from PostgreSQL to Snowflake, the leading cloud data warehouse. This comprehensive guide presents an easy-to-follow 8-step process using Estuary Flow, an open-source data operations platform designed to simplify data pipelines.
Discover how to seamlessly transfer your PostgreSQL data to Snowflake, leveraging Estuary Flow's intuitive interface and powerful real-time replication capabilities. Harness the power of both platforms to create a robust data ecosystem that drives business intelligence, analytics, and data-driven decision-making.
Key Takeaways:
1. Effortless Migration: Learn how to migrate your PostgreSQL data to Snowflake in 8 simple steps, even with limited technical expertise.
2. Real-Time Insights: Achieve near-instantaneous data syncing for up-to-the-minute analytics and reporting.
3. Cost-Effective Solution: Lower your total cost of ownership (TCO) with Estuary Flow's efficient and scalable architecture.
4. Seamless Integration: Combine the strengths of PostgreSQL's transactional power with Snowflake's cloud-native scalability and data warehousing features.
Don't miss out on this opportunity to unlock the full potential of your data. Read & Download this comprehensive guide now and embark on a seamless data journey from PostgreSQL to Snowflake with Estuary Flow!
Try it Free: https://dashboard.estuary.dev/register
A Comparative Analysis of Functional and Non-Functional Testing.pdfkalichargn70th171
A robust software testing strategy encompassing functional and non-functional testing is fundamental for development teams. These twin pillars are essential for ensuring the success of your applications. But why are they so critical?
Functional testing rigorously examines the application's processes against predefined requirements, ensuring they align seamlessly. Conversely, non-functional testing evaluates performance and reliability under load, enhancing the end-user experience.
NBFC Software: Optimize Your Non-Banking Financial CompanyNBFC Softwares
NBFC Software: Optimize Your Non-Banking Financial Company
Enhance Your Financial Services with Comprehensive NBFC Software
NBFC software provides a complete solution for non-banking financial companies, streamlining banking and accounting functions to reduce operational costs. Our software is designed to meet the diverse needs of NBFCs, including investment banks, insurance companies, and hedge funds.
Key Features of NBFC Software:
Centralized Database: Facilitates inter-branch collaboration and smooth operations with a unified platform.
Automation: Simplifies loan lifecycle management and account maintenance, ensuring efficient delivery of financial services.
Customization: Highly customizable to fit specific business needs, offering flexibility in managing various loan types such as home loans, mortgage loans, personal loans, and more.
Security: Ensures safe and secure handling of financial transactions and sensitive data.
User-Friendly Interface: Designed to be intuitive and easy to use, reducing the learning curve for employees.
Cost-Effective: Reduces the need for additional manpower by automating tasks, making it a budget-friendly solution. Benefits of NBFC Software:
Go Paperless: Transition to a fully digital operation, eliminating offline work.
Transparency: Enables managers and executives to monitor various points of the banking process easily.
Defaulter Tracking: Helps track loan defaulters, maintaining a healthy loan management system.
Increased Accessibility: Cutting-edge technology increases the accessibility and usability of NBFC operations. Request a Demo Now!
React and Next.js are complementary tools in web development. React, a JavaScript library, specializes in building user interfaces with its component-based architecture and efficient state management. Next.js extends React by providing server-side rendering, routing, and other utilities, making it ideal for building SEO-friendly, high-performance web applications.
Ansys Mechanical enables you to solve complex structural engineering problems and make better, faster design decisions. With the finite element analysis (FEA) solvers available in the suite, you can customize and automate solutions for your structural mechanics problems and parameterize them to analyze multiple design scenarios. Ansys Mechanical is a dynamic tool that has a complete range of analysis tools.
COMPSAC 2024 D&I Panel: Charting a Course for Equity: Strategies for Overcomi...Hironori Washizaki
Hironori Washizaki, "Charting a Course for Equity: Strategies for Overcoming Challenges and Promoting Inclusion in the Metaverse", IEEE COMPSAC 2024 D&I Panel, 2024.
Explore the rapid development journey of TryBoxLang, completed in just 48 hours. This session delves into the innovative process behind creating TryBoxLang, a platform designed to showcase the capabilities of BoxLang by Ortus Solutions. Discover the challenges, strategies, and outcomes of this accelerated development effort, highlighting how TryBoxLang provides a practical introduction to BoxLang's features and benefits.
What is OCR Technology and How to Extract Text from Any Image for FreeTwisterTools
Discover the fascinating world of Optical Character Recognition (OCR) technology with our comprehensive presentation. Learn how OCR converts various types of documents, such as scanned paper documents, PDFs, or images captured by a digital camera, into editable and searchable data. Dive into the history, modern applications, and future trends of OCR technology. Get step-by-step instructions on how to extract text from any image online for free using a simple tool, along with best practices for OCR image preparation. Ideal for professionals, students, and tech enthusiasts looking to harness the power of OCR.
Software development... for all? (keynote at ICSOFT'2024)miso_uam
Our world runs on software. It governs all major aspects of our life. It is an enabler for research and innovation, and is critical for business competitivity. Traditional software engineering techniques have achieved high effectiveness, but still may fall short on delivering software at the accelerated pace and with the increasing quality that future scenarios will require.
To attack this issue, some software paradigms raise the automation of software development via higher levels of abstraction through domain-specific languages (e.g., in model-driven engineering) and empowering non-professional developers with the possibility to build their own software (e.g., in low-code development approaches). In a software-demanding world, this is an attractive possibility, and perhaps -- paraphrasing Andy Warhol -- "in the future, everyone will be a developer for 15 minutes". However, to make this possible, methods are required to tweak languages to their context of use (crucial given the diversity of backgrounds and purposes), and the assistance to developers throughout the development process (especially critical for non-professionals).
In this keynote talk at ICSOFT'2024 I presented enabling techniques for this vision, supporting the creation of families of domain-specific languages, their adaptation to the usage context; and the augmentation of low-code environments with assistants and recommender systems to guide developers (professional or not) in the development process.
CViewSurvey Digitech Pvt Ltd that works on a proven C.A.A.G. model.bhatinidhi2001
CViewSurvey is a SaaS-based Web & Mobile application that provides digital transformation to traditional paper surveys and feedback for customer & employee experience, field & market research that helps you evaluate your customer's as well as employee's loyalty.
With our unique C.A.A.G. Collect, Analysis, Act & Grow approach; business & industry’s can create customized surveys on web, publish on app to collect unlimited response & review AI backed real-time data analytics on mobile & tablets anytime, anywhere. Data collected when offline is securely stored in the device, which syncs to the cloud server when connected to any network.
5. 5
Skills | Knowledge | Collaboration
Why performance testing is important
✓ 50% of frustrated users will visit another
website to accomplish their activity and 22%
won't return.
✓ 49% of respondents expect web pages to load
in under 2 seconds.
✓ 30% expect a 1-second response.
✓ 18% expect a site to load immediately.*
* http://docplayer.net/29696161-Performance-matters-key-consumer-insights.html
6. 6
Skills | Knowledge | Collaboration
From user perception point of view
What is a fast web-side?
< 1 second very fast
< 2 seconds quite fast
2-4 seconds acceptable
5-15 seconds slow
>15 seconds too slow
* https://www.hobo-web.co.uk/your-website-design-should-load-in-4-seconds/
8. 8
Skills | Knowledge | Collaboration
How smartphone users react to slow web-sites
Curse at their
phone, 23%
Scream at
their phone,
11%
Throw their
phone, 4%
Behave more
or less
normally,
62%
9. 9
Skills | Knowledge | Collaboration
Cost of poor performance
* http://www.webperformancetoday.com/2010/06/15/everything-you-wanted-to-know-about-web-performance/
If your average sales per hour is
$5,000
1 minute of downtime is
costing over $80
then
10. 10
Skills | Knowledge | Collaboration
Cost of poor performance
A 1-second page load delay equals*:
* http://www.webperformancetoday.com/2010/06/15/everything-you-wanted-to-know-about-web-performance/
16 % decrease in customer
satisfaction
11 % fewer page
views
7 % loss in
conversions
11. 11
Skills | Knowledge | Collaboration
Cost of poor performance
For example:
✓ John Lewis’s website went down around 3.20 pm on Black Friday - analysts estimated it could
cost £ 2.8m pounds.
✓ Web giant like Amazon would lose as much as $120 000 per minute of downtime.
✓ Microsoft Bing found that a two-second slowdown caused a 4.3 percent reduction in
revenue per user.
✓ Website Shopzilla reduced page load times from 7 seconds to 2 seconds, resulting in a 7–12
percent increase in revenue and 50 percent reduction in hardware costs.
* http://www.webperformancetoday.com/2010/06/15/everything-you-wanted-to-know-about-web-performance/
12. 12
Skills | Knowledge | Collaboration
When performance testing is needed
✓ The solution is already struggling from performance
problems.
✓ The solution have to deal with big amount of concurrent
users presently or in future (~ 300-500 concurrent users and
higher).
✓ The solution have a large database or should transfer or
process big amount of data in real time (~ 10 and more
concurrent users).
✓ The solution has complex architecture and a lot of internal
and external integrations running concurrently.
14. 14
Skills | Knowledge | Collaboration
✓ Performance Testing - the process of testing to determine the
performance of a software product (ISTQB Foundation).
✓ Performance testing – is a non-functional software testing technique
which determines responsiveness, stability, reliability and resource
usage of system under a certain user load (Wikipedia).
✓ Web load testing is nothing more than exercising a website under a
variety of production-like conditions to determine how it’s going to work
and to identify (and hopefully resolve) problems before your customers
find them (WebLoadTestingForDummies).
Performance testing. Definition
15. 15
Skills | Knowledge | Collaboration
Determine responsiveness, stability, reliability and resource usage of system
under a certain user load
Demonstrate that the system meets performance criteria (KPIs)
Compare different system configurations and versions to evaluate
performance improvement/degradation.
Performance trends tracking during the time
Determine system behavior under different load
Evaluate the system capacity
Scalability. Determine ability of a system to handle a growing amount of workload
Prepare the application for planned load (e-commerce: Black Friday, Marketing
Campaign. Finance: quarter/annual reporting, etc.)
Find which components of the system perform poorly under certain workload
Goals of performance testing:
16. 16
Skills | Knowledge | Collaboration
Slow sub-systems / functions (poor response)
Low capacity point
Configuration problems (web-server,
load balancers, db etc)
Dead-lock while simultaneous load
Flawed queue logic
Incorrect synchronization of recourses
Database issues e.g. size, indexing, replication
Memory, space and connections leaks
Poor network configuration
CPU, Memory utilization
Functionality bugs (how system should behave under overload, others).
Performance bottlenecks:
17. 17
Skills | Knowledge | Collaboration
Functional & Performance testing comparison
# Functional testing Performance testing
1
To verify the accuracy of the system against
expected results
To verify the behavior of the system at various
load conditions
2 Manual or automated Automated only
3 Could be done without special tools
Special set of tools is used including analyzing
and monitoring ones
4 One user performing all operations Several users performing desired operations
5
Involvement required from Customer, Tester
and Developer
Involvement required from Customers, Tester,
Developer, DB admins, DevOps
6
Test environment capacity/size could differ
from Production
Requires close to Production Test
environment!!!
based on: http://www.softwaretestinghelp.com/introduction-to-performance-testing-loadrunner-training-tutorial-part-1/
24. 24
Skills | Knowledge | Collaboration
Types of Performance Testing (load profiles)
Stress/Capacity test
Max Designed Operation Capacity
Volume test
8-72 hours or longer
+ Component Test
+ Reliability /
Recovery Test
Server-side performance
26. 26
Skills | Knowledge | Collaboration
Client-side performance testing - testing of one separate page load from client/browser side
Client-side Performance
27. 27
Skills | Knowledge | Collaboration
More detailed process of page load
Client-side Performance
28. 28
Skills | Knowledge | Collaboration
First thing the user sees
Client-side Performance
29. 29
Skills | Knowledge | Collaboration
Visual Experience
Client-side Performance
1. First Paint 2. First Contentful Paint
3. First Meaningful Paint 4. Visually Complete
based on: https://www.slideshare.net/nicjansma/measuring-real-user-performance-in-the-browser
31. 31
Skills | Knowledge | Collaboration
1. Run client-side performance of the page when there is 0 load.
2. Set performance metrics baselines based on step 1
3. Run client-side performance when there is different load on the server –
during Capacity/Load/Spike etc. tests
4. Compare results with baselines, analyze and summarize possible issues
Flow we suggest:
Client-side Performance
33. 33
Skills | Knowledge | Collaboration
Load testing metrics (synthetic monitoring)
Application-side Server-side
- Response time
- Throughput (rps/tps/tpm)
- Concurrent users
- Error rate (response code)
- Number of transactions passed/failed
- Network traffic
- CPU
- Memory
- Network
- Disk
- DB connections
- Logs error, warnings
34. 34
Skills | Knowledge | Collaboration
Response time = Latency (travelling across a network) + Processing time (system processing
of request)
• Average response time
• Peak response time (max)
• Response time with 95% or other percentile
Response time
Application-side metrics
35. 35
Skills | Knowledge | Collaboration
Throughput - how many simultaneous
requests/transactions per second/minute application can
handle
Note:
TPS could correlate with response time if requests are
consequent. The longer response – the lower tps
TPS does not correlate directly with response time if
requests are parallel.
consequent > 1 request – 1 sec response – 1 tps
parallel > 10 requests – 1 sec response – 10 tps
TPS could be improved by improving response time or
by increasing concurrent users
Throughput (rps/tps/tpm)
Application-side metrics
36. 36
Skills | Knowledge | Collaboration
Network traffic, Response codes, Error rate
Application-side metrics
Network traffic – shows how much data is flowing back
and forth from your servers (Kbytes or Mbytes / sec).
We can compare this metric to the response-time metric
to see how the throughput affects transaction
performance.
Error rate – is the mathematical calculation that
produces a percentage of problem requests compared to
all requests.
It is no standard for tolerable error rate. Some projects
consider 1% error rate successful in case the system can
handle maximum load without crash. Others consider
any errors. Still, few errors is not uncommon, especially
for large load.
37. 37
Skills | Knowledge | Collaboration
• You can monitor server-side metrics directly on the server (Linux, Windows)
• You can automate this process creating some monitoring agent to track metrics
• You can use one of monitoring tools:
- CloudWatch (Amazon)
- AppDynamics
- DynaTrace
- NewRelic
- Graylog, etc.
Server-side monitoring tools
*In addition to the server metrics monitoring during the load test (synthetic monitoring), monitoring tools allow
Real User Monitoring (RUM).
RUM is a type of performance monitoring that captures and analyzes each transaction by real users of a website or
application. Unlike synthetic monitoring, RUM never rests. It collects data from each user using every browser
across each request.
39. 39
Skills | Knowledge | Collaboration
Strong load testing tools should be able
Traffic recorder
Have IDE (console or GUI) which allows:
a) Create Requests of required protocol (HTTP, HTTPs, WS, WSS, JDBC, TCP, AJAX, etc.)
b) Support Transactions – to track time for all static data loading / redirections
c) Create Load Scenarios with ability of parametrization
d) Build different Load Profiles with rump-up and shut-down
e) Have debugger
Load runner engine
Distributed testing
Load test data saving (distributed), including client and server-side metrics
Load test data monitoring in real time
40. 40
Skills | Knowledge | Collaboration
- JMeter
- Gatling
- Locust
- The Grinder
- Apachebench
- Artillery
- Tsung
- Vegeta
- Siege
- Boom
- Wrk
Open source load testing tools
According to Load Impact tools comparison research*:
JMeter, Gatling, Grinder, Tsung and Boom all offer good performance, accuracy and reliability
Artillery, Locust and Siege have various issues with performance, accuracy and/or reliability
Performance-wise, Wrk and Apachebench are in a class of their own
NOTE: None of the tools tested can simulate thousands of VUs on a single machine without significant
degradation in measurement accuracy
* http://blog.loadimpact.com/open-source-load-testing-tool-benchmarks
45. 45
Skills | Knowledge | Collaboration
1. Planning
Identify Performance Acceptance Criteria and KPIs
• Response time for different type of transactions -> a user concern
• Throughput (tps/rps) -> a business concern
• Resource utilization -> a system concern
• Concurrent users number
• Accepted Error rate
• Accepted deviation for response time, resource utilization
• System behavior when overloaded
46. 46
Skills | Knowledge | Collaboration
1. Planning
Performance requirement analysis
Bad sample of requirements:
1. Response time should be no more than 4 seconds
2. System should be able to deal with 10 thousands
concurrent users
It is not clear whether:
• Response time should be 4 seconds for all requests?
• What if all response time will be 3.99 seconds?
• What if most request will response with 2 seconds but several with 6 seconds?
• What users do and how often? Open Main page or more?
• What if more than 10 ths. users? System should scale, or new users will be rejected or response time
just increases?
47. 47
Skills | Knowledge | Collaboration
1. Planning
Performance requirement analysis
Good sample of requirements:
1. Set average response time for different types of transactions:
• Simple navigation requests - 1 second
• Logging - 2 seconds
• Search and buy - 4 seconds
2. Set deviation - no more than 15%
3. Set failure rate - no more than 1%
4. Set CPU, memory, network, other server-metrics thresholds
5. Set response times, deviation, server-side metrics for different number of users - for
1k, 5k, 10k
6. System should reject new users in case of overload, showing informing message
48. 48
Skills | Knowledge | Collaboration
1. Planning
Identify the Test Environment
• Identify the logical and physical production architecture for performance testing
• Compare the both test and production environments while identifying the testing environment
TEST environment MUST BE the same as PROD (or run tests on PROD)
• Get resolve the environment-related concerns if any – using stabs for 3-rd parties or others
• Analyze whether additional tools are required for performance testing, like monitoring tools. Install such
tools.
Identify scope of load testing (product parts, 3-rd party services in/out of scope)
Identify technical nuances
• Scheduling services?
• Ping calls
• Client’s internet connection speed?
• Static content hosting: CDN or own servers?
• Target region (USA, Europe, etc.)?
49. 49
Skills | Knowledge | Collaboration
Your Functional Testing
Environment
TEST environment MUST BE the same as
PROD (or run tests on PROD)
Your Production
Your Real Users
50. 50
Skills | Knowledge | Collaboration
1. Planning
Plan and Design Tests
• Identify key usage scenario and workload (Load, Capacity, Spike, etc.)
• Define test data
• Establish metrics to be collected
What transactions to include:
1. Critical transactions
Example: System login, session support
2. Mostly used transactions by real users
Example: System login/logout, main page
3. Business required
Example: some specific feature
4. Risky transactions
Example: Checkout, payment
5. Heavy transactions
Example: File download/upload
51. 51
Skills | Knowledge | Collaboration
User Community Modeling Language (UCML) for Performance Test Workloads
1. Planning
52. 52
Skills | Knowledge | Collaboration
Prerequisites solving
1. Planning
• Add your load generator machine IP(s) into whitelisting (if needed)
• Request test users or create them by yourself
• Request test data (products names, files, etc.)
• Request 3-rd party dependent data (payment cards, etc.)
Configure load-generation environment
The output of this stage is prepared Performance Test Plan
• Environment capacity (CPU, memory) should be enough to run required number of
users during some time
• Setup Master-Slave architecture in case of distributed load testing
• Setup monitoring tools to track load server(s) health during tests running
53. 53
Skills | Knowledge | Collaboration
2. Implementing
• Develop performance scripts
• Put assertions points and wait timers to make it a real time scenario
• Run several smoke runs to calibrate scripts to the target environments in accordance with test
design
Simulate real users behavior basing on Production usage statistic
54. 54
Skills | Knowledge | Collaboration
3. Executing
It very depends on a project, but still we can suggest next
How to choose and run correct load profiles
55. 55
Skills | Knowledge | Collaboration
0. Warm up your product servers before load tests run
- Run some short smoke load test for several users
3. Executing
56. 56
Skills | Knowledge | Collaboration
1. Run several baseline Load tests to determine benchmark performance metrics
levels:
- virtual users: 5-100
- duration: 30-60m
- rump-up – 10m; run – 20m; shutdown – 5m
!!! Run each test at least 2 but better 3 times to be sure in results
3. Executing
57. 57
Skills | Knowledge | Collaboration
2. Run the Capacity test profile and determine when Capacity point (system limit)
happens:
- virtual users: a lot, depends on the system
- duration: 2-3 hours
- rump-up – 2-3 hour; run – NA; shutdown – NA
!!! Run each test at least 2 but better 3 times to be sure in results
3. Executing
58. 58
Skills | Knowledge | Collaboration
How to understand that you reach Capacity point:
When transactions Response time increases sharply with load increase.
Note:
- although load increase, response time should not increase sharply. However, small
deviation is acceptable;
- response time also could start decrease dramatically, what means that you start receive
responses with errors.
When the Error rate increases with load increase
Note: often, small error rate is acceptable (1-3%), especially during large load tests.
Crash of the servers or one of them (web, application, DB, etc.).
3. Executing
59. 59
Skills | Knowledge | Collaboration
Capacity point example
V users: 10 ths.
Status: failed
Error rate: test stopped on 28%
60. 60
Skills | Knowledge | Collaboration
Capacity point example
V users: 9 ths.
Status: failed
Error rate: test stopped on 49%
61. 61
Skills | Knowledge | Collaboration
Capacity point example
V users: 8 ths.
Status: passed
Error rate: 0.21% (500 errors
from 236 ths. transactions
62. 62
Skills | Knowledge | Collaboration
3. Run the Load test profile with:
- virtual users: 50-80% of Capacity point or/and required load number
- duration: 2-3 hours
- rump-up – 60m; run – 60m; shutdown – 20m
!!! Run each test at least 2 but better 3 times to be sure in results
3. Executing
63. 63
Skills | Knowledge | Collaboration
4. Run Component tests for each service / micro-service / function separately:
- could be run before or after Endurance test, especially if issues for some
service(s) were found
- virtual users: firstly, like for Capacity test, then like for Load profile
- duration: 1-2 hours
- rump-up, run and shutdown timings depend on component test profile
!!! Run each test at least 2 but better 3 times to be sure in results
3. Executing
64. 64
Skills | Knowledge | Collaboration
5. Run the Endurance test profile:
- virtual users: 20-40% of Capacity point
- duration: 8-72 hours
- rump-up – 30m; run – 23р; shutdown – 30m
!!! Run each test at least 2 but better 3 times to be sure in results
3. Executing
65. 65
Skills | Knowledge | Collaboration
6. Run other types of load testing and play with your scenarios combinations.
- volume test, Spike, Reliability/Recovery, Volume tests
- each of these tests could be run on the first stages, if it is prioritized be business
- run load tests for different user flows combinations
!!! Run each test at least 2 but better 3 times to be sure in results
3. Executing
66. 66
Skills | Knowledge | Collaboration
• In ongoing development -> Verifying and validating
component, queue of components, and integration
related performance & robustness
• Before release -> Verifying and validating the whole
product performance & robustness before release.
• Maintenance -> Verifying and validating architectural,
configurational, capacity-related, db-related, and
integration-related changes
When to perform
3. Executing
67. 67
Skills | Knowledge | Collaboration
• Collect and analyze load tests data
• Investigate possible bottlenecks (memory, disk, processor,
process, cache, network, etc.) resource usage like (memory,
CPU, network, etc.)
• Generate the Performance analysis reports
Note: The report form is
- a performance test summary report
- transactions details, hardware utilization, etc.
- the comparison of actual and expected KPIs
• Based on the analysis prepare recommendation report
• Share report with the team and stakeholders
4. Analyzing and reporting
69. 69
Skills | Knowledge | Collaboration
5. Continuous Integration
or
Development
Continuous Integration
Build. Unit test
Deploy to Dev Server
Integration Test
Deploy to Test/QA
Server
Automated Functional
Tests
Deploy to Load Server
Performance test in CI +
monitoring
Manual QA Test
Promote to
Stage / Pre-Prod
Ideally, if you have separate
environment for load testing
Note: you can implement
load tests on Stage too, if
it is the same as Prod
77. 77
Skills | Knowledge | Collaboration
Your PC
WARNING: your PC is not enough
Testing Environment
You need such load
generation environment!
78. 78
Skills | Knowledge | Collaboration
Web-site:
• opencart/ e-commerce website (open-source shopping cart solution)
User flows:
1. Registered user:
Main page -> Login -> Home Page -> Product Search -> Open Product -> Add to Cart ->
View Cart -> Logout
2. Guest user:
Main Page -> Product Search -> Open Product -> Add to Cart -> View Cart
Web-site details and user flow
79. 79
Skills | Knowledge | Collaboration
• https://jmeter.apache.org/download_jmeter.cgi
Download JMeter and Plugins to it
https://jmeter-plugins.org/install/Install/
80. 80
Skills | Knowledge | Collaboration
JMeter: Main Elements
1. ThreadGroup(user) ->Testplan
2. LogicControllers:
a) TransactionController->Testsuite/Testcase
b) Loop/While/ForEachController
c) If Controlleretc.
3. Samplers->Requests(HTTP(s), TCP, SOAP,etc.)
4. Assertions->Testyourresponse/Customize/Parametrize
5. Timers
6. Configurationelements
1
2
3
4
5
6
81. 81
Skills | Knowledge | Collaboration
http://jmeter.apache.org/usermanual/component_reference.html#HTTP%28S%29_Test_Script_
Recorder
Install certificate to record HTTPS traffic
82. 82
Skills | Knowledge | Collaboration
1. Use WorkBench to recored the traffic - temporary
storage of working files. Even if you save it, it is not run.
Only Test Plan is running
2. Add Thread Group to Test Plan
3. Add Non-Test Elements -> HTTP(S) Test Script
Recorder
- set Grouping – Put each group in a new transaction
controller
4. Configure proxy port in the browser
and in JMeter recorder
5. Start recording your flow
Record the traffic via JMeter
83. 83
Skills | Knowledge | Collaboration
Add next config elements to make your script behavior like real user ones:
- HTTP Cookie manager
- HTTP Cache manager
Add next config elements to make your scripts more flexible:
- HTTP Request Defaults
- HTTP Header Manager
Config Elements
84. 84
Skills | Knowledge | Collaboration
Use JMeter Listener -> View
Results Tree
or plugin
Go to JMeter
Options -> Plugins Manager
and install BlazeMeter Step-
by-step Debugger
+ You can combine Debugging
process with Fiddler.
For this, set Fiddler Port 8888
in HTTP Request Defaults
Scenario debugging
85. 85
Skills | Knowledge | Collaboration
Embedded recourses – static data load together with web-site page, like .css, .js, .png or other
images, .woff2. video etc.
Use HTTP Requests Defaults to manage embedded resources loading in all samplers
Best practices:
- Embedded recourses are included in load tests to emulated real users behavior
- Use HTTP Cache Manager in your Test Plan to emulate real browser cache handling
behavior
or
- Embedded recourses should NOT be included in load tests when CDN (Content Delivery
Network) is used (external load)
- Do NOT record embedded resources calls
Embedded recourses
86. 86
Skills | Knowledge | Collaboration
• Add Custom JMeter functions using JMeter Plugins Manager
JMeter functions / Custom JMeter Functions
more details is here: https://jmeter-plugins.org/wiki/Functions/
87. 87
Skills | Knowledge | Collaboration
Use controllers to control your flow:
• While controller
• Loop controller
• ForEach controller
• OnceOnly controller, etc.
JSR223 Sampler to parametrize, customize your flow (in Python/Jython):
• Install jython-standalone.jar into /lib folder
• http://www.jython.org/downloads.html
Controllers and JSR223 Sampler
89. 89
Skills | Knowledge | Collaboration
• Timeout check:
- Set timeout in HTTP
Request or in HTTP
Requests Defaults (to avoid
server stuck)
- Duration Assertion (to track
KPIs)!!!
• Size Assertion (in case when for
error response 200 is returned)
• Code response (checked
automatically)
• Response Assertion
• XPath Assertion, etc.
Assertions
90. 90
Skills | Knowledge | Collaboration
To simulate real users behavior user timers before request
• Constant Timer
• Poisson Random Timer
• Uniform Random Timer, etc.
Timers
91. 91
Skills | Knowledge | Collaboration
jpgc – Standard Set in Plugins Manager
Ultimate Thread Group
92. 92
Skills | Knowledge | Collaboration
• User Simple Data Writer to save log in file and then use for graphs building
• During test run comment out graphs and leave only Summary Report to track test
status. Graphs takes a lot of sources.
You can use next graphs:
• Transactions per Second
• Response Time Over Time
• Bytes Throughput Over Time (plugin 5 Additional Graphs)
• Response Codes per Second
• Active Thread Over Time
• Composite (Timeline) Graph (plugin)
Reporting
Highlight a value, combination of benefits and approaches
- Endurance test – тестирование стабильносты. Профиль тот же что при лоад, но время дольше.
При пиках интересно сможет ли система восставновится после пика, освободить ресурсы и т.д. Пики можно делать только для какого-то сценария, либо для всех.
Стресс тестирование – найти точку насыщения, когда нагрузка достигла критической, время отклика начинает расти хоть до этого не увеличивалась.
Не ограничивайтесь только этими профилями, стройте зависимо от вашей системы
- TPS could be improved by improving response time or by increasing concurrent users (кассира заставить работать быстрее, либо увеличить количество рабочих кас)
- TPS could be improved by improving response time or by increasing concurrent users (кассира заставить работать быстрее, либо увеличить количество рабочих кас)