Non-functional testing is the testing of a software application or system for its non-functional requirements: the way a system operates, rather than specific behaviours of that system.
2. www.kanoah.com
ABOUT US
Kanoah is an innovative company
providing ground-breaking solutions to
software testing professionals on the
Atlassian JIRA platform
3. About Kanoah Tests
Kanoah Tests is a full featured test management, integrated seamless into
JIRA with the same look-n-feel. No need to learn or switch between different
applications
Coordinate all test management activities including test planning, authoring,
execution, tracking and reporting from a central location
Kanoah Tests enables you to track testing progress and quality to foster
collaboration and visibility across traditional and agile teams
Get real-time insights into your testing progress with out of the box
reports
Easily integrate your automated tests and submit test results with Kanoah
Tests’ powerful REST API or use the API to automate many areas of the
application
4. After looking for several years at plugins for test management we finally found
Kanoah Tests. The other solutions were either too complex, didn't integrate well
with Jira, or were focused on a single project. Kanoah Tests proved to be an
elegant solution that allowed linking between any project. Kanoah has been
very responsive to feedback, requests, suggestions as well bugs. The customer
service is awesome. I'd highly recommend Kanoah Tests to teams of any size
looking to simplify test management and consolidate tools.
Don
Pierce
Robert
Murhamer
Liked Kanoah from the moment I discovered it. Integrates nicely with JIRA and
especially with Agile. Test cases can be authored right from the story level, but has
all other functionality a Test Case Management solution would need to have,
ranging from creating test plans, executing test cases, importing test cases, API for
automation, a.s.o. Additionally the team at Kanoah is amazing and responding to
any question very quickly. It wasn't hard to sell to my management to purchase
Kanoah. Will highly recommend Kanoah to anybody.
Zour
Brosh
I just start working with Kanoah and I am impressed how it's simple to manage tests
without unlimited non-used features like in most of the test management tools and
still to get the needed functionality and results. The integration with Jira is a great
working solution that enable to share testing and development in simple way on
one system. I recommend to use Kanoah for testing management. It will help to do
Kanoah as much as possible customizable like Jira to match each group
methodology
For more reviews, visit: https://marketplace.atlassian.com/plugins/com.kanoah.test-manager/server/reviews
Reviews
5. Key Features
Native seamless integration with JIRA
No need to learn or switch between
different applications
Perfect for agile & traditional testing
approaches
Manage, organize and track all your testing
efforts in a central place
Reuse test cases across your projects
Powerful REST API
Establish clear traceability between
requirements, test cases, and defects
Execute test cases and track results that
matter
Get real-time insights into your testing
progress with out of the box reports
Live statistics accessible to your entire
team
6. Benefits for the testers
No need to lear or switch between
different applications
Reuse test cases across projects for
regression
Link test cases to requirements and
defects
API support for automated efforts
Benefits for the teams
Informed decisions based on real-time
insights
End-to-end traceability and impact
analysis
Centralized Test Management
Save time and increase productivity
7. Why choose Kanoah Tests
Coordinate all test
management activities right
inside JIRA
1Testing right
inside JIRA
Take advantage of the built-in
reports to track the results and
measure progress
2Make informed
decisions
Kanoah Tests users receive
priority support, even
during trials
3Stelar
support
9. What is Non-Functional Testing?
Non-functional Testing is the testing of a software application or system for its
non-functional requirements: the way a system operates, rather than specific
behaviours of that system. [1]
Non-functional Testing is concerned with the non-functional requirements and
is designed specifically to evaluate the readiness of a system according to the
various criteria which are not covered by functional testing. [2]
Non-functional requirements tend to be those that reflect the quality of the
product, particularly in the context of the suitability perspective of its users. It
can be started after the completion of Functional Testing. The non functional
tests can be effective by using testing tools. [3]
10. What is Non-Functional Testing?
Non-functional Testing implies the testing of software attributes which are not
related to any specific function or user action like performance, scalability,
security or behavior of application under certain constraints.
Non-functional testing has a great influence on customer and user
satisfaction with the product. Non functional testing should be expressed in a
testable way, not like “the system should be fast” or “the system should be
easy to operate” which is not testable. [3]
11. Types of Non-Functional Testing
The following types and definitions have been recompiled from these authors [1]
[2] [3].
Reliability testing: Reliability Testing is about exercising an application so that
failures are discovered and removed before the system is deployed. The
purpose is to determine product reliability, and to determine whether the
software meets the customer’s reliability requirements.
Efficiency testing: Efficiency testing test the amount of code and testing
resources required by a program to perform a particular function. Software
Test Efficiency is number of test cases executed divided by unit of time
(generally per hour).
12. Usability testing: The testers tests the ease with which the user interfaces can
be used. It tests that whether the application or the product built is user-
friendly or not. It also includes five components:
1. Learnability: How easy is it for users to accomplish basic tasks the first
time they encounter the design?
2. Efficiency: How fast can experienced users accomplish tasks?
3. Memorability: When users return to the design after a period of not using
it, does the user remember enough to use it effectively the next time, or
does the user have to start over again learning everything?
4. Errors: How many errors do users make, how severe are these errors and
how easily can they recover from the errors?
5. Satisfaction: How much does the user like using the system?
Types of Non-Functional Testing
13. Types of Non-Functional Testing
Maintainability testing: It basically defines that how easy it is to maintain the
system. This means that how easy it is to analyze, change and test the
application or product.
Portability testing: It refers to the process of testing the ease with which a
computer software component or application can be moved from one
environment to another, e.g. moving of any application from Windows 2000 to
Windows XP. This is usually measured in terms of the maximum amount of
effort permitted. Results are measured in terms of the time required to move
the software and complete the and documentation updates.
Baseline testing: It refers to the validation of documents and specifications on
which test cases would be designed. The requirement specification validation
is baseline testing.
14. Types of Non-Functional Testing
Compliance testing: It is related with the IT standards followed by the
company and it is the testing done to find the deviations from the company
prescribed standards.
Documentation testing: As per the IEEE Documentation describing plans for, or
results of, the testing of a system or component, Types include test case
specification, test incident report, test log, test plan, test procedure, test
report. Hence the testing of all the above mentioned documents is known as
documentation testing.
Endurance testing: Endurance testing involves testing a system with a
significant load extended over a significant period of time, to discover how the
system behaves under sustained use. For example, in software testing, a
system may behave exactly as expected when tested for 1 hour but when the
same system is tested for 3 hours, problems such as memory leaks cause the
system to fail or behave randomly.
15. Types of Non-Functional Testing
Load testing: Load testing is performed to determine a system’s behavior
under both normal and at peak conditions. It helps to identify the maximum
operating capacity of an application as well as any bottlenecks and determine
which element is causing degradation.
Activities of Load Testing might include:
Validate that the system performs as expected when concurrent users
access the application and gets the expected response time.
This test is repeated with multiple users to get the response time and
throughput.
At the time of testing, the data base should be realistic.
The test should be conducted on a dedicated server which stimulates the
actual environment.
16. Types of Non-Functional Testing
Performance testing: Performance testing is performed to determine how fast
some aspect of a system performs under a particular workload. It can serve
different purposes like it can demonstrate that the system meets performance
criteria. It can compare two systems to find which performs better. Or it can
measure what part of the system or workload causes the system to perform
badly.
Activities in Performance Testing might include:
Validate that the system meets the expected response time.
Evaluate that the significant elements of the application meets the desired
response time.
It can also be conducted as a pert of integration testing.
It can also be conducted as a part of systems testing.
17. Types of Non-Functional Testing
Compatibility testing: Compatibility testing is basically the testing of the
application or the product built with the computing environment. It tests
whether the application or the software product built is compatible with the
hardware, operating system, database or other system software or not.
Activities in Compatibility Testing might include:
Test with each hardware with minimum and maximum configuration.
Test with different browsers.
Test cases are the same which were executed during functional testing.
In case the number of hardware and software are too many, we can use
OATS techniques to arrive at the test cases to have maximum coverage.
18. Types of Non-Functional Testing
Security testing: Security testing is basically to check that whether the
application or the product is secured or not. Can anyone came tomorrow and
hack the system or login the application without any authorization? It is a
process to determine that an information system protects data and maintains
functionality as intended.
Scalability testing: It is the testing of a software application for measuring its
capability to scale up in terms of any of its non-functional capability like load
supported, the number of transactions, the data volume etc.
19. Types of Non-Functional Testing
Volume testing: Volume testing refers to testing a software application or the
product with a certain amount of data. E.g., if we want to volume test our
application with a specific database size, we need to expand our database to
that size and then test the application’s performance on it.
Some key points of Volume Testing:
Subject the software to large amounts of data and check the limit where the
software fails.
Maximum database size is created and multiple client query the database or
create larger report.
If the application is processing database to create a report, a volume test
would be to use a large result set and check that the report is printed
correctly.
20. Types of Non-Functional Testing
Stress Testing: It is a form of testing that is used to determine the stability of a
given system. It puts greater emphasis on robustness, availability, and error
handling under a heavy load, rather than on what would be considered
correct behavior under normal circumstances. The goals of such tests may be
to ensure the software does not crash in conditions of insufficient
computational resources (such as memory or disk space).
Activities of Stress Testing might include:
Test on low memory or low disc space on clients or servers that reveals the
defects which cannot be found under normal conditions.
Multiple user performance the same transactions on the same data.
Multiple clients connected to servers with different workloads.
Reduce the Think Time to “Zero” to stress the servers to their maximum
stress.
21. Types of Non-Functional Testing
Recovery testing: Recovery testing is done in order to check how fast and
better the application can recover after it has gone through any type of crash
or hardware failure (etc). Recovery testing is the forced failure of the software
in a variety of ways to verify that recovery is properly performed.
Recovery Testing activities might include:
Power interruption, to the client while doing CURD activities.
Invalid database pointers and keys
Database process are aborted or prematurely terminated.
Database pointers, fields and keys are corrupted manually and directly
within database.
Physically disconnect the communication wires, power turn off, turn down
the routers and network servers.
22. Types of Non-Functional Testing
Internationalization testing: Internationalization is a process of designing a
software application so that it can be adapted to various languages and
regions without any changes.
Localization Testing: Localization is a process of adapting internationalized
software for a specific region or language by adding local specific
components and translating text.
23. Estimating Non-Functional Requirements
Once the team accepts a non-functional requirement into the project they
need to remain in compliance with that non-functional requirement for the
remainder of the project.
Doing performance testing (or staying in compliance with any non-functional
requirement) creates some amount of overhead on the team. This overhead,
or tax, must be paid regularly. In some cases, the team and product owner will
decide the tax must be paid every sprint. In other cases the team and
product owner may agree to pay the tax every few sprints.
Estimate the cost of initial compliance just like any other user story or product
backlog item. The team and product owner will need to incorporate an
estimate of when they'll do this. Adding performance testing after five sprints
is different than adding it after 20 sprints. [5]
24. Distinction between Functional and Non Functional Testing
Functional testing ensures that your product meets customer and business
requirements, and doesn’t have any major bugs.
Functional testing is designed to determine that the application’s features and
operations perform the way they should.
Non-functional testing wants to see if the product stands up to customer
expectations.
Non-functional testing wants to know that the product “behaves” correctly.
Failure to release a working product that meets the needs of consumer demands
can damage your company’s reputation and reduce overall product sales. [6]
25. Functional Testing
The following list has been extracted from here [7].
Tests how well the system performs.
Based on client requirements.
Means testing the application against business requirements.
It is a part of System Testing.
Validates the behavior of the application.
26. Non-Functional Testing
The following list has been extracted from here [7].
Tests how well the system responds.
Based on client expectations.
Means testing the application against the client and performance
requirements.
Also a part of System Testing.
Validates the performance of the application.
28. Next Steps
Getting Started Guide
GETTING STARTED RESOURCES FREE TRIAL
Documentation
Support
Tutorials
What we're reading this week
Blog
Start you 30-day free trial now