The main objective of testing is to find defects in requirements, design, documentation & in code, so that  the defects will be fixed  and the product delivered to the customer is defect less. So with the help of testing, it is possible to measure the quality of software in terms of defects found. Testing can give confidence in the quality of the software if it finds few or no defects.

Historical Definitions of Testing:

  • Primary role of testing is not demonstration of correct performance, but the exposure of hidden defects. -> G. J. Myers
  • Establishing confidence that a program does what it is supposed to do. ->Hetzel
  • The process of executing a program or system with the intent of finding errors. -> Myers
  • Testing shows the presence, not the absence, of bugs. -> E. W. Dijkstra.

Importance of Testing:

  • Detects the existence of defects so that defects will be fixed.
  • Delivering the right product.
  • Satisfying customer’s needs.
  • Meeting the customer’s expectations.

Non functional testing requirements don’t describe the function, but it describe the attribute of the system as a whole. i.e, how well the system should carry out it’s function. Non functional testing has a great influence on customer and user satisfaction with the product. As per ISO 9126, characteristics of Non functional testing are Reliability, Usability and Efficiency. And the characteristics are measured when executing test scenario.

Examples of Non Functional testing:

Some of the examples of non functional testing are mentioned below (By Myers 1979). These testing should be considered when performing system testing for non functional testing.

-> Load Test: Measuring of the system behavior for increasing system loads (e.g., the number of users that work simultaneously, number of transactions)
-> Performance Test: Measuring of the processing speed and response time for particular use cases, usually dependent on increasing load.
-> Volume Test: Observation of the system behavior dependent on the amount of the data (e.g, processing of very large files)
-> stress Test: Observation of the system behavior when it is over loaded.
-> (Testing of security) against unauthorized access, denial of service attacks etc.
-> (Stability) or reliability test during permanent operation (e.g, mean time between failures or failure rate with a given user profile).
-> Robustness Test: Measuring the system’s response to operating errors, wrong programming, hardware failure etc as well as examination of exception handling and recovery.
-> Testing of compatibility and data conversion: Examination of compatibility to given systems, import/export of data etc.
-> Testing of different configuration of the system, e.g, different versions of the operating system, user interface language, hardware platform etc ( This is back-to-back testing).
-> Usability test: Examination of the ease of learning the system, ease and efficiency of operation, understandability of the system output, etc., always with respect to the needs of a specific group of users (ISO 9241, ISO 9126)
-> (Checking the documentation) for compliance with system behavior (e.g, user manual & GUI)
-> Checking of maintainability: Assessing the understandability of the system documentation and whether it is up to date, checking if the system has a modular structure etc.

Note: Non functional testing should be expressed in a testable way, not like “the system should be fast” or “the system should be easy to operate” which are not testable.