Читать книгу Software Testing Foundations - Andreas Spillner - Страница 13

Testing is not debugging

In order to remedy a software fault it has to be located. To start with, we only know the effect of the fault, but not its location within the code. The process of finding and correcting faults is called debugging and is the responsibility of the developer. Debugging is often confused with testing, although these are two distinct and very different tasks. While debugging pinpoints software faults, testing is used to reveal the effect a fault causes (see figure 2-1).

Confirmation testing

Correcting a fault improves the quality of the product (assuming the correction doesn’t cause additional, new faults). Tests used to check that a fault has been successfully remedied are called confirmation tests. Testers are often responsible for confirmation testing, whereas developers are more likely to be responsible for component testing (and debugging). However, these roles can change in an agile development environment or for other software lifecycle models.

Unfortunately, in real-world situations fault correction often leads to the creation of new faults that are only revealed when completely new input scenarios are used. Such unpredictable side effects make testing trickier. Once a fault has been corrected you need to repeat your previous tests to make sure the targeted failure has been remedied, and you also need to write new tests that check for unwanted side effects of the correction process.

Objectives of testing

Static and dynamic tests are designed to achieve various objectives:

 A qualitative evaluation of work products related to the requirements, the specifications, user stories, program design, and code

 Prove that all specific requirements have been completely implemented and that the test object functions as expected for the users and other stakeholders

 Provide information that enables stakeholders to make a solid estimate of the test object’s quality and thus generate confidence in the quality provided3

 The level of quality-related risk can be reduced through identification and correction of software failures. The system will then contain fewer undiscovered faults.

 Analysis of the program and its documentation in order to avoid unwanted faults, and to document and remedy known ones

 Analyze and execute the program in order to reproduce known failures

 Receive information about the test object in order to decide whether the component in question can be committed for integration with other components

 Demonstrate that the test object adheres and/or conforms to the necessary contractual, legal and regulatory requirements and standards

Objectives depend on context

Test objectives can vary depending on the context. Furthermore, they can vary according to the development model you use (agile or otherwise) and the level of test you are performing—i.e., component, integration, system, or acceptance tests (see section 3.4).

When you are testing a component, your main objective should be to reveal as many failures as possible and to identify (i.e., debug) and remedy the underlying faults as soon as possible. Another primary objective can be to select tests that achieve the maximum possible level of code coverage (see section 2.3.1).

One objective of acceptance testing is to confirm that the system works and can be used as planned, and thus fulfills all of its functional and non-functional requirements. Another is to provide information that enables stakeholders to evaluate risk and make an informed decision about whether (or not) to go live.

Side Note: Scheme for naming different types of testing

The various names used for different types of tests can be confusing. To understand the naming of tests it is useful to differentiate between the following naming categories:

1 Test objectiveThe naming of a test type is based on the test objective (for example, a “load test”).

2 Test method/techniqueA test is named according to the method or technique used to specify and/or perform the test (i.e., “state transition testing”, as described in section 5.1.3)

3 Test objectA test is named according to the type of object to be tested (for example, “GUI test“ or “database test“)

4 Test levelA test is named according to the corresponding level of the development model being used (for example, a “system test“)

5 Test personA test is named after the person or group who perform the test (for example, “developer test“, “user test“)

6 Test scopeA test is named according to its scope (for example, a “partial regression test“)

As you can see, not all of these terms define a distinct type of test. Instead, the different names highlight different aspects of a test that are important or in focus in a particular context or with regard to a particular testing objective.