Читать книгу Software Testing Foundations - Andreas Spillner - Страница 19

ISO 25010: Quality in Use and Product Quality models

According to the ISO 25010 standard [ISO 25010], software quality can be classified in two major ways⁷:

 The Quality in Use Model, and

 The Product Quality Model

The quality in use model comprises the following five characteristics:

 Effectiveness

 Efficiency

 Satisfaction

 Freedom from risk

 Context coverage

The product quality model comprises eight characteristics:

 Functional suitability

 Performance efficiency

 Compatibility

 Usability

 Reliability

 Security

 Maintainability

 Portability

The product quality model has the most similarities with the previous ISO 9126 standard. Details of the Data Quality Model can be found in the ISO 25012 standard [ISO 25012].

In order to effectively judge the quality of a software system, all of these characteristics and quality criteria need to be considered during testing. The level of quality that each characteristic of the test object is intended to fulfill has to be defined in advance in the quality requirements. The fulfillment of these requirements and their criteria then has to be checked using appropriate tests.

Forty (sub-)characteristics

ISO 25010 breaks down the 13 quality characteristics listed above into a total of 40 further sub-characteristics. It is beyond the scope of this text to go into detail on all 40 sub-characteristics of the quality in use and product quality models. More details are available online at [ISO 25010]. Some of the more important characteristics are summarized below:

Functional suitability/functionality

The functional suitability (or, more simply, functionality) of the product quality model covers all the characteristics involved in describing the planned functionality of the system.

A quality characteristic is divided into three sub-characteristics:

 Functional completenessDoes the function set cover all specified tasks and user objectives?

 Functional correctnessDoes the product/system deliver correct results with the required degree of accuracy?

 Functional appropriatenessTo what degree do the available functions fulfill the required tasks and specified objectives?

Appropriate tests can be used to check whether specified and implicit requirements are mirrored in the available functionality, thus answering the questions posed above.

Functionality is usually described in terms of specified input/output behavior and/or a specific system reaction to specified input. Tests are designed to demonstrate that each required functionality has been implemented in such a way that the specified input/output behavior or system behavior is complied with.

Reliability

The reliability aspect of the product quality model describes a system’s ability to perform at a specific level under specified circumstances for a specified period of time.

This quality characteristic has four sub-characteristics:

 MaturityTo what degree does a system, product, or component provide the required reliability under normal operating conditions?

 AvailabilityIs the system, product, or component always ready for use, and how easily is it available when it is required?

 Fault toleranceHow well does the system, product, or component function in spite of the presence of known hard- or software faults?

 RecoverabilityHow long does it take to recover specific data and normal service following a system or product failure or crash?

Satisfaction

The satisfaction aspect of the quality in use model addresses the degree to which user needs are fulfilled when the product or system is used under specified circumstances.

This quality characteristic has four sub-characteristics:

 UsefulnessHow happy is the user with the perceived fulfillment of pragmatic objectives, including the results and consequences of using the system?

 TrustHow certain is the user (or other stakeholder) that the product or system will behave as intended?

 PleasureHow much pleasure does the user experience when using the system to fulfill his/her personal requirements?

 ComfortHow comfortable does the user find the system—also in terms of physical well- being?

Most of the characteristics of the quality in use model have a strong personal element and can only be objectively viewed and precisely evaluated under exceptional circumstances. In order to test this quality characteristic you will need to refer to multiple users (or user groups) in order to obtain usable test results.

A software system cannot fulfill all quality characteristics to the same degree. Fulfilling one characteristic often means not fulfilling another. A highly efficient software system is not easily portable, as its developers will have designed it to utilize specific attributes of the platform it runs on.

Prioritizing quality characteristics

It is therefore necessary to prioritize these characteristics. The resulting priorities will also act as a guideline for the testing thoroughness for each characteristic.

Case Study: Applying ISO 25010 to VSR-II

The VSR-II testing/QA lead suggests using the product quality model described in ISO 25010 to the project steering committee. The committee agrees and asks the testing/QA lead to prepare a concept paper on how to apply the standard in the context of the VSR-II project. The core of the draft is a matrix that illustrates the relevance of each quality attribute to each product component and which interpretations to apply. The initial draft of the matrix looks like this:

Table 2-1 Classifying quality characteristics

These risk classifications are to be interpreted relative to one another and are justified by the testing/QA lead for each quality characteristic, e.g.:

 Functional suitability/all modulesEvery module serves large numbers of users and processes a lot of data, so functional failures have the potential to produce considerable costs. The requirement is therefore classified as “high” for all modules.

 Compatibility/ConnectedCarThere are no requirements, as this module is to be built from scratch.

 Usability/FactBookThere are no requirements, as this is a back-end module and the API already exists.

 Portability/DreamCarThis characteristic is classified as “low” because the framework in use covers it without the application of additional measures.

Which checks and tests are required will be established during QA/test planning for each module. This top-level classification can be used to establish basic parameters—for example, automated continuous integration tests (see section 3.2) are required for a “high” attribute, a single round of acceptance testing is sufficient for “mid” attributes, while a written design guideline on how the team approaches an issue is sufficient for “low” attributes. The QA/test lead is sure to have to go through a number of assessment rounds with the teams to arrive at an agreement on these high-level rules.