Function coverage and call coverage are both highly recommended by ISO 26262. But what is the relationship between the two?

There is an ever-widening range of automotive electrical and/or electronic (E/E/PE) systems, including adaptive driver assistance systems, anti-lock braking systems, steering and airbags. ISO 26262 “Road vehicles – functional safety” is an international standard written in response to this burgeoning demand. It specifies exacting functional safety development processes, from requirements specification, through design, implementation, integration, verification, validation, and on to configuration.

ISO 26262 mandates the measurement of structural coverage both at the software unit level and the architectural level. Structural coverage provides a mechanism to ensure the completeness of testing, and to identify any code that is surplus to requirements.

Functional coverage v function coverage

Firstly, it is useful to address a misunderstanding that a quick search on a favoured browser will confirm is commonplace.

Functional coverage is a measure of how much of the design functionality has been exercised. It is a black-box, high-level test regime, designed to demonstrate that each feature of the software works as it should. It involves exercising tests defined in a test plan, each of which demonstrates that some aspect of the software as defined in the specification is correctly implemented. The more tests that are completed, the higher the functional coverage.

Despite the phonetic similarity, function coverage serves a different purpose. It is concerned with the structural coverage of code. The function coverage of a program is a percentage of the number of functions (sub-programs) were called, as a proportion of how many there are – irrespective of whether they fulfil their intended purpose.

In this blog, we are concerned with function coverage rather than functional coverage.

Function coverage v call coverage

Like function coverage, call coverage is a metric concerned with the structural coverage of code.

To reiterate, Function coverage is a measure of the number of functions (sub-programs) that have been called during the execution of a program, as a percentage of the number of functions that exist in that program.

Call coverage is a measure of the number of calls to  functions (sub-programs) that have been exercised during the execution of a program, as a percentage of the number of calls that exist in that program.

100% function coverage doesn’t imply 100% call coverage. The opposite is not necessarily true either. Moreover, it is not possible to infer a function coverage percentage from a call coverage metric alone.

Redundant functions

A literal interpretation of call coverage takes no account of redundant functions. By definition, these are not called at all! But redundant functions are important. They not only impact the maintenance of the code, but they are also likely to result from programming error. It is therefore wise to consider a metric alongside call coverage that will identify them – perhaps Branch/Decision Coverage or Modified Condition/Decision Coverage (MC/DC).

Relevance to ISO 26262

To evaluate the completeness of test cases and provide evidence that the test objectives for integration testing are adequately achieved, ISO 26262 requires that the structural coverage shall be assessed by the methods listed in Table 12.

ISO 26262:6 §10.4.5 recommends the use of both function coverage and call coverage for less critical applications (ASIL A & B), and highly recommends them where criticality is higher (ASIL C & D).

ISO 26262:6 Table 12 ISO 26262:6 Table 12 recommends the use of both function coverage and call coverage

Fulfilling the objectives of ISO 26262:6 Table 12

The ISO 26262 standard does not require the use of any tools. However, a certified testing tool like the LDRA tool suite^® makes the development process much more efficient and less error-prone for all but the most trivial of applications. Automating the measurement of structural coverage during the requirements-based testing at the software integration level helps to identify untestable code, unintended functionality, dead code, or deactivated code. Function coverage and call coverage reports provide evidential artefacts for that work.

Function and call coverage reports Function and call coverage reports provide evidential artefacts

Taking function coverage as an example, suppose that an anti-lock braking system has four major sub-components (speed sensors, valves, ECU, and HCU). Achieving 100% function coverage for the sensors would show that every function (sub-program) has been exercised for that component. Repeating that process for all other components would achieve 100% function coverage for the ABS system as a whole.

The graphical representation of coverage in the LDRA tool suite helps users to identify those parts of the system that have not been exercised, and to devise new tests in order to access those parts of the code.

Conclusions

Depending on the criticality of the application, ISO 26262:6 Table 12 recommends or highly recommends the use of both function and call coverage. Both are measures of code coverage and should not be confused with functional coverage, which is a measure of the degree to which software features have been tested and proven.

Fulfilling each of these objectives can be achieved through “bottom-up” testing, showing that each sub-component has been adequately tested to ultimately achieve coverage for the system as a whole.