Today’s software-based, safety-critical systems depend on certified software tools and processes for development. For many applications, tool qualification is a necessary step in ensuring the tool chain produces quality code to fulfill the needs of applicable safety standards. In many cases, the use of TÜV certified tools is sufficient but there’s an increasing number of very high-stakes applications where the functional safety standards demand more.
For critical applications where TÜV certification falls short, the ISO 26262 and DO-330 standards offer a viable path to qualification. By understanding how other standards fail to meet qualification expectations and exploring how ISO 26262 and DO-330 can be adapted beyond their original intended industries, development teams can plan a tool qualification process beyond TÜV.
Figure 1: Most functional safety standards are direct derivatives of IEC 61508 – part of the “IEC 61058 family”
Tool qualification is the practice of ensuring the risk of a tool error impacting the safety of a system is acceptably low, either because the errors are few or because the function does not impact safety. From IEC 61508, “Functional safety of electrical/electronic/programmable electronic safety-related systems”:
“Examples include the deactivation of a medical infusion pump should it malfunction or the automatic activation of an overflow valve when a certain liquid or pressure level has been reached.”
Most functional safety standards define qualification processes to ensure potential errors are either avoided or detected in the application of the tool chain, but few provide any details for the most critical applications. While these standards have a common goal of building confidence and trust in the tool, there is little clarity on what development teams should actually do.
This presents a difficult challenge, as tool qualification is too much of a complex and time-consuming activity to start off on the wrong foot.
Figure 2: Classifying software such that the verification and validation activity is proportionate to risk
ISO 26262, “Road vehicles – Functional safety”, is a derivation of the functional safety standard IEC 61508 that’s specific to the automotive industry. For tool qualification, ISO 26262 requires a level of confidence that is dependent upon the circumstances of its deployment, as stated in §11.2:
“…the possibility that the malfunctioning software tool and its corresponding erroneous output can introduce or fail to detect errors in a safety-related item or element being developed; and the confidence in preventing or detecting such errors in its corresponding output.”
For most critical applications, this standard provides a path to dramatically reduce the overhead involved with qualification by allowing the evaluation of the tool based upon evidence of the application in a suitable software development process. One way that development teams take advantage of this is by using tools approved by a TÜV-certifying organization, such as the LDRA tool suite.
For the applications where standards demand more rigorous qualification in the context of the project development environment, vendors often supply tool qualification support packs (suites, packages, kits, etc.) Applied correctly, these packs can demonstrate whether the tool has been configured appropriately to provide the correct results in the tool chain and the environment in which it will be deployed.
Figure 3: Extracts from LDRA Tool Qualification Plan as provided in the DO-330 TQSP
RTCA/DO-330, “Software Tool Qualification Consideration,” was released to supplement the tool qualification requirements for airborne software development specified in the RTCA/DO‑178C standard, “Software Considerations in Airborne Systems and Equipment Certification.” Under the terms of DO-330, every project requires tool qualification regardless of the level of criticality.
The principles described in both standards can apply to other industries, and in fact, DO-330 explicitly states so in section §1.2:
“This document provides guidance for airborne and ground-based software. It may also be used by other domains, such as automotive, space, systems, electronic hardware, aeronautical databases, and safety assessment processes.”
While industry-specific safety standards usually have demanding tool qualification requirements for the most critical applications, many do not provide specific details on how to meet them. Examples include:
The reality is that development teams are responsible for figuring it out, by determining what constitutes adequate “assessment”, “justification”, and “verification” for standards certification. These choices aren’t easy to make, let alone defend, so it’s worthwhile to consider the case for using either ISO 26262 or DO-330 tool qualification processes for filling these gaps.
There is nothing about the qualification processes defined by either standard that makes them sector specific, and tool qualification support is available from a multitude of vendors for both. The ideal path is dependent on the characteristics of the project but it’s useful to compare five key similarities and differences between ISO 26262 and DO-330 to inform the best choice:
Most functional safety standards require some form of project-specific tool qualification for high-stakes applications but only ISO 26262 and DO-330 offer details on how to go about doing it. Moreover, there’s nothing implicit about their defined qualification processes that restricts them to certain industries, making them ideal candidates for use in other sectors too.
Deciding between the two standards is a matter of understanding their common principles and where they differ to best fit the target application. Vendor qualification packs can streamline and reduce effort. A typical set of artifacts for a test tool would include test code appropriate for the functionality under test, expected results, and associated documentation to fulfil the objectives of the chosen standard.
Technical Specialist
LDRA
August 16, 2022
Email: info@ldra.com
EMEA: +44 (0)151 649 9300
USA: +1 (855) 855 5372
INDIA: +91 80 4080 8707