Today’s complex safety critical systems depend on tools for automation and efficiency. Tool qualification ensures that the tools used for their development are dependable to an extent that is proportionate to the risks associated with any failures or errors they may introduce. RTCA DO-330/EUROCAE ED 215 is relevant to all safety-critical development activities involving software tools, including software development.
DO-330/ED-215 was primarily written as a technology supplement to support the civil aviation standards typified by DO-178C/ED-12C and DO-278A/ED-209. However, it also asserts that “It may also be used by other domains, such as automotive, space, systems, electronic hardware, aeronautical databases, and safety assessment processes.”
RTCA DO-330 and EUROCAE ED-215 are effectively the same document, co-authored by an RTCA/EUROCAE joint committee. DO-330/ED-215 is one of four technology supplements (or five including DO-200) introduced when DO-178C replaced DO-178B
DO-330 “Software Tool Qualification Considerations” states that “Software tools are widely used in multiple domains, to assist in developing, verifying, and controlling other software… Examples are automated code generators, compilers, test tools, and modification management tools. This document explains the process and objectives for qualifying tools.”
DO-330/ED-215 is a domain-independent, stand-alone document. Before the release of DO-178C, the objectives it describes were contained as an integral part of the DO-178 document. In its current format, DO-330/ED-215 is intended for use not only in support of DO 178C/ED 12C, DO-278/ED-109, DO-254/ED-80 and DO-200 in the aviation sector, but also in other safety-critical domains.
According to the document itself, the motivation behind developing DO-330/ED-215 was threefold.
RTCA DO-178C (DO-178C/ED-12C)“Software Considerations in Airborne Systems and Equipment Certification” is the principal document referenced by certification authorities (FAA, EASA, TCCA, ANAC…) to approve all commercial software-based aerospace systems. It is a formal process standard that covers the complete software lifecycle to ensure correctness and robustness in software systems for civil airborne applications.
DO-178C §12.2.3 states that “The objectives, guidance, and life cycle required for each Tool Qualification Level are described in DO-330/ED-215, “Software Tool Qualification Considerations.”
RTCA DO-278A (DO-278A/ED-190A). “Software Integrity Assurance Considerations for Communication, Navigation, Surveillance and Air Traffic Management (CNS/ATM) Systems” is a sister document to DO-178C/ED-12C and adheres to similar principles to ensure correctness and robustness in software systems for CNS/ATM applications.
DO-278A §12.2.3 states that “The objectives, guidance, and life cycle required for each Tool Qualification Level are described in DO-330, “Software Tool Qualification Considerations.”
RTCA DO-254 (DO-254/ED-80)-, “Design Assurance Guidance for Airborne Electronic Hardware”, requires that the principles described in DO‑330/ED-215 are to be applied to software tools to be used to support the design and development of hardware.
RTCA DO-200A, (DO-200A/ED-76)“Standards for Processing Aeronautical Data” provides minimum requirements for all phases of the data process applicable to the processing of aeronautical data, including quality assurance and QMS. DO-200A cites the application of DO-330/ED-215 as prerequisites for software tools used in support of those processes.
RTCA DO-331 (DO331/ED-216) “Model Based Development and Verification” describes adjustments to the principles described in DO-178C that apply when using model-based design.
Like DO-330/ED-215, DO-331/ED-216 is one of the technology supplements introduced when DO-178C replaced DO-178B. However, DO-331/ED-216 is not applicable to domains other than aviation.
RTCA DO-332 (DO-332/ED-217). “Object Oriented Technology and related technologies” includes additional objectives and advice applicable to object-oriented programming and complementary practices. Like DO-330/ED-215, DO-332/ED-217 is one of the technology supplements introduced when DO-178C replaced DO-178B – but it is not applicable to domains other than aviation.
RTCA DO-333 (DO-333/ED-218). “Formal Methods Supplement to DO-178C and DO-278A” identifies additional objectives and advice pertinent to the use of formal methods in compliant applications. Like DO-330/ED-215, DO-333/ED-218 is one of the technology supplements introduced when DO-178C replaced DO-178B – but it is not applicable to sectors outside aviation.
Tool qualification is a generic term to describe a process designed to ensure that the risk of a tool error impacting the safety of a system is acceptably low – either because the errors are few, or because they cannot impact safety. Many standards define processes to achieve tool qualification by considering the application of the tool, and the environment in which it is deployed.
The application considerations are designed to ensure that the tool is used in such a way that potential errors are either avoided or detected. The environmental perspective looks to ensure that the installed tool works as part of the broader tool chain to which it contributes, building confidence and trust in the tool and its use.
There are four primary approaches to tool qualification that are recommended by one or more standards:
Only tool verification is pertinent to DO-330.
Tool verification is the most thorough but most time-consuming approach to tool qualification. It is the only permitted approach for civil aviation projects, and is the only approach discussed in DO-330/ED-215. It is often the only viable approach for the most critical applications in other fields, too.
Tool verification activities involve analysing each feature of the tool using the environment in which the tool is to be deployed, and documenting the results – often using a vendor validation suite.
For static analysis, that means ensuring that features such as endianness and size of integer are correctly configured. Dynamic analysis is more dependent on the toolchain because it involves building and executing instrumented code on target (below).
Any potential errors in these features with the potential to impact the safety of the product are further assessed to determine the probability of them being detected or avoided within the process.
Although DO-330/ED-215 is written to be applicable to domains other than civil aviation, that remains its primary focus. The principles it describes are not new, but they were separated out from DO-178 with the arrival of the DO-178C version of that standard. Its use in civil aviation applications is therefore well proven – and DO-330/ED-215 is framed in terms that will be familiar to practitioners in that sector.
DO-330 defines some tool qualification activities that are to be performed by the tool developer, but the primary responsibility rests with the tool users to show that the tool is appropriate and sufficiently reliable for their application. More information is available here.
Many safety-critical applications outside the civil aviation sector leverage one of the less demanding approaches to tool qualification, often citing evidence of acceptable assessment and leveraging tools that have been TUV certified to achieve it. However, most functional safety standards demand qualification to the level of tool verification for the most critical application classes.
Many of these functional safety standards (IEC 61508, EN 50128, IEC 62304….) do not detail just what needs to be done. The application of DO-330 principles in these cases is one way to address that void.
Some DO-330 tool qualification activities are required to be performed by the tool developer. However, the primary responsibility rests with the tool user to show that the tool is appropriate and sufficiently reliable, given the criticality of their application. More information is available here.
Tool qualification is a vital part of the certification process for airborne systems and equipment, as documented in the DO-330/ED-215 Software Tool Qualification Considerations. DO-330/ED-215 introduces the concept of Tool Qualification Levels (TQL) which are assigned according to three criteria:
Criterion 1
A tool whose output is part of the airborne software and thus could insert an error
Criterion 2
A tool that automates verification processes and thus could fail to detect an error, and whose output is used to justify the elimination or reduction of:
Criterion 3
A tool that, within the scope of its intended use, could fail to detect an error.
An unqualified compiler or an auto-code generator from an UML tool would therefore fit criterion 1. A qualified version of the same UML tool would fit criterion 2, because its use is designed to reduce the overhead of code verification processes. The LDRA tool suite is a verification tool and therefore a criterion 3 tool.
Irrespective of the application software level (for DO-178C, read Design Assurance Level or DAL) such a verification tool is always assigned Tool Qualification Level 5 – the least demanding of the five levels (below).
Civil aviation certification authorities require tool qualification on a project-by-project basis. The responsibility for showing the suitability of any tool falls on to the organization developing a civil aviation application. However, they can make use of Tool Qualification Support Packs (TQSP) provided by the vendor.
Under the terms of DO-330/ED-215, tool qualification is required for every project. TÜV and similar approvals have no bearing on projects to which DO-330/ED-215 applies.
Many vendors provide a collection of documentation including test cases with expected results and reporting processes. Usually known as Tool Qualification Kits or Tool Qualification Support Packs, these artefacts can be used to show whether a tool has been configured appropriately to provide the correct results in the tool chain in which it will be deployed.
Referencing the LDRA tool suite as an example, the DO-330/ED-215 Tool Qualification Support Pack (TQSP) consists of five sub-packs, each of which can be specified as an “operational requirement” for the pertinent development project:
The TQSP includes four key documents designed to guide the user through the validation process. The process defined by these documents ensures the creation of evidential artefacts and the compilation of reports designed to summarize findings in a form appropriate to the standard.
These four documents are as follows:
The Tool Verification Plan provided by LDRA for configuration to suit the application includes source code, test cases, and expected results, for use in verifying the effectiveness of the tool in the pertinent installation environment.
LDRA provides a generic Tool Accomplishment Summary for customization by the user in accordance with the instructions in the TVP. The TAS describes the tool and its architecture, details the tool chain and other environmental conditions under which it is operating, and provides the results of the exercise associated with the PRC and TVP documents. Extracts from LDRA Tool Accomplishment Summary as provided in the DO-330/ED-215 TQSP are shown below.
For the tool to be qualified in accordance with RTCA DO-330 / EUROCAE ED-215, tool operational requirements (TORs) need to be defined. To achieve compliance, the TORs must be verifiable, consistent, and include enough detail to demonstrate that the functionality and the resulting output from the tool correspond to the activities that the tool is replacing. An extract from the template TOR document provided as part of the TQSP is presented below.
The Tool Qualification Plan document includes project specified information as identified in the Tool Verification Plan (below). DO-330 is usually applied to aeronautical application development, in which case it also captures all the requirements specified in the Plan for Software Aspects of Certification (PSAC).
The use of traceability, test management, and static & dynamic analysis tools in safety-critical software development projects offers significant productivity and cost benefits. Some tools make compliance checking easier, less error-prone, and more cost-effective. Others streamline the creation, management, maintenance, and documentation of requirements traceability across the software lifecycle.
When selecting tools for a tool chain to assist in achieving compliance with applicable safety or certification standards, several criteria should be considered:
Beyond these generic considerations, it is critical to recognize the specific requirements of DO-330.
In the context of DO-178C and DO-278A compliant projects, compliance with DO-330 is a fundamental requirement when tools are used. Every tool must be evaluated according to DO-330 processes to determine whether formal qualification is required.
Where a tool affects certification-relevant artefacts, full qualification must be performed in accordance with DO-330. Compliance with DO-330 is therefore inseparable from compliance with DO-178C or DO-278A.
When selecting a tool for civil aviation projects, it is important to ensure that:
Choosing a tool that has already been successfully qualified in aviation projects can significantly reduce certification effort, risk, and time to market.
Although DO-330 was developed primarily for the aviation domain, it is designed to be applicable to other safety-critical industries such as automotive, rail, industrial automation, and medical devices. However, the role of tool qualification outside aviation is generally less demanding.
In most functional safety standards, including ISO 26262, IEC 61508, and IEC 62304, selecting a TÜV-certified tool can greatly simplify compliance and reduce the need for additional qualification activities.
Only in the most critical applications — such as ASIL D or SIL 4 levels — might a DO-330 qualification approach be appropriate to provide more detailed evidence. Where that route is selected, the criteria relating to qualification support material, development processes and verification, and qualification assessment will apply here as for aviation applications.
DO-330/ED-215 “Software tool qualification considerations” is a self-contained document that describes the principles of software tool qualification as recommended for civil aviation projects. Prior to the release of DO-178C, these principles were described as an integral part of DO-178.
Although one of four supplements released at that time, DO-330/ED-215 is unique in that it is intended for use not only in support of DO-‑178C/ED‑12C, DO-278/ED-109, DO-254/ED-80, and DO-200 in the aviation sector, but also in other safety-critical domains.
Email: info@ldra.com
EMEA: +44 (0)151 649 9300
USA: +1 (855) 855 5372
INDIA: +91 80 4080 8707