Modern embedded software is a complex conglomerate of code components that originate from numerous suppliers. Standards such as ISO 26262 and IEC 62304 provide guidance on the acceptable use of third-party software and help software developers mitigate risks with best practices.
Once upon a time, long, long ago, embedded software development consisted of writing a simple, bare metal application in C or assembly code. A standalone project was built for a specific task by a small team often based in a single office. But today, bare metal applications generally exist only as part of a much bigger picture.
Times have changed, and software development and maintenance has grown to become the costliest and most time consuming part of many modern embedded systems. Modern embedded systems are built on layers and layers of technology, requiring software teams of hundreds or thousands of developers who contribute millions of lines of code to a release image. It is rare for a single organization to own the entire software stack. How can anyone manage the potential risks introduced by including code from other entities and organizations to live happily ever after?
A modern, high-level software architecture for an embedded system will likely include many components. Operating systems, device drivers, communication stacks, bootloaders, hypervisors … and that’s before anyone adds any application code.
In most cases, the system codebase is a combination of handwritten code, autogenerated code, commercial off the shelf software (COTS), and open-source software (OSS). Throw in legacy code, compiler related software, template libraries, and third party tooling and there’s quite a risky looking mix.
For example, many parties in the supply chain participate in the development of vehicle software. Vehicle manufacturers rely on hundreds of first and second tier suppliers to design and manufacture the vehicle sub-systems and electronic control units (ECUs) that are integrated into the vehicle. These suppliers rely on semiconductor IP companies and hardware suppliers to produce microcontrollers along with the necessary software development toolchains, run time libraries and hardware interfaces.
The complexity of the resulting software stack consisting of code from a plethora of sources makes it imperative that risks are managed carefully.
It is clearly important to be sure that third party software will fulfil its purpose, and will save a justifiable amount of development effort. However, there is much more to think about.
Consider software licensing, for example. While open-source software is usually provided at no cost, it often comes with restrictions and obligations. It is important to consider whether those restrictions limit the commercial use of the software, whether modifications are permissible, and if so, whether those modifications have to be delivered back to the open-source software community or your users.
Failing to understand and comply with these restrictions and obligations can result in damage to an organization’s reputation, hefty penalties and fines and possibly result in prison time.
COTS typically comes with a license agreement that spells out what use of the software is permissible, and what royalties or fees, if any, must be paid to the software supplier. If there are violations of the license agreement from either side, then the other side may take legal action and claim damages. However, COTS licensing models can be more difficult to navigate than their open-source counterparts. They often contain complex licensing metrics and conditional clauses, and the license models can vary widely.
It is also important to ensure that the intellectual property associated with the software is actually the property of the COTS vendor. Their intellectual property infringement may result in apparently blameless customers being subjected to additional fees, or being forced to stop using the software.
No software is perfect, and failures in software sourced externally are clearly more challenging to deal with than for in-house projects. These failures may impact safety, or result in costly product recalls, which can also negatively impact the reputation of the manufacturer. Functional standards such as ISO 26262 Road vehicles – Functional safety, (ISO, 2018) contain guidance on best practice for developing a safe product. The standard dictates that if the product is safety critical, scrutiny of any applicable third party software is required by the integrator, preferably with assistance of the supplier.
Now that connectivity is a commonplace requirement in addition to failures that result in safety issues, a growing concern – especially with connected products now common place across many industries – is security issues. Unfortunately, these have become common place with data breaches of system hacking being reported in the media on an almost daily basis. Privacy breaches and data leaks can attract large government fines, negatively impact the reputation of the enterprise, and may result in litigation and costly damages to offset and remediate the repercussions on end users.
For example, OpenSLL is used by most internet users and website software. The Heartbleed security issue found in the OpenSLL cryptographic library led to data breaches as disclosed in 2014 [1]. If any such vulnerabilities are found in safety critical systems, then security issues can also compromise safety.
There are proven techniques that can be applied to mitigate these risks. Functional safety standards provide guidelines for the use of third party software, also referred to as software of unknown provenance (SOUP). For example, IEC 62304 Medical device software – Software life cycle processes, (IEC, 2006) details the actions that are to be taken if third party software is to be used that is not of proven pedigree (perhaps because details of the software development process used are unavailable).
ISO 26262 also suggests that “a set of measures can be:
One approach to mitigate risks of integrating third party software is to isolate it from the rest of the code base. Isolation may be achieved by using a hypervisor or firewall, through partitioning and segmentation, or using a physical air gap.
Figure 1 provides an overview of the isolation approach. The third party software on the right side of the figure has been successfully isolated. As illustrated on the left side of the figure, there may be situations where native and open-source software reside in close proximity.
Consider the recommended software development lifecycle practices where native code includes references to external Standard Template Library header files (Listing).
The starting point here would be to derive a set of requirements for the findWeekday method, accompanied by a set of test cases used to validate the requirements and to check the robustness of the code when dealing with boundary cases.
Any failed test cases are then analyzed, the code checked to ensure that it is not compromised, and the requirements are supplemented and amended to reflect the improved understanding of the code. With the SOUP requirements understood, test cases to validate these requirements can be constructed and any further failures addressed in an iterative manner.
The process is straightforward for such a small example, but the application of the SOUP methodology is potentially a time-consuming activity. It depends on the extent to which its provenance can be proven.
The STL Library is an example of where the SOUP methodology is practical because it is provided by the GNU GCC project which uses documented planning, configuration and change control processes, has a documented requirements specification, and has associated maintained test suites that can be executed by the integrator.
If the practices are carefully reviewed and understood to be appropriate by the integrator, then an argument can be made for trusting the STL library and concluding the software is fit for purpose. In cases where the risk of failure is critical, or the behavior of the third party software is not documented or clear, extra confidence may be obtained by applying the SOUP methodology.
Design and code reviews represent another established technique that can identify weaknesses in architectural design and source code. When available, it is recommended to check third party provided source code for known coding issues. Coding guidelines and coding issue databases are readily accessible and automatable.
Figure 2 shows a typical output of a C code analysis tool measuring compliance with various coding standards.
When source code is not available, for example if COTS software is delivered in a system image format, an alternative is to use the Compliance Guidance within MISRA Compliance:2020. Following this guidance, third party software suppliers can provide evidence that their software development process is of the highest quality.
One popular approach to minimizing integration effort is to perform the integration once and then “freeze” the codebase so that no changes can be made for the duration of the project. Each release of software development process is then considered a “certified” version of the software in accordance with ISO 26262, for example. If the version of the third party software is updated, then validation and verification processes must be repeated using the new code base.
There are some pitfalls, the most notable being that it may not be valid to assume that changes to the third party software will never be required. For example, any security related system must assume that a breach will occur and have effective remediation methods planned. These are likely to involve code changes.
Another approach that is becoming a best practice in some industries is to integrate third party software frequently, taking advantage of the latest features and security updates. Figure 3 presents a typical third party software integration process within a wider iterative software development process.
IEC 62304 suggests that Medical Device Software satisfy requirements and include functional capability to realize “compatibility with upgrades or multiple SOUP or other device versions”.
3 November 2021
Design & Elektronik
https://www.elektroniknet.de/international/managing-the-risk-of-adopting-third-party-code.191076.html
Email: info@ldra.com
EMEA: +44 (0)151 649 9300
USA: +1 (855) 855 5372
INDIA: +91 80 4080 8707