The reliability of the embedded software used in medical devices and the risk associated with it has become a vital concern. IEC 62304, “Medical device software – Software life cycle processes,” has thus emerged as an internationally recognized mechanism for the demonstration of compliance with relevant local requirements. Part 1 of this article (MDB, October 2017) examined the development of detailed requirements and associated design of medical devices specified by IEC 62304, culminating in a detailed software design in accordance with clause 5.4 of the standard and shown in context in Figure 1. Part 2 details the process applicable to the implementation of that design in source code, the verification and validation of that code, and the ongoing maintenance of the system after release.
5.5 Software Unit Implementation and Verification
Next comes the process of translating the detailed design into source code. To consistently achieve the desirable code characteristics, coding standards should be used to specify a preferred coding style, aid understandability, apply language usage rules or restrictions, and manage complexity. The code for each unit should be verified using a static analysis tool to ensure that it complies in a timely and cost-effective manner.
Verification tools largely offer support for a range of coding standards such as MISRA C and C++, JSF++ AV, HIS, CERT C, and CWE. The better tools will be able to confirm adherence to a very high percentage of the rules dictated by each standard, whereas more lightweight tools will be unable to detect the subtler violations. More comprehensive solutions will also support the creation of and adherence to inhouse standards from both user-defined and industry standard rule sets.
IEC 62304 also states that the manufacturer shall establish strategies, methods, and procedures for verifying each software unit. Where verification is done by testing, the test procedures shall be evaluated for correctness. Among the acceptance criteria are considerations such as the verification of the proper event sequence, data and control flow, fault handling, memory management and initialization of variables, memory overflow detection, and checking of all software boundary conditions.
Unit test tools often provide a graphical user interface for unit test specification, which is used to create tests according to the defined specification and to present a list of all defined test cases with appropriate pass/fail status. The ability to automatically present the graphical presentation of control flow graphs, and to create test harnesses, stub functions, and cover for missing member or global variables means that unit test execution and interpretation becomes a much quicker and easier process, requiring a minimum of specialist knowledge. By extending the process to the automatic generation of test vectors, the tools provide a straightforward means to analyze boundary values without creating each test case manually. Test sequences and test cases are retained so that they can be repeated (regression tested), and the results compared with those generated when they were first created.
Thorough verification also requires static and dynamic data and control flow analysis. Static data flow analysis produces a cross reference table of variables, which documents their type and where they are utilized within the source files or system under test. It also provides details of data flow anomalies, procedure interface analysis, and data flow standards violations.
Dynamic data flow analysis builds on that accumulated knowledge, mapping coverage information onto each variable entry in the table for current and combined datasets and populating flow graphs to illustrate the control flow of the unit under test.
5.6 Software Integration and Integration Testing
Software integration testing focuses on the transfer of data and control across a software module’s internal interfaces and external interfaces such as those associated with medical device hardware, operating systems, and third-party software applications and libraries. This activity requires the manufacturer to plan and execute integration of software units into ever larger aggregated software items, ultimately verifying that the resulting integrated system behaves as intended.
Integration testing can also be used to demonstrate program behavior at the boundaries of its input and output domains and confirms program responses to invalid, unexpected, and special inputs. The program’s actions are revealed when given combinations of inputs or unexpected sequences of inputs are received, or when defined timing requirements are violated. The test requirements in the plan should include, as appropriate, the types of white box testing and black box testing to be performed as part of integration testing.
Testing tools need to have the capability to provide dynamic structural coverage analysis, both at system test level and at unit test level, which is a mechanism to show which parts of the code base have been exercised during testing. The coverage data derived from unit and system test can be combined to provide the most effective way of working for the particular needs of a development project. A common approach is to operate them in tandem, so that, for instance, coverage can be generated for most of the source code through a dynamic system test and complemented using unit tests to exercise defensive code and other aspects.
Testing of changed source code can again be performed using regression testing. It is advisable to validate test cases as a matter of course and perhaps automatically, to ensure that any changed code has not affected proven functionality elsewhere.
The requirements for structural coverage metrics like statement, branch, condition, procedure/function call, and data flow coverage varies depending on device classification, and all are provided by both the unit test and system test facilities within tools such as the LDRA tool suite.
5.7 Software System Testing
Testing at the system level requires the manufacturer to verify the software’s functionality by verifying that the requirements for the software have been successfully implemented. Software system testing demonstrates that the specified functionality exists in the system as it will be deployed, and that performance of the program is as specified.
Software system testing is in many ways the ultimate integration test, and comments relating to integration testing still apply here. Functional (black box) testing with no code coverage can also be performed although it might be desirable to use white box methods to more efficiently accomplish certain tests, initiate stress conditions or faults, or increase code coverage of the qualification tests.
Software system testing tests the integrated software and can be performed either in a simulated environment, on actual target hardware, or on the implemented medical device.
Software System Maintenance
These devices require maintenance when out in the field, and that also involves risks that must be tracked, managed, and mitigated in accordance with the processes and procedures laid out in the standard, as Figure 2 illustrates.
A high percentage of medical device software defects are introduced after product release, and many are related to the application of inappropriate software updates and upgrades. For that reason, the software maintenance process is considered to be as important as the software development process. Clause 6 of the IEC 62304 standard spells out details for the software maintenance process, which has much in common with the software development process.
Subclause 6.3.1 states that the manufacturer shall apply the software development process as mentioned in clause 5 (5.3, 5.4, 5.5, 5.6, 5.7, and 5.8), or an established maintenance process to implement the modifications. Either way, it is important to manage changes in accordance with subclause 7.4.1, which requires that changes be analyzed to prevent introducing any harmful causes that could contribute to a hazardous situation. Furthermore, those changes are to be assessed to determine whether additional software risk control measures are required.
Given the similarity of the required processes before and after release, and assuming successful deployment of high assurance software in the first place, it makes sense to continue to apply the same tool chain as was used in development.
Establishing a software maintenance plan requires the manufacturer to create or identify procedures for implementing maintenance activities and tasks. To implement corrective actions, control changes during maintenance, and manage the release of revised software, the manufacturer should document and resolve reported problems and requests from users, as well as manage upgrades and associated modifications to the functionality of the medical device software.
The impact of each modification can therefore vary enormously, from a minor correction to the migration of an application to a new environment or platform. In each case, the objective is to modify the released medical device software while preserving its integrity. Where the changes require new development work, they should be managed in accordance with clauses from the body of the standard as well as those specific to maintenance.
Just like the requirements specified in the development of the device, achieving a format that lends itself to bidirectional traceability helps to achieve compliance with the standard. Bigger projects, perhaps with contributors in geographically diverse locations, are likely to benefit from an application life cycle management tool such as IBM® Rational® DOORS®, Siemens® Polarion® PLM®, or more generally, similar tools offering support for standard Requirements Interchange Formats.1–3 Smaller projects can cope admirably with carefully worded Microsoft® Word® or Microsoft® Excel®documents, written to facilitate links between phases the development life cycle, whatever model is applied.
Identifying Necessary Retest
Many software modifications will require changes to the existing software functionality — perhaps with regard to additional utilities in the software. In such circumstances, it is important to ensure that any changes made or additions to the software do not adversely affect the existing code.
A requirements traceability tool such as LDRA TBmanager® helps alleviate this concern by automatically maintaining the connections between the requirements, development, and testing artifacts and activities. In the example in Figure 1, a change is proposed to the system-level requirement “Installation and configuration.” The traceability established at development time between requirements, code, and tests mean that the tool can show which parts of the code are impacted by the proposed change, as highlighted in the example.
The existing code as launched should also have undergone quality control measures such as static analysis to assess whether coding standards have been met, unit tests to confirm functionality of each code module, and dynamic coverage analysis to show that all parts of the code have been exercised.
Figure 2 shows a display from a requirements traceability tool. In this example, a system has been subject to a change request for the “Add products” requirement. Those parts of the system that are potentially affected by the change are easily identified by means of a red dot, whereas unaffected functions carry a green dot.
In the example, there is a test case file (tcf) associated with each of four low-level requirements: “add”, “display”, “too many,” and “check against not null.” Those files retain the test vectors associated with each of these low-level requirements, meaning that they can all be rerun at the touch of a button and their functionality confirmed.
Of course, some tests will require changes to reflect the new functionality dictated by the updated requirement, but in many cases, it is enough to confirm that things work as they did previously. Automating the process in this way means that doing so requires minimal effort.
Development process artifacts such as coding standards compliance reports and code coverage reports are also retained and associated with requirements, so that as the tests are rerun (regression tested), the artifacts are regenerated to go with them (see Figure 3).
A software functional safety standard such as that prescribed by IEC 62304 with its many sections, clauses, and subclauses may at first seem intimidating. However, once broken down into digestible pieces, its guiding principles offer sound guidance in the establishment of a high-quality software development process, not only leading up to initial product release but also into maintenance and beyond.
Such a process is paramount for the assurance of true reliability and quality — and above all — the safety and effectiveness of medical devices. When used with a complementary and comprehensive suite of tools for analysis and testing, it can smooth the way for development teams to work together to effectively complete large projects with confidence in their quality.
This article was written by Mark Pitchford, Technical Specialist for LDRA, Wirral, UK. For more information, visit here.