An embedded system consists of integrated computer hardware and software, with the purpose of performing a specific function. Embedded systems may also function within a larger system, and systems may be fixed function or programmable. Applications for embedded systems can be found across many walks of life – in microwave ovens, mobile phones, cars, industrial plant, medical scanners, aeroplanes, toys… the list is almost endless.
Testing plays a crucial part in the development of an embedded system. There are many motivations for “getting it right”, including brand reputation and cost of recall. The more critical the system, the stronger the motivation – and given that lives are often at stake, there is every reason to take testing seriously.
There are many validation and verification activities associated with software development. Some of these involve “static analysis” – that is, ensuring that the software source code meets certain criteria.
Embedded software unit, system and integration testing are all examples of dynamic analysis. Their focus is to ensure that the software performs in accordance with requirements when it is executed.
The process reflects the testing approach used in other engineering disciplines such that components or modules are tested, followed by subassemblies, and then assemblies. Software engineering uses its own terminology to describe those stages.
Functional testing focuses on verifying that the software functions as intended and meets its specified requirements. It typically involves testing the system’s inputs, outputs, and the behaviour of its functions.
Functional testing is crucial for ensuring the correct operation of embedded systems, which are often used in safety-critical environments such as automotive, medical devices, aerospace, and industrial applications. Testers use various techniques, including manual testing and automated testing tools such as the LDRA tool suite to perform functional testing and ensure that the embedded software meets the specified functional requirements – in short, to ensure that the system works as specified.
For embedded systems, black box testing can involve testing the overall functionality of the embedded software without delving into the specifics of the underlying hardware or software architecture.
Testers focus on input-output behaviour, response to external stimuli, and adherence to functional requirements.
In embedded systems, white box testing involves exercising and examining the internal logic and code of individual modules or components using structural coverage analysis.
It also ensures that the embedded software modules interact correctly and that they also interact properly with hardware components, while internal code paths are thoroughly tested.
Both black box and white box testing play important roles in ensuring the reliability, functionality, and performance of embedded software. The choice between these methodologies depends on the testing objectives, the level of detail required, and the testing phase in the software development life cycle.
An embedded system software unit is a block of organized, reusable code that is used to perform a single, related action. Units are given different names in different programming languages, including functions, methods, subroutines, and procedures.
Embedded system unit testing is a software development test process that isolates and tests units for compliance with requirements.
Other objectives often include exercising all paths through the code base using structural coverage analysis and demonstrating the effective handling of boundary and failure conditions.
Low-level testing is the name given to unit testing in avionics systems. The name derives from it use in demonstrating the fulfilment of low-level requirements
Regression testing is a type of software testing that aims to ensure that recent changes to a software application, such as bug fixes, new features, or code enhancements, do not negatively impact the existing functionality of the system. It involves re-running previously executed test cases to verify that the existing features still work as intended after the introduction of new code.
Embedded system integration testing involves combining units into modules and testing those modules or groups of modules. The aim of integration testing is to expose deficiencies as units are tested in combination. As for unit testing, other objectives often include exercising all paths through the code base using structural coverage analysis (illustrated below). Similar attention is also paid to the effective handling of boundary and failure conditions.
In context, while unit testing focuses on individual components in isolation, integration testing ensures that these components collaborate successfully. Traditionally integration methods test the modules or components in groups or sub-systems nd use drivers or stubs to stimulate the interactions between them.
Both unit and integration testing are crucial for achieving a robust and reliable software system. The combination of unit and integration testing contributes to a comprehensive testing strategy within the software development life cycle.
Big-bang integration testing consolidates all the modules or components of a system into a singular unit and tests them collectively. The big-bang approach skips the use of intermediate stages or stubs to imitate the behaviour of absent or incomplete modules. Typically conducted following the completion of unit testing for individual modules, big-bang integration testing occurs prior to system testing and acceptance testing.
Proponents of big-bang integration testing present it as a practical option that is quicker and more cost effective compared to more traditional integration testing approaches, eliminating the need for creating and managing drivers or stubs and avoiding a specified integration sequence.
LDRA tools support both traditional and big-bang integration testing. They also automate the creation and management of stubs, drivers, and integration sequences, reducing the overhead associated with the former.
Embedded system testing (or embedded system integration testing) is the ultimate stage of system integration testing (SIT). It brings together and tests the whole code base as a working system, with the aim of exposing deficiencies or as part of User Acceptance Testing (UAT). Other objectives often include exercising all paths through the code base using structural coverage analysis (illustrated above), testing performance, assessing usability, and load testing.
To put that into context, integration testing focuses on the interactions between integrated components, while system testing evaluates the overall functionality and compliance of the entire system. Both testing levels are essential for ensuring the reliability and correctness of a software system before it is deployed to production.
Target testing (also known as on-target testing or target-based testing) involves testing software on hardware that is identical to, or representative of, the “target” hardware to be deployed in production.
Host testing (also known as on-host testing or host-based testing) involves testing software on different hardware to that intended for use in production. That “host” environment is usually the hardware used for the development of the application.
LDRA tools facilitate target testing by leveraging Target Licence Packages (TLPs). Each TLP module provides configuration with a specified tool chain and target platform.
Host target can also be made more representative of the production environment using simulators. These model the target hardware, allowing target-dependent code components to be executed on a host machine without using the hardware reference boards.
One popular approach is to combine the advantages of host and target testing by performing initial tests on-host, and later confirming that the results of those tests remain acceptable on target.
Implementing host and target testing traditionally involves writing code for test harnesses, vectors, and stubs for both environments. Automating that process doesn’t only save time and alleviate pain-points. It also ensures that test harnesses are robust and reliable from the start, so that any issues raised are much more likely to be with the code under test.
Tools typified by the TBrun component of the LDRA tool suite are known collectively as unit testing tools for embedded systems, and for automating regression testing. But despite the name, they make it possible to test units in isolation, or to perform integration testing in embedded systems as part of a call tree (below). They are, therefore, just as much integration test tools, as unit test tools.
LDRA’s software testing tools provide facilities for the practical unit testing of embedded systems. They provide an intuitive user interface that allows the same tests to be executed in the host or target environments, and as part of unit testing or integration testing. Where code re-use is in consideration, they allow those same tests to be re-run automatically in different target environments.
Test generation (test harness, test vectors, code stubs) and result-capture support is available for a wide range of host and target platforms. Optimized instrumentation technology supports even highly constrained 8- and 16-bit microcontrollers, and through to high-performance 32- and 64-bit processors.
Website: LCSAJs (Linear Code Sequence and Jump) – the most effective code coverage testing mechanism
Technical white paper: implementing IEC 61508:2010 with the LDRA tool suite
Blog: The ISO 26262 dilemma Functional coverage and call coverage
Case Study: Functional safety with legacy software
YouTube playlist: Unit (or low level) test and the LDRA tool suite
YouTube playlist: Structural coverage and the LDRA tool suite
YouTube playlist: Target integration and the LDRA tool suite
Email: info@ldra.com
EMEA: +44 (0)151 649 9300
USA: +1 (855) 855 5372
INDIA: +91 80 4080 8707