Test Verification

Overview

Test Verification can be used by the Unit Test component to provide coverage metrics for test cases that execute at various levels, including function/method, module/class and component/system levels. Test Verification provides test planning documents and colour-coded graphs to help you achieve the desired coverage, ranging from procedure/function calls to safety-critical modified condition/ decision coverage.

Test Verification also supports the verification of systems running customer provided test cases (outside of the Unit Test Component). These system tests can execute within LDRA Testbed, in a simulator or some other host-based environment or on a target platform.

For users interested in LDRA’s Zero Defect Software Development methodology, Test Verification provides feasibility coverage or the percentage of feasible paths executed by a test case.

Features

Coverage Metrics

LDRA Testbed provides the following industry standard coverage metrics:

• Statement
• Branch/Decision
• Procedure/Function Call

LDRA also provides feasibility metrics to support its Feasibility Testing methodology.

TBsafe adds:

• Branch Decision Condition
• Branch Condition Combination
• Modified Condition / Decision Coverage (MC/DC for DO-178B Level A)
• Dynamic Data Flow

Test Planning

Test Verification produces two test case planning documents:

• Feasibility Test Case Planner - Shows the source code associated with the branch points within uncovered feasible paths (also called LCSAJ’s) together with the logical condition necessary to execute that branch.
• MC/DC Test Case Planner - Details the steps necessary to achieve DO-178B Level A, Modified Condition/Decision Coverage.

The LDRA tool suite enables visualisation of your system by:

• Callgraphs provide the ability to drill down to procedure/functions, graphically display procedure/function call coverage and dynamic callgraphs allow visualisation of test coverage.
• Flowgraphs provide annotations which display decision criteria, as well as graphical displays of statement and branch coverage

Reports

The automatically generated reports can be configured for pass/fail coverage levels, in a procedure by procedure layout or file layout. The details available are:

• Annotated source code listings
• System overview reports
• Coverage metrics provided for:
  
  - Procedures
  - Files
  - Systems

Data Set/Profile Analysis

Data Set Analysis can be used in two ways. The first is to obtain information on which data sets actually executed each line or set of lines of the source code. This is valuable information for regression testing.

The second use is to find which lines of code were executed by any particular test data sets. This can be used for documentation purposes, or for obtaining inspiration on how to execute a particular line of code.

Profile Analysis

Profile Analysis obtains a minimal set of test data sets which yield the greatest coverage. This minimal set can then be used to make regression testing more efficient, cutting time and costs.

Benefits

• Effectively manage and measure code verification progress and gauge remaining effort
• Improve confidence in code reliability and robustness
• Quickly pinpoint inadequately tested software
• Reduce regression testing costs
• Improve code efficiency
• Test to a high standard
• Consequence: Resource savings, increased product quality & shortened time-to-market

Deliverable

The key deliverable is the measurement of testing effectiveness, which details the statement, branch and test path coverage levels within the software.