The automated test case generation provided by TBextreme and TBextremePLUS offers numerous advantages. Automation produces comprehensive test patterns that ensure extensive code coverage, identifying more potential issues, reducing the time and cost associated with manual test case creation, and accelerating the testing process. As a result, it frees up human resources to concentrate on more complex testing tasks and analysis, minimizes the potential for human error, and delivers consistent, repeatable test results for routine test work.
In general, extreme testing is concerned with pushing the limits of testing to ensure the highest possible quality and reliability of the software.
In the LDRA context, the term “extreme testing” refers to the ability of LDRA’s unit test tools to create a sequence of test cases (“extreme tests”) and automatically creating test vectors for each of them.
There is form of extreme testing (eXtreme Testing, or XT) that complements the principles and practices of eXtreme Programming (XP). It involves testing software or products beyond their normal operating conditions to identify their breaking points. Despite the commonalities between the techniques, the LDRA implementation has no association with XP.
Automatic test case generation (aka automatic test case vector generation) is the practice of generating test cases for a software function, file, or application based on minimal user specified testing parameters and configuration.
Automatic test case generation is not a replacement for Requirements-Based Testing but is complementary to it. Additionally, automatically generated test cases can be useful for assessing software robustness and uncovering cybersecurity weaknesses and vulnerabilities.
Robustness testing is a technique designed to evaluate how well a system can handle invalid inputs and stressful conditions without failing. It ensures that the software can function correctly under unexpected or adverse scenarios, such as high load, limited resources, or incorrect data.
In the context of unit and integration testing, fuzz testing involves feeding unexpected or malformed data into individual units or subsystems to uncover vulnerabilities and bugs. In the former case it is used to ensure that individual functions handle invalid inputs gracefully. In integration testing, it provides a means to examine how gracefully different components interact under unexpected conditions.
Equivalence class (or partitioning) testing is a technique used to reduce the number of test cases for a software module, especially for calculation-intensive modules with many variables. ECT defines a model of a software module that aggregates the input and output values into partitions that are functionally equivalent according to the requirements of the module’s behaviour, such that each partition can be tested by a single test case. ECT can be applied to all levels of software testing including unit, integration, and system testing.
Boundary Value Analysis (BVA) is a test case design technique used to identify errors at the boundaries of input values rather than within the permissible range of those values. Boundary Value Testing (BVT) is the process of executing the test cases derived from BVA.
BVT reduces the number of test cases for a software module by focusing on critical boundaries where errors are more likely to be found. BVT is often combined with ECT to test partitions at their minimum and maximum boundaries, in addition to testing at the middle of the boundary (which is often the only test case when ECT is used alone). BVT can be applied to all levels of software testing including unit, integration, and system testing.
There are many reasons why ECT and BVT are important for standards compliance. Functional safety standards such as DO-178C, ISO 26262 and IEC 61508 all have requirements to perform Requirements-Based Testing (RBT). The goal of RBT is to ensure that software functions exactly as specified by the requirements. In many cases, exhaustive testing of software to test all the possible combinations of inputs and outputs is impossible, so applying techniques like ECT are essential for meeting RBT requirements. In addition to RBT, these standards have requirements for ensuring software is robust. To address this challenge, the technique of BVT can be applied to ensure many of the common sources of software defects – including those that are common cybersecurity weaknesses and vulnerabilities – are exposed with additional test cases.
Test case profiling is a technique used to analyse and optimize test cases by examining their characteristics and performance to reduce redundancy and improve coverage. Additionally, it prioritizes test cases based on their importance and impact, ensuring that the most critical tests are executed first.
Automatic test case generation can complement unit testing in a few ways. It can quickly generate a set of test cases that users can review and link to requirements, reducing the time needed to generate a set of RBTs. Additionally, automatic test case generation can be used to create robustness tests – as is the case in applying ECT and BVT. Another common use case is to utilize automatic test case generation to create test cases that provide additional structural code coverage beyond what an initial set of RBTs provides. In all cases the automatically generated test cases will need to be linked to requirements – if an appropriate requirement for a test case does not exist, then it will need to be added. Together with manually created test cases, automatically generated test cases are part of the full set of test cases used to perform unit, integration and system testing as well as regression testing.
TBextreme is an optional module available for use with the LDRA tool suite and LDRAunit. The TBextreme module revolutionizes unit testing by enabling a totally automated solution for test case generation called ”extreme testing”.
Through an easy-to-use interface, associated test case generation parameters can be tuned to meet a broad set of testing needs.
These parameters vary the nature of those cases to suit the task in hand.
TBextreme provides two types of extreme test. Standard extreme tests create one test per function in the code under test, whereas tabular extreme tests create several tests per function.
Both have several uses. For example, standard extreme tests can be used to provide test templates for more manual test process, or to demonstrate the effectiveness of parameter settings by performing tabular extreme tests. Tabular extreme tests are sometimes deployed with the intention of maximising code coverage, or to test code robustness.
TBextreme can help maximize code coverage for quick and effective structural coverage analysis where requirements traceability is not a priority.
TBextreme automates robustness testing, ensuring that a system can handle unexpected inputs and stressful conditions such as permissible and inadmissible boundary values without failing.
TBextreme automates fuzz testing by feeding unexpected or malformed data into the code under test via the test tool to uncover vulnerabilities and bugs.
Boundary Value Analysis (BVA) is a test case design technique used to identify errors at the boundaries of input values rather than within the permissible range of those values. Boundary Value Testing (BVT) is the process of executing the test cases derived from BVA.
TBextreme automates both Boundary Value Analysis (BVA) and Boundary Value Testing (BVT). It generates and executes tests that target the extreme ends of input ranges, including minimum and maximum values, as well as values just inside and just outside the boundaries.
TBextreme can be used to demonstrate the absence of unexpected functionality changes resulting from legacy code modifications.
TBextremePLUS is an enhancement to TBextreme that extends its capabilities, both by providing more user configured parameters, and by allowing combinations of these and standard TBextreme parameters to be stored and associated with types of tasks. Users of TBextremePLUS can call upon predefined combinations of parameters or store their own combinations, making switching between modes easier.
TBextremePLUS introduces a third extreme test type, called linear extreme test. Linear extreme tests create a predefined number of tests per variable handled by the pertinent function under test. This test type provides a more focused approached to the achievement of thorough test code coverage which minimizes execution time and so optimizes the process.
TBextremePLUS extends TBextreme robustness testing through the application of additional requirements-based values in the extreme tests.
Requirements Based Testing (RBT) is achieved through the application of user defined test values as dictated by project requirements which are leveraged by TBextremePLUS to create appropriate test cases.
Equivalence class testing (a.k.a. equivalence testing or Equivalence Partitioning, EP) is achieved by means of user defined test data. This is leveraged by TBextremePLUS to create equivalence classes based on requirements.
TBextremePLUS automates both Boundary Value Analysis (BVA) and Boundary Value Testing (BVT). It extends the functionality of TBextreme by offering preconfigured options to include or exclude pointers with NULL values.
Test case profiling analyses and optimizes test cases to reduce redundancy and improve coverage. It is automated within TBextremePLUS by means of a code coverage configuration option.
Video: Accelerate Requirements-Based Testing (RBT) with TBextremePLUS
Video: Introducing TBextremePLUS
Data Sheet : TBextreme with TBextremePLUS
Video: Automatic generation of test cases with TBextreme
Video: Robustness testing for safety and security
Email: info@ldra.com
EMEA: +44 (0)151 649 9300
USA: +1 (855) 855 5372
INDIA: +91 80 4080 8707