You can capture functional snapshots in test cases for regression testing. Test cases can also identify conditions that could result in exceptions, which may result in system and application instability, security vulnerabilities (such as denial of service attacks), poor performance and application response time, and frequent down time.
Sections include:
About Automated Test Case Generation
C++test automatically generates test cases according to the parameters defined in the Test Configuration’s Generation tab. These test cases use a format similar to the popular CppUnit format.
With the default settings, C++test generates one test suite per source/header file. It can also be configured to generate one test suite per function or one test suite per source file (see Customizing Generation Options for details).
Safe stub definitions are automatically-generated to replace "dangerous" functions, which includes system I/O routines such as rmdir()
, remove()
, rename()
, etc. In addition, stubs can be automatically generated for missing function and variable definitions (see Understanding and Customizing Automated Stub Generation for details). User-defined stubs can be added as needed (see Adding and Modifying Stubs for details).
Generating Test Cases
The general procedure for test case generation is:
- Identify or create a Test Configuration with your preferred test generation settings.
- For a description of preconfigured Test Configurations, see Built-in Test Configurations.
- For details on how to create a custom Test Configuration, see the Configuring Test Configurations and Rules for Policies. Details on C++test-specific options are available at Configuring Test Configurations.
- Run the Test Configuration
For details on testing from the GUI, see Testing from the GUI.
For details on testing from the command line, see Testing from the Command Line Interface.Tip - Generating Tests from the Test Case Explorer
You can generate tests for a project directly from the Test Case Explorer (which can be opened by choosing Parasoft> Show View> Test Case Explorer). Just right-click the project node in the Test Case Explorer, then choose the desired test generation Test Configuration from the Test History or Test Using shortcut menu.
For details about the Test Case Explorer, see Exploring the C++test UI.
- Review the generated test cases.
- For details, see Reviewing Automatically-Generated Test Cases.
- (Optional) Fine-tune test generation settings as needed.
- For details, see Generation Tab Settings - Defining How Test Cases are Generated.
Customizing Generation Options
You can control a number of generation options by customizing the options in the Test Configuration’s Generation tab.
Controlling the Test Suite’s File Name, Location, and Layout
The generated test suite’s file name, location, and granularity/layout can be controlled by customizing options in the Test Configuration’s Generation> Test suite tab.
To change the default test suite output settings, first select one of the following three pre-defined output and layout options from the Test suite output file and layout box:
- Create one test suite per function
- Create one test suite per source/header file
- Create one test suite per source file
If the Create one test suite per function option is selected, a test suite generated for the sample ATM project (included in the examples directory) would look like this:
After selecting one of these options, you can customize the pattern as needed (for instance, to generate tests into the source location). You can use the following variables when you are customizing the pattern:
- ${test_ext} - C++test-specific extension of a test suite file (.cpp).
- ${file_name-} File name.
- ${file_base_name-} File name without extension.
- ${file_ext-} File extension.
- ${file_loc-} File location.
- ${file_loc_rel-} File location relative to the project root.
- ${file_uid-} File unique identifier.
- ${function_name-} Tested function name.
- ${function_uid-} Tested function unique identifier (hash-code computed from the function signature/mangled name).
- ${src_file_name-} Name of context (source) file. (A "context file" is a source file that describes the compilation unit in which the given tested function is defined).
- ${src_file_base_name-} Name of context (source) file without extension.
- ${src_file_ext-} Extension of context (source) file.
- ${src_file_loc-} Context (source) file location.
- ${src_file_loc_rel-} Context (source) file location relative to the project root.
- ${src_file_uid-} Source (context) Context file unique identifier (hash-code computed from the source file location).
Warning
Removing some variables can lead to overlapping test suites. C++test will alert you to this by displaying the error message "Test suite output file pattern is ambiguous."
C++test uses the following internal checks and restrictions related to changing the basic patterns for location of generated tests:
- All automatically generated tests are of "included" type (the test suite file is "glued "together with the given source file/compilation unit).
- It tries to prevent cases where test cases for functions from different compilation units would be placed in the same test suite file (because it is not possible to correctly glue such a test suite file with original source file).
- C++test has different variables (which are resolved based on the file under test, its name, its location etc.) that can be used to make the test suite file pattern unambiguous (in terms of the item above).
- One commonly-used strategy is to generate a test suite file into a file/location that has the original file name/location in it. This the default pattern:
${project_loc}/tests/autogenerated/${file_loc_rel}/
TestSuite_${file_base_name}_${file_ext}.${test_ext}
This is not ambiguous because ${file_loc_rel} and ${file_base_name} variables are used (even though there are a number of files with the same name in the project, their location will be different—and that location will be a part of the test suite file name/location). - There are also other available variables—for example, ${file_uid}, ${src_file_uid}—that can be used instead of the ${file_loc_rel} / ${file_base_name} pair while keeping the pattern unambiguous. These variables are resolved into a hash code of the original file location. For example, a pattern like
${project_loc}/tests/autogenerated/
TestSuite_${file_base_name}_${file_uid}_${file_ext}.${test_ext}
will result in the following test suite:
ATM/tests/autogenerated/TestSuite_Account_d7a5efc6_hxx.cpp
Appending or Replacing Existing Tests
You can also control whether C++test will append or replace existing tests if the generated test file has the same name and location as an existing test suite file. This behavior is determined by the When generating tests for code with an existing test suite setting, which provides the following options:
- Add tests for functions without tests: C++test will generate test cases for functions without tests. The existing tests will not be affected or modified.
- Add tests for all functions: C++test will generate test cases for all functions. The existing tests will not be affected or modified.
- Replace the existing test suite: C++test will generate test cases for all functions. The existing test suite will be removed and then replaced with the new one.
Common Test Generation Goals
The following table explains how to configure the Test Configuration’s generation options to accomplish common test generation goals. Options covered include the Generation> General tab’s Generate tests for code option and the Generation> Test suite tab’s When generating tests for a code with an existing test suite option.
Goal | Settings |
---|---|
To generate an initial set of tests | For Generate tests for code, enable Without test suites. Specify additional parameters (function access level, output file location/name etc.). |
To update an existing automatically-generated test suites with tests for new functions (do not generate new test suites) | For Generate tests for code, enable With up-to-date test suites With out-of-date test suites. For When generating tests for a code with an existing test suite, enable Add tests for functions without tests. Specify additional parameters (function access level, output file location/name etc.). |
To synchronize automatically-generated tests with the current code - append missing tests, create missing test suites | For Generate tests for code, enable Without test suites, With up-to-date test suites, and With out-of-date test suites. For When generating tests for a code with an existing test suite, enable Add tests for functions without tests. Specify additional parameters (function access level, output file location/name etc.). |
To fully reset existing automatically-generated tests | For Generate tests for code, enable Without test suites, With up-to-date test suites, and With out-of-date test suites. For When generating tests for a code with an existing test suite, enable Replace the existing test suite. Specify additional parameters (function access level, output file location/name etc.). |
Choosing the Layout Option That Suit Your Goals
This section explains how to configure the Test suite output file and layout option (in the Test Configuration’s Generation> Test suite tab) to suit various layout needs. To help you understand how each option discussed translates to actual projects, we show how it would affect the following sample project:
MyProject headers MyClass.h // contains foo() definition sources MyClass.cpp // contains bar() and goo() definitions
To generate a single test suite file for each function, keep tests in a separate directory
Use ${project_loc}/tests/${file_loc_rel}/${file_name}/
TestSuite_${function_name}.${test_ext}
Sample layout:
MyProject headers MyClass.h sources MyClass.cpp tests headers MyClass.h TestSuite_foo.cpp // contains tests for foo() sources MyClass.cpp TestSuite_bar.cpp // contains tests for bar() TestSuite_goo.cpp // contains tests for goo()
${project_loc}/tests/${file_loc_rel}/${file_name}/
TestSuite_${function_name}.${test_ext}
Sample layout:
MyProject Header Files MyClass.h Source Files MyClass.cpp tests Header Files MyClass.h TestSuite_foo.cpp // contains tests for foo() Source Files MyClass.cpp TestSuite_bar.cpp // contains tests for bar() TestSuite_goo.cpp // contains tests for goo()
To generate a single test suite file for each source/header file, keep tests in a separate directory
Use ${project_loc}/tests/${file_loc_rel}/
TestSuite_${file_base_name}_${file_ext}.${test_ext}
Sample layout:
MyProject headers MyClass.h sources MyClass.cpp tests headers TestSuite_MyClass_h.cpp // contains tests for foo() sources TestSuite_MyClass_cpp.cpp // contains tests for bar() and goo()
To generate a single test suite file for each source/header file, keep tests with the tested files
Use ${project_loc}/${file_loc_rel}/tests/
TestSuite_${file_base_name}_${file_ext}.${test_ext}
Sample layout:
MyProject headers MyClass.h tests TestSuite_MyClass_h.cpp // contains tests for foo() sources MyClass.cpp tests TestSuite_MyClass_cpp.cpp // contains tests for bar() and goo()
To keep auto-generated test suite files with original source files
Use ${project_loc}/tests/${file_loc_rel}/${file_name}/
TestSuite_${function_name}.${test_ext}
Sample layout:
MyProject module1 sources MyClass.cpp tests TestSuite_MyClass_cpp.cpp headers MyClass.h tests TestSuite_MyClass_h.cpp
To keep the auto-generated test files in an intuitive structure (outside of my original files)
Use ${project}/tests/${source_loc_rel:/MyProject/module1}/
TestSuite_${source_base_name}_${source_ext}.${test_ext}
Sample layout:
MyProject module1 sources MyClass.cpp headers MyClass.h tests TestSuite_MyClass_cpp.cpp headers TestSuite_MyClass_h.cpp
To keep test files flat in a single directory (if the project does not have duplicate names)
Use ${project}/tests/TestSuite_${source_base_name}_${source_ext}.${test_ext}
Sample layout:
MyProject module1 sources MyClass.cpp headers MyClass.h tests TestSuite_MyClass_cpp.cpp TestSuite_MyClass_h.cpp
To use "One test suite per function" mode
Use ${project}/tests/${source_base_name}_${source_ext}/
TestSuite_${function_name}_${function_uid}.${test_ext}
Sample layout:
MyProject module1 sources MyClass.cpp headers MyClass.h tests MyClass_cpp TestSuite_foo_1234abcd.cpp TestSuite_foo_4321abcd.cpp TestSuite_goo_4321dcba.cpp MyClass_h TestSuite_bar_2143badc.cpp