Versions Compared

Old Version 1

changes.mady.by.user Izabela Jarosz

Saved on

compared with

New Version Current

changes.mady.by.user Izabela Jarosz

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • <INSTALL_DIR>/cpptestcc - A coverage tool that integrates into your build process to instrument your application to collect raw coverage data.
  • <INSTALL_DIR>/engine/coverage/runtime - A coverage runtime library that needs to be integrated with the instrumented application.

Collecting coverage with cpptestcc involves three phases:


  1. Instrumenting the application by integrating the cpptestcc tool into your build.
  2. Executing instrumented code and collecting raw coverage data.
  3. Reviewing the coverage with C/C++test by importing the raw coverage data into C/C++test with a built-in test configuration.

...

  1. Add the path to <INSTALL_DIR> the to the PATH system variable to enable execution of the cpptestcc tool.

  2. Update your compilation command to include the cpptestcc executable as a prefix to the compiler command using -- as a separator.  For example:

    Original compilation command line

    Code Block
    cc -I app/includes -D defines -c source.cpp

    Updated compilation command line

    Code Block
    cpptestcc -compiler gcc_9-64 -line-coverage -- cc -I app/includes -D defines -c source.cpp
    Info
    At a minimum, the cpptestcc tool requires the following parameters to be configured on the command line:

    - a compiler identifier: -compiler <COMPILER_ID>

    - a coverage metric (for example, -decision-coverage)

    See Command Line Reference for cpptestcc for information about other options.

  3. Update your linker command with the path to the pre-built coverage runtime library shipped with C/C++test to add the library to your application. For example:

    Original command line

    Code Block
    lxx -L app/lib app/source.o somelib.lib -o app.exe

    Updated command line

    Code Block
    lxx -L app/lib app/source.o somelib.lib [INSTALL_DIR]/engine/coverage/runtime/lib/cpptest.lib -o app.exe
    Info
    iconfalse
    titleImportant

    If the coverage runtime library is linked as a shared (dynamic-load) library, you must ensure that it can be loaded when the instrumented application is started. This typically requires adding <INSTALL_DIR>/engine/coverage/runtime/bin to the PATH environment variable (on Windows) or <INSTALL_DIR>/engine/coverage/runtime/lib  to the  LD_LIBRARY_PATH environment variable (on Linux).

    Info
    iconfalse

    C/C++test provides the pre-built coverage runtime library for native Windows and Linux applications. For cross-platform and embedded testing, the runtime library needs to built from sources that are available in <INSTALL_DIR>/engine/coverage/runtime. See Coverage Runtime Library for details.

  4. Build the application. When instrumenting the code, cpptestcc creates the .cpptest/cpptestcc folder where important coverage-related data ("coverage maps") are stored. By default, the folder is located in the working directory of the current compilation. You can change the default location using the -workspace <path> option;  see Command Line Reference for cpptestcc for details. 
  5. Run the application. The coverage data ("coverage log") will be stored it the cpptest_results.clog file.
  6. In your IDE where C/C++test is installed, create a new project that includes all the source files of the application.
    (info) Ensure that the files and all the paths remain unchanged.
  7. Select the project and choose Parasoft> Test Configurations> Utilities> Load Application Coverage from your IDE menu to import the coverage data (see Importing the Coverage Data for details).
  8. Review the coverage information (see Reviewing Coverage Information).

...

C/C++test includes a Python script launcher to run the included tools, which can be found at:

Code Block
<CPPTEST_INSTALL_DIR>/engine/bin/cpptestpy.exe

The MSBuild integration script is in the following location:

...

    1. Setup the CPPTEST_INSTALL_DIR variable.

      Code Block
      set CPPTEST_HOME=<CPPTEST_INSTALL_DIR>

    2. Setup the PATH variable.

      Code Block
      set PATH=%PATH%;%CPPTEST_HOME\engine\coverage\runtime\bin

  2. (CMake only) Generate your MSBuild files using CMake. 

    Code Block
    cd <PROJECT_DIR>
mkdir build
cd build
cmake ..

  3. Run the integration script. 

    Code Block
    <cpptestpy.exe> <msbuild_cpptest.py> -c -f project.vcxproj

  4. Build the project using MSBuild.

    Code Block
    msbuild -t:Rebuild project.sln

  5.  To collect coverage, go to the build directory and run the project.

    Code Block
    cd x64\Debug
project.exe

    A .clog file that contains coverage information will be generated in your current working directory. 
    The default name of the file is cpptest_results.clog.

  6. Generate the coverage report.

    Code Block
    cpptestcli.exe \
-solution Sensor.sln \
-config "builtin://Load Application Coverage"

Anchor
Command Line Reference for cpptestcc
Command Line Reference for cpptestcc
Command Line Reference for cpptestcc

...

Table of Content Zone
maxLevel2
minLevel2
locationtop

-compiler <name|path>

Specifies the name of the compiler configuration you want to use for code analysis and instrumentation. See Compilers for the list of supported compilers or use the -list-compilers command line option to print out the list of supported compilers to the console.

Examples:

cpptestcc -compiler gcc_9-64 

cpptestcc -compiler vc_11_0

Configuration file format (see -psrc): cpptestcc.compiler <name>

-list-compilers

Prints out the names of all supported compiler configurations.

Configuration file format (see -psrc): cpptestcc.listCompilers

-include <file|pattern> and -exclude <file|pattern>

Includes into or excludes from the instrumentation scope all the file(s) that match the specified pattern.

Final filtering is determined only after all include/exclude entries have been specified in the order of their specification.

The following wildcards are supported:

  • ? - Any character
  • * - Any sequence of characters

To prevent shells from expanding * wildcards to the list of files or directories, you can use the regex: prefix to specify the value.

(info) These options can be specified multiple times.

Configuration file format (see -psrc): cpptestcc.include <path|pattern>

Example 1:

Sample project layout:

Code Block
<project root>
 + external_libs
 + src
 + include

If your project has the above layout, the following command will exclude all the files in the external_libs directory from instrumentation scope:

cpptestcc -include regex:*/<project root>/* -exclude regex:*/<project root>/external_libs <other command line options>

Example 2:

Sample project layout:

Code Block
<project root> 
<sourcefiles>.cpp 
<headerfiles>.hpp

If your project has the above layout, the following command will only instrument the header files (the source files will not be instrumented):

cpptestcc -include regex:* -exclude regex:*.cpp <remaining part of cmd>

-ignore <pattern>

Specifies the source files that will be ignored during processing. The files that match the specified pattern will be compiled, but they will not be parsed or instrumented.

Info
title-ignore vs. -exclude

The -ignore option completely removes the specified file from processing so that it is not parsed by the coverage engine.

The -include/-exclude filters are applied after source code is parsed, which allows you to selectively instrument or not instrument header files.

You can use the -ignore option to reduce build time overhead by ignoring coverage analysis on some sections of the code (such as external libraries) or to ignore specific file that expose parse errors or other problems during processing.

The following wildcards are supported:

  • ? - Any character
  • * - Any sequence of characters

To prevent shells from expanding * wildcards to the list of files or directories, you can use the regex: prefix to specify the value.

(info) This option can be specified multiple times.

Configuration file format (see -psrc): cpptestcc.ignore <path|pattern>

Examples:

cpptestcc -ignore "*/Lib/*" <remaining part of cmd>
cpptestcc -ignore regex:*/file.c <remaining part of cmd>
cpptestcc -ignore c:/proj/file.c <remaining part of cmd>
cpptestcc -ignore "*/MyLib/*.cpp" -ignore file.cpp <remaining part of cmd>

-line-coverage

Enables collecting line coverage.

Runtime coverage results are being written to the results log as the code is executed. This imposes some overhead on the tested code execution time, it but it allows you to ensure that that coverage data is collected even if the application crashes.

Configuration file format (see -psrc): cpptestcc.lineCoverage [true|false]

-optimized-line-coverage

Enables collecting optimized line coverage.

Runtime coverage results are stored in memory and then written to the results log either after the application finishes or on user request. This results in better performance, but results may be lost if the application crashes.

Configuration file format (see -psrc): cpptestcc.optimizedLineCoverage [true|false]

-function-coverage

Enables collecting function coverage.

Configuration file format (see -psrc): cpptestcc.functionCoverage [true|false]

-optimized-function-coverage

Enables collecting optimized function coverage. Configuration file format (see -psrc):

cpptestcc.optimizedFunctionCoverage [true|false]

-statement-coverage

Enables collecting statement coverage. Configuration file format (see -psrc):

cpptestcc.statementCoverage [true|false]

-optimized-statement-coverage

Enables collecting statement coverage.

Configuration file format (see -psrc): cpptestcc.optimizedStatementCoverage [true|false]

-block-coverage

Enables collecting block coverage.

Configuration file format (see -psrc): cpptestcc.blockCoverage [true|false]

-optimized-block-coverage

Enables collecting optimized block coverage.

Configuration file format (see -psrc): cpptestcc.optimizedBlockCoverage [true|false]

-path-coverage

Enables collecting path coverage.

Configuration file format (see -psrc): cpptestcc.pathCoverage [true|false]

-decision-coverage

Enables collecting decision coverage.

Configuration file format (see -psrc): cpptestcc.decisionCoverage [true|false]

-optimized-decision-coverage

Enables collecting optimized decision coverage.

Configuration file format (see -psrc): cpptestcc.optimizedDecisionCoverage [true|false]

-simple-condition-coverage

Enables collecting simple condition coverage.

Configuration file format (see -psrc): cpptestcc.simpleConditionCoverage [true|false]

-optimized-simple-condition-coverage

Enables collecting optimized simple condition coverage.

Configuration file format (see -psrc): cpptestcc.optimizedSimpleConditionCoverage [true|false]

-mcdc-coverage

Enables collecting MC/DC coverage.

Configuration file format (see -psrc): cpptestcc.mcdcCoverage [true|false]

-call-coverage

Enables collecting call coverage.

Configuration file format (see -psrc): cpptestcc.callCoverage [true|false]

-optimized-call-coverage

Enables collecting optimized call coverage.

Configuration file format (see -psrc): cpptestcc.optimizedCallCoverage [true|false]

-coverage-early-init

Enables initializing the coverage module at the beginning of the application entry point.

Configuration file format (see -psrc): cpptestcc.coverageEarlyInit [true|false]

-coverage-auto-finalization

If enabled, collecting coverage will be automatically finalized at the application exit. This option is enabled by default.

Configuration file format (see -psrc): cpptestcc.coverageAutoFinalization [true|false]

-optimized-coverage-corruption-detection

Enables corruption detection algorithms for optimized coverage metrics.

Configuration file format (see -psrc): cpptestcc.optimizedCoverageCorruptionDetection [true|false]

-template-coverage

Enables collecting coverage for template classes and functions.

Configuration file format (see -psrc): cpptestcc.templateCoverage [true|false]

Anchor
constexpr coverage
constexpr coverage
-constexpr-coverage

Enables collecting coverage for constexpr functions.

Note: This feature is available for selected compilers only. See Support for C++ Constant Expressions for details.

Configuration file format (see -psrc): cpptestcc.constexprCoverage [true|false]

-coverage-data-variants

Enables storing multiple variants of the coverage data for different compilations of the same source file.

By default, cpptestcc collects coverage data for each source file once per build. In rare scenarios when a source file is compiled more than once during a build, you can use this option to enable storing multiple variants of coverage data for the same source file – one variant for each compilation.

This option does not support scenarios where conflicting code is introduced by multiple compilations of the same source file.

-disable-auto-recovery-mode

Disables the auto recovery mode for coverage instrumentation.

By default, when a file cannot be successfully instrumented, the original (non-instrumented) version of that file is used in a build. With this option specified, the build is failed with an error if instrumentation problems occur.

Anchor
-workspace
-workspace
-workspace <path>

Specifies a custom directory where information about code structure will be stored during code analysis and instrumentation. The cpptestcli tool will use the information to generate the final coverage report.

By default, the information is stored in the working directory of the current compilation. If your compilation uses more than one working directory, we recommend that you specify a custom directory to ensure that all coverage data is stored in the same location.

Configuration file format (see -psrc): cpptestcc.workspace <path>

-techsupport

 Creates an archive with diagnostic files collected from the workspace (see -workspace). The archive will be stored in the current working directory.

Examples:

cpptestcc -techsupport

cpptestcc -techsupport -workspace path/to/workspace-dir

Anchor
psrc
psrc
-psrc <file>

Specifies the path to a configuration file where you can configure additional cpptestcc options.

By default, cpptestcc attempts to read the.psrc file located in either the current working directory or in the user HOME directory. This option allows you to specify a custom location of the configuration file.

(info) If an option is configured both in the command line and in the configuration file, cpptestcc will use the value specified in the command line.

-status

-status-verbose

Displays diagnostic data collected from the workspace (see -workspace), including error and warning information, the number of files, etc.

Use -status-verbose to display more detailed information.

Examples:

cpptestcc -status

cpptestcc -status -workspace path/to/workspace-dir

cpptestcc -status-verbose

cpptestcc -status-verbose -workspace path/to/workspace-dir

-version

Prints out information about the version

-help

Prints out the help message and exits.

...

FileDescription
<INSTALL_DIR>/engine/coverage/runtime/lib/cpptest.a32-bit import library to be used with Cygwin GNU GCC compilers. To be added to linking command line.
<INSTALL_DIR>/engine/coverage/runtime/lib/cpptest64.a64-bit import library to be used with Cygwin GNU GCC compilers. To be added to linking command line.
<INSTALL_DIR>/engine/coverage/runtime/lib/cpptest.lib32-bit import library to be used with Microsoft Visual C++ compilers. To be added to linking command line.
<INSTALL_DIR>/engine/coverage/runtime/lib/cpptest64.lib64-bit import library to be used with Microsoft Visual C++ compilers. To be added to linking command line.
<INSTALL_DIR>/engine/coverage/runtime/bin/cpptest.dll32-bit dynamic-link library. <INSTALL_DIR>/engine/coverage/runtime/bin should be added to PATH environmental variable.
<INSTALL_DIR>/engine/coverage/runtime/bin/cpptest64.dll64-bit dynamic-link library. <INSTALL_DIR>/engine/coverage/runtime/bin should be added to PATH environmental variable.

Linux (x86 and x86-64)

FileDescription
<INSTALL_DIR>/engine/coverage/runtime/lib/libcpptest.so32-bit shared library. To be added linking command line. <INSTALL_DIR>/engine/coverage/runtime/lib should be added to LD_LIBRARY_PATH
<INSTALL_DIR>/engine/coverage/runtime/lib/libcpptest64.so64 bit shared library. To be added linking command line. <INSTALL_DIR>/engine/coverage/runtime/lib should be added to LD_LIBRARY_PATH

...

  1. Locate the linker command line in your build scripts

  2. Modify the build scripts so that the coverage runtime library is specified somewhere in the linker command line - preferably after all object files. For example:

    Code Block
    $(LXX) $(PRODUCT_OBJ) $(OFLAG_EXE)$(PROJ_EXECUTABLE) $(LXXFLAGS) $(SYSLIB) $(EXECUTABLE_LIB_LXX_OPTS) <INSTALL_DIR>/engine/coverage/runtime/lib/cpptest.lib

  3. Ensure that the path to the lib directory is added to the PATH environment variable so that the library can be located when the tested program is started. You may also consider copying cpptest.dll (or cpptest64.dll) file to the same directory as your executable file or to another location that is scanned for dynamic-link libraries during tested application startup.

...

  1. Locate the linker command line in your build scripts

  2. Modify the build scripts so that the coverage runtime library is specified somewhere in the linker command line - preferably after all object files. For example:

    Code Block
    $(LXX) $(PRODUCT_OBJ) $(OFLAG_EXE)$(PROJ_EXECUTABLE) $(LXXFLAGS) $(SYSLIB) $(EXECUTABLE_LIB_LXX_OPTS) -L <INSTALL_DIR>/engine/coverage/runtime/lib -lcpptest

    Note that the -L and -lcpptest options are added.

  3. Ensure that the path to the lib directory is added to the LD_LIBRARY_PATH environmental variable to allow the tested executable to find the path to the shared library.

...