Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Collecting coverage with cpptestcc involves three phases:

Make sure that shared library can be found by tested executable by modifying LD_LIBRARY_PATH environmental variable to


  1. Instrumenting the application by integrating the cpptestcc tool into your build.
  2. Executing instrumented code and collecting raw coverage data.
  3. Reviewing the coverage with C/C++test by importing the raw coverage data into C/C++test with a built-in test configuration.

Quick Start with cpptestcc

  1. Add the path

    to the

    to <INSTALL_DIR>/bin the to the PATH system variable to enable execution of the cpptestcc tool.

  2. Update your compilation command to include the cpptestcc executable as a prefix to the compiler command using -- as a separator.  For example:

    Original compilation command line

    Code Block
    cc -I app/includes -D defines -c source.cpp

    Updated compilation command line

    Code Block
    cpptestcc -compiler gcc_7 -line-coverage -- cc -I app/includes -D defines -c source.cpp
    Info
    At a minimum, the cpptestcc tool requires the following parameters to be configured on the command line:

    - a compiler identifier: -compiler <COMPILER_ID>

    - a coverage metric (for example, -decision-coverage)

    See Command Line Reference for cpptestcc for information about other options.

  3. Update your linker command with the path to the pre-built coverage runtime library shipped with C/C++test to add the library to your application. For example:

    Original command line

    Code Block
    lxx -L app/lib app/source.o somelib.lib -o app.exe

    Updated command line

    Code Block
    lxx -L app/lib app/source.o somelib.lib [INSTALL_DIR]/bin/engine/coverage/runtime/lib/cpptest.lib -o app.exe
    Info
    iconfalse
    titleImportant

    If the coverage runtime library is linked as a shared (dynamic-load) library, you must ensure that it can be loaded when the instrumented application is started. This typically requires adding adding <INSTALL_DIR>/bin/engine/coverage/runtime/bin to  to the PATH environment variable (on Windows) or <INSTALL_DIR>/bin/engine/coverage/runtime/lib  to the  LD_LIBRARY_PATH environment variable (on Linux).

    Info
    iconfalse

    C/C++test provides the pre-built coverage runtime library for native Windows and Linux applications. For cross-platform and embedded testing, the runtime library needs to built from sources that are available

    in

    in <INSTALL_DIR>/bin/engine/coverage/runtime.

    See

     See Coverage Runtime Library for details.

  4. Build the application. When instrumenting the code, cpptestcc creates the .cpptest/cpptestcc folder where important coverage-related data ("coverage maps") are stored. By default, the folder is located in the working directory of the current compilation. You can change the default location using the -workspace <path> option;  see Command Line Reference for cpptestcc for details. 
  5. Run the application. The coverage data ("coverage log") will be stored it the cpptest_results.clog file.
  6. In your IDE where C/C++test is installed, create a new project that includes all the source files of the application.
    (info) Ensure that the files and all the paths remain unchanged.
  7. Select the project and choose Parasoft> Test Configurations> Utilities> Load Application Coverage from your IDE menu to import the coverage data (see Importing the Coverage Data for details).
  8. Review the coverage information (see Reviewing Coverage Information).

...

We recommend that you keep consistency with the metrics enabled for the cpptestcc tool in the compilation command line.

Anchor
cmake_coverage_extension
cmake_coverage_extension
Collecting Application Coverage for CMake Projects

C/C++test ships with an extension for CMake that allows you to integrate C/C++test's code coverage analysis directly into your CMake project. The extension automatically modifies your compiler/linker command lines to use the cpptestcc coverage tool when building your project. As a result, after you run the instrumented application or execute functional or unit tests, a C/C++test coverage log file (.clog) is created. The coverage log file can then be used to generate a complete code coverage report.

...

Table of Content Zone
maxLevel2
minLevel2
locationtop

-compiler <name|path>

Specifies the name of the compiler configuration you want to use for code analysis and instrumentation. See Compilers for the list of supported compilers or use the -list-compilers command line option to print out the list of supported compilers to the console.

Examples:

  • cpptestcc -compiler gcc_3_4 
  • cpptestcc -compiler vc_11_0

Configuration file format (see -psrc): cpptestcc.compiler <name>

-list-compilers

Prints out the names of all supported compiler configurations.

Configuration file format (see -psrc): cpptestcc.listCompilers

-include <file|pattern> and -exclude <file|pattern>

Includes into or excludes from the instrumentation scope all the file(s) that match the specified pattern.

Final filtering is determined only after all include/exclude entries have been specified in the order of their specification.

The following wildcards are supported:

  • ? - Any character
  • * - Any sequence of characters

To prevent shells from expanding * wildcards to the list of files or directories, you can use the regex: prefix to specify the value.

(info) These options can be specified multiple times.

Configuration file format (see -psrc): cpptestcc.include <path|pattern>

Example 1:

Sample project layout:

Code Block
<project root>
 + external_libs
 + src
 + include

If your project has the above layout, the following command will exclude all the files in the external_libs directory from instrumentation scope:

cpptestcc -include regex:*/<project root>/* -exclude regex:*/<project root>/external_libs <other command line options>

Example 2:

Sample project layout:

Code Block
<project root> 
<sourcefiles>.cpp 
<headerfiles>.hpp

If your project has the above layout, the following command will only instrument the header files (the source files will not be instrumented):

cpptestcc -include regex:* -exclude regex:*.cpp <remaining part of cmd>

-ignore <pattern>

Specifies the source files that will be ignored during processing. The files that match the specified pattern will be compiled, but they will not be parsed or instrumented.

Info
title-ignore vs. -exclude

The -ignore option completely removes the specified file from processing so that it is not parsed by the coverage engine.

The -include/-exclude filters are applied after source code is parsed, which allows you to selectively instrument or not instrument header files.

You can use the -ignore option to reduce build time overhead by ignoring coverage analysis on some sections of the code (such as external libraries) or to ignore specific file that expose parse errors or other problems during processing.

The following wildcards are supported:

  • ? - Any character
  • * - Any sequence of characters

To prevent shells from expanding * wildcards to the list of files or directories, you can use the regex: prefix to specify the value.

(info) This option can be specified multiple times.

Configuration file format (see -psrc): cpptestcc.ignore <path|pattern>

Example:

cpptestcc -ignore "*/Lib/*" <remaining part of cmd>
cpptestcc -ignore regex:*/file.c <remaining part of cmd>
cpptestcc -ignore c:/proj/file.c <remaining part of cmd>
cpptestcc -ignore "*/MyLib/*.cpp" -ignore file.cpp <remaining part of cmd>

-line-coverage

Enables collecting line coverage.

Runtime coverage results are being written to the results log as the code is executed. This imposes some overhead on the tested code execution time, it but it allows you to ensure that that coverage data is collected even if the application crashes.

Configuration file format (see -psrc): cpptestcc.lineCoverage [true|false]

-optimized-line-coverage

Enables collecting optimized line coverage.

Runtime coverage results are stored in memory and then written to the results log either after the application finishes or on user request. This results in better performance, but results may be lost if the application crashes.

Configuration file format (see -psrc): cpptestcc.optimizedLineCoverage [true|false]

-function-coverage

Enables collecting function coverage.

Configuration file format (see -psrc): cpptestcc.functionCoverage [true|false]

-optimized-function-coverage

Enables collecting optimized function coverage. Configuration file format (see -psrc):

cpptestcc.optimizedFunctionCoverage [true|false]

-statement-coverage

Enables collecting statement coverage. Configuration file format (see -psrc):

cpptestcc.statementCoverage [true|false]

-optimized-statement-coverage

Enables collecting statement coverage.

Configuration file format (see -psrc): cpptestcc.optimizedStatementCoverage [true|false]

-block-coverage

Enables collecting  block coverage.

Configuration file format (see -psrc): cpptestcc.blockCoverage [true|false]

-optimized-block-coverage

Enables collecting  optimized block coverage.

Configuration file format (see -psrc): cpptestcc.optimizedBlockCoverage [true|false]

-path-coverage

Enables collecting path coverage.

Configuration file format (see -psrc): cpptestcc.pathCoverage [true|false]

-decision-coverage

Enables collecting decision coverage.

Configuration file format (see -psrc): cpptestcc.decisionCoverage [true|false]

-optimized-decision-coverage

Enables collecting optimized decision coverage.

Configuration file format (see -psrc): cpptestcc.optimizedDecisionCoverage [true|false]

-simple-condition-coverage

Enables collecting simple condition coverage.

Configuration file format (see -psrc): cpptestcc.simpleConditionCoverage [true|false]

-optimized-simple-condition-coverage

Enables collecting optimized simple condition coverage.

Configuration file format (see -psrc): cpptestcc.optimizedSimpleConditionCoverage [true|false]

-mcdc-coverage

Enables collecting MC/DC coverage.

Configuration file format (see -psrc): cpptestcc.mcdcCoverage [true|false]

-call-coverage

Enables collecting call coverage.

Configuration file format (see -psrc): cpptestcc.callCoverage [true|false]

-optimized-call-coverage

Enables collecting optimized call coverage.

Configuration file format (see -psrc): cpptestcc.optimizedCallCoverage [true|false]

-coverage-early-init

Enables initializing the coverage module at the beginning of the application entry point.

Configuration file format (see -psrc): cpptestcc.coverageEarlyInit [true|false]

-coverage-auto-finalization

If enabled, collecting coverage will be automatically finalized at the application exit. This option is enabled by default.

Configuration file format (see -psrc): cpptestcc.coverageAutoFinalization [true|false]

-optimized-coverage-corruption-detection

Enables corruption detection algorithms for optimized coverage metrics.

Configuration file format (see -psrc): cpptestcc.optimizedCoverageCorruptionDetection [true|false]

-template-coverage

Enables collecting coverage for template classes and functions

Configuration file format (see -psrc): cpptestcc.templateCoverage [true|false]

-workspace <path>

Specifies a custom directory where information about code structure will be stored during code analysis and instrumentation. The cpptestcc cpptestcli tool will use the information to generate the final coverage report.

By default, the information is stored in the working directory of the current compilation. If your compilation uses more than one working directory, we recommend that you specify a custom directory to ensure that all coverage data is stored in the same location.

Configuration file format (see -psrc): cpptestcc.workspace <path>

Anchor
psrc
psrc
-psrc <file>

Specifies the path to a configuration file where you can configure additional cpptestcc options.

By default, cpptestcc attempts to read the.psrc file located in either the current working directory or in the user HOME directory. This option allows you to specify a custom location of the configuration file.

(info) If an option is configured both in the command line and in the configuration file, cpptestcc will use the value specified in the command line.

-version

Prints out information about the version

-help

Prints out the help message and exits.

...

The coverage runtime library is a collection of helper functions and services used by source code instrumentation to emit coverage information at application runtime. Instrumented applications cannot be linked without the library. The runtime library can be linked to the final testable binary in multiple ways depending on the tested project type.

In addition to providing basic services for instrumented code, the library is also used to adapt the code coverage solution to particular development environments, such as supporting non-standard transport for coverage results between tested embedded device and development host.

...

C/C++test ships with pre-built versions of the runtime library, which are suitable for use on the same platform on which C/C++Test is installed. In most of the cases, collecting code coverage information from natively developed applications can use pre-built versions of the runtime library.

...

Windows (x86 and x86-64) 

FileDescription
<INSTALL_DIR>/bin/engine/coverage/runtime/lib/cpptest.a32-bit import library to be used with Cygwin GNU GCC compilers. To be added to linking command line.
<INSTALL_DIR>/bin/engine/coverage/runtime/lib/cpptest64.a64-bit import library to be used with Cygwin GNU GCC compilers. To be added to linking command line.
<INSTALL_DIR>/bin/engine/coverage/runtime/lib/cpptest.lib32-bit import library to be used with Microsoft Visual C++ compilers. To be added to linking command line.
<INSTALL_DIR>/bin/engine/coverage/runtime/lib/cpptest64.lib64-bit import library to be used with Microsoft Visual C++ compilers. To be added to linking command line.
<INSTALL_DIR>/bin/engine/coverage/runtime/bin/cpptest.dll32-bit dynamic-link library. <INSTALL_DIR>/bin/engine/coverage/runtime/bin should be added to PATH environmental variable.
<INSTALL_DIR>/bin/engine/coverage/runtime/bin/cpptest64.dll64-bit dynamic-link library. <INSTALL_DIR>/bin/engine/coverage/runtime/bin should be added to PATH environmental variable.

Linux (x86 and x86-64)

FileDescription
<INSTALL_DIR>/bin/engine/coverage/runtime/lib/libcpptest.so32-bit shared library. To be added linking command line. <INSTALL_DIR>/bin/engine/coverage/runtime/lib should be added to LD_LIBRARY_PATH
<INSTALL_DIR>/bin/engine/coverage/runtime/lib/libcpptest64.so64 bit shared library. To be added linking command line. <INSTALL_DIR>/bin/engine/coverage/runtime/lib should be added to LD_LIBRARY_PATH

If you need to use the runtime library in a form not provided as an out-of-the-box solution, prepare a custom build of the coverage runtime library that matches specific development environment requirements. For more details, see see Customizing the Runtime Library.

Integrating with the Linker Command Line

...

  1. Locate the linker command line in your build scripts
  2. Modify the build scripts so that the coverage runtime library is specified somewhere in the linker command line - preferably after all object files. For example:

    Code Block
    $(LXX) $(PRODUCT_OBJ) $(OFLAG_EXE)$(PROJ_EXECUTABLE) $(LXXFLAGS) $(SYSLIB) $(EXECUTABLE_LIB_LXX_OPTS) <INSTALL_DIR>/bin/engine/coverage/runtime/lib/cpptest.lib
  3. Make sure that the <INSTALL_DIR>/bin/engine/coverage/runtime/bin Ensure that the path to the lib directory is added to your the PATH environment variable so that the library can be located when the tested program is started. You may also consider copying cpptest.dll (or cpptest64.dll) file to the same directory as your executable file or to another location that is scanned for dynamic-link libraries during tested application startup.

...

  1. Locate the linker command line in your build scripts
  2. Modify the build scripts so that the coverage runtime library is specified somewhere in the linker command line - preferably after all object files. For example:

    Code Block
    $(LXX) $(PRODUCT_OBJ) $(OFLAG_EXE)$(PROJ_EXECUTABLE) $(LXXFLAGS) $(SYSLIB) $(EXECUTABLE_LIB_LXX_OPTS) -L <INSTALL_DIR>/bin/engine/coverage/runtime/lib -lcpptest

    Note that the addition of the-L [INSTALL_DIR]/bin/engine/coverage/runtime/lib and  and -lcpptest options are added.

    Make sure that shared library can be found by tested executable by modifying
  3. Ensure that the path to the lib directory is added to the LD_LIBRARY_PATH environmental variable to include [INSTALL_DIR]/bin/engine/coverage/runtime/lib locationallow the tested executable to find the path to the shared library.

Anchor
customizing_runtime_library
customizing_runtime_library
Customizing the Runtime Library

...

  • Different form of binary file is required
  • Enabling a non-default communication channel for results transport
  • Installing custom implementation of communication channel for results transport
  • Enabling a non-default support for multithreaded applications
  • Installing custom implementation of support for multithreaded applications

...

ComponentDescription
include

Directory that contains the library include files.
include/cpptest.h - library public interface
include/cpptest/* - library private interface
The content of the include directory is not designed for environment-specific modifications.

src

Directory that contains the library source code.
src/cpptest.c - the main and single source file of the runtime library
This file is designed for modifications and customizations.

MakefileBasic Makefile provided for building the runtime library.
targetDirectory that contains a set of Makefile include files with compiler-specific options for preparing runtime library builds for the most popular development environments.
channelDirectory that contains a set of Makefile include files with configuration for supported communication channels.

...

If none of the communication channel implementations fit into your development environment, then a custom implementation can be provided. The following instructions describe how to customize the runtime library so that it uses a custom implementation of a communication channel: 

  1. Make a copy

    of

    of [INSTALL_DIR]/bin/engine/coverage/runtime/src/cpptest.c and open the file for editing.

  2. Locate the section 1.13 "Custom Communication Implementation".
    The custom communication implementation section contains empty templates for four different methods:

    FunctionDescription
    void cpptestInitializeStream(void)This function is responsible initializing the communication channel, for example creating and connecting to a socket or the initialization of UART device.
    void cpptestFinalizeStream(void)This function is responsible for finalizing the communication channel. For example, it may be responsible for closing TCP/IP socket.

    int cpptestSendData(const char *data, unsigned size)

    This function is responsible for sending size bytes from a data buffer.
    void cpptestFlushData(void)This function is responsible for flushing the data. Its meaning depends on the particular transport type. It may have a limited application in some implementations. In this case, it should be left empty.
  3. Provide the implementation for these methods that match your environment requirements.
  4. Compile cpptest.c with the following macro definition added to compilation command line:
    -DCPPTEST_CUSTOM_COMMUNICATION
  5. If the generated object file is insufficient, you can process the file even further to meet your needs (e.g., to create a shared library).

...

If you are using C/C++test's coverage engine with multithreaded applications that do not use a supported multithreading API, you can customize the runtime library to work with your multithreading API. There are following steps required:

  1. Make a copy

    of

    of [INSTALL_DIR]/bin/engine/coverage/runtime/src/cpptest.c and open the file for editing.

  2. Locate the section 2.5 "Custom Multithreading Implementation".
    Custom multithreading implementation section contains empty templates for two different methods:

    FunctionDescription
    static int cpptestLock(void)This function ensures synchronized operations inside the coverage tool runtime library. If a thread locks access to the runtime library service, it means an atomic operation is in progress and no other thread can use runtime library services. Once the lock is released other threads can use runtime library services
    static int cpptestUnlock(void)Releases the lock on runtime library services.
  3. Provide the implementation for the methods that matches your environment requirements.

  4. Compile cpptest.c with the following macro added to compilation command line:
    -DCPPTEST_CUSTOM_THREADS

  5. If the generated object file is insufficient, you can process the file even further to meet your needs (e.g.for example, to create a shared library).

...

Building the Runtime Library Using the Provided Makefile

  1. Copy

    Copy <INSTALL_DIR>/bin/engine/coverage/runtime

    to

     to a local

    directory

    director.

  2. If compilation flags need to be modified (e.g., adding specific for example, to add a cross-compiler specific or definitions to enforce runtime library reconfiguration), provide a new make configuration file in the target subdirectory. For your convenience, you can copy one of the existing target configuration files and modify its contents to fit your needs.
  3. Invoke the following command line to create a build subdirectory that contains a single object cpptest.<OBJ_EXT>, which can be used to link with the instrumented application.

    Code Block
    languagetext
    make TARGET_CFG=<target config file name> CHANNEL_FILE=<channel config file name>

    Your command line may resemble the following:

    Code Block
    languagetext
    make TARGET_CFG=gcc-static.mk CHANNEL_FILE=channel/unix-socket.mk

    Alternatively, you can provide the channel type:

    Code Block
    languagetext
    make TARGET_CFG=gcc-static.mk CHANNEL_TYPE=unix-socket
  4. If the coverage runtime library needs to be linked to from a shared library, dynamic link library, or any other type of binary, the Makefile needs to be customized for this purpose or a custom build needs to be setup.

User Build of the Runtime Library

To setup set up a user build of coverage runtime library perform the following steps:

  1. Copy the cpptest.c file

    from

    from <INSTALL_DIR>/bin/engine/coverage/runtime/src/cpptest.c

    to

     to your preferred location.

  2. Introduce any customizations as described in Customizing the Runtime Library.
  3. Set up a build system of your preference (e.g., IAR Embedded Workbench project or any other type of source code builder).
  4. Modify the compilation flags to contain the compiler include a flag (typically -I) with the following value:


    -I <INSTALL_DIR>/bin/engine/coverage/runtime/include

  5. Add any required configuration defines (typically -D), for example:
    -DCPPTEST_FILE_COMMUNICATION -DCPPTEST_NO_THREADS
  6. Invoke the command to run your builder (for example, select build command in the IDE).
  7. Locate the resulting object file and use it to link with your instrumented application.

...