In this section:

Introduction

C/C++test's multi-metric coverage analysis allows you to monitor code coverage when executing a standalone application or library outside C/C++test.

C/C++test ships with a standalone coverage package that consists of the following components:

• <INSTALL_DIR>/bin/cpptestcc - A coverage tool that integrates into your build process to instrument your application to collect raw coverage data.
  
• <INSTALL_DIR>/bin/engine/coverage/runtime - A coverage runtime library that needs to be integrated with the instrumented application.

1. Instrumenting the application by integrating the cpptestcc tool into your build.
2. Executing instrumented code and collecting raw coverage data.
3. Reviewing the coverage with C/C++test by importing the raw coverage data into C/C++test with a built-in test configuration.

Quick Start with cpptestcc

1. Add the path to <INSTALL_DIR>/bin  the to the PATH system variable to enable execution of the cpptestcc tool.

2. Update your compilation command to include the cpptestcc executable as a prefix to the compiler command using -- as a separator.  For example:
   

   Original compilation command line

   cc -I app/includes -D defines -c source.cpp

   Updated compilation command line
   

   cpptestcc -compiler gcc_9-64 -line-coverage -- cc -I app/includes -D defines -c source.cpp

   At a minimum, the cpptestcc tool requires the following parameters to be configured on the command line:

   - a compiler identifier: -compiler <COMPILER_ID>

   - a coverage metric (for example, -decision-coverage)

   See Command Line Reference for cpptestcc for information about other options.


3. Update your linker command with the path to the pre-built coverage runtime library shipped with C/C++test to add the library to your application. For example:
   

   Original command line
   

   lxx -L app/lib app/source.o somelib.lib -o app.exe

   Updated command line
   

   lxx -L app/lib app/source.o somelib.lib [INSTALL_DIR]/bin/engine/coverage/runtime/lib/cpptest.lib -o app.exe

   Important

   If the coverage runtime library is linked as a shared (dynamic-load) library, you must ensure that it can be loaded when the instrumented application is started. This typically requires adding <INSTALL_DIR>/bin/engine/coverage/runtime/bin to the PATH environment variable (on Windows) or <INSTALL_DIR>/bin/engine/coverage/runtime/lib  to the  LD_LIBRARY_PATH environment variable (on Linux).

   C/C++test provides the pre-built coverage runtime library for native Windows and Linux applications. For cross-platform and embedded testing, the runtime library needs to built from sources that are available in <INSTALL_DIR>/bin/engine/coverage/runtime. See Coverage Runtime Library for details.
   

4. Build the application. When instrumenting the code, cpptestcc creates the .cpptest/cpptestcc folder where important coverage-related data ("coverage maps") are stored. By default, the folder is located in the working directory of the current compilation. You can change the default location using the -workspace <path> option;  see Command Line Reference for cpptestcc for details. 
   
5. Run the application. The coverage data ("coverage log") will be stored it the cpptest_results.clog file.
   
6. In your IDE where C/C++test is installed, create a new project that includes all the source files of the application.
   Ensure that the files and all the paths remain unchanged.
   
7. Select the project and choose Parasoft> Test Configurations> Utilities> Load Application Coverage from your IDE menu to import the coverage data (see Importing the Coverage Data for details).
   
8. Review the coverage information (see Reviewing Coverage Information).

Importing the Coverage Data

The Load Application Coverage test configuration assumes that both .cpptest/cpptestcc folder and cpptest_results.clog file are stored in the default location. To customize the location, configure the following execution details in the test configuration (Execution> General> Execution details):
- Coverage map files root location - default: ${project_loc}/.cpptest/cpptestcc
- Coverage log files - default: ${project_loc}/*.clog

By default, the Load Application Coverage test configuration tries to load information about all the supported coverage metrics. To customize the list of supported coverage metricts, go to Execution> General> Execution details> Instrumentation Mode> Instrumentation features> C/C++ Code Coverage metrics and select the metrics you want to report.

We recommend that you keep consistency with the metrics enabled for the cpptestcc tool in the compilation command line.

Collecting Application Coverage for CMake Projects

C/C++test ships with an extension for CMake that allows you to integrate C/C++test's code coverage analysis directly into your CMake project. The extension automatically modifies your compiler/linker command lines to use the cpptestcc coverage tool when building your project. As a result, after you run the instrumented application or execute functional or unit tests, a C/C++test coverage log file (.clog) is created. The coverage log file can then be used to generate a complete code coverage report.

Support for CMake integration includes:

• The <CPPTEST_INSTALL_DIR>/integration/cmake/cpptest-coverage.cmake extension – The C/C++test extension file for CMake you need to add to your CMakeFiles.txt build file.
• The templates in <CPPTEST_INSTALL_DIR>/integration/cmake/cpptest.templates/* – A set of C/C++test templates for generating C/C++test (Eclipse) projects.
• The CPPTEST_COVERAGE=ON and CPPTEST_HOME=<CPPTEST_INSTALL_DIR> options – The options provided by the extension for activating the extension when building the application with CMake.
• The cpptest_coverage_report target – The target provided by the extension for generating the coverage report using the .clog file as an input.

In addition, the <CPPTEST_INSTALL_DIR>/examples/Timer directory includes an example project to demonstrate collecting code coverage for a CMake project.

Requirements

• CMake 3.10 or newer with Unix Makefiles or Ninja generator

Additional prerequisites:

• Full build of the application must be performed to collect code coverage for CMake projects. Incremental builds are not supported.
• On Windows, your CMake project and toolchain need to be configured for using Windows paths with forward slashes or backslashes (for example, c:/folder/source.cpp or c:\folder\source.cpp). Unix-style paths (for example, /c/folder/source.cpp) are not supported on Windows.

Workflow Overview

1. Copy <CPPTEST_INSTALL_DIR>/integration/cmake/cpptest-coverage.cmake and <CPPTEST_INSTALL_DIR>/integration/cmake/cpptest.templates/*to your CMake project.

2. Review the coverage configuration details in the cpptest-coverage.cmake file and update the options if needed.  At a minimum, you must ensure that the compiler configuration specified with the CPPTEST_COMPILER_ID option matches your compiler. See Customizing the Coverage Extension for CMake for available options.

3. Include the cpptest-coverage.cmake extension to your main CMakeLists.txt build file. The extension must precede all build target definitions to ensure that the compiler/linker command lines are automatically modified.

4. Activate the extension with the CPPTEST_COVERAGE and CPPTEST_HOME options when configuring and building your CMake project:

   > cmake -DCPPTEST_COVERAGE=ON -DCPPTEST_HOME=<CPPTEST_INSTALL_DIR> ..

   By default, C/C++test's coverage data files will be created in <CMAKE_SOURCE_DIR>/../cpptest-coverage/<CMAKE_PROJECT_NAME>/.cpptest.

5. Run your application or execute your functional or unit tests.
   By default, the C/C++test coverage log file (.clog) will be created in <CMAKE_SOURCE_DIR>/../cpptest-coverage/<CMAKE_PROJECT_NAME>/<CMAKE_PROJECT_NAME>.clog.

6. Run the cpptest_coverage_report helper target to generate the coverage report:

   > make cpptest_coverage_report

   The C/C++test target will create:
   - a workspace you can open directly in the IDE: <CMAKE_SOURCE_DIR>/../cpptest-coverage
   - a C/C++test (Eclipse) project in that workspace: <CMAKE_SOURCE_DIR>/../cpptest-coverage/<CMAKE_PROJECT_NAME>
   - the coverage report in the reports directory: <CMAKE_SOURCE_DIR>/../cpptest-coverage/<CMAKE_PROJECT_NAME>/reports

   We recommend generating the coverage report using the cpptest_coverage_report target. Alternatively, you can use C/C++test with the Load Application Coverage test configuration to generate report from the coverage data collected in <CMAKE_SOURCE_DIR>/../cpptest-coverage/<CMAKE_PROJECT_NAME>. See Importing the Coverage Data for details.

Customizing the Coverage Extension for CMake

To customize collecting coverage with the C/C++test extension, open the cpptest-coverage.cmake file you copied to your CMake project and modify the C/C++test options. 

In addition, you may want to review:

• the definition of cpptest_coverage_report target that specifies the parameters for cpptestcli for generating the coverage report.
• the # Build C/C++test coverage runtime library section, which includes configuration for the C/C++test’s runtime library (the library is automatically built by the coverage extension).
• the CPPTEST_LINKER_FLAGS option, which defines how the C/C++test coverage runtime will be linked.

Example of Integrating the Coverage Extension with CMake

This section demonstrates how to use C/C++test's coverage extension for CMake to collect coverage data for the example project located in the <CPPTEST_INSTALL_DIR>/examples/Timer directory. The Timer project is configured to use the cpptest-coverage.cmake extension shipped in <CPPTEST_INSTALL_DIR>/integration/cmake.

To collect coverage for the example project:

1. If you use a compiler other than the default GNU GCC 9 (x64), go to <CPPTEST_INSTALL_DIR>/integration/cmake/cpptest-coverage.cmake and modify the default value of the CPPTEST_COMPILER_ID to match your compiler.

2. Build the example project:
   

   > cd <CPPTEST_INSTALL_DIR>/examples/Timer
   > mkdir build
   > cd build
   > cmake -DCPPTEST_COVERAGE=ON -DCPPTEST_HOME=<CPPTEST_INSTALL_DIR> ..
   > make

3. Run the application:
   

   > ./timer

4. Generate the coverage report:
   

   > make cpptest_coverage_report

   The report will be created in <CPPTEST_INSTALL_DIR>/examples/cpptest-coverage/Timer/reports.

5. (Optionally) Launch the C/C++test GUI and open the <CPPTEST_INSTALL_DIR>/examples/cpptest-coverage workspace to review the coverage details in the IDE.
   

Collecting Application Coverage for MSBuild Projects

C/C++test ships with a script that allows you to quickly reconfigure your MSBuild project to integrate coverage analysis. This script supports both Visual Studio and CMake-generated MSBuild projects. The script creates a backup of the target .vcxproj file and then modifies it to enable the requested Parasoft features on all build targets.

C/C++test includes a Python script launcher to run the included tools, which can be found at:

<CPPTEST_INSTALL_DIR>/bin/engine/bin/cpptestpy.exe

<CPPTEST_INSTALL_DIR>/integration/msbuild/msbuild_cpptest.py

To collect coverage for an MSBuild project:

1. Setup the environmental variables.

Be sure to not use double quotes after = which will cause incorrect string interpretation. 

1. 1. Setup the CPPTEST_INSTALL_DIR variable.

      set CPPTEST_HOME=<CPPTEST_INSTALL_DIR>

   2. Setup the PATH variable.

      set PATH=%PATH%;%CPPTEST_HOME\bin\engine\coverage\runtime\bin

2. (CMake only) Generate your MSBuild files using CMake. 

   cd <PROJECT_DIR>
   mkdir build
   cd build
   cmake ..

3. Run the integration script. 

   <cpptestpy.exe> <msbuild_cpptest.py> -c -f project.vcxproj

4. Build the project using MSBuild.

   msbuild -t:Rebuild project.sln

5.  To collect coverage, go to the build directory and run the project.

   cd x64\Debug
   project.exe

   A .clog file that contains coverage information will be generated in your current working directory. 
   The default name of the file is cpptest_results.clog.

6. Generate the coverage report.

   cpptestcli.exe \
   -data ../workspace \
   -bdf cpptestscan.bdf \
   -property bdf.import.location=BDF_LOC \
   -config "builtin://Load Application Coverage"

Command Line Reference for cpptestcc

You can run the following command to print out the available options to the console: cpptestcc -help

The following options are available:

<project root>
 + external_libs
 + src
 + include

<project root> 
<sourcefiles>.cpp 
<headerfiles>.hpp

while(1) {} // no decision / branch
if(false) {} // no decision / branch

Coverage Runtime Library

The coverage runtime library is a collection of helper functions and services used by source code instrumentation to emit coverage information at application runtime. Instrumented applications cannot be linked without the library. The runtime library can be linked to the final testable binary in multiple ways depending on the tested project type.

In addition to providing basic services for instrumented code, the library is also used to adapt the code coverage solution to particular development environments, such as supporting non-standard transport for coverage results between tested embedded device and development host.

Pre-built Versions and Customized Builds

C/C++test ships with pre-built versions of the runtime library, which are suitable for use on the same platform on which C/C++Test is installed. In most cases, collecting code coverage information from natively developed applications can use pre-built versions of the runtime library.

All users developing cross-platform applications will need to prepare a custom build of the coverage runtime library using a suitable cross compiler and possibly linker. Source code of the code coverage runtime library is shipped with C/C++test.

The process of preparing the coverage runtime library custom build is typically limited to the compilation of coverage runtime library source code. In some situations, you may need to install some fragments of source code to adapt code coverage to a particular development platform. This process is described in the following sections.

Using the Pre-built Runtime Library

The following binary files are included with the C/C++test:

Windows (x86 and x86-64) 

If you need to use the runtime library in a form not provided as an out-of-the-box solution, prepare a custom build of the coverage runtime library that matches specific development environment requirements. For more details, see Customizing the Runtime Library.

Integrating with the Linker Command Line

Integrating the coverage runtime library with a tested application linking process usually requires modifying the linker command line and, in some cases, the execution environment. This section describes how to modify the linking process when using the pre-built versions shipped with C/C++test.

Static library for Windows Cygwin GNU GCC compilers:

1. Locate the linker command line in your build scripts
2. Modify the build scripts so that the coverage runtime library is specified somewhere in the linker command line - preferably after all object files.

Dynamic-link library for Microsoft Visual C++ compilers:

1. Locate the linker command line in your build scripts

2. Modify the build scripts so that the coverage runtime library is specified somewhere in the linker command line - preferably after all object files. For example:

   $(LXX) $(PRODUCT_OBJ) $(OFLAG_EXE)$(PROJ_EXECUTABLE) $(LXXFLAGS) $(SYSLIB) $(EXECUTABLE_LIB_LXX_OPTS) <INSTALL_DIR>/bin/engine/coverage/runtime/lib/cpptest.lib

3. Ensure that the path to the lib directory is added to the PATH environment variable so that the library can be located when the tested program is started. You may also consider copying cpptest.dll (or cpptest64.dll) file to the same directory as your executable file or to another location that is scanned for dynamic-link libraries during tested application startup.

Shared library for Linux GNU GCC compilers:

1. Locate the linker command line in your build scripts

2. Modify the build scripts so that the coverage runtime library is specified somewhere in the linker command line - preferably after all object files. For example:

   $(LXX) $(PRODUCT_OBJ) $(OFLAG_EXE)$(PROJ_EXECUTABLE) $(LXXFLAGS) $(SYSLIB) $(EXECUTABLE_LIB_LXX_OPTS) -L <INSTALL_DIR>/bin/engine/coverage/runtime/lib -lcpptest

   Note that the -L and -lcpptest options are added.

3. Ensure that the path to the lib directory is added to the LD_LIBRARY_PATH environmental variable to allow the tested executable to find the path to the shared library.

Customizing the Runtime Library

You may need to customize the runtime library as a result of the following conditions:

• Different form of binary file is required
• Enabling a non-default communication channel for results transport
• Installing custom implementation of communication channel for results transport
• Enabling non-default support for multithreaded applications
• Installing custom implementation of support for multithreaded applications
• Multiple instrumented binaries writing to the same coverage log file

Library Source Code Structure

The runtime library source code is shipped with C/C++test in the [INSTALL_DIR]/bin/engine/coverage/runtime directory. The following table describes the structure: 

Switching Communication Channel Support

The runtime library supports data collection through various communication channels. The communication channel used depends on the development environment. In most cases, storing results in a file or files is appropriate, but in other TCP/IP sockets or RS232 transport may be required. Specific communication channels can be enabled by setting the value to a dedicated macro during cpptest.c library source file compilation. Add -D<MACRO> to the compilation command line to set the value. The following table provides the full list of communication channel control macros:  

If the coverage runtime library is being built with the provided Makefile, then one of the make configuration files provided in the [INSTALL_DIR]/bin/engine/coverage/runtime/channel directory can be used.

Installing Support for Custom Communication Channel

If none of the communication channel implementations fit into your development environment, then a custom implementation can be provided. The following instructions describe how to customize the runtime library so that it uses a custom implementation of a communication channel: 

1. Make a copy of [INSTALL_DIR]/bin/engine/coverage/runtime/src/cpptest.c and open the file for editing.

2. Locate section 1.13 "Custom Communication Implementation".
   The custom communication implementation section contains empty templates for four different methods:

   Function	Description
   void cpptestInitializeStream(void)	This function is responsible initializing the communication channel, for example creating and connecting to a socket or the initialization of UART device.
   void cpptestFinalizeStream(void)	This function is responsible for finalizing the communication channel. For example, it may be responsible for closing TCP/IP socket.
   int cpptestSendData(const char *data, unsigned size)	This function is responsible for sending size bytes from a data buffer.
   void cpptestFlushData(void)	This function is responsible for flushing the data. Its meaning depends on the particular transport type. It may have a limited application in some implementations. In this case, it should be left empty.

3. Provide the implementation for these methods that match your environment requirements.
4. Compile cpptest.c with the following macro definition added to compilation command line:
   -DCPPTEST_CUSTOM_COMMUNICATION
5. If the generated object file is insufficient, you can process the file even further to meet your needs (e.g., to create a shared library).

Switching Multithreading API Support

The runtime library contains support for multithreaded applications. POSIX, Windows, and VxWorks APIs are supported. You can enable support for a specific multithreading API by adding -D<MACRO> to the compilation command line during cpptest.c compilation. The following table describes the full list of multithreading API support control macros:

Installing Support for Custom Threading API

If you are using C/C++test's coverage engine with multithreaded applications that do not use a supported multithreading API, you can customize the runtime library to work with your multithreading API. There are following steps required:

1. Make a copy of [INSTALL_DIR]/bin/engine/coverage/runtime/src/cpptest.c and open the file for editing.

2. Locate the section 2.5 "Custom Multithreading Implementation".
   Custom multithreading implementation section contains empty templates for two different methods:

   Function	Description
   static int cpptestLock(void)	This function ensures synchronized operations inside the coverage tool runtime library. If a thread locks access to the runtime library service, it means an atomic operation is in progress and no other thread can use runtime library services. Once the lock is released other threads can use runtime library services
   static int cpptestUnlock(void)	Releases the lock on runtime library services.

3. Provide the implementation for the methods that matches your environment requirements.

4. Compile cpptest.c with the following macro added to compilation command line:
   -DCPPTEST_CUSTOM_THREADS

5. If the generated object file is insufficient, you can process the file even further to meet your needs (for example, to create a shared library).

Multiple Instrumented Binaries Writing to the Same Coverage Log File

When the same coverage log file (.clog) is used by multiple instrumented binaries running in parallel, it is required to enable multi-process support in the Runtime Library.

In order to enable multi-process support, compile cpptest.c with the following macro added to the compilation command line:
"-DCPPTEST_MULTI_PROCESSES_LOG=1"

Building the Runtime Library

C/C++test ships with a simple Makefile (see Library Source Code Structure) which simplifies the process of building the runtime library. In many instances, however, the make file provided will not be required because the source code is already optimized for the building process. The only step that is always required is the compilation of the main cpptest.c source file. Any additional processing of the produced object file will depend on the particular development environment and its requirements, such as providing the runtime library as a shared library.

Building the Runtime Library Using the Provided Makefile

1. Copy <INSTALL_DIR>/bin/engine/coverage/runtime to a local director.

2. If compilation flags need to be modified (for example, to add a cross-compiler or definitions to enforce runtime library reconfiguration), provide a new make configuration file in the target subdirectory. For your convenience, you can copy one of the existing target configuration files and modify its contents to fit your needs.

3. Invoke the following command line to create a build subdirectory that contains a single object cpptest.<OBJ_EXT>, which can be used to link with the instrumented application.

   make TARGET_CFG=<target config file name> CHANNEL_FILE=<channel config file name>

   Your command line may resemble the following:

   make TARGET_CFG=gcc-static.mk CHANNEL_FILE=channel/unix-socket.mk

   Alternatively, you can provide the channel type:

   make TARGET_CFG=gcc-static.mk CHANNEL_TYPE=unix-socket

4. If the coverage runtime library needs to be linked to from a shared library, dynamic link library, or any other type of binary, the Makefile needs to be customized for this purpose or a custom build needs to be setup.

User Build of the Runtime Library

To set up a user build of coverage runtime library perform the following steps:

1. Copy the cpptest.c file from <INSTALL_DIR>/bin/engine/coverage/runtime/src/cpptest.c to your preferred location.

2. Introduce any customizations as described in Customizing the Runtime Library.
3. Set up a build system of your preference (e.g., IAR Embedded Workbench project or any other type of source code builder).

4. Modify the compilation flags to contain the compiler include a flag (typically -I) with the following value:-I <INSTALL_DIR>/bin/engine/coverage/runtime/include

5. Add any required configuration defines (typically -D), for example:
   -DCPPTEST_FILE_COMMUNICATION -DCPPTEST_NO_THREADS
6. Invoke the command to run your builder (for example, select build command in the IDE).
7. Locate the resulting object file and use it to link with your instrumented application.

Troubleshooting Code Coverage

cpptestcc -disable-auto-recovery-mode [. . .]

cpptestcc -coverage-data-variants [. . .]