Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space CPPTDESKDEV and version 10.4.2

In addition to industry-standard C/C++ code coverage metrics, C/C++test can collect code coverage at the assembly level (also called known as object code coverage), which provides visibility into which sections of assembly code were covered during test execution, as well as details on branching points execution. For details on supported assembly metrics, see "Parasoft cpptestasm User Guide".

Unlike C/C++ code coverage, assembly coverage results are not integrated with C/C++test views. Reports are available as external HTML, XML, or CVS files. See Generating the Assembly Coverage Report for more details.

This section provides information on collecting object code coverage in your IDE.  For details about Parasoft tools for collecting coverage at the assembly level, see the Parasoft ASMtools documentation shipped with C/C++test in [INSTALL_DIR]\engine\asmtool\manuals.

Anchor
Supported Compilers
Supported Compilers
Supported Compilers

On Windows

Assembly code coverage is supported for the following GHS compilers:

  • Green Hills Software C/C++ Compiler v2013.1.x for PowerPCCompiler for PPC v. 3.5.x
  • Green Hills Software C/C++ Compiler v5.0 for PowerPCCompiler for PPC v. 4.0.x
  • Green Hills Software C/C++ Compiler v4.2 for PowerPCCompiler for PPC v. 4.2.x
  • Green Hills Software C/C++ Compiler v4.0 for PowerPCGreen Hills Software Compiler for PPC v. 2017.1.x

C/C++

...

C++test is preconfigured to collect assembly level coverage data from Green Hills software Software PPC simulator. If you wish to configure collecting   Contact Parasoft Support for more information if you want to collect object coverage data from on-target tests execution please contact Parasoft Support for more information.

Unlike C/C++ code coverage, assembly coverage results are not integrated with C++test views, reports are available as external html, xml or text documents. See Generating the Assembly Coverage Report for more details.

.

On Linux 64-bit:

Assembly code coverage is supported for the following GNU GCC compilers for x86 (32-bit):

  • GNU GCC 4.6.x
  • GNU GCC 4.7.x
  • GNU GCC 4.8.x
  • GNU GCC 4.9.x
  • GNU GCC 5.x
  • GNU GCC 6.x
  • GNU GCC 7.x
  • GNU GCC 8.x

Configuring Project Settings

Assuming that your project is already prepared configured for unit tests test execution or application memory monitoring , there are no additional configuration activities required to prepare it for assembly coverage monitoring. Note, however, that the project must be setup for a supported compilerwith C/C++test and set up for a supported compiler (see Supported Compilers), no other configuration steps are required.

You can exclude test-harness initializers (added by C/C++test) from the assembly coverage report. Go to Project Properties> Parasoft> C++test> Build Settings> Compiler Options and add the following option:

Code Block
-DCPPTEST_USE_GLOBAL_OBJECTS_TO_INIT_RUNTIME=0

This will disable adding the test-harness initializers to compilation units under test. Instead, C/C++test will generate a separate "test runner" for performing test-harness initialization.

Collecting Assembly Code
Anchor
collect_assem
collect_assem

C/C++test ships with a built-in test configuration configurations for executing unit test cases with assembly coverage monitoring.  To view the test configuration settings:

  1. Choose Parasoft> Test Configurations Choose form your IDE menu.
  2. Select of the following built-in test configuration:
    On Windows: Builtin> Embedded Systems> Green Hills Software> Run GHS Tests with Assembly Coverage Monitoring
    On Linux: Builtin>Embedded Systems>GNU GCC>Run GNU GCC Tests with Assembly Coverage Monitoring

When you run the test configuration, coverage data is stored in the tested program's memory buffers, and when . When the scheduled tests are executed and test executable exits, the collected coverage information is saved via the defined communication channel (typically, directly to the file). Buffers with coverage information may become corrupted or may not be saved if, for example,

(info)  If the test executable crashes or the memory becomes corrupted. We recommend reviewing all unit test cases and exclude the ones that may cause application crash from the testing session.

The following Test Execution Flow properties are available in the test configuration for assembly coverage reports configuration (also see Generating the Assembly Coverage Report):

...

Available formats: html, xml, txt, flattxt, csv

html is a default value

...

Encoding used for html reports.

default is UTF-8

...

Root directory for assembly code coverage reports.

Reports are generated in the designated location. Separate subdirectories are created for tested contexts.

Default location is: ${workspace_loc}/.cpptest/${project_name}/cpptestasm/reportdata

Contact Parasoft Support for more information if you want to collect object coverage data from on-target tests execution.

Enabling Assembly Code Coverage for User-defined Test Configuration

You can duplicate the built-in Run GHS Tests with Assembly Coverage Monitoring test configuration (right-click the on the test configuration and choose Duplicate) as a base for project specific customizations.

...

In addition to enabling assembly level code instrumentation you also need to modify the test execution flow to include a dedicated step for assembly coverage report generation. This step is already configured in the built-in Run GHS Tests with Assembly Coverage Monitoring test configuration.

...

Code Block
<ReadTestLogStep
	     testLogFile="${cpptest:testware_loc}/
cpptest_results.tlog"
	     timeoutInfoProperty="test_exec_timeouted"
	/>
	<ReadDynamicCoverageStep
		covLogFile="${cpptest:testware_loc}/
cpptest_results.clog"
	/>

...

Insert the following flow step:

Code Block
<GenASMToolReportFlowStep 
	asmToolCmdPattern="$(asmReportTool) --generate-report=on --workspace=$(asmWorkspace) 
--psrc=$(additionalOptions) --report-format=$(asmReportFor-mat) --results-directory=$(asmReportdir)
--project-name=$(asmProjectName)  --input-file=$(asmInputFile) --report-encoding=$(asmReportEncoding) --psrc=$(advancedOp-
tionsFile)" 
	asmReportFormat="${cpptestproperty:repor_format}"
	asmReportEncoding="${cpptestprop-erty:repor_encoding}"
	asmReportdir="${cpptestproperty:repor_loc}"   
	asmInputFile="${cpptest:testware_loc}/${cpptestprop-erty:oc_result_file_name}" 
/>

The asmToolCmdPattern attribute defines the command line for too that generates the report. This attribute uses the variables to facilitate report generation configuration. To make the variables editable via the test execution flow properties you can add the following at the begging of your test execution flow:

Code Block
<SetProperty key="repor_format" value="html||xml||flat-
txt||txt||csv" uiEditable="true"
displayName="Assembly coverage report format" />
	<SetProperty key="repor_encoding" value="UTF-8" uiEdit-
able="true" displayName="Assembly
coverage report encoding" />
	<SetProperty key="report_loc"
value="${workspace_loc}/.cpptest/${project_name}/cpptestasm/
report-data" uiEditable="true"
displayName="Assembly coverage report location" />

, the buffers that store coverage data may become corrupted or may not be saved. For this reason, we recommend reviewing the tests cases prior to execution and excluding from the testing session all tests that may crash the application.

Generating the Assembly Coverage Report
Anchor
generate_assem
generate_assem

Assembly language coverage reports are generated after tests execution and are available for review in html, text or csv formatin the format specified in the test configuration (see Customizing Report Options). There is no assembly coverage data visible in C/C++test’s coverage view, source code editor, or any other component of the C/C++test Eclipse or Visual Studio plug-in. If you are generating assembly coverage from a user-defined test configuration, make sure to configure the test execution flow described above.UI.

  1. After executing the test configuration, click Generate Report button.



  2. Click Preferences and choose HTML (C++test’s Assembly CoverageUnit Testing details) from the Report Format drop-down menu.Image Removed
  3. Apply your settings and click OK to exit the Preferences dialog
  4. Click OK on the Report & Publish dialog.

The link to the assembly coverage report will be available in the dedicated Assembly Coverage column included in the Additional Reports section at the bottom of main report. See the table under Collecting Assembly Code Coverage, for Test Execution Flow properties that are available for assembly coverage reports configuration:

Image Added

Info
iconfalse

The HTML report links include relative paths that will not be broken when the reports are moved to another location.

Anchor
Customizing Report Options
Customizing Report Options
Customizing Report Options

You can customize the format and encoding of the assembly coverage report by modifying the test configuration you will use to execute tests cases (see Collecting Assembly Code). Go to General> Test execution flow and configure the following options:

Option NameValues
Assembly coverage report format

Available formats: html, xml, txt, flattxt, csv

The default is html.

Assembly coverage report encoding

Encoding used for html reports.

The default is UTF-8

Contact Parasoft Support for more information if you want to collect object coverage data from on-target tests execution.

Additional Notes About Assembly Coverage

  • Assembly coverage metrics are only supported for select GHS PPC the selected compilers (see Assembly Code Coverage). The C/C++test console will print a message to the console if you run assembly coverage monitoring with a project configured for an unsupported compiler.
  • Assembly coverage is collected through assembler source code instrumentation generated from C/C++ instrumented code. You should not therefore enable other instrumentation features that will make C/C++ instrumented code significantly different from original code. The recommended configuration of instrumentation features is shown on the screen shot below.