You can configure the report settings in the GUI or from the command line interface (using settings). In addition to generating local reports, you can also send reports to Parasoft DTP. Sending reports to DTP enables you to apply sophisticated analytics to better understand the risk associated with the application under test and your overall development and testing processes.
Sections include:
You can specify the report settings for all tests, whether they are run from the command-line interface or the UI using the GUI controls. Before configuring report settings, you should review the settings on the following preference pages to ensure that task authorship is being calculated correctly, that results are being sent to the appropriate system, that the correct email host is used, and so on:
The settings specified in the UI can be fully or partially overwritten by those specified via settings.
To specify reporting settings from the GUI:
Functional Report for Project A
.Setting | Description |
---|---|
Detailed report for developers | Determines whether customized, detailed reports are generated for each team member (in addition to a summary report for managers). These reports contain only the tasks assigned to that specific team member. |
Overview of tasks by authors | Determines whether the report includes an overview of the number and type of tasks assigned to each developer. |
Overview of checked files and executed tests | Specifies whether the report provides details about all checked files and executed tests. For static analysis, this results in a list of all the files that were checked. For each file, it lists the number of rule violations. If the file has a violation, it also lists the line number, rule name, and rule ID for that violation. For test execution, this results in a list of all executed test cases and their outcomes (pass or fail). For each test suite, it lists the total number of test cases and the number of passed test cases. If a task is reported for a test case, additional details (stack trace, outcome, etc.) are presented. For code review, this results in a list of all pending issues with messages between author - reviewer - monitor (filtered by session tag by default). |
Task details | Determines whether the report includes details about all of the reported tasks. |
Test case details | Determines whether the report includes details about all executed test cases. |
Requirement/defect details | Specifies whether the report shows requirements, defects, tasks, and feature requests that are associated with a test. Only top-level test suites (below) must be disabled for this setting to take effect. |
Only top-level test suites | Determines whether the Test Suite Summary report section only lists the .tst files (with this option enabled) or displays a tree-like view of the individual tests in each .tst file (with this option disabled). |
Active static analysis rules | Determines whether the report lists the static analysis rules that were enabled for the test. |
Only tests that failed | Specifies whether the report lists only tests that failed. |
Generate formatted reports in command-line mode | Determines whether formatted reports are generated for tests run in command line mode. |
Cutoff date for graphs | Specifies the start date for trend graphs that track different categories of tasks over a period of time. |
If you are specifying a URL for the artifact (for example, as a SOAtest test property), that URL will be available as a hyperlink in HTML reports. If want to create hyperlinks and you are specifying artifact associations using only the artifact ID as specified in Associating Tests with Development Artifacts, you can use local properties to specify how the ID maps to a URL. For example:
|
Setting | Description |
---|---|
Format | Specifies the desired report format:
|
XSL file | If you chose custom XSL as the report format, specify the path to the XSL file that defines your custom format. |
Report file extension | If you want to use a file extension other than the default .html extension, specify that extension here. |
Generate additional archive with entire report content | Enable this option to generate an additional compressed archive (.zip) file in the specified report location. The ZIP file contains all the files generated to build the report. This option can generate an archive for any report format (for example, HTML, CSV, PDF, and so forth). By generating an archive, you can also perform custom transformations of the report because all of the elements are generated to the specified destination folder. |
Setting | Description |
---|---|
Session Tag | Session tags are unique identifiers for test runs and are used to distinguish specific runs from similar runs in DTP. The results from each of the team’s regular test runs should have a unique tag. Variables can be used as described in Using Preference Settings for Command-Line Execution. For instance, if your team runs static analysis, unit testing, and code review, you might use the session tags Static , Execution , and Code Review . Or you might use variables such as session.tag=${config_name} or session.tag=${analysis_type} |
Build ID | Specifies a build identifier used to label results. It may be unique for each build but may also label more than one test sessions that were executed during a specified build. The default is |
See Reporting Settings for details.
There are some limitations in the default reports produced by desktop installations from the GUI or command line. See the list below for details.
These limitations can be addressed by enabling the legacy report format. To enable the legacy report format, set the system property |
You can customize locally generated and emailed reports by building a custom XSL transformer that specifies how you want the results formatted. For instance, you could use a custom transformer to map data into the format you need to demonstrate compliance with an internal security policy.
To specify a custom report format, you need to:
Alternatively, you can specify this information in a settings file via the settings:(results.)report.custom.extension and (results.)report.custom.xsl.file.
The following parameters can be used in custom XSL files:
Parameter | Notes | |
---|---|---|
report_type=disk_report|email_report | Determines if the generated report will be sent by email or saved on the local disk drive. | |
test_params | The command line that was used to start the product that generated this report. Example:
| |
test_config_name | The name of the Test Configuration that was executed to produce this report. | |
output_dir=[dir] | The directory where the report is created. This can be used to generate developer reports. Example:
| |
rules_dir_path=[path] | The directory where rules documentation is saved. This can be used to generate rule popups/links. Example:
| |
dev_errors=true|false | The value of the "Task details" option. See Report Configuration Settings for details. | |
dev_reports=true|false | The value of the "Detailed report for developers" option. See Report Configuration Settings for details. | |
show_active_rules=true|false | The value of the "Active static analysis rules" option. See Report Configuration Settings for details. | |
associations=true|false | The value of the "Requirement/defect details" option. See Report Configuration Settings for details. | |
dev_reports_prefix=[prefix] | The prefix used to name developer reports. Example:
| |
exec_cvg_details=true|false | The value of the "Detailed report for" option. See Report Configuration Settings for details. | |
attachments=true|false | The value of the settings key report.mail.attachments. See Configuring Settings for details. | |
test_suites_only=true|false | The value of the settings key report.test_suites_only. See Configuring Settings for details. | |
test_cases_details=true|false | The value of the "Test case details" option. See Report Configuration Settings for details. | |
failed_tests_only=true|false | The value of settings key report.failed_tests_only. See Configuring Settings for details. | |
authors_details=true|false | The value of the "Overview of tasks by authors" option. See Report Configuration Settings for details. | |
contexts_details=true|false | The value of the "Overview of checked files and executed tests" option. See Report Configuration Settings for details. |