Data Collector is the component that accepts reports from testing and code analysis tools. You can change Data Collector’s default storage and upload limits to meet data requirements for your organization.

In this section:

Configuring the Report Storage Threshold

When Data Collector receives and processes files from analyzers, a report file is processed and moved to the <DTP_DATA_DIR>/data/_DEFAULT_/dc/stored folder. This folder contains subfolders that are structured according to a YEAR/MONTH/DAY/XXX/YYY format where XXX and YYY are integer values. The values are calculated based on file name.

You can set a maximum size for the stored folder and DTP will automatically prune older entries.

Open the DCConfig.xml file located in the <DTP_DATA_DIR>/grs/config/ directory and locate the following entries:
```
<stored-folder>
	<max-size>20</max-size>
	<size-to-clean>5</size-to-clean>
</stored-folder>
```
Change the value for the <max-size> property (in GB) to an acceptable maximum storage capacity; the default is 20 GB.
Change the value for the <size-to-clean> property (in GB) to the amount of data you want pruned when the maximum capacity is reached; the default is 5 GB.
Save the file.

If these entries do not appear in the DCConfig.xml file, then the defaults will be used.

Configuring the Maximum Size of Report Uploads

By default, the maximum size of the XML report file that Data Collector can accept is 512M. You can change the limit by adding the following Data Collector JVM argument:

-Dcom.parasoft.sdm.api.rawstorage.datacollector.uploadMaxSize=512

For Linux installations, you can modify the following string located in the <DTP_INSTALL>/bin/variables file:

JAVA_DC_CONFIG_ARGS=" -Dcom.parasoft.sdm.api.rawstorage.datacollector.uploadMaxSize=512"

Configuring Data Retention Settings

You can configure the amount of unit testing, coverage, resource coverage, and metrics data that DTP stores. Test, coverage, and metrics data are collectively referred to as "test details" because together they provide critical testing information about the build. The data from these practices consume substantial storage, so DTP preserves space by regularly pruning the data associated with older builds in the filter.

About Resource Coverage

DTP collects coverage at the method level and at the resource level. Method-level coverage data is required to view coverage in the Coverage Explorer. Unless explicitly stated, "coverage" in this documentation means method-level coverage.

Resource coverage data refers to the code coverage data associated with resource groups. A resource group is a collection of files and/or folders defined by a set of one or more ANT file patterns. Resource groups are subsets of a project that you can associate with a filter. They enable you to view software quality information for specific parts of the code. See Adding Resource Groups to Projects for additional information about resource groups.

Default Data Retention Settings

The amount of data stored is based on the number of historical builds, as opposed to a hard limit, such as gigabytes. By default, DTP keeps the following data:

Unit test data: two (2) historical builds
Coverage data: two (2) historical builds
Resource coverage data: ten (10) historical builds
Metrics data: ten (10) historical builds.

Configuration

To configure the data retention settings:

Stop the Data Collector service. See Stopping DTP Services.

Open the <DTP_DATA_DIR>/grs/config/DCConfig.xml configuration file and locate the <details-retention-builds-count> group of settings:

<!-- <details-retention-builds-count>
        <tests>2</tests>
        <coverage>2</coverage>
		<resource-coverage>10</resource-coverage>
        <metrics>8</metrics>
 </details-retention-builds-count>-->

Uncomment the settings and specify how much data should be stored for each practice. The values refer to number of builds. Specifying 1, for example, means storing one build worth of data for the practice.
```
<details-retention-builds-count>
	<tests>3</tests>
    <coverage>4</coverage>
	<resource-coverage>12</resource-coverage>
    <metrics>9</metrics>
</details-retention-builds-count>
```
Save the file and restart Data Collector and DTP Server.

Configuring Test Failures Threshold

By default, Data Collector will reject reports that contain more than 5000 reported test failures. You can change the limit by adding the following Data Collector JVM argument:

-Dcom.parasoft.sdm.rawstorage.failures.limit=5000

For Linux installations, you can modify the following string located in the <DTP_INSTALL>/bin/variables file:

JAVA_DC_CONFIG_ARGS=" -Dcom.parasoft.sdm.rawstorage.failures.limit=5000"

If a negative value is provided, Data Collector will not reject reports due to number of test failures.

Disabling Security Layer Protocols and Cipher Suites

SSL is enabled by default in DTP, but you can disable SSL protocols and/or cipher suites if you experience security-related issues associated with your Parasoft installation:

Stop the Data Collector service. See Stopping DTP Services.

Open the the <DTP_DATA_DIR>/conf/DCServerConfig.xml configuration file and specify the protocols and/or cipher suites to exclude in the appropriate elements:

<!-- comma separated list of protocols to exclude
 By default, TLSv1, TLSv1.1 are excluded -->
<excluded-ssl-protocols>TLSv1,TLSv1.1</excluded-ssl-protocols>
<!-- comma separated list of cipher suites to exclude -->
<excluded-cipher-suites></excluded-cipher-suites>

Restart Data Collector.

Reverse Proxy Configuration

Reverse proxies are sometimes used to ensure high availability or strengthen network security. If your organization uses a reverse proxy, you can configure Data Collector to work in your reverse proxy environment. In addition to configuring the Data Collector configuration file, you must also configure the reverse proxy server to forward requests to the correct ports. See Reverse Proxy Support for additional details.

Open the DCServerConfig.xml configuration file located in the <DTP_DATA_DIR>/conf directory.
Specify values for the following elements:
- <dc-reverse-proxy-protocol> : The protocol (HTTP or HTTPS) on the reverse proxy server that users connect to.
- <dc-reverse-proxy-port>: The port on the reverse proxy server that users connect to.
- <dc-reverse-proxy-host>: The host name of the reverse proxy server that users connect to.
- <dc-reverse-proxy-path>: (Optional) Specify a path that the reverse proxy server can use to upload data to DTP through Data Collector. This option enables you to provide access to Data Collector, Report Center, and other DTP applications if they have been configured to use a single context path. You will only need to configure the path if your reverse proxy is configured to support single context paths. The path must begin with /.
Save the file and restart Data Collector (see Starting DTP Services).

Example Configuration

In the following example, Parasoft reports are sent to https://reverse-proxy123.example.com:7777/dtp/dc, which forward to http://your-dtp-server.example.com:8082:

<root-config>
<!-- fields that always need to be are not part of the reverse proxy setup -->
<dc-server-protocol>http</dc-server-protocol>
<dc-server-port>8082</dc-server-port>
<!-- reverse proxy setup -->
<dc-reverse-proxy-protocol>https</dc-reverse-proxy-protocol>
<dc-reverse-proxy-port>7777</dc-reverse-proxy-port>
<dc-reverse-proxy-host>reverse-proxy123.example.com</dc-reverse-proxy-host>
<dc-reverse-proxy-path>/dtp/dc</dc-reverse-proxy-path>
... other fields ...
</root-config>

If these fields are not set, then a Parasoft tool (Jtest, etc.) will fail when uploading a report to Data Collector.

You should also configure your reverse proxy to handle large request payloads. This is because Parasoft tools can generate reports that are megabytes in size. Refer to the documentation for your reverse proxy server for details.

Page tree

Configuring Data Collector