Data Collector is the component that accepts reports from DTP Engines, as well as from third-party testing and code analysis tools. You can change Data Collector’s default storage and upload limits to meet data requirements for your organization.

In this section:

Changing Report Storage Threshold

When Data Collector receives and processes files from analyzers, a report file is processed and moved to the [DTP_HOME]/data/_DEFAULT_/dc/stored folder. This folder contains subfolders that are structured according to a YEAR/MONTH/DAY/XXX/YYY format where XXX and YYY are integer values. The values are calculated based on file name.

You can set a maximum size for the stored folder and DTP will automatically prune older entries.

  1. Open the DCConfig.xml file located in the [DTP_HOME]/grs/config/ directory and locate the following entries:

    <stored-folder>
	<max-size>20</max-size>
	<size-to-clean>5</size-to-clean>
</stored-folder>
  2. Change the value for the <max-size> property (in GB) to an acceptable maximum storage capacity; the default is 20 GB.
  3. Change the value for the <size-to-clean> property (in GB) to the amount of data you want pruned when the maximum capacity is reached; the default is 5 GB.
  4. Save the file.

If these entries do not appear in the DCConfig.xml file, then the defaults will be used.

Changing Maximum Size of Report Upload

By default, the maximum size of the XML report file that Data Collector can accept is 512M. You can change the limit by adding the following Data Collector JVM argument:

-Dcom.parasoft.sdm.api.rawstorage.datacollector.uploadMaxSize=512

For Linux installations, you can modify the following string located in the [DTP_HOME]/bin/variables file:

JAVA_DC_CONFIG_ARGS=" -Dcom.parasoft.sdm.api.rawstorage.datacollector.uploadMaxSize=512"

Setting Retention Period for Metrics Details

You can set the number of days that metric data from DTP Engines will be stored in the DTP database.

  1. Stop the Data Collector service. See Stopping DTP Services.

  2. Open the DTP_HOME/grs/config/DCConfig.xml configuration file and locate the following setting:

    <!-- Number of days that metric details should be kept in database. -->
<!--metric-details-retention-period>8</metric-details-retention-period-->

  3. Uncomment the setting and change the value to reflect how many days you want to store metric data in the database. The default is eight (8) days:

    <metric-details-retention-period>16</metric-details-retention-period>

    If the setting is commented-out or no value is specified, then the default value is used.

  4. Save the file and restart Data Collector.

Changing Test Failures Threshold

By default, Data Collector will reject reports that contain more than 5000 reported test failures. You can change the limit by adding the following Data Collector JVM argument:

-Dcom.parasoft.sdm.rawstorage.failures.limit=5000

For Linux installations, you can modify the following string located in the [DTP_HOME]/bin/variables file:

JAVA_DC_CONFIG_ARGS=" -Dcom.parasoft.sdm.rawstorage.failures.limit=5000"

If a negative value is provided, Data Collector will not reject reports due to number of test failures.

Controlling Coverage Data Processing for DTP Enterprise Pack Artifacts

By default, Data Collector does not process coverage data for the Change Based Testing (CBT) or Risky Code Changes (RCC) DTP Enterprise Pack artifact. This is to reduce the potential performance impact associated with the running the artifacts. If you want to use these artifacts, you must enable coverage data processing in the ReducedCoverageConfig.xml configuration file located in the DTP_HOME/conf/ directory. This file enables you to specify a list of coverage images that you want Data Collector to process. 

If your coverage report files exceed 1 GB and you send multiple reports per build, you may experience an extra five minutes per GB in Data Collector processing time.


  1. Open the file and set the value of the type parameter in the <images> element to either accept or reject:

    <images type="accept">
     
    Setting the parameter to accept means that Data Collector will only accept coverage data for the coverage images in the list. Setting the parameter to reject means Data Collector will accept coverage data for all coverage images except for those in the list. 

  2. Specify the coverage images to accept or reject by adding them in <image> elements within the <images> element:

    <images type="accept">
   <image>Example Coverage</image>
   <image>Test Coverage</image>
</images>
 
  3. Save the file.

See the XML file for more examples of accept or reject settings.