This topic explains how to use Load Test Continuum, which you can use (along with the command line interface) to set up an automated continuous performance testing process and easily monitor its results. In this section:

Introduction

Load Test Continuum is a Web application that allows users to organize data from multiple Load Test runs and access it through the browser interface.

The Load Test Continuum Web application works in conjunction with the Load Test command line scripting interface; this allows load test reports to be automatically sent to the Load Test Continuum after the completion of a load test.

The Load Test Continuum Web application, together with the relevant Load Test command line interface allow software development and testing teams to set up an automated continuous performance testing process and easily monitor its results.

Incorporating performance testing into the SDLC and running performance tests continuously as a part of the daily/nightly testing process offers the following benefits:

Eliminate SDLC uncertainty by performing load tests early and in a continuous automated fashion
Track performance of services/applications throughout the SDLC, thus discovering and resolving issues before they become a problem
Establish a QoS (Quality of Service) baseline for performance then detect deviations from that baseline as soon as they happen.

Deploying the Load Test Continuum Web Application

The Load Test Continuum Web application is packaged in a ltc.war archive, which is available in the SOAtest/LoadTest root installation directory.

To deploy Load Test Continuum on Tomcat Web server, place the ltc.war archive in the ${TOMCAT_HOME}/webapps directory. If you are using a different Web server that supports Java Web applications put the ltc.war in the appropriate Web application deployment directory of your server.

Configuring the Load Test Continuum Data Root

The Load Test Continuum Web application looks for the Load Test report data within a specific file path on the machine where it is installed. By default, this data root path is set to C:\\CONTINUOUS_LOAD_TEST.

Once the Load Test Continuum Web application is deployed, you can change the data root path. Open the ${TOMCAT_HOME}\webapps\ltc\WEB-INF\web.xml file in a text editor and set the param-value of the data_source_location parameter to the path where Load Test Continuum should look for load test data. You can also unjar the ltc.war, change the data_source_location in web.xml and jar the Web application if you want to reuse this configuration in multiple Load Test Continuum installations.

Sending Reports to Load Test Continuum

You can send load test reports to the Load Test Continuum by running Load Test from the command line (as described in Load Test Command Line Interface - cli).

The following example command from a Load Test script sends reports to Load Test Continuum. It is important that you keep your report path in the sequence illustrated by the script below:

${ltc-data-root}/${project-name}/%d/${category-name}/${test-name}

The ${ltc-data-root} variable should point to the Load Test Continuum data root (see Configuring the Load Test Continuum Data Root).

Set the value of the ${project-name} variable to the name of your project. This name will appear in the Project Selection View as described in the Exploring Load Test Continuum below.

%d is a wild card for current date, it can be replaced with a custom date of the YYYY-MM-DD format (for instance 2017-03-25).

You can combine the ${ltc-data-root} and ${project-name} into a single variable:

var ltc-data-root = C:\CONTINUOUS_LOAD_TEST
var project-name = YOUR_PROJECT_NAME
var project-home = ${ltc-data-root}/${project-name}

or

var project-home = C:\CONTINUOUS_LOAD_TEST\YOUR_PROJECT_NAME

The following example shows what the complete script would look like:

#
# Set variable values according to your configuration
#
var ltc-data-root = C:\CONTINUOUS_LOAD_TEST
var project-name = YOUR_PROJECT_NAME
var project-home = ${ltc-data-root}/${project-name}

var scenario = "Steady Load"
var base = tests/loadtester/accuracytest/tests
var category = BackComp.SOAtest.Accuracy
var minutes = 1

var test-name = 1-Profile-HPS
open ${base}/${test-name}.lt
loadtest -minutes ${minutes} -allReports ${project-home}/%d/${category}/${test-name}
${scenario}

Load Test Continuum organizes test report categories, reports and report metrics into a tree structure. Report categories can be organized into sub-categories of a desired number of levels. The "." (dot) character in the category path is interpreted as a category/subcategory delimiter.

For example, the string BackComp.SOAtest.Accuracy will be interpreted as a path with "BackComp" as a top level category, "SOAtest" as the first level sub-category and "Accuracy" as the second level sub-category. It will be displayed as a three-level tree structure in the Load Test Continuum Project view (see the screen shot in the Project View section below).

You can specify Team Server settings in a local settings file. You can create different local settings files for different projects, then use the -localSettings option of the loadtest command to indicate which file should be used for the current command line test. See Local Settings Files for more details.

Exploring Load Test Continuum

Open Load Test Continuum in a browser (i.e., http://yourserver:yourport/ltc) to begin exploring the interface.

Project Selection View

In the project selection view, click the link for the project whose results you want to explore. The appropriate Load Test Continuum project page will open.

Project View

The top of the Load Test Continuum Project View contains the Project History view, which includes:

Report Calendar
Test History Graph
Metric History Graph

Report Calendar

The calendar view in the upper left corner of the home page displays a monthly summary view.

To cycle through different months, clicking the arrow buttons:

To reload report contents, click the "Reload project data" icon:

Red and green highlights are used to indicate whether a given day’s tests failed or succeeded. You can click a highlighted date to view the Category/Test/Metric tree view for that day.

The daily report summary and tree view are displayed under the Project History view. You can click on the disclosure triangle at each node to expand and contract the report tree.

The Load Test report metrics are hidden by default. Click Show Metrics and Hide Metrics to toggle the metrics data open and closed.

Within the Load Test Reports view, there are various links for Test Reports and QoS Metric Graphs.

Test Reports

To view individual test report details, click the appropriate link:

History—For the history of that specific test and its QoS metrics.
HTML—For the Load Test load test HTML report.
Binary—For the binary Load Test load test report.

Individual QoS Metric Graphs

If a QoS metric Status Details string contains a number, this metric is considered as a "numeric" metric. A numeric value history graph is available for each numeric QoS metric.

To view the numeric metric history for a single test, click the "Metric history graph" graph icon next to the metric you want to view:

A numeric value history graphs will display:

The numeric QoS metric graph page displays the metric value graph in the center of the page and a navigation graph under the value graph. The navigation graph displays all the available data and a highlighted sliding window, which defines the time boundaries of the data graph. To navigate these graphs, you can:

Change the slider window width by clicking one of the Last 7|15|30 days and Last 12|26|52 weeks links in the panel above the value graph.
Drag the highlighted sliding window of the navigation graph at the bottom of the page; after the mouse is released, the value graph will reload. If the sliding window highlights the entire navigation graph, select a smaller time window interval.

Multiple QoS metric Graphs

If a QoS metric with a certain name was applied to multiple tests, you can see the history graphs of that metric for all the tests it was applied to...all on a single page.

To view history graphs for multiple tests, click the "All graphs of this metric" icon next to the metric you want to view:

Multiple QoS metric graphs of the same name display:

This type of view is useful when you want to see how a certain metric applies to all of the tests. For example, an "Average Execution Time" metric or a "Failure Count" metric. You can click on each summary graph to view further details and navigation options.

To view all metric graphs of a certain test, click the "All graphs of this test" icon:

Multiple QoS metric graphs of the same test display:

You can click on each summary graph to view further details and navigation options.

Test History and Metric History Graphs

These graphs are located next to the Report Calendar graph at the top of the Load Test Continuum home page. These graphs provide a high level view of how all tests and QoS metrics perform over time.

You can click on these graphs and navigate them in the same fashion as the Individual and Multiple QoS Metric graphs.

Page tree

Load Test Continuum