This topic explains how to use Load Test Continuum, which you can use (along with the command line interface) to set up an automated continuous performance testing process and easily monitor its results. In this section:
Load Test Continuum is a Web application that allows users to organize data from multiple Load Test runs and access it through the browser interface.
The Load Test Continuum Web application works in conjunction with the Load Test command line scripting interface; this allows load test reports to be automatically sent to the Load Test Continuum after the completion of a load test.
The Load Test Continuum Web application, together with the relevant Load Test command line interface, lets software development and testing teams set up an automated continuous performance testing process and easily monitor its results.
Incorporating performance testing into the SDLC and running performance tests continuously as a part of the daily/nightly testing process offers the following benefits:
Load Test Continuum is designed to function on many modern servers, but different combinations of server platform and Java version may affect the performance or usability of LTC. |
The Load Test Continuum Web application is packaged in a ltc.war archive, which is available in the SOAtest/LoadTest root installation directory.
To deploy Load Test Continuum on Tomcat Web server, place the ltc.war archive in the <TOMCAT_HOME>/webapps directory. If you are using a different Web server that supports Java Web applications, put the ltc.war in the appropriate Web application deployment directory of your server.
If you run Load Test Continuum outside the United States, do one of the following as appropriate for your setup:
Pass the following system properties to the web server JVM:
-Duser.country=US -Duser.language=en |
If you use Tomcat and launch it with a catalina.sh or catalina.bat script, add the following system properties to the script:
set "JAVA_OPTS=%JAVA_OPTS% -Duser.country=US -Duser.language=en" |
If you use Tomcat installed as a Windows service, launch the Tomcat9w.exe from the bin directory of your Tomcat installation. In the main window, select the Java tab and append the following lines to the Java Options text area and restart the Tomcat service:
-Duser.country=US -Duser.language=en |
The Load Test Continuum Web application looks for the Load Test report data within a specific file path on the machine where it is installed. By default, this data root path is set to C:\\CONTINUOUS_LOAD_TEST.
Once the Load Test Continuum Web application is deployed, you can change the data root path. Open the web.xml file in the ${TOMCAT_HOME}\webapps\ltc\WEB-INF\ directory in a text editor and set the param-value of the data_source_location parameter to the path where Load Test Continuum should look for load test data. You can also unjar the ltc.war file, change the data_source_location in web.xml and jar the Web application if you want to reuse this configuration in multiple Load Test Continuum installations.
You can send load test reports to the Load Test Continuum by running Load Test from the command line (as described in Load Test Command Line Interface - cli).
The following example command from a Load Test script sends reports to Load Test Continuum. It is important that you keep your report path in the sequence illustrated by the script below:
${ltc-data-root}/${project-name}/%d/${category-name}/${test-name}
The ${ltc-data-root} variable should point to the Load Test Continuum data root (see Configuring the Load Test Continuum Data Root).
Set the value of the ${project-name} variable to the name of your project. This name will appear in the Project Selection View as described in the Exploring Load Test Continuum below.
%d is a wild card for current date, it can be replaced with a custom date of the YYYY-MM-DD format (for instance 2023-03-25).
You can combine the ${ltc-data-root} and ${project-name} into a single variable:
var ltc-data-root = C:\CONTINUOUS_LOAD_TEST
var project-name = YOUR_PROJECT_NAME
var project-home = ${ltc-data-root}/${project-name} |
or
var project-home = C:\CONTINUOUS_LOAD_TEST\YOUR_PROJECT_NAME
The following example shows what the complete script would look like:
#
# Set variable values according to your configuration
#
var ltc-data-root = C:\CONTINUOUS_LOAD_TEST
var project-name = YOUR_PROJECT_NAME
var project-home = ${ltc-data-root}/${project-name}
var scenario = "Steady Load"
var base = tests/loadtester/accuracytest/tests
var category = BackComp.SOAtest.Accuracy
var minutes = 1
var test-name = 1-Profile-HPS
open ${base}/${test-name}.lt
loadtest -minutes ${minutes} -allReports ${project-home}/%d/${category}/${test-name}
${scenario} |
Load Test Continuum organizes test report categories, reports and report metrics into a tree structure. Report categories can be organized into sub-categories of a desired number of levels. The "." (dot) character in the category path is interpreted as a category/subcategory delimiter.
For example, the string BackComp.SOAtest.Accuracy will be interpreted as a path with "BackComp" as a top-level category, "SOAtest" as the first level sub-category and "Accuracy" as the second level sub-category. It will be displayed as a three-level tree structure in the Load Test Continuum Project view (see the screen shot in the Project View section below).
You can create different settings files for different projects, then use the -settings option of the loadtest command to indicate which file should be used for the current command line test. See Settings Files for more details.
Open Load Test Continuum in a browser (for example, http://<yourserver>:<yourport>/ltc) to begin exploring the interface.
In the project selection view, click the link for the project whose results you want to explore. The appropriate Load Test Continuum project page will open.
The top of the Load Test Continuum Project View contains the Project History view, which includes:
The calendar view in the upper left corner of the home page displays a monthly summary view.
To cycle through different months, click the arrows:
To reload report contents, click Reload project data:
Red and green highlights are used to indicate whether a given day’s tests failed or succeeded. You can click a highlighted date to view the Category/Test/Metric tree view for that day.
The daily report summary and tree view are displayed under the Project History view. You can click on the disclosure triangle at each node to expand and contract the report tree.
The Load Test report metrics are hidden by default. Click Show Metrics and Hide Metrics to toggle the metrics data open and closed.
Within the Load Test Reports view, there are various links for Test Reports and QoS Metric Graphs.
To view individual test report details, click the appropriate link:
If a QoS metric Status Details string contains a number, this metric is considered as a "numeric" metric. A numeric value history graph is available for each numeric QoS metric.
To view the numeric metric history for a single test, click the Metric history graph icon next to the metric you want to view:
A numeric value history graph will be shown:
The numeric QoS metric graph page displays the metric value graph in the center of the page and a navigation graph under the value graph. The navigation graph displays all the available data and a highlighted sliding window, which defines the time boundaries of the data graph. To navigate these graphs, you can:
If a QoS metric with a certain name was applied to multiple tests, you can see the history graphs of that metric for all the tests it was applied to...all on a single page.
To view history graphs for multiple tests, click the All graphs of this metric icon next to the metric you want to view:
Multiple QoS metric graphs of the same name are shown:
This type of view is useful when you want to see how a certain metric applies to all of the tests. For example, an "Average Execution Time" metric or a "Failure Count" metric. You can click on each summary graph to view further details and navigation options.
To view all metric graphs of a certain test, click the All graphs of this test icon:
Multiple QoS metric graphs of the same test are shown:
You can click on each summary graph to view further details and navigation options.
These graphs are located next to the Report Calendar graph at the top of the Load Test Continuum home page. These graphs provide a high-level view of how all tests and QoS metrics perform over time.
You can click on these graphs and navigate them in the same fashion as the Individual and Multiple QoS Metric graphs.