This topic explains how to use Load Test Continuum, which you can use (along with the command line interface) to set up an automated continuous performance testing process and easily monitor its results. In this section:
Table of Contents | ||
---|---|---|
|
Introduction
Load Test Continuum is a Web application that allows users to organize data from multiple Load Test runs and access it through the browser interface.
The Load Test Continuum Web application works in conjunction with the Load Test command line scripting interface; this allows load test reports to be automatically sent to the Load Test Continuum after the completion of a load test.
The Load Test Continuum Web application, together with the relevant Load Test command line interface allow software development and testing teams to set up an automated continuous performance testing process and easily monitor its results.
Incorporating performance testing into the SDLC and running performance tests continuously as a part of the daily/nightly testing process offers the following benefits:
- Eliminate SDLC uncertainty by performing load tests early and in a continuous automated fashion
- Track performance of services/applications throughout the SDLC, thus discovering and resolving issues before they become a problem
- Establish a QoS (Quality of Service) baseline for performance then detect deviations from that baseline as soon as they happen.
Info | ||
---|---|---|
| ||
Load Test Continuum is designed to function on many modern servers, but different combinations of server platform and Java version may affect the performance or usability of LTC. |
Deploying the Load Test Continuum Web Application
The Load Test Continuum Web application is packaged in a ltc.war archive, which is available in the SOAtest/LoadTest root installation directory.
To deploy Load Test Continuum on Tomcat Web server, place the ltc.war archive in the ${TOMCAT_HOME}/webapps directory. If you are using a different Web server that supports Java Web applications put the ltc.war in the appropriate Web application deployment directory of your server.
Configuring the Load Test Continuum Data Root
The Load Test Continuum Web application looks for the Load Test report data within a specific file path on the machine where it is installed. By default, this data root path is set to C:\\CONTINUOUS_LOAD_TEST.
Once the Load Test Continuum Web application is deployed, you can change the data root path. Open the ${TOMCAT_HOME}\webapps\ltc\WEB-INF\web.xml file in a text editor and set the param-value
of the data_source_location
parameter to the path where Load Test Continuum should look for load test data. You can also unjar the ltc.war, change the data_source_location
in web.xml and jar the Web application if you want to reuse this configuration in multiple Load Test Continuum installations.
Sending Reports to Load Test Continuum
You can send load test reports to the Load Test Continuum by running Load Test from the command line (as described in Load Test Command Line Interface - cli).
The following example command from a Load Test script sends reports to Load Test Continuum. It is important that you keep your report path in the sequence illustrated by the script below:
${ltc-data-root}/${project-name}/%d/${category-name}/${test-name}
The ${ltc-data-root}
variable should point to the Load Test Continuum data root (see Configuring the Load Test Continuum Data Root).
Set the value of the ${project-name} variable to the name of your project. This name will appear in the Project Selection View as described in the Exploring Load Test Continuum below.
%d
is a wild card for current date, it can be replaced with a custom date of the YYYY-MM-DD format (for instance 2017-03-25
).
You can combine the ${ltc-data-root}
and ${project-name}
into a single variable:
Code Block |
---|
var ltc-data-root = C:\CONTINUOUS_LOAD_TEST var project-name = YOUR_PROJECT_NAME var project-home = ${ltc-data-root}/${project-name} |
or
var project-home = C:\CONTINUOUS_LOAD_TEST\YOUR_PROJECT_NAME
The following example shows what the complete script would look like:
Code Block |
---|
# # Set variable values according to your configuration # var ltc-data-root = C:\CONTINUOUS_LOAD_TEST var project-name = YOUR_PROJECT_NAME var project-home = ${ltc-data-root}/${project-name} var scenario = "Steady Load" var base = tests/loadtester/accuracytest/tests var category = BackComp.SOAtest.Accuracy var minutes = 1 var test-name = 1-Profile-HPS open ${base}/${test-name}.lt loadtest -minutes ${minutes} -allReports ${project-home}/%d/${category}/${test-name} ${scenario} |
Load Test Continuum organizes test report categories, reports and report metrics into a tree structure. Report categories can be organized into sub-categories of a desired number of levels. The "." (dot) character in the category path is interpreted as a category/subcategory delimiter.
For example, the string BackComp.SOAtest.Accuracy
will be interpreted as a path with "BackComp
" as a top level category, "SOAtest
" as the first level sub-category and "Accuracy
" as the second level sub-category. It will be displayed as a three-level tree structure in the Load Test Continuum Project view (see the screen shot in the Project View section below).
You can create different settings files for different projects, then use the -settings option of the loadtest command to indicate which file should be used for the current command line test. See Settings Files for more details.
Exploring Load Test Continuum
Open Load Test Continuum in a browser (i.e., http://yourserver:yourport/ltc) to begin exploring the interface.
Project Selection View
In the project selection view, click the link for the project whose results you want to explore. The appropriate Load Test Continuum project page will open.
Project View
Anchor | ||||
---|---|---|---|---|
|
The top of the Load Test Continuum Project View contains the Project History view, which includes:
- Report Calendar
- Test History Graph
- Metric History Graph
Report Calendar
The calendar view in the upper left corner of the home page displays a monthly summary view.
To cycle through different months, clicking the arrow buttons:
To reload report contents, click the "Reload project data" icon:
Red and green highlights are used to indicate whether a given day’s tests failed or succeeded. You can click a highlighted date to view the Category/Test/Metric tree view for that day.
The daily report summary and tree view are displayed under the Project History view. You can click on the disclosure triangle at each node to expand and contract the report tree.
The Load Test report metrics are hidden by default. Click Show Metrics and Hide Metrics to toggle the metrics data open and closed.
Within the Load Test Reports view, there are various links for Test Reports and QoS Metric Graphs.
Test Reports
To view individual test report details, click the appropriate link:
- History—For the history of that specific test and its QoS metrics.
- HTML—For the Load Test load test HTML report.
- Binary—For the binary Load Test load test report.
Individual QoS Metric Graphs
If a QoS metric Status Details string contains a number, this metric is considered as a "numeric" metric. A numeric value history graph is available for each numeric QoS metric.
To view the numeric metric history for a single test, click the "Metric history graph" graph icon next to the metric you want to view:
A numeric value history graphs will display:
The numeric QoS metric graph page displays the metric value graph in the center of the page and a navigation graph under the value graph. The navigation graph displays all the available data and a highlighted sliding window, which defines the time boundaries of the data graph. To navigate these graphs, you can:
- Change the slider window width by clicking one of the Last 7|15|30 days and Last 12|26|52 weeks links in the panel above the value graph.
- Drag the highlighted sliding window of the navigation graph at the bottom of the page; after the mouse is released, the value graph will reload. If the sliding window highlights the entire navigation graph, select a smaller time window interval.
Multiple QoS metric Graphs
If a QoS metric with a certain name was applied to multiple tests, you can see the history graphs of that metric for all the tests it was applied to...all on a single page.
To view history graphs for multiple tests, click the "All graphs of this metric" icon next to the metric you want to view:
Multiple QoS metric graphs of the same name display:
This type of view is useful when you want to see how a certain metric applies to all of the tests. For example, an "Average Execution Time" metric or a "Failure Count" metric. You can click on each summary graph to view further details and navigation options.
To view all metric graphs of a certain test, click the "All graphs of this test" icon:
Multiple QoS metric graphs of the same test display:
You can click on each summary graph to view further details and navigation options.
Test History and Metric History Graphs
These graphs are located next to the Report Calendar graph at the top of the Load Test Continuum home page. These graphs provide a high level view of how all tests and QoS metrics perform over time.
You can click on these graphs and navigate them in the same fashion as the Individual and Multiple QoS Metric graphs.