This topic explains how SOAtest test cases can be mapped to specific requirements in ALM / Quality Center. Once a test case is mapped to a requirement, the testing of the requirement becomes as simple as clicking a button inside ALM. Parasoft test cases can then be run from inside ALM and the results will be reported back to ALM.

ALM can run SOAtest test suites using the SOAtest command line interface (soatestcli), which can be used to run tests using soatestcli on the client's local machine.

In this section:

VAPI-XP Test Type Configuration and Test Creation

Creating a Test in the Test Plan Module

  1. After logging in to ALM, select the Test Plan module. The Test Plan module allows you to set up your automated test and custom scripts.
  2. In the Test Plan Tree, select Subject and click the New Folder toolbar button.
  3. In the New Folder dialog box, enter a folder name (for example, "SOAtest") in the Test Folder Name field.
  4. Select the newly created folder, then click the New Test icon from the above toolbar.
  5. In the Create New Test dialog box, select VAPI-XP-TEST in the Test Type menu, enter a test name (for example, "SOAtest Demo") in the Test Name field, then click OK.

  6. In the VAPI-XP Wizard select JavaScript as the Script Language, enter "SOAtest" for Script Name, then click Finish.
  7. Open the Test Script tab. Notice that a template has been created from the VAPI-XP wizard.
  8. Open the customized SOAtest script VAPI-XP-SOAtest.js.
    Note: the script assumes the default install location is c:/Progra~1/Parasoft/SOAtest/[SOAtest version number]/. The script is configured to run a test suite called example.tst inside the TestAssets project. 
  9. Copy the VAPI-XP-SOAtest.js script and replace it in the template.
  10. Modify the script as needed (see the following section for details).
  11. Save the script.

Modifying the Script

The VAPI-XP-SOAtest.js integration script is used in your test's Test Script tab in order to tell SOAtest to run your project file and report the test results back to ALM / QC. Modify testSuite and workspaceLocation as needed.

Variable Descriptions

The following table describes the variables that can be used to configure the script's behavior:

VariableDescriptionExample
testConfigurationThe name of the Test Configuration (same as the -config CLI argument).var testConfiguration = "builtin://Run Automated Server Tests";
testSuiteThe relative path to your SOAtest project file from the workspace.

var testSuite = "TestAssets/example.tst";

By default, the script is configured to run a test suite called "example.tst" inside the TestAssets project.

testNameThe name of the test in the test suite to run. (same as the -testName CLI argument).

var testName = "";
(leave as empty string "" to run all tests)

dataSourceRow dataSourceNameRuns tests with a single data source row (same as the -dataSourceRow and -dataSourceName CLI arguments).

var dataSourceRow = "";
var dataSourceName = "";
(leave as empty string "" to use all data source rows)

environmentThe name of the environment to enable during testing (same as the -environment CLI argument).

var environment = "";
(leave as empty string "" to use default active environment)

dataGroupConfig

The full path to the XML file that specifies the active data source for each data group within each .tst file contained in the test run.

Equivalent to the -dataGroupConfig command line argument.

var dataGroupConfig = "C:/configFiles/dataGroupConfig.xml";
environmentConfig

The full path to the XML file that specifies the environment variables to use for each .tst file contained in the test run.

Equivalent to the -environmentConfig command line argument.

var environmentConfig = "C:/configFiles/environmentConfig.xml";
reportDirThe location of the report directory.var reportDir = "%TEMP%/parasoft/soatest/reports";
clearReportDirIf true, removes all files from the report directory before running SOAtest. var clearReportDir = true;
detailedReportingControls the level of reporting. If true, test results will show detailed information about each test run. If false, only information about test failures will be reported.var detailedReporting = true;
SOAtestExeThe full path to soatestcli.exe in your SOAtest installation location.

var SOAtestExe = "C:/Progra~1/Parasoft/SOAtest/<version>/soatestcli.exe";

workspaceLocation

The full path to the workspace var workspaceLocation = "C:/Users/name/soatest/workspace";
Use "" to specify the default workspace location.

var workspaceLocation = "C:/Docume~1/soatest-workspace/";

additionalArgs

Passes additional command line arguments to SOAtest. For example, set this to "-testName Custom" to only run the test named Custom. To run the entire test suite, leave this as empty string ""
var additionalArgs = "";

var additionalArgs = ""-testName Custom";

Running a Test in the Test Lab Module

To run a test in the Test Lab module:

  1. Open the Test Lab module.
  2. Create a new folder from the Root node.
  3. Add a new Test Set to the new folder.
  4. From the Test Plan Tree (on some ALM versions, open it by clicking Select Tests at the top left of the toolbar), find the test that was created from Test Plan and add it to the new test set.

  5. Once the test is added, select and run the test. Assuming that SOAtest is installed locally, ensure that the Run All Tests Locally option is checked, then click the Run All or Run button.

Understanding Test Results

After a test run is completed, you can see results summaries and details as you would for other tests. 

For each test run, Test Center’s status column will indicate the overall success of your SOAtest test suite. For each test run, it will also show a list of run steps that correspond to each test that was executed within the SOAtest test suite. The status column for the run step will indicate whether that test passed or failed.

Detailed information about a test can be found by looking at the run step's Description field, which contains the following:

  • Test Information: Includes test name, test duration, and status.
  • Traffic: Only available for traffic-generating tests. Will show response time, the request message, and response message.
  • Error Source, Error Summary, Error Detail: These fields will only appear if the test failed. The Error Source will include the name of the tool that reported the error and the corresponding data source column, if applicable. The Error Summary and Error Detail sections show the brief and detailed messages corresponding to the error.

Troubleshooting

When running tests with ALM, you may encounter the following error messages.

Error MessageResolution
You do not have the required permissions to execute this action

This error is thrown from within ALM when the QC user running a test from test lab module does not have one or more permissions turned on. The QC script that is shipped with SOAtest requires specific permissions to be turned on in order for the script to successfully perform certain tasks.

If you see this error, you might not have the following permission set for the group that the logged in QC user belongs to: Status [RUN] (Modify Run > Status [RUN]).

To view/change permissions for a group that a QC user belongs to in Quality Center, do the following:

  1. Choose Tools > Customize.
  2. Select Groups.
  3. In the left, select the group that the QC user belongs to.
  4. Click the Change button on the right.

To check that permission Status [RUN] is enabled:

  1. From the gear icon in the top-right, choose Customize.
  2. Choose Groups and Permissions.
  3. Choose the QC user's group.
  4. Click Permissions.
  5. Choose TestLab.
  6. Expand the Run/Update node.
  7. Find the Status nested node and ensure the Status [RUN] permission is enabled.
     
SOAtest XML test results not found!Verify the xmlReportLocation variable is correct. SOAtest must also have write permissions on the folder where the XML file is to be written.
  • No labels

Need assistance? Visit our support page