Parasoft Selenic analyzes the use of locators in your Selenium tests and provides recommendations for updating the locator to maintain test stability. After executing the tests and processing the results, you can view the recommendations in the Parasoft report or load them into your IDE.
In this section:
Viewing Recommendations in the IDE
The Selenic Recommendations view is an interface for improving the maintainability of your Selenium tests. It contains information about element locators used in the tests, as well as actions that you can take to view and update the locators. This section demonstrates viewing recommendations in Eclipse, but the same functionality is available in IntelliJ.
Selenic will generate and load recommendations into the IDE after executing a Selenic run configuration. By default, Selenic only generates recommendations for failed element locators, but you can configure the run configuration to generate recommendations for all locators in the Selenic Run Configurations screen.
You can also import recommendations from the report generated by an automated test run. Click the Import Recommendation link and specify the location of the JSON report file.
When the report loads, you can review the locator in the page object file by clicking the Go to Locator icon.
If your test does not follow the page object model, you can click the Go to Method icon to open the test method identified by the locator in the test file.
If your tests execute Cucumber feature files, you can click the Go to Step Definition icon to open the file containing the step definitions.
Manual Updates
- Click the View Recommended Locators link in the Selenic Recommendations view to access the recommended locators. Refer to About Locator Recommendations for information about understanding locators.
- Choose the locator you want to use and copy it to your clipboard.
- If your test follows the page object model, you can close the overlay and click the Go to Locator icon. If your test does not follow the page object model, click the Go to Method icon to open the test method identified by the locator in the test file.
- When the page or test file opens, you can paste the recommended locator into the file to update it. If your test does not follow the page object model, you will need to manually locate the bad locator within the test method in order to make the update.
Quick-fix Updates
Click the Update Locator link in the Selenic Recommendations view to access the recommended locators. You can choose the best locator for your test and click Update to instantly update the test. Refer to About Locator Recommendations for information about understanding locators.
You can also copy the locator to your clipboard and paste it into the test manually.
Viewing the Report
The report will open after running Selenic in the IDE if the Open HTML report after execution option is enabled (see Selenic Run Configurations). If the report fails to launch, you can open it from the <ECLIPSE_WORKSPACE>/.metadata/.plugins/etc directory.
Session Summary
The Started field marks the start of the analysis.
The values at the Tests to Review link indicate how many tests failed during the run per the total number of tests executed. Click the link to open the Details - Tests to Review section.
Click the Execution Summary link to view the Details - Execution Summary section. The Execution Summary section includes test names, execution times, and other details.
Summary - Tests to Review
This section provides an overview of the tests that passed and failed, as well as any issues that were addressed if the self-healing option was enabled.
If your team uses Applitools for visual testing, visual changes will be logged under Issues. See Command Line for information about integrating with Applitools.
If you are using the performance benchmarking feature, exceeded execution times will be logged under Issues. See Command Line for information about enabling performance benchmarking.
Details - Tests to Review
This section shows the names of any failed tests and a description of the issues that caused the failure.
If the screenshots option was enabled during execution, a link to the screenshot associated with the failure will be available.
Click on the failed test to open the test details view, which shows the following information:
- Recommendations for fixing issues
- Stacktrace of test failure
- Console errors (if logged)
If your team uses Applitools for visual testing, visual changes will appear this section. Links to the Applitools web application will also be included. See Command Line for information about integrating with Applitools.
If you are using the performance benchmarking feature, this section will also include information about tests that exceeded the average execution time. See Command Line for information about enabling performance benchmarking.
Details - Execution Summary
This section shows details about the pass/fail state for each test in the execution, as well as the duration for each test. This information is intended to help you determine a baseline for test or application performance.
The Average Time column shows the average of previous passing test runs not including the most recent run. If at least one test method fails, the entire test class is marked as failed.
If you are using the performance benchmarking feature, this section will also include information about tests that exceeded the average execution time. This information is intended to help you diagnose potential problems with your tests or application. See Command Line for information about enabling performance benchmarking. If at least one test method exceeds the average execution time, the entire test class is marked as exceeding the average execution time.
Viewing Results in DTP
If you configured Selenic to report results to DTP, you can add test widgets to your DTP dashboard to view the data. Reporting data to DTP is supported by running the Selenic analyzer from the command line. See Reporting Results to DTP for information about how to configure Selenic to report results to DTP. Refer to the Parasoft DTP documentation for information about how to create, configure, and view dashboards.
Any test failure (e.g., assertion failures, thrown exceptions, etc.) will be marked as failed in DTP. Healed tests will be marked as incomplete.
The following example Test widget shows a total of 11 tests.
Two of the tests failed. Of the nine passing tests, one test was automatically healed by Selenic. Clicking on the widget opens the DTP Test Explorer view, which shows additional information about the tests and provides an interface for prioritizing test remediation tasks.
Applitools Results Sent to DTP
Test runs will be marked as incomplete in DTP if Applitools reports visual differences. If the tests fail, Applitools execution will be aborted and the tests will be marked as failed in DTP. Issues related to Applitools failures will not be imported into the Recommendations view in the IDE.
About Locator Recommendations
Selenic assigns each locator a confidence factor, weight, and stability. The weight and stability are based on previous test runs. The confidence factor is calculated from the weight and stability and represents how confident Selenic is that a recommended locator should be used. This section provides background information for understanding Selenic locator recommendations.
Locator Recommendation Types
The Selenic Analyzer can recommend various types of locators to repair or improve existing locators. Each type of locator matches an element based on its text, name, ID, position, class name, or other criteria. Some recommendation types, such as the table locator, which locates a table cell based on adjacent text in a row and column, also combine criteria. The full list of locator recommendation types is described in the "locator_weights.properties" file in the Selenic installation directory. This file provides a description for each type, including examples.
Each locator recommendation type can also have multiple representations depending on the circumstances. For example, Selenic may recommend a locator that finds hyperlinks based on its text. This locator type has multiple representations in Selenium, including link text, partial link text, and XPath. For exact text matches, a link text locator would be recommended. For partial text matches, either a partial link text or an XPath locator would be recommended.
Locator Stability
Stability is an estimate of how well a type of locator recommendation is expected to work in the future given how well it would have worked in previous executions of the test. In order to leverage the test execution history, the tests must have been executed with Selenic.
For example, if a Selenium test has a locator that matches on a hyperlink, and all or part of the link's text has been constant from run to run, then Selenic would produce a link text recommendation with a high stability number. If the hyperlink, however, has also moved around the page from run to run, then a position-based locator recommendation would have very low stability reported.
Locator Confidence and Weight
Each recommendation has a confidence factor that is calculated from the locator's stability and weight. The weight is used to reduce the confidence of certain locator recommendations, even if the stability is high. For example, if an element did not move on the page from run to run, then Selenic may recommend position-based locators. The position-based locators would have a very high stability, but may have a very low confidence because positional locators tend to be more fragile than other types of locators. As a result, position-based locators have a low weight the reduce the overall confidence factor.
Configuring Locator Weights
The weights for each type of locator recommendation are configured in the "locator_weights.properties" file. The value should be between 0 and 1. A value of 1 means that any locator recommendations of that type will have a confidence factor identical to the stability number. A value of 0 is used to prevent locators of a particular type from ever being recommended.
# Locates an element by its text content. Builds a "link text" or # "partial link text" locator for a hyperlink, otherwise builds an "xpath" # locator. # examples: # link text: click me # partial link text: click # xpath: descendant::div[normalize-space(.)="The quick brown fox"] text_locator.link_text=.98 text_locator.partial_link_text=.98 text_locator.xpath=.98
