Details

Recommendation type: Potential setup problems

Configurable in Preferences: Yes

Filter category: Potential setup problems

Description

If tests fail in bulk with the same exception or for the same reason, it is an indicator that there are potential setup problems in the project. The Potential setup problems recommendation is dedicated to diagnosing the root cause of such behavior. The Potential setup problems recommendation may highlight one or more exceptions that hide the root causes of test failures and may not be necessarily discovered by the Exceptions and assertion errors recommendation.

UTA displays this recommendation when:

• the same exception is thrown in two or more unit tests that were executed for a class in the same place in the code
• an exception is not detected in the unit test, but is caught e.g. by a try-catch block

The following examples illustrate these scenarios:

• For non-void methods, an exception is thrown in the try block, caught and logged in the catch block, and null is returned. Usually, in such cases, subsequent calls on the null object raise a NullPointerException.
• An exception is thrown in the try block, caught, and boxed in another exception (for example, a user-defined exception). In this case, only the last thrown exception will be observed in the test, and the real cause will remain hidden.
• An exception is thrown in the try block, caught, and rethrown as another exception (for example, a user-defined exception). In this case, only the rethrown exception will be observed in the test, only indirect indications of the real cause may be observed.
• For void methods, an exception is thrown in the try block, caught, and logged. In such cases, there will be no exception in the test. However, due to the unusual behavior, the tests may fail in bulk during the run

Usually, the cause of an unexpected exception is related to:

• Missed dependencies or libraries that may not be discovered in very complex projects.
• Misconfiguration of the project setup.
• Misconfiguration between the project language version and the language version supported by third-party libraries, among others.

For bulk creation, this may lead to a large number of removed tests since they will not contribute to coverage. The Potential setup problems recommendation is designed to discover such issues that may affect test generation or their execution. 

For information on diagnosing potential setup problems, see How can I diagnose potential setup problems?

Depending on your code, the following action links may be available:

• Highlight - Highlights the issue in the execution flow displayed in the Unit Test Assistant view.
• Go to - Highlights the code associated with the recommendation in the editor (for example, the code that throws the exception).Examples

Example 1 – Multiple Exceptions Thrown

The following two tests have been executed:

The tests receive the same exception from the same place in the code:

Example 2 - Exception Caught by Try-Catch

The following test has been created:

An exception is not detected in the unit test, but it is caught by a try statement for the following line of code:

In this example another exception, shown in the catch statement, is rethrown in the unit test.

UTA detects the caught exception: