When your team has hundreds or thousands of manual tests defined, it is difficult to know which ones need to be run when testing recent changes or doing regression testing for release. Without a good optimization strategy, your team ends up running more tests than needed and/or neglecting to run some important tests needed to validate the recent changes. The Manual Testing module identifies an optimized subset of manual tests that need to be run to sufficiently test application changes, allowing your team to focus testing on only a specific set of tests while still having the confidence that sufficient testing was done.
The Manual Testing module provides a real-time view of the execution status of each manual test for a given application in a given environment—passed, failed, incomplete, impacted (needs retesting), or not executed. This view automatically updates as you execute tests or whenever the application changes. If the underlying code that is associated with a previously executed manual test changes, the view will automatically update the test status to show that it is impacted and needs retesting. With this summary, you can focus your efforts on manual tests that are impacted or haven't been run yet. If a test is shown as passing and is not impacted, you know that you don't need to run the test again, even if the build has been updated since you ran the test.
To enable this workflow, first you import or create your manual test scenarios within CTP. You also configure Java or .NET coverage agents on each component within your application. Then you start a test session within CTP. As you execute each manual test, you start and stop each test in CTP to tell the system which test you are running and to collect the associated code coverage. CTP creates a mapping between each manual test and the specific code exercised by that test, which is continuously updated as tests are run. When a new build is delivered, the system automatically compares the source code or application binaries to identify the specific code that was changed, and then highlights specific manual tests that need to be run to validate those changes.
The Manual Testing module also allows you to measure the code coverage obtained by your manual test scenarios, and to aggregate that code coverage data with the code coverage data collected by your other testing practices such as automated unit testing or functional testing. Parasoft DTP is required for code coverage reporting.
Prerequisites
Before using the Manual Testing module, ensure that:
- You have set up a System and Environment in CTP representing the application under test (AUT). See Defining Systems and Environments for more information.
- Your application under test (AUT) has been configured for a coverage agent. See Configuring the Application Under Test for Coverage for more information.
- You have connected the components in your system to coverage agents. See Connecting CTP to Coverage Agents for more information.
- (Optional) CTP is connected to DTP. This is necessary if you want to send manual testing data to DTP for viewing results and coverage details. See Connecting to DTP for more information.
