In this section:

Introduction

You can monitor and collect coverage data from managed code during manual or automated functional tests performed on a running web application that is deployed on IIS server. 

The dotTEST coverage agent is attached to the application under test (AUT) and monitors the code being executed as the AUT runs. When the coverage agent is attached to the AUT, a REST API is exposed that enables you to mark the beginning and end of each test and test session. During test execution, interactions with the coverage agent and AUT are written to a dynamic coverage data file, which contains markers that specify which lines of code were touched. dotTEST processes the collected coverage data and creates a coverage.xml file, which can be sent to DTP. 

Test results are also sent to DTP from the tool that executes the tests (i.e., SOAtest, tests executed by dotTEST, manual tests, etc.) in a report.xml file. If the build IDs for the coverage.xml file and the report match, DTP is able to correlate the data and display the coverage information.

Prerequisites

To collecting coverage, Internet Information Services (IIS) version 7.5 or higher is required.

Configuring the Application Under Test for Coverage

The following steps are required to prepare the application under test (AUT):

1. Optional: Generate the static coverage file with source code information. The static coverage file contains metadata about user classes, methods, and lines; see Generating the Static Coverage File with Source Code Information.
2. Attach the coverage agent to the AUT; see Attaching the Coverage Agent to the AUT.

Attaching the Coverage Agent to the AUT 

1. Copy the <INSTALLATION_DIR>\integration\coverage\agent directory to the machine where IIS is installed and the web application is deployed.
2. Run a console as an Administrator

3. Invoke the Agent Client tool on this machine to enable runtime coverage collection inside IIS:

   agent_client.exe

   You may need to configure the Agent Client with additional options, see Agent Client Options.

   agent_client initializes the environment for the web server (IIS) and behaves like a service, enabling you to execute tests and collect coverage. The service is ready and waiting for commands as long as the following message is printed to the output:  

   Write 'exit' and hit Enter to close agent_client 

   A test session and test can be started even if the tested website or application has not been loaded yet.

4. Ensure that port 8050 (default port for the coverage agent) allows HTTP traffic in firewall settings on this machine. You can change the coverage agent port number if the default port is unavailable.
5. Open the website or application.

6. Go to the following address to check the status of the coverage agent: http://host:8050/status
   You should receive the following response:

   {"session":null,"test":null}

Connecting with the Coverage Agent via HTTPS

By default, the coverage agent is available via the HTTP protocol. To connect via HTTPS, you need to bind an SSL certificate to the port used by the coverage agent. This can be achieved in one of the following ways:

• By adding your SSL certificate to a storage accessible from the Netsh.exe tool and passing the hash of the certificate and the port number to agent_client.exe:

  agent_client.exe -port 8050 -useSsl -certificateHash <SSL certificate hash>

• By passing the path and password to an SSL certificate stored in a .pfx file and the port number to agent_client.exe:

  agent_client.exe -port 8050 -useSsl -certificatePath <path to PFX> -certificatePassword <password>

• By configuring a port with an SSL certificate and the appid of the Agent Client for the Netsh.exe tool. The Agent Client has the following ID:

  {ed344e08-fab7-4dfb-9d07-68f2c2c9e373}

  When the SSL certificate is configured in the Netsh.exe tool, you can connect via HTTPS by launching the Agent Client with the -port and -useSsl switches:

  agent_client.exe -port 8050 -useSsl

To use a self-signed certificate, you must add the -skipvalidation option.

You can use an SSL certificate in the in PFX or PKCS12 format.

Removing an SSL Certificate

By default, the Agent Client automatically handles certificates configured for the coverage agent. To manually remove a certificate, launch the Certmgr.exe tool and remove the certificate stored in Personal> Certificates.

If you manually unregister the coverage service port using the Agent Client with the -unregister switch, you need to add the -useSsl and -removeCertificate switches to remove the SLL certificate:

agent_client.exe -unregister -port 8050 -useSsl -removeCertificate

This will delete the SSL certificate from the port number. You can now manually remove the certificate from the storage.

Collecting Coverage from Multiple Users

You can collect coverage information for multiple users that are simultaneously accessing the same web application server. This requires launching the Agent Client with the -multiuser switch:

agent_client.exe -multiuser

Changing Agent Client Idle Timeouts

By default, the Agent Client application pool processes are recycled after 20 minutes of idle time, which can have negative consequences on a test session. You can prevent this behavior by changing the default value so that people working with the application do not experience unexpected stops and restarts during a test session.

1. Start the IIS Manager.
2. Open the Application Pools node.
3. Choose the pool for your web application.
4. Click Advanced Settings in the Actions panel.
5. In Process Model section, change the Idle Time-out (minutes) setting to a value better-suited to your testing practices.

Agent Client Options

Executing Tests to Collect Runtime Coverage

To collect runtime coverage:

• You can use SOAtest to run functional tests. When tests are being performed SOAtest will collect runtime coverage, and upload the data to DTP. Refer to the SOAtest user guide for details about performing test sessions.
• You can execute tests on an application with the attached coverage agent. Coverage will be collected in the runtime coverage directory. Next, you need to merge the runtime coverage with static coverage and upload the merged data to DTP. For details, see Creating a Coverage Report and Uploading the Data to DTP.

Stopping Runtime Coverage Data Collection

You can stop the process of collecting runtime coverage data in one of the following ways:

1. Write exit in the open console when the following message will be printed to the output to stop agent_client:

   Write 'exit' and hit Enter to close agent_client

2. Send a request to the service by entering the following URL in the browser: http://host:port/shutdown

If any errors occur when agent_client exits, which prevent the clean-up of the Web Server environment, then execute agent_client with the -stop option to bring back the original Web Server environment and settings: 

agent_client.exe -stop

Creating a Coverage Report and Uploading the Data to DTP

See Creating a Coverage Report and Uploading the Data to DTP.

Known Limitations

• You can download coverage information that was collected in a test session. Coverage data collected when no test session was active cannot be downloaded.
• If multiple users are simultaneously accessing the same web application, the coverage data they collect may be mixed. To ensure that coverage is properly associated with individual users, the multiuser mode must be enabled (see Collecting Coverage from Multiple Users).
• The HTTP or HTTPS protocols are required to enable the multiuser mode, as the user-specific information must be provided within the HTTP header.
• In the multiuser mode, collecting coverage for WCF-based applications requires that they have the ASP.NET compatibility mode enabled.
• In the multiuser mode, the "default" user (the user who has not specified their ID) may collect extra coverage information from other users who are accessing the same web application.
• Coverage data collected web initialization is not assigned to a specific user, but to the "default" user.
• The application coverage scope file cannot be used for WebSite projects as they may get recompiled by IIS server and change the name of the target assembly. Scope files can be safely used for Web Applications.