API coverage results are included in HTML, PDF, and XML reports.
HTML and PDF Reports
HTML and PDF reports that are generated for API-Coverage-enabled test runs will include an API Coverage section that shows API coverage against service definitions. They detail the level of coverage achieved for each applicable resource and operation, as well as the pass/fail status of the tests responsible for that coverage. They also pinpoint which .tsts and tests covered each operation (with a pass/fail breakdown at this level as well).
The initial view of the Coverable Resource column shows each known "coverable resource," which is a service definition (RAML, OpenAPI/Swagger, WADL, WSDL).
The Successful/Total column indicates how many times a test invoked the coverable resource (the service, a particular REST resource/method or SOAP method, and so on), and how many of those tests actually succeeded.
The Coverage column indicates the number of resources that were invoked vs. the total number of resources. Resources that do not map to any service definition are listed under "Unknown Service." The graphical coverage bar indicates the overall coverage (the total length of the colored portion of the bar) as well as the pass/fail breakdown for the tests that achieved that coverage.
The table is hierarchical. Coverage is calculated for a service as a whole, as well as each resource/ method and operation. When an HTML report is viewed in a browser, the tree branches can be collapsed and expanded.
If you want to see what .tsts and specific tests covered a certain method, drill down into the tree, then click the Show tests link. For quick expansion and collapsing, you can use the Expand All / Collapse All links at the top left of the report.
Note that test cases are shown only when the Only top-level test suites reporting option is disabled in Preference> Reports.
XML Reports
XML reports generated for API-Coverage-enabled test runs include an API Coverage section that reports API coverage against service definitions.
The report contains an APICoverage
element, which includes a list of coverable service definitions. For each operation that was covered during test during execution, there will be a list of test-ref
elements. The id
of each test-ref
element indicates which tests covered the operation.
Here is a very simple example of the API coverage section of an XML report:
<APICoverage> <service definition-uri="http://machine:8080/petstore.yaml" type="REST"> <resource id="/pets-multipart/{petId}"> <operation name="POST" /> </resource> <resource id="/pets-urlencode/{petId}"> <operation name="POST" /> </resource> </service> <service type="Unknown Service"> <resource id="http://ws1.parasoft.com:8080/examples/servlets/Echo/v2/pets-multipart/"> <operation name="POST"> <test-ref id="wk:///TestAssets/Diff3 Swagger.tst#01#02#03" /> </operation> </resource> <resource id="http://ws1.parasoft.com:8080/examples/servlets/Echo/v2/pets-urlencode/"> <operation name="POST"> <test-ref id="wk:///TestAssets/Diff3 Swagger.tst#00#01#04#05" /> </operation> </resource> </service> </APICoverage>