Generating reports on JUnit tests and JUnit plug-in tests

After you have run your tests and generated execution histories, you can generate a report on those test results to summarize the data.

Generate the report:

Right-click on a TPTP JUnit test in the Test Navigator and select Report... from the pop-up menu.
The New Report dialog launches.
Select the Time Frame Historic option then click on Next.
Provide the parent folder and a name for the report, then click on Next and provide the report time frame Start and End dates.
Click on Finish to generate the report.

Time Frame Historic report

There are 4 tabs in the report: Overview, By Test, By Log, and Details.

Overview Tab

The Overview report is a 3D pie chart showing the overall breakdown across all test runs during the specified period. In this example two-thirds (66.67%) of the tests failed and one-third (33.33%) were in error.

Pie chart showing 66.67% of the tests failed and 33.33% of the tests were in error.

By Test Tab

The By Test report shows the overall relative proportions of the categories Pass, Inconclusive, Fail, and Error.

Part of a bar chart showing that a portion of the test failed and a portion of the test was in error.

By Test Log Tab

The By Test Log report shows, for each test log, the overall relative proportions of the categories Pass, Inconclusive, Fail, and Error.

Part of a bar chart summarizing two tests. Both tests have part fail and part error.

Details Tab

The Details report is in 2 sections: Summary and Results by Test.

Summary

The Summary lists the following:

name of the tests that ran
verdict across all of those tests
creation date of the report
start time of the execution histories
end time of the execution histories
number of pass results
number of fail results
number of inconclusive results
number of error results

This section summarizes the outcomes for test runs taking place during the specified period. In this example, out of a total of 6 tests, there were 4 failed tests and 2 tests in error.

Results by Test

The Results by Test section is a table with eight columns:

Test Log (the name of the test)
Date (the day and time of the execution history)
Pass (the number of tests that passed)
Inconclusive (the number of tests that were inconclusive)
Fail (the number of tests that failed)
Error (the number of tests that had an error)
Attempted (the number of tests attempted)
Failure Rate (the percentage of tests that failed)

This section tabulates the the test outcomes for each test log.