Part 2 of the Clover Tutorial focuses on the creation and interpretation of 'Historical' Clover reports. Historical reports display graphical and numerical data relating to sets of coverage data collected over time for the project. This tutorial covers the generation of a set of historical data, interpretation of the information displayed in the Historical reports and customisation of the reports for your particular reporting preferences.

In the first step, we'll edit the Ant build file to generate a History Point.

A history point is a snapshot of code coverage and metrics data for the project at a particular point in time. By running tests with Clover over time and creating a series of history points, it is possible to compare code coverage and metrics by viewing results in a single Clover report and enabling you to track the development of your project. The generation of historical reports is discussed in latersections. In the meantime, this section demonstrates how to set up the relevant Ant target and run the command so that a history point can be created.

Adding a history point target

Add the following target to your build.xml file:

 <target name="record.point" depends="with.clover">
    <clover-historypoint historyDir="clover_history"/>
 </target>

When this target is run, a history point will be created with the timestamp value of the coverage run.

The value of historyDir is the directory where the history points will be stored. You should create this directory before executing this target.

Recording a history point

Ensure that the source code has been instrumented and the tests run with the commands ant with.clover code and ant test respectively.

Run the command ant record.point. Output should be similar to the following:

 $ ant record.point
 Buildfile: build.xml
 with.clover:
 record.point:
  [clover-historypoint] Clover Version 1.x, built on ...
  [clover-historypoint] Merged results from 2 coverage recordings.
  [clover-historypoint] Writing report to
 'C:\tutorial\clover_history\clover-20030307111326.xml'
  [clover-historypoint] Done.
 BUILD SUCCESSFUL
 Total time: 2 seconds

In the next step we'll add more tests to improve coverage of the Money Library, recording Clover history points along the way.

In Part 1 of the tutorial we made additions to the testing suite to improve code coverage. In order to show the historical reporter in use, we will now continue to add tests and periodically record history points which will later be used as code coverage and metrics data by the historical reporter.

The Money.java file is at 100% coverage, however there are several sections of code that remain untested in the MoneyBag.java file. These uncovered lines of code are shown below in red. This section will focus on bringing the coverage of this class to 100% as well as creating historical data in the form of history points.

Open the source file MoneyTest.java in your favourite text editor and make the following additions shown in bold:

Declare the variables f0CHF and fMB3:

 public class MoneyTest extends TestCase {
  private Money f12CHF;
  private Money f14CHF;
  private Money f7USD;
  private Money f21USD;
  private Money f0USD;
  private Money f0CHF;
  private IMoney fMB1;
  private IMoney fMB2;
  private IMoney fMB3;
     ...

Initialise f0CHF and fMB3 in the setUp() method:

 protected void setUp()   {
  f12CHF = new Money(12, "CHF");
  f14CHF = new Money(14, "CHF");
  f7USD = new Money( 7, "USD");
  f21USD = new Money(21, "USD");
  f0USD = new Money(0, "USD");
  f0CHF = new Money(0, "CHF");
  fMB1 = MoneyBag.create(f12CHF, f7USD);
  fMB2 = MoneyBag.create(f14CHF, f21USD);
  fMB3 = MoneyBag.create(f0CHF, f0USD);
     ...

Add the following test:

 public void testMoneyBagEqualsZero(){
	assertTrue(!fMB3.equals(null));
	IMoney expected = MoneyBag.create(new Money(0, "CHF"),
                                      new Money(0, "USD"));
	assertTrue(fMB3.equals(expected));
 }

After making the above changes, reinstrument and test your code by running ant with.clover code and ant test respectively. Then record a new history point by running ant record.point. By recording a history point now, Clover will capture the new state of code coverage and metrics for comparison with past or future runs.

Add the following tests to bring the coverage of the Money project to 100%:

 public void testToString(){
	String expected="{[12 CHF][7 USD]}";
 	assertEquals(expected, fMB1.toString());
 }
 public void testVectorSize(){
	IMoney other = MoneyBag.create(new Money(2, "CHF"),
                                   new Money(2, "USD"));
	assertTrue(!other.equals(fMB3));
 }

Once again, reinstrument your code, test and record a new history point.

We have now created a series of history points for the Money library. The next section discusses how to generate a Clover historical report which will display the historical data that has been collected.

Now that we have recorded several history points, the next step is to add a target to the build file which will call the historical reporter and generate a historical report.

Add a historical report target

Add the following target to build.xml:

 <target name="hist.report" depends="with.clover">
     <clover-report>
           <historical outfile="historical.pdf"
                  historyDir="clover_history"/>
     </clover-report>
 </target>

The historical reporter needs to be able to find the coverage history files in order to create the report so the historyDir value must be the same as the historyDir defined for the history points. The format of the report can be either PDF or HTML as specified by the <format> element. The <format> element is optional and is not included in the example above. When the <format> element is omitted, a PDF report is produced by default. Depending on the chosen format, the outfile value may represent a single file as in the case of the PDF format, or the name of a directory (in the case of the HTML format).

Generating a historical report

Create a historical report by using the command ant hist.report. Output should be similar to the following:

 $ ant hist.report
 Buildfile: build.xml
 with.clover:
 hist.report:
  [clover-report] Clover Version 1.x, built on ...
  [clover-report] Writing report to 'C:\tutorial\historical.pdf'
  [clover-report] Merged results from 2 coverage recordings.
  [clover-report] Done. Processed 1 packages.
  [clover-report] Writing historical report to 'C:\tutorial\historical.pdf'
  [clover-report] Read 3 history points.
  [clover-report] Done.
 BUILD SUCCESSFUL
 Total time: 8 seconds

The report can now be viewed by opening the file tutorial\historical.pdf in a PDF viewer such as Adobe Acrobat Reader. We'll look at how to interpret this report in the next section.

We will now look at interpreting the report that you generated in the previous step by enabling the report in an appropriate PDF viewer. When you view the report you should see a picture similar to the screenshot below, although it is likely that the the graphs that you produce will contain different values.

Like the 'current' report, the historical report begins with a header containing relevant project information. This includes the report title, the project metrics and the period for which history points are included in the report. Below this header is the Project Overview Chart which shows the branch, statement, method and total coverage percentages for the project for the most recent history point included in the report.

The 'Coverage over time' graph shows the percentage values of branch, statement, method and total coverage for each history point and plots them against time in an easy-to-read chart.

The 'Metrics over time' graph shows the project statistics for each history point plotted against time. It is therefore possible to observe changes in metrics such as the number of methods. In the example below, the number of methods can be seen shown in green.

The final section, 'Movers', displays classes that have increased or decreased in coverage by more than a specified percentage point threshold over a particular time interval, the default being 1 percentage point over the two latest history points. In this case there have not been any classes which have lost more than 1 percentage point coverage, hence the only item displayed here is the Money package which has gained 10.6 percentage points coverage over the two latest history points.

The next section of this tutorial will discuss how you can customise many aspects of the historical report.

A custom title can also be displayed for your report by using the title attribute in the <historical> element as above.

Chart Selection

The historical reporter allows you to specify which charts to include in your report and also allows you to configure further options in the charts themselves.

The default reporting mode is to include all four report elements: <overview>, <coverage>, <metrics> and <movers>. But to include some and not the others is a simple matter of nesting the desired elements within the <historical> element. Try adding the following target to your build.xml file as an example:

 <target name="hist.report.coverage" depends="with.clover">
      <clover-report>
            <historical outfile="histCoverage.pdf"
                  title="My Project"
                  historyDir="clover_history">
	      <overview/>
	      <coverage/>
            </historical>
      </clover-report>
 </target>

The above code will produce a historical PDF report with the title 'My Project' which includes only two sections: the 'Overview' and the 'Coverage over time' charts.

Chart Configuration

The 'Coverage over time' and 'Metrics over time' charts also allow you to choose which metrics information should be included. The default elements for the coverage chart are branches, statements, methods and total, while the default elements for the metrics chart are loc, ncloc, methods and classes. By using the include attribute you can specify the required configuration:

<target name="hist.report.select" depends="with.clover">
      <clover-report>
            <historical outfile="histSelect.pdf"
                  title="My Project"
                  historyDir="clover_history">
	      <coverage include="total"/>
	      <metrics include="methods, packages"/>
            </historical>
      </clover-report>
</target>

This will produce a PDF file with the filename 'histSelect.pdf' with two sections: the 'Coverage over time' chart with total coverage information; and the 'Metrics over time' chart with method and package information. You can also specify whether or not a chart uses a log scale by adding the logscale attribute:

     <metrics include="methods, packages" logscale="false"/>

'Movers' Configuration

The 'Movers' section of the historical report shows you the classes whose coverage has changed the most recently. This is useful for spotting classes that have had sudden changes in coverage, perhaps the unintended result of changes to the unit test suite.

The 'Movers' chart allows you to specify the threshold of point change a class must satisfy, the maximum number of gainers and losers to display and the period across which the gains and losses are calculated. Add the following target to your build.xml file as an example of this feature in use:

 <target name="hist.report.movers" depends="with.clover">
      <clover-report>
            <historical outfile="histMovers.pdf"
                  title="My Project"
                  historyDir="clover_history">
 	      <movers threshold="5%" range="20" interval="2w"/>
            </historical>
      </clover-report>
 </target>

In this case, the configuration values selected state that classes must have a change in coverage of at least 5 percentage points to be included in the chart, a maximum of 20 gainers and 20 losers can be displayed, and the initial valuation point for class coverage is 2 weeks prior to the most recent history point. Should there be greater than 20 gainers in this period, then the classes with the biggest percentage point gain will be displayed, and the same for the losers.

See Interval Format for details on the syntax for specifying interval values.

The next section of this tutorial will discuss how you can automate the coverage checking of your project.