In situations where the sensor data of interest is contained in an XML data file produced by the tool, an easy way to implement a sensor is through the use of an XSL stylesheet in combination with the XmlData sensor. This section shows how this can be accomplished for the Clover coverage tool.
Clover is a tool that can calculate the coverage associated with test cases. Hackystat already provides a sensor data type called "Coverage" to represent the results of a coverage tool, and analyses such as the Coverage reduction function to support interpretation of the results. As Clover outputs the coverage results in XML format, it is an excellent candidate for sensor implementation using the XmlData sensor. This section illustrates the steps involved in creating a sensor for this type of situation.
The first step is to invoke the tool and have it generate an example XML data file to see how the tool represents its data. Example 17.1, “clover.build.xml file” illustrates a very simple Ant task that invokes Clover to produce coverage for the StackyHack system. Ant is a convenient way to generate Clover coverage data, but it is not required.
Example 17.1. clover.build.xml file
<project name="stackyhack.clover" default="clover">
<description>
Provides the clover tool and the Hackystat clover sensor.
</description>
<import file="build.xml"/>
<property environment="env"/>
<property name="clover.dir" location="${build.dir}/clover" />
<property name="junit.dir" location="${build.dir}/junit" />
<taskdef resource="clovertasks"/>
<clover-setup initString="mycoverage.db"/>
<target name="clover" depends="clean, compile"
description="Cleans, compiles, instruments byte codes, runs unit tests, generates clover report.">
<mkdir dir="${clover.dir}" />
<mkdir dir="${junit.dir}" />
<junit printsummary="withOutAndErr" fork="yes">
<classpath path="${build.dir}/classes;${java.class.path}">
<pathelement location="${ant.home}/lib/clover.jar" />
<path refid="compile.classpath"/>
</classpath>
<formatter type="xml" />
<batchtest todir="${junit.dir}">
<fileset dir="${src.dir}">
<include name="**/Test*.java" />
</fileset>
</batchtest>
</junit>
<clover-report>
<current outfile="${clover.dir}/coverage.xml">
<format type="xml"/>
</current>
</clover-report>
</target>
</project>
The details of how this Ant file works are not important for this chapter; the point is to illustrate one way to generate an XML file containing coverage data generated by Clover.
The second step is to review the XML data and determine what data to extract and send to Hackystat. Example 17.2, “An example Clover XML report file” illustrates the XML file generated by Clover for the StackyHack example system.
Example 17.2. An example Clover XML report file
<?xml version="1.0" encoding="UTF-8"?>
<coverage generated="1167892240109" clover="1.3.13">
<project timestamp="1167892239971">
<metrics coveredelements="55" packages="1" coveredconditionals="3" ncloc="117" statements="47" loc="267" files="5"
conditionals="4" coveredmethods="10" coveredstatements="42" methods="13" classes="5" elements="64"/>
<package name="edu.hawaii.stack">
<metrics coveredelements="55" files="5" coveredconditionals="3" conditionals="4" ncloc="117" statements="47"
coveredstatements="42" coveredmethods="10" loc="267" methods="13" classes="5" elements="64"/>
<file name="/Users/johnson/svn-csdl/StackyHack/src/edu/hawaii/stack/ClearStack.java">
<class name="ClearStack">
<metrics coveredelements="2" coveredconditionals="0" conditionals="0" statements="2" coveredstatements="1"
coveredmethods="1" methods="2" elements="4"/>
</class>
<metrics coveredelements="2" coveredconditionals="0" conditionals="0" ncloc="11" statements="2"
coveredstatements="1" coveredmethods="1" loc="37" methods="2" classes="1" elements="4"/>
<line num="23" type="method" count="1"/>
<line num="24" type="stmt" count="1"/>
<line num="34" type="method" count="0"/>
<line num="35" type="stmt" count="0"/>
</file>
<file name="/Users/johnson/svn-csdl/StackyHack/src/edu/hawaii/stack/EmptyStackException.java">
<class name="EmptyStackException">
<metrics coveredelements="2" coveredconditionals="0" conditionals="0" statements="1" coveredstatements="1"
coveredmethods="1" methods="1" elements="2"/>
</class>
<metrics coveredelements="2" coveredconditionals="0" conditionals="0" ncloc="7" statements="1" coveredstatements="1"
coveredmethods="1" loc="23" methods="1" classes="1" elements="2"/>
<line num="20" type="method" count="1"/>
<line num="21" type="stmt" count="1"/>
</file>
<file name="/Users/johnson/svn-csdl/StackyHack/src/edu/hawaii/stack/TestStack.java">
<class name="TestStack">
<metrics coveredelements="15" coveredconditionals="0" conditionals="0" statements="14" coveredstatements="13"
coveredmethods="2" methods="2" elements="16"/>
</class>
<metrics coveredelements="15" coveredconditionals="0" conditionals="0" ncloc="28" statements="14"
coveredstatements="13" coveredmethods="2" loc="52" methods="2" classes="1" elements="16"/>
<line num="26" type="method" count="1"/>
<line num="27" type="stmt" count="1"/>
<line num="28" type="stmt" count="1"/>
<line num="29" type="stmt" count="1"/>
<line num="30" type="stmt" count="1"/>
<line num="31" type="stmt" count="1"/>
<line num="32" type="stmt" count="1"/>
<line num="33" type="stmt" count="1"/>
<line num="34" type="stmt" count="1"/>
<line num="35" type="stmt" count="1"/>
<line num="37" type="stmt" count="1"/>
<line num="42" type="method" count="1"/>
<line num="43" type="stmt" count="1"/>
<line num="44" type="stmt" count="1"/>
<line num="45" type="stmt" count="1"/>
<line num="46" type="stmt" count="0"/>
</file>
<file name="/Users/johnson/svn-csdl/StackyHack/src/edu/hawaii/stack/TestClearStack.java">
<class name="TestClearStack">
<metrics coveredelements="14" coveredconditionals="2" conditionals="2" statements="11" coveredstatements="11"
coveredmethods="1" methods="1" elements="14"/>
</class>
<metrics coveredelements="14" coveredconditionals="2" conditionals="2" ncloc="23" statements="11"
coveredstatements="11" coveredmethods="1" loc="41" methods="1" classes="1" elements="14"/>
<line num="25" type="method" count="1"/>
<line num="26" type="stmt" count="1"/>
<line num="27" type="stmt" count="1"/>
<line num="28" type="stmt" count="1"/>
<line num="29" type="stmt" count="1"/>
<line num="30" type="stmt" count="1"/>
<line num="32" type="stmt" count="1"/>
<line falsecount="1" num="32" type="cond" truecount="3"/>
<line num="33" type="stmt" count="3"/>
<line num="34" type="stmt" count="3"/>
<line num="37" type="stmt" count="1"/>
<line num="38" type="stmt" count="1"/>
<line num="39" type="stmt" count="1"/>
</file>
<file name="/Users/johnson/svn-csdl/StackyHack/src/edu/hawaii/stack/Stack.java">
<class name="Stack">
<metrics coveredelements="22" coveredconditionals="1" conditionals="2" statements="19" coveredstatements="16"
coveredmethods="5" methods="7" elements="28"/>
</class>
<metrics coveredelements="22" coveredconditionals="1" conditionals="2" ncloc="48" statements="19"
coveredstatements="16" coveredmethods="5" loc="114" methods="7" classes="1" elements="28"/>
<line num="30" type="method" count="6"/>
<line num="31" type="stmt" count="6"/>
<line num="41" type="method" count="4"/>
<line num="42" type="stmt" count="4"/>
<line num="43" type="stmt" count="4"/>
<line num="44" type="stmt" count="3"/>
<line num="45" type="stmt" count="3"/>
<line num="48" type="stmt" count="1"/>
<line num="59" type="method" count="1"/>
<line num="60" type="stmt" count="1"/>
<line num="61" type="stmt" count="1"/>
<line num="62" type="stmt" count="1"/>
<line num="65" type="stmt" count="0"/>
<line num="75" type="method" count="1"/>
<line num="76" type="stmt" count="1"/>
<line num="77" type="stmt" count="1"/>
<line num="85" type="method" count="1"/>
<line num="86" type="stmt" count="1"/>
<line num="87" type="stmt" count="1"/>
<line num="88" type="stmt" count="1"/>
<line num="89" type="stmt" count="1"/>
<line num="94" type="stmt" count="1"/>
<line falsecount="0" num="94" type="cond" truecount="1"/>
<line num="102" type="method" count="0"/>
<line num="103" type="stmt" count="0"/>
<line num="111" type="method" count="0"/>
<line num="112" type="stmt" count="0"/>
</file>
</package>
</project>
</coverage>
With a little bit of study, you can see that the coverage report concerns five Java classes (ClearStack, EmptyStackException, TestStack, TestClearStack, and Stack), contained in five .java files. The XML file is structured in a five-level hierarchy of tags: the top-level tag is <coverage>, which contains a <project> tag, which contains one or more <package> tags, which contains one or more <file> tags, which contains one or more <class> tags. With the exception of the top-level <coverage> tag, each of these tags contains an interior <metrics> tag containing data appropriate to that level of the system description.
Once you understand how the XML file is structured, you must compare it to the structure of the Sensor Data Type used to represent this type of data. The Coverage sensor data type, which is documented in the Reference Guide, contains seven required fields: tstamp, tool, runTime, granularity, fileName, covered, and uncovered. To use this sensor data type, you must first decide what granularity or granularities to represent: method, statement, or conditional? For this example, let's assume that we are only interested in statement-level coverage. Second, we see that the sensor data type requires the "uncovered" value, must be calculated as the total number of statements minus the covered number of statements.
Through review of the XmlData sensor specification, we can see that the XML file must consist of a top-level node (which can be anything) followed by a set of <entry> nodes whose attributes provide the required and optional fields for a single sensor data instance. For the Coverage SDT, each sensor data entry provides the coverage information associated with a single file, and so we would represent the coverage data in Example 17.2, “An example Clover XML report file” by five <entry> nodes. Example 17.3, “An example XML input file to the XmlData sensor” illustrates a reasonable version of the XML file we must produce for input into the XmlData sensor from the example above.
Example 17.3. An example XML input file to the XmlData sensor
<xmldata>
<entry sdt="Coverage" tool="Clover" granularity="statement" covered="1" uncovered="1"
fileName="/Users/johnson/svn-csdl/StackyHack/src/edu/hawaii/stack/ClearStack.java" />
<entry sdt="Coverage" tool="Clover" granularity="statement" covered="1" uncovered="0"
fileName="/Users/johnson/svn-csdl/StackyHack/src/edu/hawaii/stack/EmptyStackException.java" />
<entry sdt="Coverage" tool="Clover" granularity="statement" covered="13" uncovered="1"
fileName="/Users/johnson/svn-csdl/StackyHack/src/edu/hawaii/stack/TestStack.java" />
<entry sdt="Coverage" tool="Clover" granularity="statement" covered="11" uncovered="0"
fileName="/Users/johnson/svn-csdl/StackyHack/src/edu/hawaii/stack/TestClearStack.java" />
<entry sdt="Coverage" tool="Clover" granularity="statement" covered="16" uncovered="3"
fileName="/Users/johnson/svn-csdl/StackyHack/src/edu/hawaii/stack/Stack.java" />
</xmldata>
The ordering of attributes is unimportant. Note that each <entry> node in the XML file contains one field not specified in the Coverage sensor data type ("sdt") and lacks two of its required fields ("tstamp" and "runTime"). The "sdt" field is a special field for the XmlData sensor that tells it the sensor data type to be associated with each entry for the purpose of validity checking. The XmlData sensor can thus process a single XML file containing data for many kinds of sensor data types, although in this example all of the entries are associated with the single Coverage SDT.
The "tstamp" field in an SDT indicates a "timestamp" to be associated with each sensor data type entry, which must be unique. (If you send data of a given SDT with the same timestamp as a previously sent entry, then the new entry will overwrite the old data on the server side.) If a field called "tstamp" is included in the <entry> node, then that value will be used, otherwise the XmlData sensor will automatically generate a unique timestamp for each <entry> using the Java code "String.valueOf(new Date().getTime())". In this case, we will not include the required "tstamp" field in our XML file, and let the XmlData sensor insert it for us. This is an appropriate design strategy when the XmlData sensor will be invoked on the tool data right after the tool data has been generated, so that these timestamps will closely approximate the actual time that the data was collected. If XmlData sensor won't be invoked on the tool's XML file until hours or days later, then it's better to design the XSL transform to generate a unique tstamp field explicitly in each <entry> node, perhaps using the file creation time as a basis.
Finally, the Coverage SDT, like several other SDTs, requires a "runTime" field as a way of identifying a collection of Coverage entries as having been generated at the same time. Consider the situation where Coverage data is being generated frequently, and the following sequence of events occurs: (a) coverage data is generated; (b) class Foo is deleted; (c) coverage data is regenerated. On the server side, the coverage data generated in (a) would contain data regarding class Foo, while the coverage data in (c) would not. By associating a single unique runTime value with each run of the Coverage tool, analyses on the server side can separate the data generated during run (a) from the data generated during run (b), in most cases discarding all but the latest data generated during a day.
Because the runTime field is a popular attribute in sensor data types, the XmlData sensor provides an option called "createRunTime" to automate the generation of these timestamps. When enabled, the createRunTime option will automatically generate a runTime timestamp and provide that value with each sensor data entry generated from the XML data file.
Now that we know what kind of file we need to produce from the Clover XML file, the next task is to write the XSLT file to produce it, and invoke it on the Clover XML data file. There are many tools available for XML data transformation using XSLT stylesheets. Example 17.4, “The initial clover.sensor task” illustrates one approach using Ant. This Ant task takes the Clover XML data file as input and runs an XSL stylesheet called "CloverToHackystat.xsl" to transform it into a new XML data file called "hackystat.xml".
Example 17.4. The initial clover.sensor task
<target name="clover.sensor"
description="Transforms the XML file produced by Clover and invokes XmlData sensor.">
<xslt style="CloverToHackystat.xsl"
in="${clover.dir}/coverage.xml" out="${clover.dir}/hackystat.xml" />
<concat>
<fileset file="${clover.dir}/hackystat.xml"/>
</concat>
</target>
As you can see, this target is quite simple; it just invokes the <xslt> task followed by the <concat> task so that the output is immediately available for review. The <concat> task is provided for debugging purposes and would generally be deleted or commented out in the production version. Example 17.5, “The initial CloverToHackystat.xsl stylesheet” shows an initial version of the XSLT stylesheet.
Example 17.5. The initial CloverToHackystat.xsl stylesheet
<?xml version="1.0"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output method="xml" indent="yes"/>
<xsl:template match="/">
<xmldata>
<xsl:apply-templates select="coverage/project/package/file"/>
</xmldata>
</xsl:template>
<xsl:template match="file">
<entry sdt="Coverage" tool="Clover" granularity="statement" fileName="{@name}" covered="{class/metrics/@coveredstatements}"/>
</xsl:template>
</xsl:stylesheet>
As you can see, this first version of the stylesheet is extremely simple: it just finds all of the <file> nodes and transforms selected attributes from them (or their interior nodes) into <entry> nodes.
Finally, let's see what happens when we invoke the clover.sensor task, as illustrated in Example 17.6, “Output from running the initial clover.sensor task”.
Example 17.6. Output from running the initial clover.sensor task
admin01:~/svn-csdl/StackyHack johnson$ ant -f clover.build.xml clover.sensor Buildfile: clover.build.xml [clover-setup] Clover Version 1.3.13, built on September 04 2006 [clover-setup] loaded from: /Users/johnson/java/apache-ant-1.7.0RC1/lib/clover.jar [clover-setup] Open Source License registered to the Hackystat Project. This license of Clover is provided to support the development of Hackystat only. [clover-setup] [clover-setup] Clover is enabled with initstring '/Users/johnson/svn-csdl/StackyHack/mycoverage.db' clover.sensor: [concat] <?xml version="1.0" encoding="UTF-8"?> [concat] <xmldata> [concat] <entry sdt="Coverage" tool="Clover" granularity="statement" fileName="/Users/johnson/svn-csdl/StackyHack/src/edu/hawaii/stack/ClearStack.java" covered="1"/> [concat] <entry sdt="Coverage" tool="Clover" granularity="statement" fileName="/Users/johnson/svn-csdl/StackyHack/src/edu/hawaii/stack/EmptyStackException.java" covered="1"/> [concat] <entry sdt="Coverage" tool="Clover" granularity="statement" fileName="/Users/johnson/svn-csdl/StackyHack/src/edu/hawaii/stack/TestStack.java" covered="13"/> [concat] <entry sdt="Coverage" tool="Clover" granularity="statement" fileName="/Users/johnson/svn-csdl/StackyHack/src/edu/hawaii/stack/TestClearStack.java" covered="11"/> [concat] <entry sdt="Coverage" tool="Clover" granularity="statement" fileName="/Users/johnson/svn-csdl/StackyHack/src/edu/hawaii/stack/Stack.java" covered="16"/> [concat] </xmldata> BUILD SUCCESSFUL Total time: 0 seconds
The output prefixed with [concat] shows the content of the hackystat.xml file. As you can see, it's quite close to the desired output. The only missing attribute is the uncovered attribute, which requires the XSL stylesheet to do some simple arithmetic by subtracting the total number of statements from the covered number of statements.
A little bit of googling reveals that you can do arithmetic on XPath expressions, store the results in a variable, then refer to that variable using the $ escape character. Example 17.7, “The improved CloverToHackystat.xsl stylesheet” shows the completed version of the XSLT stylesheet that includes the uncovered value.
Example 17.7. The improved CloverToHackystat.xsl stylesheet
<?xml version="1.0"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output method="xml" indent="yes"/>
<xsl:template match="/">
<xmldata>
<xsl:apply-templates select="coverage/project/package/file"/>
</xmldata>
</xsl:template>
<xsl:template match="file">
<xsl:variable name="uncovered" select="class/metrics/@statements - class/metrics/@coveredstatements" />
<entry sdt="Coverage" tool="Clover" granularity="statement" fileName="{@name}" covered="{class/metrics/@coveredstatements}" uncovered="{$uncovered}"/>
</xsl:template>
</xsl:stylesheet>
Now that we have the hackystat.xml file containing XML data in the form required by the XmlData sensor, we must next download and install the XmlData sensor, then invoke it on the hackystat.xml file. Downloading is quite simple and will not be discussed here; see the Reference Guide for details. The XmlData sensor provides both a command line interface as well as an Ant interface. Example 17.8, “XmlData invocation using its command line interface” illustrates the invocation of the XmlData sensor on the hackystat.xml file using its command line interface.
Example 17.8. XmlData invocation using its command line interface
admin01:~/svn-csdl/StackyHack johnson$ java -jar ~/java/sensor.xmldata-cli.jar -file /Users/johnson/svn-csdl/StackyHack/build/clover/hackystat.xml -createRunTime runTime -verbose true Verbose mode on: true Runtime name is: runTime, value is: 1168192491004 Sensor enabled: true Processing file: /Users/johnson/svn-csdl/StackyHack/build/clover/hackystat.xml Sending to sensorshell: Coverage [add, fileName=/Users/johnson/svn-csdl/StackyHack/src/edu/hawaii/stack/ClearStack.java, uncovered=1, granularity=statement, tool=Clover, covered=1, runTime=1168192491004] Sending to sensorshell: Coverage [add, fileName=/Users/johnson/svn-csdl/StackyHack/src/edu/hawaii/stack/EmptyStackException.java, uncovered=0, granularity=statement, tool=Clover, covered=1, runTime=1168192491004] Sending to sensorshell: Coverage [add, fileName=/Users/johnson/svn-csdl/StackyHack/src/edu/hawaii/stack/TestStack.java, uncovered=1, granularity=statement, tool=Clover, covered=13, runTime=1168192491004] Sending to sensorshell: Coverage [add, fileName=/Users/johnson/svn-csdl/StackyHack/src/edu/hawaii/stack/TestClearStack.java, uncovered=0, granularity=statement, tool=Clover, covered=11, runTime=1168192491004] Sending to sensorshell: Coverage [add, fileName=/Users/johnson/svn-csdl/StackyHack/src/edu/hawaii/stack/Stack.java, uncovered=3, granularity=statement, tool=Clover, covered=16, runTime=1168192491004]
In production settings, you would probably omit the "verbose" mode to minimize data output.
The final step in development is to verify that the Hackystat server is properly receiving the data after you invoke the XmlData sensor. To check this, invoke the XmlData sensor, then login to the server and invoke the List Sensor Data command, as illustrated by Figure 17.1, “ List Sensor Data showing the Coverage data sent by XmlData ”.
If you run into problems using the XmlData sensor, it can sometimes be helpful to look at the log file generated while the sensor runs. This log file is located in the ~/.hackystat/logs directory, and is called XmlData.log.0. Example 17.9, “The XmlData log file” illustrates the portion of the log file corresponding to the invocation above.
Example 17.9. The XmlData log file
Hackystat Version: 7.7.107 (January 7 2007 06:57:27)
SensorShell started at: 01/07/2007 07:54:51
Type 'help' for a list of commands.
Host: http://hackystat.ics.hawaii.edu/ is available and key is valid.
Defined shell command: Activity
Defined shell command: Coverage
Defined shell command: UnitTest
Defined shell command: FileMetric
Defined shell command: SampleSdt
#> AutoSend [10]
AutoSend OK (set to 10 minutes)
AutoSend enabled every 10 minutes.
Checking for offline data to recover.
#> Coverage [add, fileName=/Users/johnson/svn-csdl/StackyHack/src/edu/hawaii/stack/ClearStack.java, uncovered=1, granularity=statement, tool=Clover, covered=1, runTime=1168192491004]
Coverage add OK (1 total)
#> Coverage [add, fileName=/Users/johnson/svn-csdl/StackyHack/src/edu/hawaii/stack/EmptyStackException.java, uncovered=0, granularity=statement, tool=Clover, covered=1, runTime=1168192491004]
Coverage add OK (2 total)
#> Coverage [add, fileName=/Users/johnson/svn-csdl/StackyHack/src/edu/hawaii/stack/TestStack.java, uncovered=1, granularity=statement, tool=Clover, covered=13, runTime=1168192491004]
Coverage add OK (3 total)
#> Coverage [add, fileName=/Users/johnson/svn-csdl/StackyHack/src/edu/hawaii/stack/TestClearStack.java, uncovered=0, granularity=statement, tool=Clover, covered=11, runTime=1168192491004]
Coverage add OK (4 total)
#> Coverage [add, fileName=/Users/johnson/svn-csdl/StackyHack/src/edu/hawaii/stack/Stack.java, uncovered=3, granularity=statement, tool=Clover, covered=16, runTime=1168192491004]
Coverage add OK (5 total)
#> send
Sending sensor data (01/07 07:54:54)
Activity: Send OK (No entries to send.)
Ping: Ping OK (contacted server http://hackystat.ics.hawaii.edu/ with valid key.)
Coverage: Send OK (5 entries)
AutoSend: AutoSend OK ('send' command ignored)
UnitTest: Send OK (No entries to send.)
FileMetric: Send OK (No entries to send.)
SampleSdt: Send OK (No entries to send.)
>>
This log file is generated by the SensorShell after it receives the data from the XmlData sensor, and can help you determine if your problem is associated with your XSL stylesheet or with some other problem in your environent, such as an incorrect hackystat key or server. For more information about the SensorShell, see Chapter 16, The SensorShell.