To conclude this chapter, let's look at a more complicated and interesting example of telemetry that we call "Unit Test Dynamics" (UTD). The basic idea of UTD is to measure three percentages: (1) The percentage of test code in the system; (2), the percentage of Active Time that was devoted to test code; and (3) The percentage of test coverage resulting from this test code and active time.

We choose these three percentages to track because they are potentially interesting proxy values for more fundamental project management concerns. For example, the percentage of test code in the system can be viewed as a proxy for the future maintenance overhead of testing: the more test code you have, the more time you are likely to spend in future keeping it working. The percentage of time spent on testing provides a sense for the human overhead involved with testing in the project. Finally, the test coverage provides a proxy for the quality of testing (and by extension, the functional correctness of the system itself). In the best of all possible worlds, we would want a very small percentage of test code, a very small percentage of active time devoted to testing, but 100% coverage. Of course, the real world of software development is rarely so accomodating.

What makes the "D" in UTD is the fact that these three values in isolation are not necessarily all that interesting: what we want to know is how these values are changing over time and in relationship to each other. For example, if coverage is decreasing while both the percentage of time spent on testing and the percentage of code is increasing, we are in trouble. The opposite situation indicates that we are doing something right: we are improving both the efficiency and effectiveness of testing.

Finally, while you can calculate UTD for the system as a whole, we have found that it can be quite interesting to compute UTD at the module level. Different modules can have quite different UTD characteristics.

Example 5.3, “Unit Test Dynamics Telemetry” shows how to define UTD using telemetry.

Example 5.3. Unit Test Dynamics Telemetry

y-axis yAxis(label) = {
  label
};

streams UTD-ActiveTime (filePattern1, filePattern2) = {
  "Active Time Percentage",
  (ActiveTime(filePattern1, "false") / ActiveTime(filePattern2, "false")) * 100
};

streams UTD-Coverage (filePattern) = {
  "Java Coverage",
  JavaCoverage("Percentage", filePattern)
};

streams UTD-Code (filePattern1, filePattern2) = {
  "Test Code Percentage",
  (FileMetric("Java", "sourceLines", filePattern1) / FileMetric("Java", "sourceLines", filePattern2)) * 100
};

chart UTD-Chart (ModuleFilePattern, ModuleTestFilePattern) = {
  "UTD Chart", 
  (UTD-ActiveTime(ModuleTestFilePattern, ModuleFilePattern), yAxis("ActiveTime %")),
  (UTD-Coverage(ModuleFilePattern), yAxis("Coverage %")),
  (UTD-Code(ModuleTestFilePattern, ModuleFilePattern), yAxis("TestCode %"))
};

report Hackystat-UTD-Report() = {
  "Unit Test Dynamics", 
  UTD-Chart("**/hackyZorro/**", "**/hackyZorro/**/Test*"),
  UTD-Chart("**/hackyTelemetry/**", "**/hackyTelemetry/**/Test*")
};

To display UTD telemetry, we must define three streams: one for Active Time percentage, one for Test Code Percentage, and one for Test Coverage percentage. Next, we put these three streams together in a Chart called UTD-Chart. Finally, we create a report, in this case passing it file patterns that single out two hackystat modules: hackyZorro and hackyTelemetry.

As should be immediately obvious, the hackyZorro and hackyTelemetry modules differ radically with respect to unit test dynamics. In the case of hackyZorro, all three UTD telemetry streams started out relatively high, and all three plummeted during the time period. This indicates that test quality was falling, but at the same time the amount of time and code devoted to testing was also falling. Interestingly, the developer of hackyZorro told us that at the beginning of this period, he was practicing Test Driven Design, but reverted to more traditional Test Last Design as time went on. This neatly explains the trends in this telemetry.

The hackyTelemetry module shows very different unit test dynamics. In this case, the test code and test coverage percentages are both rising steadily over the same time period. However, testing effort was not steady: there was one week where almost all effort was devoted to testing, which resulted in a significant increase in both test code size and coverage. Thereafter, testing effort was sporadic, but was still sufficient to maintain a gradual increase in coverage.

This concludes the User Guide introduction to the language and concepts of Software Project Telemetry. Chapter 28, Software Project Telemetry: Reduction Functions documents the available reduction functions on this server. Chapter 27, Software Project Telemetry: Filter Functions documents the available filter functions on this server. Finally, Chapter 23, Software Project Telemetry: Defining Reduction Functions and Filter Functions documents how you can write code to implement new reduction functions and filter functions, and thus extend the "alphabet" of Software Project Telemetry.