In previous chapters, we've seen that Hackystat provides a way to automatically collect raw sensor data, which can be abstracted into metrics like "Most Active File", and from which we can obtain various snapshots of the development process, such as the Daily Diary or Daily Project Details. For example, Figure 5.1, “ Daily Project Details (Summary Section) for Hackystat ” shows the Summary Section of Daily Project Details for the Hackystat project on one day in March, 2005:

Figure 5.1.  Daily Project Details (Summary Section) for Hackystat

We can see from this analysis that on the day in question, the Project hacky2004-all generated almost 15 hours of Active Time, was built over 40 times, consisted of almost 70,000 LOC, generated over 2 hours of Review Active Time, that this review generated 42 issues, and so forth.

Such snapshots can be quite useful as an empirical answer to the question of "What happened?" with a Project on a particular day. However, such an answer usually leads to a new question: "So, are these numbers good or bad?" More generally, we need to go beyond simple awareness of project statistics to their application for project management and decision-making.

A traditional approach to use of project metrics for management begins by collecting "final" values of these measures over a set of completed projects. This "historical project metrics repository" is then used as a baseline for comparison with data from a current project. While this can be a useful approach to initial project planning, it has a number of obvious limitations. First, the historical projects must be "comparable" to the current one, which can be hard when applications, personnel, and resources are changing. Second, and more fundamentally, how do you usefully compare data from a completed project to one that is still in progress?

The automated nature of metrics collection and analysis in Hackystat allows a different use of these metrics. Instead of trying to compare data from already completed projects to the current in-progress project, we can instead compare snapshots of data collected from a single Project over various points in time to each other. For example, if Active Time was consistently 30 hours per day from the start of a Project until recently, but has now dropped to 15 hours per day, then management may need to consider obtaining additional resources for this Project, or else adjust the scope. As another example, if coverage used to be almost 100% but has been gradually dropping over time, then management may need to re-allocate resources to improve quality assurance.

From very early in its development, Hackystat has provided a few special purpose analyses for displaying trends in metric data over time, such as the one illustrated in Figure 5.2, “ Project Active Time for Hackystat, Jan-March 2005 ”.

Figure 5.2.  Project Active Time for Hackystat, Jan-March 2005

This chart provides a perspective on how the Active Time generated by each member for the Project has accumulated over the course of 10 weeks. (The actual chart includes a legend mapping line colors to Project members, which is cropped from this image.) This analysis can be useful for understanding how overall Active Time for the Project is distributed among members, and how various members have allocated time to the project over the course of the 10 weeks.

However, our original approach of providing special purpose "trend" analyses such as this one has significant limitations. First, this analysis shows only a single type of measure: Active Time. Many interesting issues in software project management involve understanding the relationship between different measures. For example, we might be interested in seeing if an increased investment in code review pays off with less unit test failures, and/or increased coverage, and/or less defects reported against the reviewed modules. Such a question requires comparing a set of metrics values over time. Second, implementing each Hackystat analysis command requires writing Java code, installing it into the system, and rebuilding and reinstalling a new server. This is complex, time consuming, and requires expertise in the internals of Hackystat. With literally thousands of possible combinations of interesting metric values, building custom analyses for each one becomes infeasible. What is needed is a mechanism that facilitates interactive exploration of the relationships between the process and product data collected by Hackystat, so that developers and management can "play with the data" to see what perspectives provide insight to their particular situation. Hackystat provides this capability as part of an implementation of a general method of software project management called "Software Project Telemetry."

	Note
	The idea that Hackystat could provide a form of "telemetry" regarding software development originally arose from conversations in 2001 with Carl Puckett and Kenny Meyer of the Jet Propulsion Laboratory. We would like to thank them for providing an especially powerful and productive metaphor for software engineering metrics analysis!