Chapter 10. Zorro: Guidelines for performing validation studies

Table of Contents

10.1. Introduction
10.2. Zorro Validation Study Preparation
10.3. Data Collection
10.4. Data Analysis and Interpretation
10.5. Support materials for Zorro validation study

10.1. Introduction

Projects Zorro and SDSA (Software Development Stream Analysis) provide a means to study Test-Driven Development, one of the most well-known Extreme Programming (XP) practice. Our preliminary pilot study found that Zorro can achieve 89% accuracy on recognizing Test-Driven Development (TDD). Please refer to the technique report at here for more information.

Our preliminary Zorro pilot study shows that Zorro really works well on recognizing Test-Driven Development. We also refined Zorro on data collection and improved TDD recognition rules after the pilot study, which means that Zorro will be able to achieve much higher recogniztion accuracy than it was before. However, our small scale and internal validation is still far away from perfect. In order to prove that Zorro is applicable, it has to be validated both internally in one organization and externally in other organizations. The goal of this documentation is to have a general introduction on Zorro validation case study to help other TDD practitioners, researchers and organizations conduct external validation studies on Zorro.

The validation procedure includes installation of Hackystat Eclipse sensor and Eclipse Screen Recorder, test subject's hand-on development in TDD, development process data collection and analysis.

Section 10.2, “Zorro Validation Study Preparation” tells you how to configure development environment to get both development process data for Zorro analysis and development process video for validation. Section 10.3, “Data Collection” explains how data are collected with Hacksytat Eclipse sensor and Eclipse Screen Recorder (ESR). After finish the experiment, researchers can follow Section 10.4, “Data Analysis and Interpretation” to validate Zorro.

[Important]IDE Limitation

Eclipse is the only IDE that has full-fledged Hackystat sensor support to collect rich information of development process. Software development in the validation study has to be done within Eclipse IDE. At the time this documentation is written, the latest stable Eclipse release is version 3.1.2.