11.2. Categories of experimentation support: configuration, monitoring, and analysis

Experimentation support in Hackystat can be broken down into three categories: configuration, monitoring, and analysis.

11.2.1. Configuration

Hackystat support for experiment configuration involves administrator facilities for representing the group of users (subjects) involved in an experiment, and providing automated support for the basic activities in configuring their Hackystat account. Hackystat account configuration includes: (a) registering with the Hackystat server and obtaining the user key for each user; (b) installing sensors in the user's account along with their user key so that their data will be sent to the appropriate account, and (c) configuring their Hackystat account with Workspace Root and Project definitions.

Note that automating the second activity, installing sensors in the user's account, requires that the experiment be performed in an environment where the administrator has root-level access to the accounts. If the experiment involves the subject's use of their own computers, then the experimental protocol will generally require instructions for the user on how to install the required sensors.

11.2.2. Monitoring

Once the experiment is under way, it can be important for the administrator to be able to monitor the progress of data collection. Both client-side and server-side monitoring can useful. Client-side monitoring involves analyses that help the administrator to see whether or not data is being sent from users to the server in the types and quantities that are expected according to the experimental protocol. Server-side monitoring involves analyses to ensure that the server is running.

11.2.3. Analysis

The final component of experimentation support involves analysis. Hackystat can provide custom analyses for specific experiments, but by default provides "generic" experimentation analysis support in the form of data export facilities that enable experimenters to obtain their subjects data in XML or CSV format.