Tracking human-centric controlled experiments with biscuit

Authors:
Fernando Olivero;Michele Lanza;Marco D'ambros;Romain Robbes
Affiliations:
University Of Lugano, Lugano, Switzerland;University Of Lugano, Lugano, Switzerland;University Of Lugano, Lugano, Switzerland;University Of Chile, Santiago de Chile, Chile
Venue:
Proceedings of the ACM 4th annual workshop on Evaluation and usability of programming languages and tools
Year:
2012

Citing 6
Cited 0

Beyond the Personal Software Process: metrics collection and analysis for the differently disciplined

Proceedings of the 25th International Conference on Software Engineering
Using task context to improve programmer productivity

Proceedings of the 14th ACM SIGSOFT international symposium on Foundations of software engineering
Information Needs in Collocated Software Development Teams

ICSE '07 Proceedings of the 29th international conference on Software Engineering
SpyWare: a change-aware development toolset

Proceedings of the 30th international conference on Software engineering
Software systems as cities: a controlled experiment

Proceedings of the 33rd International Conference on Software Engineering
A Controlled Experiment for Program Comprehension through Trace Visualization

IEEE Transactions on Software Engineering

Quantified Score

Hi-index	0.00

Visualization

Abstract

Software is created by humans, for humans. For this reason, software engineering is---above all---a human activity. Acknowledging this fact, many researchers perform controlled experiments with human subjects to evaluate the performance and usability of novel approaches and software engineering tools. However, the intrinsically non-deterministic nature of humans introduces a number of threats to the validity of such experiments. One of them concerns how to record information without influencing the behavior of the subjects involved. Another one relates to providing means to assure the correctness of the gathered data, for further pristine analyses and replication. We present Biscuit, a tool that silently records relevant pieces of information regarding an experiment performed with human subjects. We present the main features and benefits of Biscuit by showcasing a controlled experiment of Gaucho, a next generation IDE. Based on our experience, we discuss the potential of Biscuit and outline future research in this direction.