Collecting Feedback during Software Engineering Experiments

  • Authors:

  • Amela Karahasanoviæ;Bente Anda;Erik Arisholm;Siw Elisabeth Hove;Magne Jørgensen;Dag I. Sjøberg;Ray Welland

  • Affiliations:

  • Simula Research Laboratory, Lysaker, Norway NO-1325;Simula Research Laboratory, Lysaker, Norway NO-1325;Simula Research Laboratory, Lysaker, Norway NO-1325;Simula Research Laboratory, Lysaker, Norway NO-1325;Simula Research Laboratory, Lysaker, Norway NO-1325;Simula Research Laboratory, Lysaker, Norway NO-1325;Department of Computing Science, University of Glasgow, Glasgow, USA G12 8RZ

  • Venue:
  • Empirical Software Engineering
  • Year:
  • 2005

Quantified Score

Hi-index 0.00

Visualization

Abstract

Objective: To improve the qualitative data obtained from software engineering experiments by gathering feedback during experiments. Rationale: Existing techniques for collecting quantitative and qualitative data from software engineering experiments do not provide sufficient information to validate or explain all our results. Therefore, we would like a cost-effective and unobtrusive method of collecting feedback from subjects during an experiment to augment other sources of data. Design of study: We formulated a set of qualitative questions that might be answered by collecting feedback during software engineering experiments. We then developed a tool to collect such feedback from experimental subjects. This feedback-collection tool was used in four different experiments and we evaluated the usefulness of the feedback obtained in the context of each experiment. The feedback data was triangulated with other sources of quantitative and qualitative data collected for the experiments. Results: We have demonstrated that the collection of feedback during experiments provides useful additional data to: validate the data obtained from other sources about solution times and quality of solutions; check process conformance; understand problem solving processes; identify problems with experiments; and understand subjects驴 perception of experiments. Conclusions: Feedback collection has proved useful in four experiments and we intend to use the feedback-collection tool in a range of other experiments to further explore the cost-effectiveness and limitations of this technique. It is also necessary to carry out a systematic study to more fully understand the impact of the feedback-collecting tool on subjects驴 performance in experiments.