Empirical evaluation in Computer Science research published by ACM

  • Authors:

  • Jacques Wainer;Claudia G. Novoa Barsottini;Danilo Lacerda;Leandro Rodrigues Magalhães de Marco

  • Affiliations:

  • Computing Institute, University of Campinas, Av. Albert Einstein 1251, 13083-852 Campinas, SP, Brazil;Department of Health Informatics, Federal University of Sao Paulo, Brazil;Computing Institute, University of Campinas, Av. Albert Einstein 1251, 13083-852 Campinas, SP, Brazil;Computing Institute, University of Campinas, Av. Albert Einstein 1251, 13083-852 Campinas, SP, Brazil

  • Venue:
  • Information and Software Technology
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper repeats part of the analysis performed in the 1995 paper ''Experimental evaluation in Computer Science: a quantitative study'' by Tichy and collaborators, for 147 papers randomly selected from the ACM, published in the year 2005. The papers published in 2005 are classified in the following way: 4% theory, 17% empirical, 4.7% hypothesis testing, 3.4% other, and 70% design and modeling (using the 1995 paper categories). Within the design and modeling class, 33% of the papers have no evaluation. The numbers of the 2005 sample are very similar to the original figures for the 1995 sample, which shows that Computer Science research has not increased significantly its empirical or experimental component.