On evaluating recommender systems for API usages

Authors:
Marcel Bruch;Thorsten Schäfer;Mira Mezini
Affiliations:
Darmstadt University of Technology;Darmstadt University of Technology;Darmstadt University of Technology
Venue:
Proceedings of the 2008 international workshop on Recommendation systems for software engineering
Year:
2008

Citing 13
Cited 2

Mining association rules between sets of items in large databases

SIGMOD '93 Proceedings of the 1993 ACM SIGMOD international conference on Management of data
Data mining library reuse patterns using generalized association rules

Proceedings of the 22nd international conference on Software engineering
Object-oriented framework-based software development: problems and experiences

ACM Computing Surveys (CSUR)
Investigating Reading Techniques for Object-Oriented Framework Learning

IEEE Transactions on Software Engineering
Jungloid mining: helping to navigate the API jungle

Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation
DynaMine: finding common error patterns by mining software revision histories

Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering
PR-Miner: automatically extracting implicit programming rules and detecting violations in large software code

Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering
MAPO: mining API usages from open source repositories

Proceedings of the 2006 international workshop on Mining software repositories
XSnippet: mining For sample code

Proceedings of the 21st annual ACM SIGPLAN conference on Object-oriented programming systems, languages, and applications
FrUiT: IDE support for framework understanding

eclipse '06 Proceedings of the 2006 OOPSLA workshop on eclipse technology eXchange
Detecting object usage anomalies

Proceedings of the the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering
Parseweb: a programmer assistant for reusing open source code on the web

Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering
A study of cross-validation and bootstrap for accuracy estimation and model selection

IJCAI'95 Proceedings of the 14th international joint conference on Artificial intelligence - Volume 2

Learning from examples to improve code completion systems

Proceedings of the the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering
On recommending meaningful names in source and UML

Proceedings of the 2nd International Workshop on Recommendation Systems for Software Engineering

Quantified Score

Hi-index	0.00

Visualization

Abstract

To ease framework understanding, tools have been developed that analyze existing framework instantiations to extract API usage patterns and present them to the user. However, detailed quantitative evaluations of such recommender systems are lacking. In this paper we present an automated evaluation process which extracts queries and expected results from existing code bases. This enables the validation of recommendation systems with large test beds in an objective manner by means of precision and recall measures. We demonstrate the applicability of our approach by evaluating an improvement of an existing API recommender tool that takes into account the framework-method context for recommendations.