Statistics to measure correlation for data mining applications

Authors:
V. J. Rayward-Smith
Affiliations:
Department of Business Systems, Monash University, Clayton, VIC 3800, Australia
Venue:
Computational Statistics & Data Analysis
Year:
2007

Citing 2
Cited 2

Mathematical statistics (4th ed.)

Mathematical statistics (4th ed.)
Data clustering: a review

ACM Computing Surveys (CSUR)

A formal and empirical analysis of the fuzzy gamma rank correlation coefficient

Information Sciences: an International Journal
A computational knowledge-based model for emulating human performance in the Iowa Gambling Task

Neural Networks

Quantified Score

Hi-index	0.03

Visualization

Abstract

Correlation is usually used in the context of real-valued sequences but, in data mining, the values of fields may be of various types-real, nominal or ordinal. Techniques for measuring correlation between any two sequences of data are reviewed, regardless of their type. In particular, a new technique for measuring the correlation between real-valued data and nominal data is proposed. The technique relies on the definition of an assignment of the nominal values to real values and hence is called A-correlation (A for assignment). The proposed assignment is defined to be the most favourable of all such assignments and can be efficiently computed. Moreover, it is shown that the resulting correlation coefficient has a natural interpretation independent of the assignment. With nominal/nominal data, Cramer's V-statistic can be used or, alternatively, a new statistic based on matching. These can also be used in the ordinal/nominal case. With just ordinal data, correlations based on rank are appropriate and these can also be used in the ordinal/ordinal and the ordinal/real cases.