Normalizing variables with too-frequent values using a Kolmogorov-Smirnov test: A practical approach

  • Authors:
  • Zvi Drezner;Ofir Turel

  • Affiliations:
  • Steven G. Mihaylo College of Business and Economics, California State University - Fullerton, Fullerton, CA 92834, United States;Steven G. Mihaylo College of Business and Economics, California State University - Fullerton, Fullerton, CA 92834, United States

  • Venue:
  • Computers and Industrial Engineering
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

Many quantitative applications in business operations, environmental engineering, and production assume sufficient normality of data, which is often, demonstrated using tests of normality, such as the Kolmogorov deemed Smirnov test. A practical problem arises when a high proportion of a too-frequent value exists in data, in which case transformation to normality that passes tests for normality may be impossible. Analysts and researchers are therefore often concerned with the question: should we bother transforming the variable to normality? Or should we revert to other approaches not requiring a normal distribution? In this study, we find the critical number of the frequency of a single value for which there is no feasible transformation to normality within a given @a of the Kolmogorov-Smirnov test. The resultant decision table can guide the effort of analysts and researchers.