Training genetic programming on half a million patterns: an example from anomaly detection

  • Authors:
  • Dong Song;M. I. Heywood;A. N. Zincir-Heywood

  • Affiliations:
  • Quest Software Inc., Halifax, NS, Canada;-;-

  • Venue:
  • IEEE Transactions on Evolutionary Computation
  • Year:
  • 2005

Quantified Score

Hi-index 0.00

Visualization

Abstract

The hierarchical RSS-DSS algorithm is introduced for dynamically filtering large datasets based on the concepts of training pattern age and difficulty, while utilizing a data structure to facilitate the efficient use of memory hierarchies. Such a scheme provides the basis for training genetic programming (GP) on a data set of half a million patterns in 15 min. The method is generic, thus, not specific to a particular GP structure, computing platform, or application context. The method is demonstrated on the real-world KDD-99 intrusion detection data set, resulting in solutions competitive with those identified in the original KDD-99 competition, while only using a fraction of the original features. Parameters of the RSS-DSS algorithm are demonstrated to be effective over a wide range of values. An analysis of different cost functions indicates that hierarchical fitness functions provide the most effective solutions.