A Preliminary Investigation of Modified XCS as a Generic Data Mining Tool

• Authors: Phillip William Dixon;David Corne;Martin J. Oates
• Venue: IWLCS '01 Revised Papers from the 4th International Workshop on Advances in Learning Classifier Systems
• Year: 2001

Wilson's XCS classifier system has recently been modified and extended in ways which enable it to be applied to real-world benchmark data mining problems. Excellent results have been reported already on one such problem by Wilson, while other work by Saxon and Barry on a tunable collection of machine learning problems has also pointed to the strong potential of XCS in this area. In this paper we test a modified XCS implementation on twelve benchmark machine learning problems, all real-world derived. XCS is compared on these benchmarks with C4.5 and with HIDER (a new and sophisticated GA for machine learning developed elsewhere). Results for both C4.5, HIDER and XCS on each problem were tenfold cross-validated, and in the case of HIDER and XCS a modest amount of preliminary parameter investigation was done to find good results in each case. We find that XCS outperforms the other techniques in eight of the twelve problems, and is second-best in two of the remaining three. Some investigation is then done of the variance in XCS performance, and we find this to be verging on significant, either when varying the data fold composition, or the algorithmic random seed. We also investigate variation of several XCS parameters around well-known default settings. We find the default settings to be generally robust, but find the mutation rates and GA selection scheme to be particularly worthy of exploration with a view to improved performance. We conclude that XCS has the potential to be a powerful general data mining tool, at least for databases without too many fields, but that considerable research is warranted to identify rules and guidelines for parameter and strategy setting.