Random multiclass classification: generalizing random forests to random MNL and random NB

Authors:
Anita Prinzie
Affiliations:
Department of Marketing, Ghent University, Ghent, Belgium
Venue:
DEXA'07 Proceedings of the 18th international conference on Database and Expert Systems Applications
Year:
2007

Citing 4
Cited 1

Random Forests

Machine Learning
Predicting home-appliance acquisition sequences: Markov/Markov for Discrimination and survival analysis for modeling sequential information in NPTB models

Decision Support Systems
Hidden naive Bayes

AAAI'05 Proceedings of the 20th national conference on Artificial intelligence - Volume 2
An analysis of Bayesian classifiers

AAAI'92 Proceedings of the tenth national conference on Artificial intelligence

Botnets: a heuristic-based detection framework

Proceedings of the Fifth International Conference on Security of Information and Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

Random Forests (RF) is a successful classifier exhibiting performance comparable to Adaboost, but is more robust. The exploitation of two sources of randomness, random inputs (bagging) and random features, make RF accurate classifiers in several domains. We hypothesize that methods other than classification or regression trees could also benefit from injecting randomness. This paper generalizes the RF framework to other multiclass classification algorithms like the well-established MultiNomial Logit (MNL) and Naive Bayes (NB). We propose Random MNL (RMNL) as a new bagged classifier combining a forest of MNLs estimated with randomly selected features. Analogously, we introduce Random Naive Bayes (RNB). We benchmark the predictive performance of RF, RMNL and RNB against state-of-the-art SVM classifiers. RF, RMNL and RNB outperform SVM. Moreover, generalizing RF seems promising as reflected by the improved predictive performance of RMNL.