Parallel implementation of classification algorithms based on MapReduce

Authors:
Qing He;Fuzhen Zhuang;Jincheng Li;Zhongzhi Shi
Affiliations:
The Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China;The Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China and Graduate University of Chinese Academy of Sciences, Bei ...;The Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China and Graduate University of Chinese Academy of Sciences, Bei ...;The Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
Venue:
RSKT'10 Proceedings of the 5th international conference on Rough set and knowledge technology
Year:
2010

Citing 10
Cited 1

Computer systems that learn: classification and prediction methods from statistics, neural nets, machine learning, and expert systems

Computer systems that learn: classification and prediction methods from statistics, neural nets, machine learning, and expert systems
C4.5: programs for machine learning

C4.5: programs for machine learning
Machine learning, neural and statistical classification

Machine learning, neural and statistical classification
Machine Learning

Machine Learning
Induction of Decision Trees

Machine Learning
Pattern Classification (2nd Edition)

Pattern Classification (2nd Edition)
MapReduce: simplified data processing on large clusters

OSDI'04 Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6
Computing in the clouds

netWorker - Cloud computing: PC functions move onto the web
Market-Oriented Cloud Computing: Vision, Hype, and Reality for Delivering IT Services as Computing Utilities

HPCC '08 Proceedings of the 2008 10th IEEE International Conference on High Performance Computing and Communications
Pairwise document similarity in large collections with MapReduce

HLT-Short '08 Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics on Human Language Technologies: Short Papers

Parallel extreme learning machine for regression based on MapReduce

Neurocomputing

Quantified Score

Hi-index	0.00

Visualization

Abstract

Data mining has attracted extensive research for several decades. As an important task of data mining, classification plays an important role in information retrieval, web searching, CRM, etc. Most of the present classification techniques are serial, which become impractical for large dataset. The computing resource is under-utilized and the executing time is not waitable. Provided the program mode of MapReduce, we propose the parallel implementation methods of several classification algorithms, such as k-nearest neighbors, naive bayesian model and decision tree, etc. Preparatory experiments show that the proposed parallel methods can not only process large dataset, but also can be extended to execute on a cluster, which can significantly improve the efficiency.