A document-based data warehousing approach for large scale data mining
ICPCA/SWS'12 Proceedings of the 2012 international conference on Pervasive Computing and the Networked World
Efficient algorithms for frequent pattern mining in many-task computing environments
Knowledge-Based Systems
| Hi-index | 0.00 |
Data indexing is common in data mining when working with high-dimensional, large-scale data sets. Hadoop, a cloud computing project using the MapReduce framework in Java, has become of significant interest in distributed data mining. A feasible distributed data indexing algorithm is proposed for Hadoop data mining, based on ZSCORE binning and inverted indexing and on the Hadoop SequenceFile format. A data mining framework on Hadoop using the Java Persistence API (JPA) and MySQL Cluster is proposed. The framework is elaborated in the implementation of a decision tree algorithm on Hadoop. We compare the data index-ing algorithm with Hadoop MapFile indexing, which performs a binary search, in a modest cloud environment. The results show the algorithm is more efficient than naïve MapFile indexing. We compare the JDBC and JPA implementations of the data mining framework. The performance shows the framework is efficient for data mining on Hadoop.