RACHET: An Efficient Cover-Based Merging of Clustering Hierarchies from Distributed Datasets

Authors:
Nagiza F. Samatova;George Ostrouchov;Al Geist;Anatoli V. Melechko
Affiliations:
Computer Science and Mathematics Division, Oak Ridge National Laboratory, P.O. Box 2008, Oak Ridge, TN 37831, USA. samatovan@ornl.gov;Computer Science and Mathematics Division, Oak Ridge National Laboratory, P.O. Box 2008, Oak Ridge, TN 37831, USA. ost@ornl.gov;Computer Science and Mathematics Division, Oak Ridge National Laboratory, P.O. Box 2008, Oak Ridge, TN 37831, USA. gst@ornl.gov;Molecular-Scale Engineering and Nanoscale Technologies Group, Oak Ridge National Laboratory, P.O. Box 2008, Oak Ridge, TN 37831, USA. melechko@unix.cas.utk.edu
Venue:
Distributed and Parallel Databases - Special issue: Parallel and distributed data mining
Year:
2002

Citing 6
Cited 12

Introduction to statistical pattern recognition (2nd ed.)

Introduction to statistical pattern recognition (2nd ed.)
Parallel algorithms for hierarchical clustering

Parallel Computing
Mining business databases

Communications of the ACM
Advances in knowledge discovery and data mining

Advances in knowledge discovery and data mining
Data clustering: a review

ACM Computing Surveys (CSUR)
Distributed clustering using collective principal component analysis

Knowledge and Information Systems

Element matching across data-oriented XML sources using a multi-strategy clustering model

Data & Knowledge Engineering
PENS: an algorithm for density-based clustering in peer-to-peer systems

InfoScale '06 Proceedings of the 1st international conference on Scalable information systems
Distributed hierarchical document clustering

ACST'06 Proceedings of the 2nd IASTED international conference on Advances in computer science and technology
Merging distributed database summaries

Proceedings of the sixteenth ACM conference on Conference on information and knowledge management
A consensus-driven fuzzy clustering

Pattern Recognition Letters
Distributed data mining and agents

Engineering Applications of Artificial Intelligence
Approximated clustering of distributed high-dimensional data

PAKDD'05 Proceedings of the 9th Pacific-Asia conference on Advances in Knowledge Discovery and Data Mining
Clustering distributed data streams in peer-to-peer environments

Information Sciences: an International Journal
Distributed data mining patterns and services: an architecture and experiments

Concurrency and Computation: Practice & Experience
Effective fuzzy semantic clustering scheme for decentralised network through multi-domain ontology model

International Journal of Metadata, Semantics and Ontologies
Effective fuzzy semantic clustering scheme for decentralised network through multi-domain ontology model

International Journal of Metadata, Semantics and Ontologies
GoSCAN: Decentralized scalable data clustering

Computing

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper presents a hierarchical clustering method named RACHET (Recursive Agglomeration of Clustering Hierarchies by Encircling Tactic) for analyzing multi-dimensional distributed data. A typical clustering algorithm requires bringing all the data in a centralized warehouse. This results in O(nd) transmission cost, where n is the number of data points and d is the number of dimensions. For large datasets, this is prohibitively expensive. In contrast, RACHET runs with at most O(n) time, space, and communication costs to build a global hierarchy of comparable clustering quality by merging locally generated clustering hierarchies. RACHET employs the encircling tactic in which the merges at each stage are chosen so as to minimize the volume of a covering hypersphere. For each cluster centroid, RACHET maintains descriptive statistics of constant complexity to enable these choices. RACHET's framework is applicable to a wide class of centroid-based hierarchical clustering algorithms, such as centroid, medoid, and Ward.