D-Swoosh: A Family of Algorithms for Generic, Distributed Entity Resolution

Authors:
Omar Benjelloun;Hector Garcia-Molina;Heng Gong;Hideki Kawai;Tait E. Larson;David Menestrina;Sutthipong Thavisomboon
Affiliations:
Stanford University;Stanford University;Stanford University;Stanford University;Stanford University;Stanford University;Stanford University
Venue:
ICDCS '07 Proceedings of the 27th International Conference on Distributed Computing Systems
Year:
2007

Citing 0
Cited 8

Repair checking in inconsistent databases: algorithms and complexity

Proceedings of the 12th International Conference on Database Theory
Swoosh: a generic approach to entity resolution

The VLDB Journal — The International Journal on Very Large Data Bases
Disambiguating authors in academic publications using random forests

Proceedings of the 9th ACM/IEEE-CS joint conference on Digital libraries
Generic Entity Resolution in Relational Databases

ADBIS '09 Proceedings of the 13th East European Conference on Advances in Databases and Information Systems
Generic entity resolution with negative rules

The VLDB Journal — The International Journal on Very Large Data Bases
A fast approach for parallel deduplication on multicore processors

Proceedings of the 2011 ACM Symposium on Applied Computing
Flexible and efficient distributed resolution of large entities

FoIKS'12 Proceedings of the 7th international conference on Foundations of Information and Knowledge Systems
Discovering denial constraints

Proceedings of the VLDB Endowment

Quantified Score

Hi-index	0.00

Visualization

Abstract

Entity Resolution (ER) matches and merges records that refer to the same real-world entities, and is typically a compute-intensive process due to complex matching functions and high data volumes. We present a family of algorithms, D-Swoosh, for distributing the ER workload across multiple processors. The algorithms use generic match and merge functions, and ensure that new merged records are distributed to processors that may have matching records. We perform a detailed performance evaluation on a testbed of 15 processors. Our experiments use actual comparison shopping data provided by Yahoo!. (keywords: Entity Resolution, Information Integration, Data Cleaning)