A vector space model for automatic indexing
Communications of the ACM
Data integration: a theoretical perspective
Proceedings of the twenty-first ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
A survey of approaches to automatic schema matching
The VLDB Journal — The International Journal on Very Large Data Bases
On schema matching with opaque column names and data values
Proceedings of the 2003 ACM SIGMOD international conference on Management of data
iMAP: discovering complex semantic matches between database schemas
SIGMOD '04 Proceedings of the 2004 ACM SIGMOD international conference on Management of data
Multi-column substring matching for database schema translation
VLDB '06 Proceedings of the 32nd international conference on Very large data bases
Ontology Matching
ACM Computing Surveys (CSUR)
Large-Scale Deduplication with Constraints Using Dedupalog
ICDE '09 Proceedings of the 2009 IEEE International Conference on Data Engineering
Automatic algorithm configuration based on local search
AAAI'07 Proceedings of the 22nd national conference on Artificial intelligence - Volume 2
Proceedings of the VLDB Endowment
The Probabilistic Relevance Framework: BM25 and Beyond
Foundations and Trends in Information Retrieval
An Introduction to Duplicate Detection
An Introduction to Duplicate Detection
On multi-column foreign key discovery
Proceedings of the VLDB Endowment
Helix: online enterprise data analytics
Proceedings of the 20th international conference companion on World wide web
Data Matching: Concepts and Techniques for Record Linkage, Entity Resolution, and Duplicate Detection
Learning expressive linkage rules using genetic programming
Proceedings of the VLDB Endowment
Instance-Based matching of large ontologies using locality-sensitive hashing
ISWC'12 Proceedings of the 11th international conference on The Semantic Web - Volume Part I
HIL: a high-level scripting language for entity integration
Proceedings of the 16th International Conference on Extending Database Technology
| Hi-index | 0.00 |
A basic step in integration is the identification of linkage points, i.e., finding attributes that are shared (or related) between data sources, and that can be used to match records or entities across sources. This is usually performed using a match operator, that associates attributes of one database to another. However, the massive growth in the amount and variety of unstructured and semi-structured data on the Web has created new challenges for this task. Such data sources often do not have a fixed pre-defined schema and contain large numbers of diverse attributes. Furthermore, the end goal is not schema alignment as these schemas may be too heterogeneous (and dynamic) to meaningfully align. Rather, the goal is to align any overlapping data shared by these sources. We will show that even attributes with different meanings (that would not qualify as schema matches) can sometimes be useful in aligning data. The solution we propose in this paper replaces the basic schema-matching step with a more complex instance-based schema analysis and linkage discovery. We present a framework consisting of a library of efficient lexical analyzers and similarity functions, and a set of search algorithms for effective and efficient identification of linkage points over Web data. We experimentally evaluate the effectiveness of our proposed algorithms in real-world integration scenarios in several domains.