Sync your data: update propagation for heterogeneous protein databases

• Authors: T. Claypool;A. Rundensteiner
• Venue: The VLDB Journal — The International Journal on Very Large Data Bases
• Year: 2005

The traditional model of bench (wet) chemistry in many life sciences domain is today actively complimented by computer-based discoveries utilizing the growing number of online data sources. A typical computer-based discovery scenario for many life scientists includes the creation of local caches of pertinent information from multiple online resources such as Swissprot [Nucleic Acid Res. 1(28), 45–48 (2000)], PIR [Nucleic Acids Res. 28(1), 41–44 (2000)], PDB [The Protein DataBank. Wiley, New York (2003)], to enable efficient data analysis. This local caching of data, however, exposes their research and eventual results to the problems of data staleness, that is, cached data may quickly be obsolete or incorrect, dependent on the updates that are made to the source data. This represents a significant challenge to the scientific community, forcing scientists to be continuously aware of the frequent changes made to public data sources, and more importantly aware of the potential effects on their own derived data sets during the course of their research. To address this significant challenge, in this paper we present an approach for handling update propagation between heterogeneous databases, guaranteeing data freshness for scientists irrespective of their choice of data source and its underlying data model or interface. We propose a middle-layer–based solution wherein first the change in the online data source is translated to a sequence of changes in the middle-layer; next each change in the middle-layer is propagated through an algebraic representation of the translation between the source and the target; and finally the net-change is translated to a set of changes that are then applied to the local cache. In this paper, we present our algebraic model that represents the mapping of the online resource to the local cache, as well as our adaptive propagation algorithm that can incrementally propagate both schema and data changes from the source to the cache in a data model independent manner. We present a case study based on a joint ongoing project with our collaborators in the Chemistry Department at UMass-Lowell to explicate our approach.