Managing data quality in a terabyte-scale sensor archive

Authors:
Bryce Cutt;Ramon Lawrence
Affiliations:
University of British Columbia Okanagan;University of British Columbia Okanagan
Venue:
Proceedings of the 2008 ACM symposium on Applied computing
Year:
2008

Citing 8
Cited 0

Striving towards Near Real-Time Data Integration for Data Warehouses

DaWaK 2000 Proceedings of the 4th International Conference on Data Warehousing and Knowledge Discovery
Chimera: AVirtual Data System for Representing, Querying, and Automating Data Derivation

SSDBM '02 Proceedings of the 14th International Conference on Scientific and Statistical Database Management
Earth System Science Workbench: A Data Management Infrastructure for Earth Science Products

SSDBM '01 Proceedings of the 13th International Conference on Scientific and Statistical Database Management
Provenance management in curated databases

Proceedings of the 2006 ACM SIGMOD international conference on Management of data
A Framework for Collecting Provenance in Data-Centric Scientific Workflows

ICWS '06 Proceedings of the IEEE International Conference on Web Services
CASA and LEAD: Adaptive Cyberinfrastructure for Real-Time Multiscale Weather Forecasting

Computer
Building a terabyte NEXRAD radar database for hydrometeorology research

Computers & Geosciences
Declarative support for sensor data cleaning

PERVASIVE'06 Proceedings of the 4th international conference on Pervasive Computing

Quantified Score

Hi-index	0.00

Visualization

Abstract

Sensor networks collect vast amounts of real-time information about the environment, business processes, and systems. Archived sensor data is valuable for long-term analysis and decision making, which requires it be suitably archived, indexed, and validated. In this paper, we describe a general approach to managing and improving data quality by the generation and validation of metadata and the logging of workflow events. The approach has been implemented within a system archiving terabytes of U.S. weather radar data. The data quality system has resulted in the detection of data errors while simplifying the administration of the complex archive system.