A programming framework for integrating web-based spatiotemporal sensor data with MapReduce capabilities

  • Authors:
  • James Horey

  • Affiliations:
  • Oak Ridge National Laboratory

  • Venue:
  • Proceedings of the ACM SIGSPATIAL International Workshop on GeoStreaming
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

Web-based sensor data, provided by organizations such as the National Oceanographic and Atmospheric Administration, provide a valuable service to the public and scientific communities. However, much of this data is locked in a variety of presentation formats and is computationally inaccessible. In addition, although these data have a spatiotemporal context, both the spatial and temporal data are usually only implicitly defined. Although storing this data in a consistent database can partially resolve this problem, a data-driven programming model coupled with MapReduce capabilities is a more flexible and extensible solution. Our implementation of this programming model allows users to parse a wide array of sensor data and express complex computation in a simple, scalable manner. In addition, our framework uses a simple key-value storage mechanism and provides convenient geospatial output mechanisms. In this paper, we discuss some early results of our programming model within the context of our current Java-oriented implementation, and demonstrate how the system can be used to create many different applications. We also discuss and evaluate our system with respect to memory usage and scalability.