A robust framework for real-time distributed processing of satellite data

Authors:
Shahram Tehranian;Yongsheng Zhao;Tony Harvey;Anand Swaroop;Keith Mckenzie
Affiliations:
AC Technologies, Inc., Advanced Computation and Information Services, 4640 Forbes Blvd. Suite 320, Lanham, MD 20706, USA;AC Technologies, Inc., Advanced Computation and Information Services, 4640 Forbes Blvd. Suite 320, Lanham, MD 20706, USA;AC Technologies, Inc., Advanced Computation and Information Services, 4640 Forbes Blvd. Suite 320, Lanham, MD 20706, USA;AC Technologies, Inc., Advanced Computation and Information Services, 4640 Forbes Blvd. Suite 320, Lanham, MD 20706, USA;National Environmental Satellite Data Information Service (NESDIS), National Oceanic and Atmospheric Administration (NOAA), US Department of Commerce, Suitland, MD 20746, USA
Venue:
Journal of Parallel and Distributed Computing
Year:
2006

Citing 9
Cited 5

Essential JNI: Java Native Interface

Essential JNI: Java Native Interface
DISCWorld: an environment for service-based matacomputing

Future Generation Computer Systems - Special issue on metacomputing
MPI-The Complete Reference, Volume 1: The MPI Core

MPI-The Complete Reference, Volume 1: The MPI Core
C++ Network Programming: Systematic Reuse with ACE and Frameworks, Vol. 2

C++ Network Programming: Systematic Reuse with ACE and Frameworks, Vol. 2
MPI/FTTM: Architecture and Taxonomies for Fault-Tolerant, Message-Passing Middleware for Performance-Portable Parallel Computing

CCGRID '01 Proceedings of the 1st International Symposium on Cluster Computing and the Grid
The ACE Programmer's Guide: Practical Design Patterns for Network and Systems Programming

The ACE Programmer's Guide: Practical Design Patterns for Network and Systems Programming
Distributed frameworks and parallel algorithms for processing large-scale geographic data

Parallel Computing - Special issue: High performance computing with geographical data
Big Wins with Small Application-Aware Caches

Proceedings of the 2004 ACM/IEEE conference on Supercomputing
MPICH-V2: a Fault Tolerant MPI for Volatile Nodes based on Pessimistic Sender Based Message Logging

Proceedings of the 2003 ACM/IEEE conference on Supercomputing

Parallel techniques for information extraction from hyperspectral imagery using heterogeneous networks of workstations

Journal of Parallel and Distributed Computing
Parallel morphological/neural processing of hyperspectral images using heterogeneous and homogeneous platforms

Cluster Computing
Clusters Versus FPGA for Parallel Processing of Hyperspectral Imagery

International Journal of High Performance Computing Applications
An interoperable spatiotemporal weather radar data dissemination system

International Journal of Remote Sensing
Improving the Performance of Hyperspectral Image and Signal Processing Algorithms Using Parallel, Distributed and Specialized Hardware-Based Systems

Journal of Signal Processing Systems

Quantified Score

Hi-index	0.01

Visualization

Abstract

It is estimated that future satellite instruments such as the Advanced Baseline Imager (ABI) and the Hyperspectral Environmental Suite (HES) on the GOES-R series of satellites will provide raw data volume of about 1.5Terabyte per day. Due to the high data rate, satellite ground data processing will require considerable computing power to process data in real-time. Cluster technologies employing a multi-processor system present the only current economically viable option. To sustain high levels of system reliability and operability in a cluster-oriented operational environment, a fault-tolerant data processing framework is proposed to provide a platform for encapsulating science algorithms for satellite data processing. The science algorithms together with the framework are hosted on a Linux cluster. In this paper we present an architectural model and a system prototype for providing performance, reliability, and scalability of candidate hardware and software for a satellite data processing system. Furthermore, benchmarking results are presented for a selected number of science algorithms for the Geosynchronous Imaging Fourier Transform Spectrometer (GIFTS) instrument showing that considerable performance can be gained without sacrificing the reliability and high availability constraints imposed on the operational cluster system.