Data-intensive e-science frontier research
Communications of the ACM - Blueprint for the future of high-performance networking
Web Services Interfaces and Open Standards Integration into the European UNICORE 6 Grid Middleware
EDOCW '07 Proceedings of the 2007 Eleventh International IEEE EDOC Conference Workshop
| Hi-index | 0.00 |
UNICORE is a state of the art and well tested Grid middleware, designed for seamless and secure access to distributed resources, applications and data, in an easy to use fashion. A wide variety of UNICORE applications for example in bio-informatics generate and compute huge amounts of data. These large amounts of data are not easy to manage reliably and efficiently with the default UNICORE storage system which is using a standard file system. Hence, the current UNICORE does not support a scalable distributed storage system so far. We have integrated Apache Hadoop and its supported distributed storage/ file systems into the UNICORE storage management service. Thus allows to build a UNICORE storage system providing data replication, disaster recovery, durability and elasticity. In this paper we will present the architecture and operation of a prototype called UniHadoop, which provides the integration of UNICORE and the distributed storage systems (DSS) supported by Hadoop and highlight its potential in usage scenarios.