MapReduce: simplified data processing on large clusters
Communications of the ACM - 50th anniversary issue: 1958 - 2008
Humanities e-Science: From Systematic Investigations to Institutional Infrastructures
ESCIENCE '10 Proceedings of the 2010 IEEE Sixth International Conference on e-Science
| Hi-index | 0.00 |
Language and literature researchers often use a variety of data resources in order to conduct their day-to-day research. Such resources include dictionaries, thesauri, corpora, images, audio and video collections. These resources are typically distributed, and comprise non-interoperable repositories of data that are often licence protected. In this context, researchers typically conduct their research through direct access to individual web-based resources. This form of research is non-scalable, time consuming and often frustrating to the researchers. The JISC funded project Enhancing Repositories for Language and Literature Researchers (ENROLLER, http://www.gla.ac.uk/enroller/) aims to address this by provision of an interactive, research infrastructure providing seamless access to a range of major language and literature repositories. This paper describes this infrastructure and the services that have been developed to overcome the issues in access and use of digital resources in humanities. In particular, we describe how high performance computing facilities including the UK e-Science National Grid Service (NGS, http://www.ngs.ac.uk) have been exploited to support advanced, bulk search capabilities, implemented using Google's MapReduce algorithm. We also describe our experiences in the use of the resource brokering Workload Management System (WMS) and the Virtual Organization Membership Service (VOMS) solutions in this space. Finally we outline the experiences from the arts and humanities community on the usage of this infrastructure.