A Large-scale Images Processing Model Based on Hadoop Platform

  • Authors:

  • Gongrong Zhang;Qingxiang Wu;Zhiqiang Zhuo;Xiaowei Wang;Xiaojin Lin

  • Affiliations:

  • College of Photonic and Electronic Engineering, Fujian Normal University Fujian, Fuzhou, 350007, China;College of Photonic and Electronic Engineering, Fujian Normal University Fujian, Fuzhou, 350007, China;College of Photonic and Electronic Engineering, Fujian Normal University Fujian, Fuzhou, 350007, China;College of Photonic and Electronic Engineering, Fujian Normal University Fujian, Fuzhou, 350007, China;College of Photonic and Electronic Engineering, Fujian Normal University Fujian, Fuzhou, 350007, China

  • Venue:
  • Proceedings of the Second International Conference on Innovative Computing and Cloud Computing
  • Year:
  • 2013

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper presents a parallel processing model based on Hadoop platform for large-scale images processing, which aims to make use of the advantages of high reliability and high scalability of Hadoop distributed platform for distributed memory and distributed computing, so as to achieve the purpose of fast processing of large-scale images. The Hadoop streaming technology is used in the model. The main operations are written on shell script as the mapper of Hadoop streaming, then an assigned filelist is used as the Hadoop streaming's input. The large numbers of image files are delivered to cluster computers for concurrent image processing. The model has been implemented using virtual machines. A set of experimental results and analysis are provided.