MapReduce for Data Intensive Scientific Analyses
ESCIENCE '08 Proceedings of the 2008 Fourth IEEE International Conference on eScience
HAIL: a high-availability and integrity layer for cloud storage
Proceedings of the 16th ACM conference on Computer and communications security
Delay scheduling: a simple technique for achieving locality and fairness in cluster scheduling
Proceedings of the 5th European conference on Computer systems
Cloud computing paradigms for pleasingly parallel biomedical applications
Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing
CloudEdge: a content delivery system for storage service in cloud environment
International Journal of Ad Hoc and Ubiquitous Computing
Dynamic proportional share scheduling in Hadoop
JSSPP'10 Proceedings of the 15th international conference on Job scheduling strategies for parallel processing
CloudBATCH: A Batch Job Queuing System on Clouds with Hadoop and HBase
CLOUDCOM '10 Proceedings of the 2010 IEEE Second International Conference on Cloud Computing Technology and Science
Study on Cloud Storage System Based on Distributed Storage Systems
ICCIS '10 Proceedings of the 2010 International Conference on Computational and Information Sciences
Cloud Technologies for Bioinformatics Applications
IEEE Transactions on Parallel and Distributed Systems
Optimizing Multiway Joins in a Map-Reduce Environment
IEEE Transactions on Knowledge and Data Engineering
A personalized mobile IPTV system with seamless video reconstruction algorithm in cloud networks
International Journal of Communication Systems
Hi-index | 0.24 |
Most users process short tasks using MapReduce. In other words, most tasks handled by the Map and Reduce functions require low response time. Currently, quite few users use MapReduce for 2D to 3D image processing, which is highly complicated and requires long execution time. However, in our opinion, MapReduce is exactly suitable for processing applications of high complexity and high computation. This paper implements MapReduce on an integrated 2D to 3D multi-user system, in which Map is responsible for image processing procedures of high complexity and high computation, and Reduce is responsible for integrating the intermediate data processed by Map for the final output. Different from short tasks, when several users compete simultaneously to acquire data from MapReduce for 2D to 3D applications, data that waits to be processed by Map will be delayed by the current user and Reduce has to wait until the completion of all Map tasks to generate the final result. Therefore, a novel scheduling scheme, Dynamic Switch of Reduce Function (DSRF) Algorithm, is proposed in this paper for MapReduce to switch dynamically to the next task according to the achieved percentage of tasks and reduce the idle time of Reduce. By using Hadoop to implement our MapReduce platform, we compare the performance of traditional Hadoop with our proposed scheme. The experimental results reveal that our proposed scheduling scheme efficiently enhances MapReduce performance in running 2D to 3D applications.