Big data, big business: bridging the gap
Proceedings of the 1st International Workshop on Big Data, Streams and Heterogeneous Source Mining: Algorithms, Systems, Programming Models and Applications
| Hi-index | 0.00 |
In this paper, we present a cloud framework to provide cloud clustering, workflow scheduling and management, fault tolerance and distributed data storage, data analytics and visualisation services. Using a practical case study, we show that in the process of analyzing multiscale climate data, typical problems plaguing data analysts are faced. These include large datasets and limited computational resources, data complexity and limited knowledge, and varying data structures/formats and the need to integrate different tools. The implementation of our framework to climate studies was a success. This can be seen in its ability to perform spatio-temporal data analysis and visualization of a large multi-dimensional climate dataset with reduced processing time. The framework demonstrates great flexibility and simplicity for end users intending to perform data analysis by aiding the integration of data and tools and enabling interactive visualization on-the-fly. This is coupled with effective utilization of computational resources and data storage systems.