Turning the tide of the data deluge
Proceedings of the first annual workshop on High performance computing meets databases
| Hi-index | 0.00 |
In the evolution of computing technology over the decades, file system capabilities have not grown in tandem to processing power. Today, scientific computing is highly data intensive and relies on workflows. Workflows developed are not portable among workflow management system. Also, scientific computation that rely on inherent workflows do not have a kernel support for workflows, and are executed essentially in a batch processing model. A file system that includes native kernel functionalities to support workflow execution would address the issue of parallel processing as well as portability. Such a file system would improve scientific computing performance. This paper describes an approach we developed to add workflow functionality to the Linux kernel and native file system to help simplify the use of workflow management systems for scientific computing.