Phoenix: Making Data-Intensive Grid Applications Fault-Tolerant
GRID '04 Proceedings of the 5th IEEE/ACM International Workshop on Grid Computing
Faults in large distributed systems and what we can do about them
Euro-Par'05 Proceedings of the 11th international Euro-Par conference on Parallel Processing
| Hi-index | 0.00 |
Grid computing brings with it additional complexities and unexpected failures. Just keeping track of our jobs traversing different grid resources before completion can at times become tricky. We introduce a client-centric grid knowledgebase that keeps track of the job performance and failure characteristics on different grid resources as observed by the client. We present the design and implementation of our prototype grid knowledgebase and evaluate its effectiveness on two real life grid data processing pipelines: NCSA image processing pipeline and WCER video processing pipeline. It enabled us to easily extract useful job and resource information and interpret them to make better scheduling decisions. Using it, we were able to understand failures better and were able to devise innovative methods to automatically avoid and recover from failures and dynamically adapt to grid environment improving fault-tolerance and performance.