Shared State for Distributed Interactive Data Mining Applications
Distributed and Parallel Databases - Special issue: Parallel and distributed data mining
Exploiting high-level coherence information to optimize distributed shared state
Proceedings of the ninth ACM SIGPLAN symposium on Principles and practice of parallel programming
| Hi-index | 0.00 |
InterWeave is a distributed middleware system that attempts to do for computer programs what the World Wide Web did for human beings: make it dramatically simpler to share information across the Internet. Specifically, InterWeave allows processes written in multiple languages, running on heterogeneous machines, to share arbitrary typed data structures as if they resided in local memory. In C, operations on shared data, including pointers, take precisely the same form as operations on non-shared data. Sharing at all levels is supported seamlessly---InterWeave can accommodate hardware coherence and consistency within multiprocessors (level-1 sharing), software distributed shared memory (SDSM) within tightly coupled clusters (level-2 sharing), and version-based coherence and consistency across the Internet (level-3 sharing). Application-specific knowledge of minimal coherence requirements is used to minimize communication. Consistency information is maintained in a manner that allows scaling to large amounts of shared data. .pp We discuss the implementation of InterWeave in some detail, with a particular emphasis on memory management; coherence and consistency; and communication and heterogeneity. We then evaluate the performance and usability of the system. Anecdotal evidence suggests that the InterWeave prototype significantly simplifies the construction of important distributed applications. Quantitative evidence demonstrates that it achieves this simplification at acceptably modest cost.