Vertical partitioning algorithms for database design
ACM Transactions on Database Systems (TODS)
An improved spectral graph partitioning algorithm for mapping parallel computations
SIAM Journal on Scientific Computing
The Coign automatic distributed partitioning system
OSDI '99 Proceedings of the third symposium on Operating systems design and implementation
Distributed and Parallel Databases
Parameterized View Definition and Recursive Relations
Proceedings of the Second International Conference on Data Engineering
J-Orchestra: Automatic Java Application Partitioning
ECOOP '02 Proceedings of the 16th European Conference on Object-Oriented Programming
A Note on Distributed Computing
MOS '96 Selected Presentations and Invited Papers Second International Workshop on Mobile Object Systems - Towards the Programmable Internet
A Compiler and Runtime Infrastructure for Automatic Program Distribution
IPDPS '05 Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium (IPDPS'05) - Papers - Volume 01
MapReduce: simplified data processing on large clusters
OSDI'04 Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6
Dynamo: amazon's highly available key-value store
Proceedings of twenty-first ACM SIGOPS symposium on Operating systems principles
Automatic Trace-Based Parallelization of Java Programs
ICPP '07 Proceedings of the 2007 International Conference on Parallel Processing
Pig latin: a not-so-foreign language for data processing
Proceedings of the 2008 ACM SIGMOD international conference on Management of data
The Demaq system: declarative development of distributed applications
Proceedings of the 2008 ACM SIGMOD international conference on Management of data
SCOPE: easy and efficient parallel processing of massive data sets
Proceedings of the VLDB Endowment
Processes Are Data: A Programming Model for Distributed Applications
WISE '09 Proceedings of the 10th International Conference on Web Information Systems Engineering
Robotics and Computer-Integrated Manufacturing
Hi-index | 0.00 |
The goal of the Demaq/TransScale system is to automate the distribution of applications to multiple hosts. Today, in order to create highly scalable architectures, developers have to manually restructure the application logic, creating balanced application fragments and data partitions to distribute across the available host machines. Our approach automates much of this manual work. We show how a novel, messaging-based programming model allows to implement distribution as a source-level transformation that turns a non-distributed application specification into a set of programs that can be executed on the various machines of a cluster. The challenge is the identification of application fragments that can be executed independently of each other, such that each fragment can be executed using only local transactions. Further, our method allows developers to increase scalability (i.e. the number of possible fragments) by choosing from a library of automatically applicable rewrites. As a running example, and for the reported experimental results, we use TPC-App as a benchmark application.