Lower bounds on communication complexity in distributed computer networks
Journal of the ACM (JACM)
On the fractional covering number of hypergraphs
SIAM Journal on Discrete Mathematics
Parallel bottom-up processing of Datalog queries
Journal of Logic Programming
Communication complexity
The space complexity of approximating the frequency moments
Journal of Computer and System Sciences
Constraint solving via fractional edge covers
SODA '06 Proceedings of the seventeenth annual ACM-SIAM symposium on Discrete algorithm
MapReduce: simplified data processing on large clusters
OSDI'04 Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6
FOCS '07 Proceedings of the 48th Annual IEEE Symposium on Foundations of Computer Science
Pig latin: a not-so-foreign language for data processing
Proceedings of the 2008 ACM SIGMOD international conference on Management of data
On determinism versus non-determinism and related problems
SFCS '83 Proceedings of the 24th Annual Symposium on Foundations of Computer Science
Size Bounds and Query Plans for Relational Joins
FOCS '08 Proceedings of the 2008 49th Annual IEEE Symposium on Foundations of Computer Science
Revisiting the Direct Sum Theorem and Space Lower Bounds in Random Order Streams
ICALP '09 Proceedings of the 36th International Colloquium on Automata, Languages and Programming: Part I
Hive: a warehousing solution over a map-reduce framework
Proceedings of the VLDB Endowment
Optimizing joins in a map-reduce environment
Proceedings of the 13th International Conference on Extending Database Technology
On distributing symmetric streaming computations
ACM Transactions on Algorithms (TALG)
A model of computation for MapReduce
SODA '10 Proceedings of the twenty-first annual ACM-SIAM symposium on Discrete Algorithms
Dremel: interactive analysis of web-scale datasets
Proceedings of the VLDB Endowment
Counting triangles and the curse of the last reducer
Proceedings of the 20th international conference on World wide web
Parallel evaluation of conjunctive queries
Proceedings of the thirtieth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
What next?: a half-dozen data management research goals for big data and the cloud
PODS '12 Proceedings of the 31st symposium on Principles of Database Systems
Worst-case optimal join algorithms: [extended abstract]
PODS '12 Proceedings of the 31st symposium on Principles of Database Systems
Designing good MapReduce algorithms
XRDS: Crossroads, The ACM Magazine for Students - Big Data
Big data begets big database theory
BNCOD'13 Proceedings of the 29th British National conference on Big Data
| Hi-index | 0.00 |
We consider the problem of computing a relational query q on a large input database of size n, using a large number p of servers. The computation is performed in rounds, and each server can receive only O(n/p1-ε) bits of data, where ε ∈[0,1] is a parameter that controls replication. We examine how many global communication steps are needed to compute q. We establish both lower and upper bounds, in two settings. For a single round of communication, we give lower bounds in the strongest possible model, where arbitrary bits may be exchanged; we show that any algorithm requires ε ≥ 1--1/τ*, where τ* is the fractional vertex cover of the hypergraph of q. We also give an algorithm that matches the lower bound for a specific class of databases. For multiple rounds of communication, we present lower bounds in a model where routing decisions for a tuple are tuple-based. We show that for the class of tree-like queries there exists a tradeoff between the number of rounds and the space exponent ε. The lower bounds for multiple rounds are the first of their kind. Our results also imply that transitive closure cannot be computed in O(1) rounds of communication.