SIAM Journal on Computing
Randomized algorithms
The space complexity of approximating the frequency moments
Journal of Computer and System Sciences
External memory algorithms
On showing lower bounds for external-memory computational geometry problems
External memory algorithms
External memory algorithms and data structures: dealing with massive data
ACM Computing Surveys (CSUR)
Models and issues in data stream systems
Proceedings of the twenty-first ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
On the Streaming Model Augmented with a Sorting Primitive
FOCS '04 Proceedings of the 45th Annual IEEE Symposium on Foundations of Computer Science
On the memory requirements of XPath evaluation over XML streams
PODS '04 Proceedings of the twenty-third ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
Buffering in query evaluation over XML streams
Proceedings of the twenty-fourth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
Lower bounds for sorting with few random accesses to external memory
Proceedings of the twenty-fourth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
Randomized computations on large data sets: tight lower bounds
Proceedings of the twenty-fifth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
Data streams: algorithms and applications
Foundations and Trends® in Theoretical Computer Science
Tight lower bounds for query processing on streaming and external memory data
Theoretical Computer Science
Lower bounds for randomized read/write stream algorithms
Proceedings of the thirty-ninth annual ACM symposium on Theory of computing
Machine models and lower bounds for query processing
Proceedings of the twenty-sixth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
Theoretical Computer Science
On the Value of Multiple Read/Write Streams for Approximating Frequency Moments
FOCS '08 Proceedings of the 2008 49th Annual IEEE Symposium on Foundations of Computer Science
Algorithms for memory hierarchies: advanced lectures
Algorithms for memory hierarchies: advanced lectures
The complexity of querying external memory and streaming data
FCT'05 Proceedings of the 15th international conference on Fundamentals of Computation Theory
Machine models for query processing
ACM SIGMOD Record
The Value of Multiple Read/Write Streams for Approximating Frequency Moments
ACM Transactions on Computation Theory (TOCT)
Grammar-based compression in a streaming model
LATA'10 Proceedings of the 4th international conference on Language and Automata Theory and Applications
Validating XML documents in the streaming model with external memory
Proceedings of the 15th International Conference on Database Theory
Strategy machines and their complexity
MFCS'12 Proceedings of the 37th international conference on Mathematical Foundations of Computer Science
Validating XML documents in the streaming model with external memory
ACM Transactions on Database Systems (TODS) - Invited papers issue
| Hi-index | 0.00 |
We consider a scenario where we want to query a large dataset that is stored in external memory and does not fit into main memory. The most constrained resources in such a situation are the size of the main memory and the number of random accesses to external memory. We note that sequentially streaming data from external memory through main memory is much less prohibitive. We propose an abstract model of this scenario in which we restrict the size of the main memory and the number of random accesses to external memory, but admit arbitrary sequential access. A distinguishing feature of our model is that it allows the usage of unlimited external memory for storing intermediate results, such as several hard disks that can be accessed in parallel. In this model, we prove lower bounds for the problem of sorting a sequence of strings (or numbers), the problem of deciding whether two given sets of strings are equal, and two closely related decision problems. Intuitively, our results say that there is no algorithm for the problems that uses internal memory space bounded by N1−ϵ and at most o(log N) random accesses to external memory, but unlimited “streaming access”, both for writing to and reading from external memory. (Here, N denotes the size of the input and ϵ is an arbitrary constant greater than 0.) We even permit randomized algorithms with one-sided bounded error. We also consider the problem of evaluating database queries and prove similar lower bounds for evaluating relational algebra queries against relational databases and XQuery and XPath queries against XML-databases.