A Design Pattern for Efficient Retrieval of Large Data Sets from Remote Data Sources

Authors:
Brad Long
Affiliations:
-
Venue:
On the Move to Meaningful Internet Systems, 2002 - DOA/CoopIS/ODBASE 2002 Confederated International Conferences DOA, CoopIS and ODBASE 2002
Year:
2002

Citing 7
Cited 2

Computer graphics: principles and practice (2nd ed.)

Computer graphics: principles and practice (2nd ed.)
Design patterns: elements of reusable object-oriented software

Design patterns: elements of reusable object-oriented software
Core J2EE Patterns: Best Practices and Design Strategies

Core J2EE Patterns: Best Practices and Design Strategies
Guest Editor's Introduction to the Special Issue on Web Technologies

IEEE Transactions on Knowledge and Data Engineering
Efficient Query Result Retrieval over the Web

ICPADS '00 Proceedings of the Seventh International Conference on Parallel and Distributed Systems
A Case Study in Testing Distributed Systems

DOA '01 Proceedings of the Third International Symposium on Distributed Objects and Applications
A Note on Distributed Computing

A Note on Distributed Computing

Distributed result set iterator: a design pattern for efficient retrieval of large result sets from remote data sources

Journal of Parallel and Distributed Computing
Where the speed matters... zero-response-time search engine for small collections

ICADL'05 Proceedings of the 8th international conference on Asian Digital Libraries: implementing strategies and sharing experiences

Quantified Score

Hi-index	0.00

Visualization

Abstract

Retrieving large amounts of information over wide area networks, including the Internet, is problematic due to issues arising from latency of response, lack of direct memory access to data serving resources, and fault tolerance. This paper describes a design pattern for solving the issues of handling results from queries that return large amounts of data. Typically these queries would be made by a client process across a wide area network (or Internet), with one or more middle-tiers, to a relational database residing on a remote server. The solution involves implementing a combination of data retrieval strategies, including the use of iterators for traversing data sets and providing an appropriate level of abstraction to the client, double-buffering of data subsets, multi-threaded data retrieval, and query slicing. This design has recently been implemented and incorporated into the framework of a commercial software product developed at Oracle Corporation.