Algorithms for scalable synchronization on shared-memory multiprocessors
ACM Transactions on Computer Systems (TOCS)
A study of integrated prefetching and caching strategies
Proceedings of the 1995 ACM SIGMETRICS joint international conference on Measurement and modeling of computer systems
A critique of ANSI SQL isolation levels
SIGMOD '95 Proceedings of the 1995 ACM SIGMOD international conference on Management of data
ACM Transactions on Computer Systems (TOCS)
File server scaling with network-attached secure disks
SIGMETRICS '97 Proceedings of the 1997 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
Prefetching using Markov predictors
Proceedings of the 24th annual international symposium on Computer architecture
Profetching and memory system behavior of the SPEC95 benchmark suite
IBM Journal of Research and Development - Special issue: performance analysis and its impact on design
Implementing cooperative prefetching and caching in a globally-managed memory system
SIGMETRICS '98/PERFORMANCE '98 Proceedings of the 1998 ACM SIGMETRICS joint international conference on Measurement and modeling of computer systems
Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility
SOSP '01 Proceedings of the eighteenth ACM symposium on Operating systems principles
A survey of web caching schemes for the Internet
ACM SIGCOMM Computer Communication Review
Guided region prefetching: a cooperative hardware/software approach
Proceedings of the 30th annual international symposium on Computer architecture
Dynamic Metadata Management for Petabyte-Scale File Systems
Proceedings of the 2004 ACM/IEEE conference on Supercomputing
Data Cache Prefetching Using a Global History Buffer
HPCA '04 Proceedings of the 10th International Symposium on High Performance Computer Architecture
Aggressive prefetching: an idea whose time has come
HOTOS'05 Proceedings of the 10th conference on Hot Topics in Operating Systems - Volume 10
Shark: scaling file servers via cooperative caching
NSDI'05 Proceedings of the 2nd conference on Symposium on Networked Systems Design & Implementation - Volume 2
Reducing file system latency using a predictive approach
USTC'94 Proceedings of the USENIX Summer 1994 Technical Conference on USENIX Summer 1994 Technical Conference - Volume 1
Software prefetching and caching for translation lookaside buffers
OSDI '94 Proceedings of the 1st USENIX conference on Operating Systems Design and Implementation
A comparison of file system workloads
ATEC '00 Proceedings of the annual conference on USENIX Annual Technical Conference
PVFS: a parallel file system for linux clusters
ALS'00 Proceedings of the 4th annual Linux Showcase & Conference - Volume 4
An analytical approach to file prefetching
ATEC '97 Proceedings of the annual conference on USENIX Annual Technical Conference
Why does file system prefetching work?
ATEC '99 Proceedings of the annual conference on USENIX Annual Technical Conference
Bigtable: A Distributed Storage System for Structured Data
ACM Transactions on Computer Systems (TOCS)
AMP: An Affinity-Based Metadata Prefetching Scheme in Large-Scale Distributed Storage Systems
CCGRID '08 Proceedings of the 2008 Eighth IEEE International Symposium on Cluster Computing and the Grid
Single global lock semantics in a weakly atomic STM
ACM SIGPLAN Notices
Communications of the ACM - Rural engineering development
Thrashing: its causes and prevention
AFIPS '68 (Fall, part I) Proceedings of the December 9-11, 1968, fall joint computer conference, part I
A Novel Weighted-Graph-Based Grouping Algorithm for Metadata Prefetching
IEEE Transactions on Computers
ZooKeeper: wait-free coordination for internet-scale systems
USENIXATC'10 Proceedings of the 2010 USENIX conference on USENIX annual technical conference
Large-scale incremental processing using distributed transactions and notifications
OSDI'10 Proceedings of the 9th USENIX conference on Operating systems design and implementation
Many-Thread Aware Prefetching Mechanisms for GPGPU Applications
MICRO '43 Proceedings of the 2010 43rd Annual IEEE/ACM International Symposium on Microarchitecture
Windows Azure Storage: a highly available cloud storage service with strong consistency
SOSP '11 Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles
Software as a service for data scientists
Communications of the ACM
Concurrent tries with efficient non-blocking snapshots
Proceedings of the 17th ACM SIGPLAN symposium on Principles and Practice of Parallel Programming
Thread reinforcer: Dynamically determining number of threads via OS level monitoring
IISWC '11 Proceedings of the 2011 IEEE International Symposium on Workload Characterization
StorkCloud: data transfer scheduling and optimization as a service
Proceedings of the 4th ACM workshop on Scientific cloud computing
| Hi-index | 0.00 |
With the overwhelming emergence of data-intensive applications in the Cloud, the wide-area transfer of metadata and other descriptive information about remote data is critically important for searching, indexing, and enumerating remote file system hierarchies, as well as for purposes of data transfer estimation and reservation. In this paper, we present a highly efficient network-aware caching and prefetching mechanism tailored to reduce metadata access latency and improve responsiveness in wide-area data transfers. To improve the maximum requests per second (RPS) handled by the system, we designed and implemented a network-aware prefetching service using dynamically provisioned parallel TCP streams. To improve the performance of accessing local metadata, we designed and implemented a non-blocking concurrent in-memory cache to handle unexpected bursts of requests. We have implemented the proposed mechanisms in the Directory Listing Service (DLS) system---a Cloud-hosted metadata retrieval, caching, and prefetching system, and have evaluated its performance on Amazon EC2 and XSEDE.