Scale and performance in a distributed file system
ACM Transactions on Computer Systems (TOCS)
Encapsulation of parallelism in the Volcano query processing system
SIGMOD '90 Proceedings of the 1990 ACM SIGMOD international conference on Management of data
The design and implementation of a log-structured file system
SOSP '91 Proceedings of the thirteenth ACM symposium on Operating systems principles
Serverless network file systems
SOSP '95 Proceedings of the fifteenth ACM symposium on Operating systems principles
Cluster-based scalable network services
Proceedings of the sixteenth ACM symposium on Operating systems principles
Frangipani: a scalable distributed file system
Proceedings of the sixteenth ACM symposium on Operating systems principles
Proceedings of the seventeenth ACM symposium on Operating systems principles
High-Performance Web Site Design Techniques
IEEE Internet Computing
A Case for NOW (Networks of Workstations)
IEEE Micro
The Gamma Database Machine Project
IEEE Transactions on Knowledge and Data Engineering
LH*LH: A scalable High Performance Data Structure for Switched Multicomputers
EDBT '96 Proceedings of the 5th International Conference on Extending Database Technology: Advances in Database Technology
RP*: A Family of Order Preserving Scalable Distributed Data Structures
VLDB '94 Proceedings of the 20th International Conference on Very Large Data Bases
Scalability Issues for High Performance Digital Libraries on the World Wide Web
ADL '96 Proceedings of the 3rd International Forum on Research and Technology Advances in Digital Libraries
Design alternatives for scalable Web server accelerators
ISPASS '00 Proceedings of the 2000 IEEE International Symposium on Performance Analysis of Systems and Software
USITS'99 Proceedings of the 2nd conference on USENIX Symposium on Internet Technologies and Systems - Volume 2
System design issues for internet middleware services: deductions from a large client trace
USITS'97 Proceedings of the USENIX Symposium on Internet Technologies and Systems on USENIX Symposium on Internet Technologies and Systems
A hierarchical internet object cache
ATEC '96 Proceedings of the 1996 annual conference on USENIX Annual Technical Conference
Flash: an efficient and portable web server
ATEC '99 Proceedings of the annual conference on USENIX Annual Technical Conference
A scalable and explicit event delivery mechanism for UNIX
ATEC '99 Proceedings of the annual conference on USENIX Annual Technical Conference
Network support for mobile multimedia using a self-adaptive distributed proxy
NOSSDAV '01 Proceedings of the 11th international workshop on Network and operating systems support for digital audio and video
SEDA: an architecture for well-conditioned, scalable internet services
SOSP '01 Proceedings of the eighteenth ACM symposium on Operating systems principles
Viceroy: a scalable and dynamic emulation of the butterfly
Proceedings of the twenty-first annual symposium on Principles of distributed computing
A composable framework for secure multi-modal access to internet services from Post-PC devices
Mobile Networks and Applications
A distributed persistent object store for scalable service
ACM SIGOPS Operating Systems Review
ACM Transactions on Computer Systems (TOCS)
Evolving RPC for active storage
Proceedings of the 10th international conference on Architectural support for programming languages and operating systems
A pluggable service-to-service communication mechanism for home multimedia networks
Proceedings of the tenth ACM international conference on Multimedia
Design and Implementation of a Distributed Crawler and Filtering Processor
NGITS '02 Proceedings of the 5th International Workshop on Next Generation Information Technologies and Systems
Ninja: A Framework for Network Services
ATEC '02 Proceedings of the General Track of the annual conference on USENIX Annual Technical Conference
Atomic Data Access in Distributed Hash Tables
IPTPS '01 Revised Papers from the First International Workshop on Peer-to-Peer Systems
Application specific data replication for edge services
WWW '03 Proceedings of the 12th international conference on World Wide Web
Distributed Computing with Load-Managed Active Storage
HPDC '02 Proceedings of the 11th IEEE International Symposium on High Performance Distributed Computing
A Decentralized, Adaptive Replica Location Mechanism
HPDC '02 Proceedings of the 11th IEEE International Symposium on High Performance Distributed Computing
Integrated resource management for cluster-based Internet services
ACM SIGOPS Operating Systems Review - OSDI '02: Proceedings of the 5th symposium on Operating systems design and implementation
Routing networks for distributed hash tables
Proceedings of the twenty-second annual symposium on Principles of distributed computing
Synchronization analysis for decentralizing composite Web services
Proceedings of the 2003 ACM symposium on Applied computing
Composing Web services on the Semantic Web
The VLDB Journal — The International Journal on Very Large Data Bases
Sharc: Managing CPU and Network Bandwidth in Shared Clusters
IEEE Transactions on Parallel and Distributed Systems
Improving availability with recursive microreboots: a soft-state system case study
Performance Evaluation - Dependable systems and networks-performance and dependability symposium (DSN-PDS) 2002: Selected papers
TODS: cluster object storage platform designed for scalable services
Future Generation Computer Systems - Special issue: Advanced services for clusters and internet computing
Decentralized orchestration of composite web services
Proceedings of the 13th international World Wide Web conference on Alternate track papers & posters
Intermediary infrastructures for the world wide web
Computer Networks: The International Journal of Computer and Telecommunications Networking
System support for pervasive applications
ACM Transactions on Computer Systems (TOCS)
Improving Availability and Performance with Application-Specific Data Replication
IEEE Transactions on Knowledge and Data Engineering
Cheap recovery: a key to self-managing state
ACM Transactions on Storage (TOS)
Quantifying and Improving the Availability of High-Performance Cluster-Based Internet Services
Proceedings of the 2003 ACM/IEEE conference on Supercomputing
Quantifying the Performability of Cluster-Based Services
IEEE Transactions on Parallel and Distributed Systems
Integrated resource management for cluster-based internet services
OSDI '02 Proceedings of the 5th symposium on Operating systems design and implementationCopyright restrictions prevent ACM from being able to make the PDFs for this conference available for downloading
LH*RS---a highly-available scalable distributed data structure
ACM Transactions on Database Systems (TODS)
Fault-scalable Byzantine fault-tolerant services
Proceedings of the twentieth ACM symposium on Operating systems principles
Autonomous recovery in componentized Internet applications
Cluster Computing
Overload management as a fundamental service design primitive
EW 10 Proceedings of the 10th workshop on ACM SIGOPS European workshop
Selective early request termination for busy internet services
Proceedings of the 15th international conference on World Wide Web
Customized Delivery of E-Government Web Services
IEEE Intelligent Systems
Efficient Updates in Highly Available Distributed Random Access Memory
ICPADS '06 Proceedings of the 12th International Conference on Parallel and Distributed Systems - Volume 1
Survey of research towards robust peer-to-peer networks: search methods
Computer Networks: The International Journal of Computer and Telecommunications Networking
HOTOS'03 Proceedings of the 9th conference on Hot Topics in Operating Systems - Volume 9
The case for a session state storage layer
HOTOS'03 Proceedings of the 9th conference on Hot Topics in Operating Systems - Volume 9
Session state: beyond soft state
NSDI'04 Proceedings of the 1st conference on Symposium on Networked Systems Design and Implementation - Volume 1
Explicit control a batch-aware distributed file system
NSDI'04 Proceedings of the 1st conference on Symposium on Networked Systems Design and Implementation - Volume 1
Performance modeling and system management for multi-component online services
NSDI'05 Proceedings of the 2nd conference on Symposium on Networked Systems Design & Implementation - Volume 2
Boxwood: abstractions as the foundation for storage infrastructure
OSDI'04 Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6
Neptune: scalable replication management and programming support for cluster-based network services
USITS'01 Proceedings of the 3rd conference on USENIX Symposium on Internet Technologies and Systems - Volume 3
Using fault injection and modeling to evaluate the performability of cluster-based services
USITS'03 Proceedings of the 4th conference on USENIX Symposium on Internet Technologies and Systems - Volume 4
Adaptive overload control for busy internet servers
USITS'03 Proceedings of the 4th conference on USENIX Symposium on Internet Technologies and Systems - Volume 4
Model-based resource provisioning in a web service utility
USITS'03 Proceedings of the 4th conference on USENIX Symposium on Internet Technologies and Systems - Volume 4
SkipNet: a scalable overlay network with practical locality properties
USITS'03 Proceedings of the 4th conference on USENIX Symposium on Internet Technologies and Systems - Volume 4
Symphony: distributed hashing in a small world
USITS'03 Proceedings of the 4th conference on USENIX Symposium on Internet Technologies and Systems - Volume 4
NPS: a non-interfering deployable web perfectching system
USITS'03 Proceedings of the 4th conference on USENIX Symposium on Internet Technologies and Systems - Volume 4
Anypoint: extensible transport switching on the edge
USITS'03 Proceedings of the 4th conference on USENIX Symposium on Internet Technologies and Systems - Volume 4
CCALoop: scalable design of a distributed component framework
Proceedings of the 16th international symposium on High performance distributed computing
Experience distributing objects in an SMMP OS
ACM Transactions on Computer Systems (TOCS)
Data management in human networks
Proceedings of the second ACM workshop on Challenged networks
Sinfonia: a new paradigm for building scalable distributed systems
Proceedings of twenty-first ACM SIGOPS symposium on Operating systems principles
Stasis: flexible transactional storage
OSDI '06 Proceedings of the 7th symposium on Operating systems design and implementation
Replication degree customization for high availability
Proceedings of the 3rd ACM SIGOPS/EuroSys European Conference on Computer Systems 2008
Service-oriented data denormalization for scalable web applications
Proceedings of the 17th international conference on World Wide Web
OLTP through the looking glass, and what we found there
Proceedings of the 2008 ACM SIGMOD international conference on Management of data
Handling flash crowds from your garage
ATC'08 USENIX 2008 Annual Technical Conference on Annual Technical Conference
LH*RSP2P: a scalable distributed data structure for P2P environment
NOTERE '08 Proceedings of the 8th international conference on New technologies in distributed systems
A practical scalable distributed B-tree
Proceedings of the VLDB Endowment
Architecture of a Database System
Foundations and Trends in Databases
Sinfonia: A new paradigm for building scalable distributed systems
ACM Transactions on Computer Systems (TOCS)
FAWN: a fast array of wimpy nodes
Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles
Proceedings of the 10th ACM/IFIP/USENIX International Conference on Middleware
VECPAR'02 Proceedings of the 5th international conference on High performance computing for computational science
Storing and accessing live mashup content in the cloud
ACM SIGOPS Operating Systems Review
Making cloud intermediate data fault-tolerant
Proceedings of the 1st ACM symposium on Cloud computing
LH*RSP2P: a fast and high churn resistant scalable distributed data structure for P2P systems
International Journal of Internet Technology and Secured Transactions
Middleware'09 Proceedings of the ACM/IFIP/USENIX 10th international conference on Middleware
Centrifuge: integrated lease management and partitioning for cloud services
NSDI'10 Proceedings of the 7th USENIX conference on Networked systems design and implementation
Piccolo: building fast, distributed programs with partitioned tables
OSDI'10 Proceedings of the 9th USENIX conference on Operating systems design and implementation
Scale and concurrency of GIGA+: file system directories with millions of files
FAST'11 Proceedings of the 9th USENIX conference on File and stroage technologies
Increasing performance in byzantine fault-tolerant systems with on-demand replica consistency
Proceedings of the sixth conference on Computer systems
CassMail: a scalable, highly-available, and rapidly-prototyped e-mail service
Proceedings of the 11th IFIP WG 6.1 international conference on Distributed applications and interoperable systems
Small cache, big effect: provable load balancing for randomly partitioned cluster services
Proceedings of the 2nd ACM Symposium on Cloud Computing
SDDSfL vs. local disk-a comparative study for Linux
Annales UMCS, Informatica
Domus – an architecture for cluster-oriented distributed hash tables
PPAM'05 Proceedings of the 6th international conference on Parallel Processing and Applied Mathematics
When multi-hop peer-to-peer lookup matters
IPTPS'04 Proceedings of the Third international conference on Peer-to-Peer Systems
All your database are belong to us
Communications of the ACM
All Your Database Are Belong to Us
Queue - Debugging
Surviving congestion in geo-distributed storage systems
USENIX ATC'12 Proceedings of the 2012 USENIX conference on Annual Technical Conference
Authenticated and persistent skip graph: a data structure for cloud based data-centric applications
Proceedings of the International Conference on Advances in Computing, Communications and Informatics
Controlling incoming connections using certificates and distributed hash tables
NEW2AN'07 Proceedings of the 7th international conference on Next Generation Teletraffic and Wired/Wireless Advanced Networking
nsdi'13 Proceedings of the 10th USENIX conference on Networked Systems Design and Implementation
Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles
ACM SIGOPS 24th Symposium on Operating Systems Principles
Tango: distributed data structures over a shared log
Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles
TAO: Facebook's distributed data store for the social graph
USENIX ATC'13 Proceedings of the 2013 USENIX conference on Annual Technical Conference
Hi-index | 0.02 |
This paper presents a new persistent data management layer designed to simplify cluster-based Internet service construction. This self-managing layer, called a distributed data structure (DDS), presents a conventional single-site data structure interface to service authors, but partitions and replicates the data across a cluster. We have designed and implemented a distributed hash table DDS that has properties necessary for Internet services (incremental scaling of throughput and data capacity, fault tolerance and high availability, high concurrency, consistency, and durability). The hash table uses two-phase commits to present a coherent view of its data across all cluster nodes, allowing any node to service any task. We show that the distributed hash table simplifies Internet service construction by decoupling service-specific logic from the complexities of persistent, consistent state management, and by allowing services to inherit the necessary service properties from the DDS rather than having to implement the properties themselves. We have scaled the hash table to a 128 node cluster, 1 terabyte of storage, and an in-core read throughput of 61,432 operations/s and write throughput of 13,582 operations/s.