Handling very large numbers of messages in distributed hash tables

Authors:
Fabius Klemm;Jean-Yves Le Boudec;Dejan Kostic;Karl Aberer
Affiliations:
School of Computer and Communication Sciences, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland;School of Computer and Communication Sciences, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland;School of Computer and Communication Sciences, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland;School of Computer and Communication Sciences, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
Venue:
COMSNETS'09 Proceedings of the First international conference on COMmunication Systems And NETworks
Year:
2009

Citing 11
Cited 0

Adaptive Deadlock- and Livelock-Free Routing in the Hypercube Network

IEEE Transactions on Parallel and Distributed Systems
A universal proof technique for deadlock-free routing in interconnection networks

Proceedings of the seventh annual ACM symposium on Parallel algorithms and architectures
Chord: A scalable peer-to-peer lookup service for internet applications

Proceedings of the 2001 conference on Applications, technologies, architectures, and protocols for computer communications
A scalable content-addressable network

Proceedings of the 2001 conference on Applications, technologies, architectures, and protocols for computer communications
Congestion control for high bandwidth-delay product networks

Proceedings of the 2002 conference on Applications, technologies, architectures, and protocols for computer communications
Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems

Middleware '01 Proceedings of the IFIP/ACM International Conference on Distributed Systems Platforms Heidelberg
Peer-to-peer information retrieval using self-organizing semantic overlay networks

Proceedings of the 2003 conference on Applications, technologies, architectures, and protocols for computer communications
Congestion Control for Distributed Hash Tables

NCA '06 Proceedings of the Fifth IEEE International Symposium on Network Computing and Applications
Designing a DHT for low latency and high throughput

NSDI'04 Proceedings of the 1st conference on Symposium on Networked Systems Design and Implementation - Volume 1
Hybrid global-local indexing for effcient peer-to-peer information retrieval

NSDI'04 Proceedings of the 1st conference on Symposium on Networked Systems Design and Implementation - Volume 1
Tapestry: a resilient global-scale overlay for service deployment

IEEE Journal on Selected Areas in Communications

Quantified Score

Hi-index	0.00

Visualization

Abstract

The principal service of Distributed Hash Tables (DHTs) is route(id, data), which sends data to a peer responsible for id, using typically O(log(# of peers)) overlay hops. Certain applications like peer-to-peer information retrieval generate billions of small messages that are concurrently inserted into a DHT. These applications can generate messages faster than the DHT can process them. To support such demanding applications, a DHT needs a congestion control mechanism to efficiently handle high loads of messages. this paper we provide an extended study on congestion control for DHTs: we present a theoretical analysis that demonstrates that congestion control for DHTs is absolutely necessary for applications that provide elastic traffic. We then present a new congestion control algorithm for DHTs. We provide extensive live evaluations in a ModelNet cluster and the PlanetLab test bed, which show that our algorithm is nearly loss-free, fair, and provides low lookup times and high throughput under cross-load.