Maelstrom: transparent error correction for lambda networks

Authors:
Mahesh Balakrishnan;Tudor Marian;Ken Birman;Hakim Weatherspoon;Einar Vollset
Affiliations:
Cornell University, Ithaca, NY;Cornell University, Ithaca, NY;Cornell University, Ithaca, NY;Cornell University, Ithaca, NY;Cornell University, Ithaca, NY
Venue:
NSDI'08 Proceedings of the 5th USENIX Symposium on Networked Systems Design and Implementation
Year:
2008

Citing 22
Cited 6

Multicast routing in datagram internetworks and extended LANs

ACM Transactions on Computer Systems (TOCS)
The performance of TCP/IP for networks with high bandwidth-delay products and random loss

IEEE/ACM Transactions on Networking (TON)
Parity-based loss recovery for reliable multicast transmission

SIGCOMM '97 Proceedings of the ACM SIGCOMM '97 conference on Applications, technologies, architectures, and protocols for computer communication
Effective erasure codes for reliable computer communication protocols

ACM SIGCOMM Computer Communication Review
Modeling TCP throughput: a simple model and its empirical validation

Proceedings of the ACM SIGCOMM '98 conference on Applications, technologies, architectures, and protocols for computer communication
PSockets: the case for application-level network striping for data intensive applications using high speed wide area networks

Proceedings of the 2000 ACM/IEEE conference on Supercomputing
Reed-Solomon Codes and Their Applications

Reed-Solomon Codes and Their Applications
Congestion control for high bandwidth-delay product networks

Proceedings of the 2002 conference on Applications, technologies, architectures, and protocols for computer communications
LT Codes

FOCS '02 Proceedings of the 43rd Symposium on Foundations of Computer Science
The End-to-End Performance Effects of Parallel TCP Sockets on a Lossy Wide-Area Network

IPDPS '02 Proceedings of the 16th International Parallel and Distributed Processing Symposium
The case for packet level FEC

PfHSN '96 Proceedings of the TC6 WG6.1/6.4 Fifth International Workshop on Protocols for High-Speed Networks V
SaTPEP: A TCP Performance Enhancing Proxy for Satellite Links

NETWORKING '02 Proceedings of the Second International IFIP-TC6 Networking Conference on Networking Technologies, Services, and Protocols; Performance of Computer and Communication Networks; and Mobile and Wireless Communications
The effects of systemic packet loss on aggregate TCP flows

Proceedings of the 2002 ACM/IEEE conference on Supercomputing
Reliable Blast UDP: Predictable High Performance Bulk Data Transfer

CLUSTER '02 Proceedings of the IEEE International Conference on Cluster Computing
AFEC: An Adaptive Forward Error Correction Protocol for End-to-End Transport of Real-Time Traffic

IC3N '98 Proceedings of the International Conference on Computer Communications and Networks
An integrated experimental environment for distributed systems and networks

OSDI '02 Proceedings of the 5th symposium on Operating systems design and implementationCopyright restrictions prevent ACM from being able to make the PDFs for this conference available for downloading
A Cross Layer Study of Packet Loss in All-Optical Networks

AICT-ICIW '06 Proceedings of the Advanced Int'l Conference on Telecommunications and Int'l Conference on Internet and Web Applications and Services
Growth codes: maximizing sensor network data persistence

Proceedings of the 2006 conference on Applications, technologies, architectures, and protocols for computer communications
OverQos: an overlay based architecture for enhancing internet Qos

NSDI'04 Proceedings of the 1st conference on Symposium on Networked Systems Design and Implementation - Volume 1
End-to-End Performance of 10-Gigabit Ethernet on Commodity Systems

IEEE Micro
Ricochet: lateral error correction for time-critical multicast

NSDI'07 Proceedings of the 4th USENIX conference on Networked systems design & implementation
Packet-level traffic measurements from the Sprint IP backbone

IEEE Network: The Magazine of Global Internetworking

Smoke and mirrors: reflecting files at a geographically remote location without loss of performance

FAST '09 Proccedings of the 7th conference on File and storage technologies
Pangolin: speeding up concurrent messaging for cloud-based social gaming

Proceedings of the Seventh COnference on emerging Networking EXperiments and Technologies
RPT: re-architecting loss protection for content-aware networks

NSDI'12 Proceedings of the 9th USENIX conference on Networked Systems Design and Implementation
Minimizing the Data Transfer Time Using Multicore End-System Aware Flow Bifurcation

CCGRID '12 Proceedings of the 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012)
High performance vehicular connectivity with opportunistic erasure coding

USENIX ATC'12 Proceedings of the 2012 USENIX conference on Annual Technical Conference
NetSlices: scalable multi-core packet processing in user-space

Proceedings of the eighth ACM/IEEE symposium on Architectures for networking and communications systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

The global network of datacenters is emerging as an important distributed systems paradigm--commodity clusters running high-performance applications, connected by high-speed 'lambda' networks across hundreds of milliseconds of network latency. Packet loss on long-haul networks can cripple application performance --a loss rate of 0.1% is sufficient to reduce TCP/IP throughput by an order of magnitude on a 1 Gbps link with 50ms latency. Maelstrom is an edge appliance that masks packet loss transparently and quickly from inter-cluster protocols, aggregating traffic for high-speed encoding and using a new Forward Error Correction scheme to handle bursty loss.