Decentralized erasure codes for distributed networked storage

Authors:
Alexandros G. Dimakis;Vinod Prabhakaran;Kannan Ramchandran
Affiliations:
Department of Electrical Engineering and Computer Science, University of California, Berkeley, CA;Department of Electrical Engineering and Computer Science, University of California, Berkeley, CA;Department of Electrical Engineering and Computer Science, University of California, Berkeley, CA
Venue:
IEEE/ACM Transactions on Networking (TON) - Special issue on networking and information theory
Year:
2006

Citing 16
Cited 23

Solving sparse linear equations over finite fields

IEEE Transactions on Information Theory
Efficient dispersal of information for security, load balancing, and fault tolerance

Journal of the ACM (JACM)
Randomized algorithms

Randomized algorithms
OceanStore: an architecture for global-scale persistent storage

ASPLOS IX Proceedings of the ninth international conference on Architectural support for programming languages and operating systems
LT Codes

FOCS '02 Proceedings of the 43rd Symposium on Foundations of Computer Science
An algebraic approach to network coding

IEEE/ACM Transactions on Networking (TON)
Power-efficient sensor placement and transmission structure for data gathering under distortion constraints

Proceedings of the 3rd international symposium on Information processing in sensor networks
A Practical Analysis of Low-Density Parity-Check Erasure Codes for Wide-Area Storage Applications

DSN '04 Proceedings of the 2004 International Conference on Dependable Systems and Networks
Network coding: an instant primer

ACM SIGCOMM Computer Communication Review
Ubiquitous access to distributed data in large-scale sensor networks through decentralized erasure codes

IPSN '05 Proceedings of the 4th international symposium on Information processing in sensor networks
Network information flow

IEEE Transactions on Information Theory
Linear network coding

IEEE Transactions on Information Theory
Distributed source coding using syndromes (DISCUS): design and construction

IEEE Transactions on Information Theory
Networked Slepian-Wolf: theory, algorithms, and scaling laws

IEEE Transactions on Information Theory
Raptor codes

IEEE Transactions on Information Theory
A Random Linear Network Coding Approach to Multicast

IEEE Transactions on Information Theory

Maximizing the data utility of a data archiving & querying system through joint coding and scheduling

Proceedings of the 6th international conference on Information processing in sensor networks
Network coding applications

Foundations and Trends® in Networking
Fountain Codes Based Distributed Storage Algorithms for Large-Scale Wireless Sensor Networks

IPSN '08 Proceedings of the 7th international conference on Information processing in sensor networks
Scalable Ubiquitous Data Access in Clustered Sensor Networks

SSDBM '08 Proceedings of the 20th international conference on Scientific and Statistical Database Management
Doped fountain coding for minimum delay data collection in circular networks

IEEE Journal on Selected Areas in Communications - Special issue on network coding for wireless communication networks
Data extraction from wireless sensor networks using distributed fountain codes

IEEE Transactions on Communications
Decentralized coding algorithms for distributed storage in wireless sensor networks

IEEE Journal on Selected Areas in Communications
Resilient coding algorithms for sensor network data persistence

EWSN'08 Proceedings of the 5th European conference on Wireless sensor networks
Pivoting algorithms for maximum likelihood decoding of LDPC codes over erasure channels

GLOBECOM'09 Proceedings of the 28th IEEE conference on Global telecommunications
On reliable transmission by adaptive network coding in wireless sensor networks

ICC'09 Proceedings of the 2009 IEEE international conference on Communications
Survivable distributed storage with progressive decoding

INFOCOM'10 Proceedings of the 29th conference on Information communications
Designing and embedding reliable virtual infrastructures

Proceedings of the second ACM SIGCOMM workshop on Virtualized infrastructure systems and architectures
Data preservation under spatial failures in sensor networks

Proceedings of the eleventh ACM international symposium on Mobile ad hoc networking and computing
Network coding for distributed storage systems

IEEE Transactions on Information Theory
Cost-bandwidth tradeoff in distributed storage systems

Computer Communications
Maximizing growth codes utility in large-scale wireless sensor networks

Euro-Par'10 Proceedings of the 16th international Euro-Par conference on Parallel processing: Part II
Application of wireless sensor networks in critical infrastructure protection: challenges and design options

IEEE Wireless Communications
Maximal recovery network coding under topology constraint

IEEE Transactions on Information Theory
Epidemic data survivability in unattended wireless sensor networks

Proceedings of the fourth ACM conference on Wireless network security
Model driven development for data-centric sensor network applications

Proceedings of the 9th International Conference on Advances in Mobile Computing and Multimedia
A novel network coding scheme for data collection in WSNs with a mobile BS

DNIS'11 Proceedings of the 7th international conference on Databases in Networked Information Systems
Distributed data collection and storage algorithms for collaborative learning vision sensor devices with applications to pilgrimage

International Journal of Sensor Networks
Dynamic Data-Centric Storage for long-term storage in Wireless Sensor and Actor Networks

Wireless Networks

Quantified Score

Hi-index	0.12

Visualization

Abstract

In this correspondence, we consider the problem of constructing an erasure code for storage over a network when the data sources are distributed. Specifically, we assume that there are n storage nodes with limited memory and k n sources generating the data. We want a data collector, who can appear anywhere in the network, to query any k storage nodes and be able to retrieve the data. We introduce decentralized erasure codes, which are linear codes with a specific randomized structure inspired by network coding on random bipartite graphs. We show that decentralized erasure codes are optimally sparse, and lead to reduced communication, storage and computation cost over random linear coding.