Data Infrastructure at LinkedIn

Authors:
Aditya Auradkar;Chavdar Botev;Shirshanka Das;Dave De Maagd;Alex Feinberg;Phanindra Ganti;Lei Gao;Bhaskar Ghosh;Kishore Gopalakrishna;Brendan Harris;Joel Koshy;Kevin Krawez;Jay Kreps;Shi Lu;Sunil Nagaraj;Neha Narkhede;Sasha Pachev;Igor Perisic;Lin Qiao;Tom Quiggle;Jun Rao;Bob Schulman;Abraham Sebastian;Oliver Seeliger;Adam Silberstein;BBoris Shkolnik;Chinmay Soman;Roshan Sumbaly;Kapil Surlaker;Sajid Topiwala;Cuong Tran;Balaji Varadarajan;Jemiah Westerman;Zach White;David Zhang;Jason Zhang
Affiliations:
-;-;-;-;-;-;-;-;-;-;-;-;-;-;-;-;-;-;-;-;-;-;-;-;-;-;-;-;-;-;-;-;-;-;-;-
Venue:
ICDE '12 Proceedings of the 2012 IEEE 28th International Conference on Data Engineering
Year:
2012

Citing 0
Cited 2

Multi-scale dynamics in a massive online social network

Proceedings of the 2012 ACM conference on Internet measurement conference
On brewing fresh espresso: LinkedIn's distributed data serving platform

Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data

Quantified Score

Hi-index	0.00

Visualization

Abstract

Linked In is among the largest social networking sites in the world. As the company has grown, our core data sets and request processing requirements have grown as well. In this paper, we describe a few selected data infrastructure projects at Linked In that have helped us accommodate this increasing scale. Most of those projects build on existing open source projects and are themselves available as open source. The projects covered in this paper include: (1) Voldemort: a scalable and fault tolerant key-value store, (2) Data bus: a framework for delivering database changes to downstream applications, (3) Espresso: a distributed data store that supports flexible schemas and secondary indexing, (4) Kafka: a scalable and efficient messaging system for collecting various user activity events and log data.