Macroscope: end-point approach to networked application dependency discovery

Authors:
Lucian Popa;Byung-Gon Chun;Ion Stoica;Jaideep Chandrashekar;Nina Taft
Affiliations:
University of California, Berkeley, Berkeley, CA, USA;Intel Labs Berkeley, Berkeley, CA, USA;University of California, Berkeley, Berkeley, CA, USA;Intel Labs Berkeley, Berkeley, CA, USA;Intel Labs Berkeley, Berkeley, CA, USA
Venue:
Proceedings of the 5th international conference on Emerging networking experiments and technologies
Year:
2009

Citing 12
Cited 2

Performance debugging for distributed systems of black boxes

SOSP '03 Proceedings of the nineteenth ACM symposium on Operating systems principles
BLINC: multilevel traffic classification in the dark

Proceedings of the 2005 conference on Applications, technologies, architectures, and protocols for computer communications
WAP5: black-box performance debugging for wide-area systems

Proceedings of the 15th international conference on World Wide Web
Semi-automated discovery of application session structure

Proceedings of the 6th ACM SIGCOMM conference on Internet measurement
Path-based faliure and evolution management

NSDI'04 Proceedings of the 1st conference on Symposium on Networked Systems Design and Implementation - Volume 1
Using magpie for request extraction and workload modelling

OSDI'04 Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6
Pip: detecting the unexpected in distributed systems

NSDI'06 Proceedings of the 3rd conference on Networked Systems Design & Implementation - Volume 3
Towards highly reliable enterprise network services via inference of multi-level dependencies

Proceedings of the 2007 conference on Applications, technologies, architectures, and protocols for computer communications
What's going on?: learning communication rules in edge networks

Proceedings of the ACM SIGCOMM 2008 conference on Data communication
Exploiting Temporal Persistence to Detect Covert Botnet Channels

RAID '09 Proceedings of the 12th International Symposium on Recent Advances in Intrusion Detection
Automating network application dependency discovery: experiences, limitations, and new solutions

OSDI'08 Proceedings of the 8th USENIX conference on Operating systems design and implementation
X-trace: a pervasive network tracing framework

NSDI'07 Proceedings of the 4th USENIX conference on Networked systems design & implementation

Traffic causality graphs: profiling network applications through temporal and spatial causality of flows

Proceedings of the 23rd International Teletraffic Congress
On the accurate identification of network service dependencies in distributed systems

lisa'12 Proceedings of the 26th international conference on Large Installation System Administration: strategies, tools, and techniques

Quantified Score

Hi-index	0.00

Visualization

Abstract

Enterprise and data center networks consist of a large number of complex networked applications and services that depend upon each other. For this reason, they are difficult to manage and diagnose. In this paper we propose Macroscope, a new approach to extracting the dependencies of networked applications automatically by combining application process information with network level packet traces. We evaluate Macroscope on traces collected at 52 laptops within a large enterprise and show that Macroscope is accurate in finding the dependencies of networked applications. We also show that Macroscope requires less human involvement and is significantly more accurate than state of the art approaches that use only packet traces. Using our rich profiles of the application-service dependencies, we explore and uncover some interesting characteristics about this relationship. Finally, we discuss several usage scenarios that can benefit from Macroscope.