A new data streaming method for locating hosts with large connection degree

  • Authors:
  • Xiaohong Guan;Pinghui Wang;Tao Qin

  • Affiliations:
  • MOE Key Lab for Int. Networks and Network Security, State Key Lab for Manufacturing System and Xian Jiaotong Univ., Xi'an, P.R. China and Dept. of Automation, Tsinghua National Lab for Inf. Scienc ...;MOE Key Lab for Intelligent Networks and Network Security, State Key Lab for Manufacturing System, Xian Jiaotong University, Xi'an, P.R. China;MOE Key Lab for Intelligent Networks and Network Security, State Key Lab for Manufacturing System, Xian Jiaotong University, Xi'an, P.R. China

  • Venue:
  • GLOBECOM'09 Proceedings of the 28th IEEE conference on Global telecommunications
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

Locating hosts with large connection degree is very important for monitoring anomalous network traffics. The in-degree (out-degree), defined as the number of distinct sources (destinations) that a network host is connected with (connects) during a given time interval. Due to massive amount of data in high speed network traffics and limit on processing capability, it is difficult to accurately locate hosts with large connection degree over high speed links on line. In this paper we present a new data streaming method for locating hosts with large connection degree based on the reversible connection degree sketch to monitor anomalous network traffics. The required memory space is small and constant, and more importantly the update/query complexity would not depend on the amount of data. The hash functions for data sketch are designed based on the remainder characteristics of the number theory so that in-degree/out-degree associated with a given host can be accurately estimated. Although the connection degree sketch does not preserve any host address information, we can analytically reconstruct the host addresses associated with large in-degree/out-degree by a simply equation purely based on the characteristics of the hash functions without using any host address information. This procedure is highly efficient since the computational time is constant and ignorable. Furthermore, this reversible connection degree sketch based method can be easily implemented in distributed systems. The experimental and testing results based on the actual network traffics show that the new method is truly accurate and efficient.