Mining frequent patterns without candidate generation
SIGMOD '00 Proceedings of the 2000 ACM SIGMOD international conference on Management of data
Deriving traffic demands for operational IP networks: methodology and experience
Proceedings of the conference on Applications, Technologies, Architectures, and Protocols for Computer Communication
New directions in traffic measurement and accounting
Proceedings of the 2002 conference on Applications, technologies, architectures, and protocols for computer communications
Fast Algorithms for Mining Association Rules in Large Databases
VLDB '94 Proceedings of the 20th International Conference on Very Large Data Bases
Proceedings of the 2003 ACM SIGMOD international conference on Management of data
Diamond in the rough: finding Hierarchical Heavy Hitters in multi-dimensional data
SIGMOD '04 Proceedings of the 2004 ACM SIGMOD international conference on Management of data
Online identification of hierarchical heavy hitters: algorithms, evaluation, and applications
Proceedings of the 4th ACM SIGCOMM conference on Internet measurement
Space complexity of hierarchical heavy hitters in multi-dimensional data streams
Proceedings of the twenty-fourth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
Keeping things simple: finding frequent item sets by recursive elimination
Proceedings of the 1st international workshop on open source data mining: frequent pattern mining implementations
Frequent pattern mining: current status and future directions
Data Mining and Knowledge Discovery
Finding hierarchical heavy hitters in data streams
VLDB '03 Proceedings of the 29th international conference on Very large data bases - Volume 29
Finding hierarchical heavy hitters in streaming data
ACM Transactions on Knowledge Discovery from Data (TKDD)
Anomaly extraction in backbone networks using association rules
Proceedings of the 9th ACM SIGCOMM conference on Internet measurement conference
Automating root-cause analysis of network anomalies using frequent itemset mining
Proceedings of the ACM SIGCOMM 2010 conference
Analysis of the impact of sampling on NetFlow traffic classification
Computer Networks: The International Journal of Computer and Telecommunications Networking
Efficient computation of frequent and top-k elements in data streams
ICDT'05 Proceedings of the 10th international conference on Database Theory
FaRNet: fast recognition of high multi-dimensional network traffic patterns
Proceedings of the ACM SIGMETRICS/international conference on Measurement and modeling of computer systems
| Hi-index | 0.00 |
Extracting knowledge from big network traffic data is a matter of foremost importance for multiple purposes including trend analysis, network troubleshooting, capacity planning, network forensics, and traffic classification. An extremely useful approach to profile traffic is to extract and display to a network administrator the multi-dimensional hierarchical heavy hitters (HHHs) of a dataset. However, existing schemes for computing HHHs have several limitations: (1) they require significant computational resources; (2) they do not scale to high dimensional data; and (3) they are not easily extensible. In this paper, we introduce a fundamentally new approach for extracting HHHs based on generalized frequent item-set mining (FIM), which allows to process traffic data much more efficiently and scales to much higher dimensional data than present schemes. Based on generalized FIM, we build and thoroughly evaluate a traffic profiling system we call FaRNet. Our comparison with AutoFocus, which is the most related tool of similar nature, shows that FaRNet is up to three orders of magnitude faster. Finally, we describe experiences on how generalized FIM is useful in practice after using FaRNet operationally for several months in the NOC of GEANT, the European backbone network.