Bro: a system for detecting network intruders in real-time
Computer Networks: The International Journal of Computer and Telecommunications Networking
Compressing Relations and Indexes
ICDE '98 Proceedings of the Fourteenth International Conference on Data Engineering
Gigascope: a stream database for network applications
Proceedings of the 2003 ACM SIGMOD international conference on Management of data
Snort - Lightweight Intrusion Detection for Networks
LISA '99 Proceedings of the 13th USENIX conference on System administration
More Netflow Tools for Performance and Security
LISA '04 Proceedings of the 18th USENIX conference on System administration
C-store: a column-oriented DBMS
VLDB '05 Proceedings of the 31st international conference on Very large data bases
Integrating compression and execution in column-oriented database systems
Proceedings of the 2006 ACM SIGMOD international conference on Management of data
Bigtable: a distributed storage system for structured data
OSDI '06 Proceedings of the 7th USENIX Symposium on Operating Systems Design and Implementation - Volume 7
Tribeca: a system for managing large databases of network traffic
ATEC '98 Proceedings of the annual conference on USENIX Annual Technical Conference
On-demand view materialization and indexing for network forensic analysis
NETB'07 Proceedings of the 3rd USENIX international workshop on Networking meets databases
Column-stores vs. row-stores: how different are they really?
Proceedings of the 2008 ACM SIGMOD international conference on Management of data
Read-optimized databases, in depth
Proceedings of the VLDB Endowment
Brighthouse: an analytic data warehouse for ad-hoc queries
Proceedings of the VLDB Endowment
Proceedings of the 4th ACM workshop on Security and artificial intelligence
Real-time creation of bitmap indexes on streaming network data
The VLDB Journal — The International Journal on Very Large Data Bases
Toward efficient querying of compressed network payloads
USENIX ATC'12 Proceedings of the 2012 USENIX conference on Annual Technical Conference
RasterZip: compressing network monitoring data with support for partial decompression
Proceedings of the 2012 ACM conference on Internet measurement conference
Efficient multidimensional aggregation for large scale monitoring
lisa'12 Proceedings of the 26th international conference on Large Installation System Administration: strategies, tools, and techniques
A scalable network forensics mechanism for stealthy self-propagating attacks
Computer Communications
| Hi-index | 0.00 |
With the increasing sophistication of attacks, there is a need for network security monitoring systems that store and examine very large amounts of historical network flow data. An efficient storage infrastructure should provide both high insertion rates and fast data access. Traditional row-oriented Relational Database Management Systems (RDBMS) provide satisfactory query performance for network flow data collected only over a period of several hours. In many cases, such as the detection of sophisticated coordinated attacks, it is crucial to query days, weeks or even months worth of disk resident historical data rapidly. For such monitoring and forensics queries, row oriented databases become I/O bound due to long disk access times. Furthermore, their data insertion rate is proportional to the number of indexes used, and query processing time is increased when it is necessary to load unused attributes along with the used ones. To overcome these problems we propose a new column oriented storage infrastructure for network flow records, called NetStore. NetStore is aware of network data semantics and access patterns, and benefits from the simple column oriented layout without the need to meet general purpose RDBMS requirements. The prototype implementation of NetStore can potentially achieve more than ten times query speedup and ninety times less storage size compared to traditional row-stores, while it performs better than existing open source columnstores for network flow data.