Optimizing bitmap indices with efficient compression
ACM Transactions on Database Systems (TODS)
The BSD packet filter: a new architecture for user-level packet capture
USENIX'93 Proceedings of the USENIX Winter 1993 Conference Proceedings on USENIX Winter 1993 Conference Proceedings
On the performance of bitmap indices for high cardinality attributes
VLDB '04 Proceedings of the Thirtieth international conference on Very large data bases - Volume 30
Hyperion: high volume stream archival for retrospective querying
ATC'07 2007 USENIX Annual Technical Conference on Proceedings of the USENIX Annual Technical Conference
Enriching network security analysis with time travel
Proceedings of the ACM SIGCOMM 2008 conference on Data communication
Position list word aligned hybrid: optimizing space and performance for compressed bitmaps
Proceedings of the 13th International Conference on Extending Database Technology
NET-FLi: on-the-fly compression, archiving and indexing of streaming network traffic
Proceedings of the VLDB Endowment
Collection and exploration of large data monitoring sets using bitmap databases
TMA'10 Proceedings of the Second international conference on Traffic Monitoring and Analysis
RasterZip: compressing network monitoring data with support for partial decompression
Proceedings of the 2012 ACM conference on Internet measurement conference
Indexing million of packets per second using GPUs
Proceedings of the 2013 conference on Internet measurement conference
| Hi-index | 0.00 |
Long-term historical analysis of captured network traffic is a topic of great interest in network monitoring and network security. A critical requirement is the support for fast discovery of packets that satisfy certain criteria within large-scale packet repositories. This work presents the first indexing scheme for network packet traces based on compressed bitmap indexing principles. Our approach supports very fast insertion rates and results in compact index sizes. The proposed indexing methodology builds upon libpcap, the de-facto reference library for accessing packet-trace repositories. Our solution is therefore backward compatible with any solution that uses the original library. We experience impressive speedups on packet-trace search operations: our experiments suggest that the index-enabled libpcap may reduce the packet retrieval time by more than 1100 times.