Assessing the quality of packet-level traces collected on internet backbone links

Authors:
Behrooz Sangchoolie;Mazdak Rajabi Nasab;Tomas Olovsson;Wolfgang John
Affiliations:
Department of Computer Science and Engineering, Chalmers University of Technology, Gothenburg, Sweden;Department of Computer Science and Engineering, Chalmers University of Technology, Gothenburg, Sweden;Department of Computer Science and Engineering, Chalmers University of Technology, Gothenburg, Sweden;Ericsson Research, Kista, Sweden
Venue:
NordSec'12 Proceedings of the 17th Nordic conference on Secure IT Systems
Year:
2012

Citing 10
Cited 0

Automated packet trace analysis of TCP implementations

SIGCOMM '97 Proceedings of the ACM SIGCOMM '97 conference on Applications, technologies, architectures, and protocols for computer communication
Prefix-Preserving IP Address Anonymization: Measurement-Based Security Evaluation and a New Cryptography-Based Scheme

ICNP '02 Proceedings of the 10th IEEE International Conference on Network Protocols
Analysis of internet backbone traffic and header anomalies observed

Proceedings of the 7th ACM SIGCOMM conference on Internet measurement
Anomaly extraction in backbone networks using association rules

Proceedings of the 9th ACM SIGCOMM conference on Internet measurement conference
TCP revisited: a fresh look at TCP in the wild

Proceedings of the 9th ACM SIGCOMM conference on Internet measurement conference
Measuring IP and TCP behavior on edge nodes with Tstat

Computer Networks: The International Journal of Computer and Telecommunications Networking
Review: Passive internet measurement: Overview and guidelines based on experiences

Computer Communications
Estimating routing symmetry on single links by passive flow measurements

Proceedings of the 6th International Wireless Communications and Mobile Computing Conference
A parameterizable methodology for Internet traffic flow profiling

IEEE Journal on Selected Areas in Communications
Packet-level traffic measurements from the Sprint IP backbone

IEEE Network: The Magazine of Global Internetworking

Quantified Score

Hi-index	0.00

Visualization

Abstract

The quality of captured traffic plays an important role for decisions made by systems like intrusion detection/prevention systems (IDS/IPS) and firewalls. As these systems monitor network traffic to find malicious activities, a missing packet might lead to an incorrect decision. In this paper, we analyze the quality of packet-level traces collected on Internet backbone links using different generations of DAG cards. This is accomplished by inferring dropped packets introduced by the data collection system with help of the intrinsic structural properties inherently provided by TCP traffic flows. We employ two metrics which we believe can detect all kinds of missing packets: i) packets with ACK numbers greater than the expected ACK, indicating that the communicating parties acknowledge a packet not present in the trace; and ii) packets with data beyond the receiver's window size, which with a high probability, indicates that the packet advertising the correct window size was not recorded. These heuristics have been applied to three large datasets collected with different hardware and in different environments. We also introduce flowstat, a tool developed for this purpose which is capable of analyzing both captured traces and real-time traffic. After assessing more than 400 traces (75M bidirectional flows), we conclude that at least 0.08% of the flows have missing packets, a surprisingly large number that can affect the quality of analysis performed by firewalls and intrusion detection/prevention systems. The paper concludes with an investigation and discussion of the spatial and temporal aspects of the experienced packet losses and possible reasons behind missing data in traces.