Building a Better Similarity Trap with Statistically Improbable Features

  • Authors:
  • Affiliations:
  • Venue:
  • HICSS '09 Proceedings of the 42nd Hawaii International Conference on System Sciences
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

One of the persistent topics in digital forensic research in recent years has been the problem of finding all things similar. Developed tools usually take on the form of similarity, or fuzzy hash. In this paper, we present a generic empirical study of the problem of finding common features in binary data. Specifically, we study the problem of false positives and demonstrate that similarity tools work only as well as the underlying data allows them to and, therefore, must be aware of the basic properties of the input. We propose a new feature selection algorithm, which is based on the notion of statistically improbable features. We also show that the proposed method, can be tuned to account for the type-specific distribution of false positives.