Efficient data structures for sparse network representation

  • Authors:

  • Jorkki Hyvonen;Jari Saramaki;Kimmo Kaski

  • Affiliations:

  • Laboratory of Computational Engineering, Helsinki University of Technology, Espoo, Finland;Laboratory of Computational Engineering, Helsinki University of Technology, Espoo, Finland;Laboratory of Computational Engineering, Helsinki University of Technology, Espoo, Finland

  • Venue:
  • International Journal of Computer Mathematics - COMPLEX NETWORKS
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

Modern-day computers are characterized by a striking contrast between the processing power of the CPU and the latency of main memory accesses. If the data processed is both large compared to processor caches and sparse or high-dimensional in nature, as is commonly the case in complex network research, the main memory latency can become a performace bottleneck. In this article, we present a cache-efficient data structure, a variant of a linear probing hash table, for representing edge sets of such networks. The performance benchmarks show that it is, indeed, quite superior to its commonly used counterparts in this application. In addition, its memory footprint only exceeds the absolute minimum by a small constant factor. The practical usability of our approach has been well demonstrated in the study of very large real-world networks.