Real-time analysis and management of big time-series data

  • Authors:
  • A. Biem;H. Feng;A. V. Riabov;D. S. Turaga

  • Affiliations:
  • IBM Research Division, Thomas J. Watson Research Yorktown Heights, NY;-;IBM Research Division, Thomas J. Watson Research Center, Yorktown Heights, NY;IBM Research Division, Thomas J. Watson Research Center, Yorktown Heights, NY

  • Venue:
  • IBM Journal of Research and Development
  • Year:
  • 2013

Quantified Score

Hi-index 0.00

Visualization

Abstract

The ability to process and analyze large volumes of time-series data is in increasing demand in various domains including health care, finance, energy and utilities, transportation, and cybersecurity. Despite the broad use of time-series data worldwide, the design of a system to easily manage, analyze, and visualize large multidimensional time series, with dimensions on the order of hundreds of thousands, is still a challenging endeavor. This paper describes the Streaming Time-Series Analysis and Management (STAM) system as a solution to this problem. STAM provides the capability to glean actionable information from continuously changing time series with thousands of dimensions, in real time. STAM exploits the IBM InfoSphere® Streams platform and allows for general-purpose large-scale time-series analytics for applications including anomaly detection, modeling, smoothing, forecasting, and tracking. In addition, the system provides user-friendly tools for managing, deploying, and initiating analytics on large-scale data streams of interest, and provides a web-based graphical visualization interface that allows highlighting of events of interest with interactive menus. In this paper, we describe the system and illustrate its use in a large-scale system-monitoring application.