Multi-core vs. I/O wall: the approaches to conquer and cooperate

  • Authors:

  • Yansong Zhang;Min Jiao;Zhanwei Wang;Shan Wang;Xuan Zhou

  • Affiliations:

  • DEKE Lab, Renmin University of China and National Survey Research Center at Renmin University of China, Beijing, China;DEKE Lab, Renmin University of China and School of Information, Renmin University of China, Beijing, China;DEKE Lab, Renmin University of China and School of Information, Renmin University of China, Beijing, China;DEKE Lab, Renmin University of China and School of Information, Renmin University of China, Beijing, China;DEKE Lab, Renmin University of China and School of Information, Renmin University of China, Beijing, China

  • Venue:
  • WAIM'11 Proceedings of the 12th international conference on Web-age information management
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

Multi-core comes to be the mainstream of processor techniques. The data-intensive OLAP relies on inexpensive disks as massive data storage device, so the enhanced processing power oppose to I/O bottleneck in big data OLAP applications becomes more critical because the latency gap between I/O and multi-core gets even larger. In this paper, we focus on the disk resident OLAP with large dataset, exploiting the power of multi-core processing under I/O bottleneck. We propose optimizations for schema-aware storage layout, parallel accessing and I/O latency aware concurrent processing. On the one hand I/O bottleneck should be conquered to reduce latency for multi-core processing, on the other hand we can make good use of I/O latency for heavy concurrent query workload with multi-core power. We design experiments to exploit parallel and concurrent processing power for multi-core with DDTA-OLAP engine which minimizes the star-join cost by directly dimension tuple accessing technique. The experimental results show that we can achieve maximal speedup ratio of 103 for multi-core concurrent query processing in DRDB scenario.