Semi-sparse algorithm based on multi-layer optimization for recommendation system

  • Authors:
  • Hu Guan;Huakang Li;Minyi Guo

  • Affiliations:
  • Shanghai Jiao Tong University, Shanghai, China;Shanghai Jiao Tong University, Shanghai, China;Shanghai Jiao Tong University, Shanghai, China

  • Venue:
  • Proceedings of the 2012 International Workshop on Programming Models and Applications for Multicores and Manycores
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

Collaborative filter (CF) is the most successful technique in recommender system, which makes personalize recommendations during online interaction. We propose a new Semi-sparse algorithm based on multi-layer optimization to speed up the basic Pearson Correlation Coefficient of CF. Semi-sparse algorithm spares out over-reduplicate accessing and judgement on selected sparse vector to accelerate the batch of similarity-comparisons in one thread. We propose a reduce-vector in thread-pool to restrict the lock using on critical resources in parallelize implementation. Thread-pool is wrapped with Pthreads on multi-core node to make semi-sparse parallelization more easily. A shared zip file is read to cut down messages with Message Passing Interface package. The performance of proposed semi-sparse with multi-layer framework achieved a brilliant speedup in the evaluation of Netflix, MovieLens and MovieLen1600.