Multi-core parallelization in Clojure: a case study

  • Authors:
  • Johann M. Kraus;Hans A. Kestler

  • Affiliations:
  • University of Ulm, Ulm;University of Ulm, Ulm and University Hospital Ulm, Ulm

  • Venue:
  • Proceedings of the 6th European Lisp Workshop
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

In recent years, the demand for computational power in data mining applications has increased due to rapidly growing data sets. As a consequence, standard algorithms need to be parallelized for fast processing of the generated data sets. Unfortunately, most approaches for parallelizing algorithms require a careful software design and a deep knowledge about thread-safe programming. As a consequence they are hardly applicable for rapid prototyping of new algorithms. We outline the process of multi-core parallelization using Clojure, a new functional programming language utilizing the Java Virtual Machine (JVM) that does not require knowledge of thread-safe programming. We provide some benchmark results for our multi-core algorithm to demonstrate its computational power. The rationale behind Clojure is combining the industry-standard JVM with functional programming, immutable data structures, and a built-in concurrency support via software transactional memory. This makes it a suitable tool for parallelization and rapid prototyping in many areas. In this case study we present a multi-core parallel implementation of the k-means cluster algorithm. The multi-core algorithm shows an increase in computation speed up to a factor of 10 compared to R or network based parallelization.