Parallel lossless data compression using the Burrows-Wheeler Transform

Authors:
Jeff Gilchrist;Aysegul Cuhadar
Affiliations:
Department of Systems and Computer Engineering, Carleton University, 1125 Colonel By Drive, Ottawa, Ontario, K1S 5B6, Canada.;Department of Systems and Computer Engineering, Carleton University, 1125 Colonel By Drive, Ottawa, Ontario, K1S 5B6, Canada
Venue:
International Journal of Web and Grid Services
Year:
2008

Citing 4
Cited 0

Parallel programming with MPI

Parallel programming with MPI
Universal Data Compression Based on the Burrows-Wheeler Transformation: Theory and Practice

IEEE Transactions on Computers
On the Performance of BWT Sorting Algorithms

DCC '00 Proceedings of the Conference on Data Compression
An FPGA Parallel Sorting Architecture for the Burrows Wheeler Transform

RECONFIG '05 Proceedings of the 2005 International Conference on Reconfigurable Computing and FPGAs (ReConFig'05) on Reconfigurable Computing and FPGAs

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper, we present parallel algorithms for lossless data compression based on the Burrows-Wheeler Transform (BWT) block-sorting technique. We investigate the performance of using data parallelism and task parallelism for both multi-threaded and message-passing programming. The output produced by the parallel algorithms is fully compatible with their sequential counterparts. To balance the workload among processors we develop a task scheduling strategy. An extensive set of experiments is performed with a shared memory NUMA system using up to 120 processors and on a distributed memory cluster using up to 100 processors. Our experimental results show that significant speedup can be achieved with both data parallel and task parallel methodologies. These algorithms will greatly reduce the amount of time it takes to compress large amounts of data while the compressed data remains in a form that users without access to multiple processor systems can still use.