Elements of information theory
Elements of information theory
Introduction to data compression
Introduction to data compression
Scattered Data Interpolation with Multilevel B-Splines
IEEE Transactions on Visualization and Computer Graphics
Data Reduction Using Cubic Rational B-Splines
IEEE Computer Graphics and Applications
Fast Lossless Compression of Scientific Floating-Point Data
DCC '06 Proceedings of the Data Compression Conference
Fast and Efficient Compression of Floating-Point Data
IEEE Transactions on Visualization and Computer Graphics
FPC: A High-Speed Compressor for Double-Precision Floating-Point Data
IEEE Transactions on Computers
Adaptable, metadata rich IO methods for portable high performance IO
IPDPS '09 Proceedings of the 2009 IEEE International Symposium on Parallel&Distributed Processing
Lossless compression of predicted floating-point geometry
Computer-Aided Design
ISABELA-QA: query-driven analytics with ISABELA-compressed extreme-scale scientific data
Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis
In-situ I/O processing: a case for location flexibility
Proceedings of the sixth workshop on Parallel Data Storage
ISOBAR hybrid compression-I/O interleaving for large-scale parallel I/O optimization
Proceedings of the 21st international symposium on High-Performance Parallel and Distributed Computing
Computer Science - Research and Development
SC '13 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis
Assessing the effects of data compression in simulations using physically motivated metrics
SC '13 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis
| Hi-index | 0.00 |
Modern large-scale scientific simulations running on HPC systems generate data in the order of terabytes during a single run. To lessen the I/O load during a simulation run, scientists are forced to capture data infrequently, thereby making data collection an inherently lossy process. Yet, lossless compression techniques are hardly suitable for scientific data due to its inherently random nature; for the applications used here, they offer less than 10% compression rate. They also impose significant overhead during decompression, making them unsuitable for data analysis and visualization that require repeated data access. To address this problem, we propose an effective method for In-situ Sort-And-B-spline Error-bounded Lossy Abatement (ISABELA) of scientific data that is widely regarded as effectively incompressible. With ISABELA, we apply a preconditioner to seemingly random and noisy data along spatial resolution to achieve an accurate fitting model that guarantees a ≥ 0.99 correlation with the original data. We further take advantage of temporal patterns in scientific data to compress data by ≈ 85%, while introducing only a negligible overhead on simulations in terms of runtime. ISABELA significantly outperforms existing lossy compression methods, such as Wavelet compression. Moreover, besides being a communication-free and scalable compression technique, ISABELA is an inherently local decompression method, namely it does not decode the entire data, making it attractive for random access.