Compressing Bitmap Indices by Data Reorganization
ICDE '05 Proceedings of the 21st International Conference on Data Engineering
Optimizing bitmap indices with efficient compression
ACM Transactions on Database Systems (TODS)
Integrating compression and execution in column-oriented database systems
Proceedings of the 2006 ACM SIGMOD international conference on Management of data
Approximate encoding for direct access and query processing over compressed bitmaps
VLDB '06 Proceedings of the 32nd international conference on Very large data bases
Multi-resolution bitmap indexes for scientific data
ACM Transactions on Database Systems (TODS)
A novel approach to revealing positive and negative co-regulated genes
Journal of Computer Science and Technology
On the performance of bitmap indices for high cardinality attributes
VLDB '04 Proceedings of the Thirtieth international conference on Very large data bases - Volume 30
RLH: bitmap compression technique based on run-length and huffman encoding
Proceedings of the ACM tenth international workshop on Data warehousing and OLAP
High performance multivariate visual data exploration for extremely large data
Proceedings of the 2008 ACM/IEEE conference on Supercomputing
Breaking the Curse of Cardinality on Bitmap Indexes
SSDBM '08 Proceedings of the 20th international conference on Scientific and Statistical Database Management
Dynamic data organization for bitmap indices
Proceedings of the 3rd international conference on Scalable information systems
Secondary bitmap indexes with vertical and horizontal partitioning
Proceedings of the 12th International Conference on Extending Database Technology: Advances in Database Technology
RLH: Bitmap compression technique based on run-length and Huffman encoding
Information Systems
Divide-and-conquer scheme for strictly optimal retrieval of range queries
ACM Transactions on Storage (TOS)
GPU-WAH: applying GPUs to compressing bitmap indexes with word aligned hybrid
DEXA'10 Proceedings of the 21st international conference on Database and expert systems applications: Part II
TI: an efficient indexing mechanism for real-time search on tweets
Proceedings of the 2011 ACM SIGMOD International Conference on Management of data
Variable length compression for bitmap indices
DEXA'11 Proceedings of the 22nd international conference on Database and expert systems applications - Volume Part II
ISABELA-QA: query-driven analytics with ISABELA-compressed extreme-scale scientific data
Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis
EDBT'06 Proceedings of the 10th international conference on Advances in Database Technology
Space-efficient structures for detecting port scans
DEXA'07 Proceedings of the 18th international conference on Database and Expert Systems Applications
Taming massive distributed datasets: data sampling using bitmap indices
Proceedings of the 22nd international symposium on High-performance parallel and distributed computing
Column imprints: a secondary index structure
Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data
SDQuery DSI: integrating data management support with a wide area data transfer protocol
SC '13 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis
Dynamic bitmap index recompression through workload-based optimizations
Proceedings of the 17th International Database Engineering & Applications Symposium
Hi-index | 0.00 |
In this paper, we study the effects of compression on bitmap indexes. The main operations on the bitmaps during query processing are bitwise logical operations such as AND, OR, NOT, etc. Using the general purpose compression schemes, such as gzip, the logical operations onthe compressed bitmaps are much slower than on the uncompressed bitmaps. Specialized compression schemes, like the byte-aligned bitmap code (BBC), are usually faster in performing logical operations than the general purpose schemes, but in many cases they are still orders of magnitude slower than the uncompressed scheme. To make the compressed bitmap indexes operate more efficiently, we designed a CPU-friendly scheme which we refer to as the word-aligned hybrid code (WAH). Tests on both synthetic and real application data show that the new scheme significantly outperforms well-known compression schemes at a modest increase in storage space. Compared to BBC, a scheme well-known for its operational efficiency, WAH performs logical operations about 12 times faster and uses only 60% more space. Compared to the uncompressed scheme, in most test cases WAH is faster while still using less space. We further verified with additional tests that the improvement in logical operation speed translates to similar improvement in query processing speed.