An approximation algorithm for space-optimal encoding of a text
The Computer Journal
A text compression scheme that allows fast searching directly in the compressed file
ACM Transactions on Information Systems (TOIS)
Fast and flexible word searching on compressed text
ACM Transactions on Information Systems (TOIS)
General-purpose compression for efficient retrieval
Journal of the American Society for Information Science and Technology
Compression and Coding Algorithms
Compression and Coding Algorithms
A Compression Scheme for Large Databases
ADC '00 Proceedings of the Australasian Database Conference
Data Compression Using Long Common Strings
DCC '99 Proceedings of the Conference on Data Compression
| Hi-index | 0.00 |
To bound memory consumption, most compression systems provide a facility that controls the amount of data that may be processed at once. In this work we consider the RE-PAIR mechanism of Larsson and Moffat [2000], which processes large messages as disjoint blocks. We show that the blocks emitted by RE-PAIR can be post-processed to yield further savings, and describe techniques that allow files of 500 MB or more to be compressed in a holistic manner using less than that much main memory. The block merging process we describe has the additional advantage of allowing new text to be appended to the end of the compressed file.