A locally adaptive data compression scheme

Authors:
Jon Louis Bentley;Daniel D. Sleator;Robert E. Tarjan;Victor K. Wei
Affiliations:
AT&T Bell Laboratories, Murray Hill, NJ;AT&T Bell Laboratories, Murray Hill, NJ;Princeton Univ., Princeton, NJ;Bell Communications Research, Murray Hill, NJ
Venue:
Communications of the ACM
Year:
1986

Citing 12
Cited 110

Amortized efficiency of list update and paging rules

Communications of the ACM
Data structures and network algorithms

Data structures and network algorithms
Self-adjusting binary search trees

Journal of the ACM (JACM)
Dynamic Huffman coding

Journal of Algorithms
Interval and recency rank source coding: two on-line adaptive variable-length schemes

IEEE Transactions on Information Theory
The art of computer programming, volume 3: (2nd ed.) sorting and searching

The art of computer programming, volume 3: (2nd ed.) sorting and searching
A Discipline of Programming

A Discipline of Programming
Information Theory and Reliable Communication

Information Theory and Reliable Communication
AVL-Trees for Localized Search

Proceedings of the 11th Colloquium on Automata, Languages and Programming
Localized search in sorted lists

STOC '81 Proceedings of the thirteenth annual ACM symposium on Theory of computing
Amortized efficiency of list update rules

STOC '84 Proceedings of the sixteenth annual ACM symposium on Theory of computing
Self-adjusting binary trees

STOC '83 Proceedings of the fifteenth annual ACM symposium on Theory of computing

Design and analysis of dynamic Huffman codes

Journal of the ACM (JACM)
Data compression

ACM Computing Surveys (CSUR)
Application of splay trees to data compression

Communications of the ACM
Data compression with finite windows

Communications of the ACM
Systolic implementations of a move-to-front text compressor

SPAA '89 Proceedings of the first annual ACM symposium on Parallel algorithms and architectures
Modeling for text compression

ACM Computing Surveys (CSUR)
An honors course in data compression

SIGCSE '91 Proceedings of the twenty-second SIGCSE technical symposium on Computer science education
Systolic implementations of a move-to-front text compressor

ACM SIGARCH Computer Architecture News - Symposium on parallel algorithms and architectures
Practical dictionary management for hardware data compression

Communications of the ACM
The relationship between greedy parsing and symbolwise text compression

Journal of the ACM (JACM)
Arithmetic coding for data compression

Communications of the ACM
Code compression

Proceedings of the ACM SIGPLAN 1997 conference on Programming language design and implementation
Code compression for embedded systems

DAC '98 Proceedings of the 35th annual Design Automation Conference
Arithmetic coding revisited

ACM Transactions on Information Systems (TOIS)
Fast searching on compressed text allowing errors

Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval
Splay trees for data compression

Proceedings of the sixth annual ACM-SIAM symposium on Discrete algorithms
Online list accessing algorithms and their applications: recent empirical evidence

SODA '97 Proceedings of the eighth annual ACM-SIAM symposium on Discrete algorithms
An analysis of the Burrows-Wheeler transform

Proceedings of the tenth annual ACM-SIAM symposium on Discrete algorithms
Fast and flexible word searching on compressed text

ACM Transactions on Information Systems (TOIS)
An experimental study of an opportunistic index

SODA '01 Proceedings of the twelfth annual ACM-SIAM symposium on Discrete algorithms
Can entropy characterize performance of online algorithms?

SODA '01 Proceedings of the twelfth annual ACM-SIAM symposium on Discrete algorithms
Biased dictionaries with fast insert/deletes

STOC '01 Proceedings of the thirty-third annual ACM symposium on Theory of computing
An analysis of the Burrows—Wheeler transform

Journal of the ACM (JACM)
Word-based block-sorting text compression

ACSC '01 Proceedings of the 24th Australasian conference on Computer science
Enhanced word-based block-sorting text compression

ACSC '02 Proceedings of the twenty-fifth Australasian conference on Computer science - Volume 4
Adding Compression to Block Addressing Inverted Indexes

Information Retrieval
The V.42bis Standard for Data-Compressing Modems

IEEE Micro
Text Compression for Dynamic Document Databases

IEEE Transactions on Knowledge and Data Engineering
Invited Lecture: The Burrows-Wheeler Transform: Theory and Practice

MFCS '99 Proceedings of the 24th International Symposium on Mathematical Foundations of Computer Science
Two New Families of List Update Algorithms

ISAAC '98 Proceedings of the 9th International Symposium on Algorithms and Computation
Compressed Text Databases with Efficient Query Algorithms Based on the Compressed Suffix Array

ISAAC '00 Proceedings of the 11th International Conference on Algorithms and Computation
Biased Skip Lists for Highly Skewed Access Patterns

ALENEX '01 Revised Papers from the Third International Workshop on Algorithm Engineering and Experimentation
Invited Lecture: Online Algorithms: A Study of Graph-Theoretic Concepts

WG '99 Proceedings of the 25th International Workshop on Graph-Theoretic Concepts in Computer Science
A General Practical Approach to Pattern Matching over Ziv-Lempel Compressed Text

CPM '99 Proceedings of the 10th Annual Symposium on Combinatorial Pattern Matching
X-MatchPRO: A ProASIC-Based 200 Mbytes/s Full-Duplex Lossless Data Compressor

FPL '01 Proceedings of the 11th International Conference on Field-Programmable Logic and Applications
Improving compression performance of block sorting coder

Journal of Computing Sciences in Colleges
Switching Between Two On-line List Update Algorithms for Higher Compression of Burrows-Wheeler Transformed Data

DCC '00 Proceedings of the Conference on Data Compression
Move-to-Front and Inversion Coding

DCC '00 Proceedings of the Conference on Data Compression
DNA Sequence Compression Using the Burrows-Wheeler Transform

CSB '02 Proceedings of the IEEE Computer Society Conference on Bioinformatics
Generalized Lempel-Ziv parsing scheme and its preliminary analysis of the average profile

DCC '95 Proceedings of the Conference on Data Compression
Searching BWT Compressed Text with the Boyer-Moore Algorithm and Binary Search

DCC '02 Proceedings of the Data Compression Conference
Parsing Strategies for BWT Compression

DCC '01 Proceedings of the Data Compression Conference
Survey of code-size reduction methods

ACM Computing Surveys (CSUR)
Gbit/s lossless data compression hardware

IEEE Transactions on Very Large Scale Integration (VLSI) Systems - Special section on the 2001 international conference on computer design (ICCD)
When indexing equals compression: experiments with compressing suffix arrays and applications

SODA '04 Proceedings of the fifteenth annual ACM-SIAM symposium on Discrete algorithms
“Book Stack” as a New Statistical Test for Random Numbers

Problems of Information Transmission
Universal Text Preprocessing for Data Compression

IEEE Transactions on Computers
Word-based text compression using the Burrows-Wheeler transform

Information Processing and Management: an International Journal
Indexing compressed text

Journal of the ACM (JACM)
Boosting textual compression in optimal linear time

Journal of the ACM (JACM)
Segmentation-based multilayer diagnosis lossless medical image compression

VIP '05 Proceedings of the Pan-Sydney area workshop on Visual information processing
Context exhumation after the Burrows-Wheeler transform

Information Processing Letters
ARC: A Self-Tuning, Low Overhead Replacement Cache

FAST '03 Proceedings of the 2nd USENIX Conference on File and Storage Technologies
Squeezing succinct data structures into entropy bounds

SODA '06 Proceedings of the seventeenth annual ACM-SIAM symposium on Discrete algorithm
Combining Structural and Textual Contexts for Compressing Semistructured Databases

ENC '05 Proceedings of the Sixth Mexican International Conference on Computer Science
Improved adaptive vector quantization algorithm using hybrid codebook data structure

Real-Time Imaging
Error resilient locally adaptive data compression

Journal of Systems and Software
Grid-based computer animation rendering

Proceedings of the 4th international conference on Computer graphics and interactive techniques in Australasia and Southeast Asia
Entropy-based bounds for online algorithms

ACM Transactions on Algorithms (TALG)
When indexing equals compression: Experiments with compressing suffix arrays and applications

ACM Transactions on Algorithms (TALG)
Using structural contexts to compress semistructured text collections

Information Processing and Management: an International Journal
User modeling for personalized Web search with self-organizing map: Research Articles

Journal of the American Society for Information Science and Technology
Lossless and Near-Lossless Compression of Ecg Signals with Block-Sorting Techniques

International Journal of High Performance Computing Applications
Fault diagnosis for airplane engines using Bayesian networks and distributed particle swarm optimization

Parallel Computing
Data structures for limited oblivious execution of programs while preserving locality of reference

Proceedings of the 2007 ACM workshop on Digital Rights Management
Burrows–Wheeler compression: Principles and reflections

Theoretical Computer Science
A simpler analysis of Burrows–Wheeler-based compression

Theoretical Computer Science
From first principles to the Burrows and Wheeler transform and beyond, via combinatorial optimization

Theoretical Computer Science
Compressing table data with column dependency

Theoretical Computer Science
RadixZip: linear time compression of token streams

VLDB '07 Proceedings of the 33rd international conference on Very large data bases
Dynamic asymmetric communication

Information Processing Letters
New adaptive compressors for natural language text

Software—Practice & Experience
Reversible information hiding for VQ indices based on locally adaptive coding

Journal of Visual Communication and Image Representation
Inverted index compression and query processing with optimized document ordering

Proceedings of the 18th international conference on World wide web
Squarepants in a tree: Sum of subtree clustering and hyperbolic pants decomposition

ACM Transactions on Algorithms (TALG)
An Application of Self-organizing Data Structures to Compression

SEA '09 Proceedings of the 8th International Symposium on Experimental Algorithms
Novel and Generalized Sort-Based Transform for Lossless Data Compression

SPIRE '09 Proceedings of the 16th International Symposium on String Processing and Information Retrieval
A real-time program trace compressor utilizing double move-to-front method

Proceedings of the 46th Annual Design Automation Conference
Real-time lossless compression for silicon debug

IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
Stream-oriented lossless packet compression in wireless sensor networks

SECON'09 Proceedings of the 6th Annual IEEE communications society conference on Sensor, Mesh and Ad Hoc Communications and Networks
Word-based text compression using the Burrows-Wheeler transform

Information Processing and Management: an International Journal
Context exhumation after the Burrows--Wheeler transform

Information Processing Letters
Optimal partitions of strings: a new class of Burrows-Wheeler compression algorithms

CPM'03 Proceedings of the 14th annual conference on Combinatorial pattern matching
Compressing semistructured text databases

ECIR'03 Proceedings of the 25th European conference on IR research
Dynamic lightweight text compression

ACM Transactions on Information Systems (TOIS)
Fractal curves to improve the reversible data embedding for VQ-indexes based on locally adaptive coding

Journal of Visual Communication and Image Representation
Move-to-Front, Distance Coding, and Inversion Frequencies revisited

Theoretical Computer Science
Post BWT stages of the Burrows–Wheeler compression algorithm

Software—Practice & Experience
ARC: a self-tuning, low overhead replacement cache

FAST'03 Proceedings of the 2nd USENIX conference on File and storage technologies
Balancing and clustering of words in the Burrows-Wheeler transform

Theoretical Computer Science
Embedded debug architecture for bypassing blocking bugs during post-silicon validation

IEEE Transactions on Very Large Scale Integration (VLSI) Systems
Improving semistatic compression via phrase-based modeling

Information Processing and Management: an International Journal
A reversible data hiding scheme for VQ indices using locally adaptive coding

Journal of Visual Communication and Image Representation
Natural Language Compression on Edge-Guided text preprocessing

Information Sciences: an International Journal
Predicting Metal-Binding Sites from Protein Sequence

IEEE/ACM Transactions on Computational Biology and Bioinformatics (TCBB)
Dynamic asymmetric communication

SIROCCO'06 Proceedings of the 13th international conference on Structural Information and Communication Complexity
A simpler analysis of burrows-wheeler based compression

CPM'06 Proceedings of the 17th Annual conference on Combinatorial Pattern Matching
Word-based fixed and flexible list compression

ISCIS'05 Proceedings of the 20th international conference on Computer and Information Sciences
Mapping words into codewords on PPM

SPIRE'06 Proceedings of the 13th international conference on String Processing and Information Retrieval
Self-Describing context-based pixel ordering

ISVC'05 Proceedings of the First international conference on Advances in Visual Computing
Trimming the tree: tailoring adaptive huffman coding to wireless sensor networks

EWSN'10 Proceedings of the 7th European conference on Wireless Sensor Networks
List update with probabilistic locality of reference

Information Processing Letters
A new perspective on list update: probabilistic locality and working set

WAOA'11 Proceedings of the 9th international conference on Approximation and Online Algorithms
Improving evolved alphabet using tabu set

HAIS'12 Proceedings of the 7th international conference on Hybrid Artificial Intelligent Systems - Volume Part I
Space-conscious compression

MFCS'07 Proceedings of the 32nd international conference on Mathematical Foundations of Computer Science
Move-to-front, distance coding, and inversion frequencies revisited

CPM'07 Proceedings of the 18th annual conference on Combinatorial Pattern Matching
Most burrows-wheeler based compressors are not optimal

CPM'07 Proceedings of the 18th annual conference on Combinatorial Pattern Matching
Implicit indexing of natural language text by reorganizing bytecodes

Information Retrieval
Chain code lossless compression using move-to-front transform and adaptive run-length encoding

Image Communication
A novel high-performance reversible data hiding scheme using SMVQ and improved locally adaptive coding method

Journal of Visual Communication and Image Representation

Quantified Score

Hi-index	48.29

Visualization

Abstract

A data compression scheme that exploits locality of reference, such as occurs when words are used frequently over short intervals and then fall into long periods of disuse, is described. The scheme is based on a simple heuristic for self-organizing sequential search and on variable-length encodings of integers. We prove that it never performs much worse than Huffman coding and can perform substantially better; experiments on real files show that its performance is usually quite close to that of Huffman coding. Our scheme has many implementation advantages: it is simple, allows fast encoding and decoding, and requires only one pass over the data to be compressed (static Huffman coding takes two passes).