Implementing sorting in database systems

Authors:
Goetz Graefe
Affiliations:
Microsoft, Redmond, WA
Venue:
ACM Computing Surveys (CSUR)
Year:
2006

Citing 42
Cited 21

Design and analysis of dynamic Huffman codes

Journal of the ACM (JACM)
Data compression

ACM Computing Surveys (CSUR)
Improving Quicksort Performance with a Codeword Data Structure

IEEE Transactions on Software Engineering
Merging sorted runs using large main memory

Acta Informatica
The effect of bucket size tuning in the dynamic hybrid GRACE hash join method

VLDB '89 Proceedings of the 15th international conference on Very large data bases
Efficient assembly for complex objects

SIGMOD '91 Proceedings of the 1991 ACM SIGMOD international conference on Management of data
Algorithms for creating indexes for very large tables without quiescing updates

SIGMOD '92 Proceedings of the 1992 ACM SIGMOD international conference on Management of data
A survey of adaptive sorting algorithms

ACM Computing Surveys (CSUR)
Query evaluation techniques for large databases

ACM Computing Surveys (CSUR)
RAID: high-performance, reliable secondary storage

ACM Computing Surveys (CSUR)
Loading databases using dataflow parallelism

ACM SIGMOD Record
A super scalar sort algorithm for RISC processors

SIGMOD '96 Proceedings of the 1996 ACM SIGMOD international conference on Management of data
Cost-based optimization for magic: algebra and implementation

SIGMOD '96 Proceedings of the 1996 ACM SIGMOD international conference on Management of data
High-performance sorting on networks of workstations

SIGMOD '97 Proceedings of the 1997 ACM SIGMOD international conference on Management of data
Approximate medians and other quantiles in one pass and with limited memory

SIGMOD '98 Proceedings of the 1998 ACM SIGMOD international conference on Management of data
Memory management during run generation in external sorting

SIGMOD '98 Proceedings of the 1998 ACM SIGMOD international conference on Management of data
The art of computer programming, volume 3: (2nd ed.) sorting and searching

The art of computer programming, volume 3: (2nd ed.) sorting and searching
Implementing radixsort

Journal of Experimental Algorithmics (JEA)
Duplicate record elimination in large data files

ACM Transactions on Database Systems (TODS)
Using Semi-Joins to Solve Relational Queries

Journal of the ACM (JACM)
Ubiquitous B-Tree

ACM Computing Surveys (CSUR)
An encoding method for multifield sorting and indexing

Communications of the ACM
On supporting containment queries in relational database management systems

SIGMOD '01 Proceedings of the 2001 ACM SIGMOD international conference on Management of data
Analysing cache effects in distribution sorting

Journal of Experimental Algorithmics (JEA)
Data Structures and Algorithms

Data Structures and Algorithms
Introduction to Algorithms

Introduction to Algorithms
Access path selection in a relational database management system

SIGMOD '79 Proceedings of the 1979 ACM SIGMOD international conference on Management of data
AlphaSort: a cache-sensitive parallel external sort

The VLDB Journal — The International Journal on Very Large Data Bases
Speeding Up External Mergesort

IEEE Transactions on Knowledge and Data Engineering
Fundamental Techniques for Order Optimization

EDBT '96 Proceedings of the 5th International Conference on Extending Database Technology: Advances in Database Technology
Order Preserving Compression

ICDE '96 Proceedings of the Twelfth International Conference on Data Engineering
System Issues in Parallel Sorting for Database Systems

Proceedings of the Sixth International Conference on Data Engineering
Compressing Relations and Indexes

ICDE '98 Proceedings of the Fourteenth International Conference on Data Engineering
B-Tree Indexes and CPU Caches

Proceedings of the 17th International Conference on Data Engineering
Block Oriented Processing of Relational Database Operations in Modern Computer Architectures

Proceedings of the 17th International Conference on Data Engineering
Dynamic Memory Adjustment for External Mergesort

VLDB '97 Proceedings of the 23rd International Conference on Very Large Data Bases
Hash Joins and Hash Teams in Microsoft SQL Server

VLDB '98 Proceedings of the 24rd International Conference on Very Large Data Bases
Buffering and Read-Ahead Strategies for External Mergesort

VLDB '98 Proceedings of the 24rd International Conference on Very Large Data Bases
Memory-Adaptive External Sorting

VLDB '93 Proceedings of the 19th International Conference on Very Large Data Bases
External Sorting: Run Formation Revisited

IEEE Transactions on Knowledge and Data Engineering
The optimization of queries in relational databases

The optimization of queries in relational databases
Adapting Radix Sort to the Memory Hierarchy

Journal of Experimental Algorithmics (JEA)

CellSort: high performance sorting on the cell processor

VLDB '07 Proceedings of the 33rd international conference on Very large data bases
The five-minute rule twenty years later, and how flash memory changes the rules

DaMoN '07 Proceedings of the 3rd international workshop on Data management on new hardware
The Five-Minute Rule 20 Years Later: and How Flash Memory Changes the Rules

Queue - Enterprise Flash Storage
The five-minute rule 20 years later (and how flash memory changes the rules)

Communications of the ACM - Barbara Liskov: ACM's A.M. Turing Award Winner
Low-cost management of inverted files for online full-text search

Proceedings of the 18th ACM conference on Information and knowledge management
Sorting improves word-aligned bitmap indexes

Data & Knowledge Engineering
Engineering burstsort: Toward fast in-place string sorting

Journal of Experimental Algorithmics (JEA)
Self-selecting, self-tuning, incrementally optimized indexes

Proceedings of the 13th International Conference on Extending Database Technology
Engineering burstsort: towards fast in-place string sorting

WEA'08 Proceedings of the 7th international conference on Experimental algorithms
Two-way replacement selection

Proceedings of the VLDB Endowment
FPGASort: a high performance sorting architecture exploiting run-time reconfiguration on fpgas for large problem sorting

Proceedings of the 19th ACM/SIGDA international symposium on Field programmable gate arrays
Which sort orders are interesting?

The VLDB Journal — The International Journal on Very Large Data Bases
New algorithms for join and grouping operations

Computer Science - Research and Development
Modern B-Tree Techniques

Foundations and Trends in Databases
Deferred maintenance of indexes and of materialized views

DNIS'11 Proceedings of the 7th international conference on Databases in Networked Information Systems
Data sorting in flash memory

Proceedings of the 27th Annual ACM Symposium on Applied Computing
A high-performance sorting algorithm for multicore single-instruction multiple-data processors

Software—Practice & Experience
Sort-sharing-aware query processing

The VLDB Journal — The International Journal on Very Large Data Bases
SliceSort: efficient sorting of hierarchical data

Proceedings of the 21st ACM international conference on Information and knowledge management
Comparison based sorting for systems with multiple GPUs

Proceedings of the 6th Workshop on General Purpose Processor Using Graphics Processing Units
Memory footprint matters: efficient equi-join algorithms for main memory data processing

Proceedings of the 4th annual Symposium on Cloud Computing

Quantified Score

Hi-index	0.00

Visualization

Abstract

Most commercial database systems do (or should) exploit many sorting techniques that are publicly known, but not readily available in the research literature. These techniques improve both sort performance on modern computer systems and the ability to adapt gracefully to resource fluctuations in multiuser operations. This survey collects many of these techniques for easy reference by students, researchers, and product developers. It covers in-memory sorting, disk-based external sorting, and considerations that apply specifically to sorting in database systems.