The design and implementation of a log-structured file system
ACM Transactions on Computer Systems (TOCS)
SIAM Journal on Computing
Journal of Algorithms
Algorithms and data structures for flash memories
ACM Computing Surveys (CSUR)
FlashDB: dynamic self-tuning database for NAND flash
Proceedings of the 6th international conference on Information processing in sensor networks
Microhash: an efficient index structure for fash-based sensor devices
FAST'05 Proceedings of the 4th conference on USENIX Conference on File and Storage Technologies - Volume 4
A flash-memory based file system
TCON'95 Proceedings of the USENIX 1995 Technical Conference Proceedings
Avoiding the disk bottleneck in the data domain deduplication file system
FAST'08 Proceedings of the 6th USENIX Conference on File and Storage Technologies
Design tradeoffs for SSD performance
ATC'08 USENIX 2008 Annual Technical Conference on Annual Technical Conference
Proceedings of the VLDB Endowment
Online maintenance of very large random samples on flash storage
Proceedings of the VLDB Endowment
Proceedings of the 14th international conference on Architectural support for programming languages and operating systems
FlashLogging: exploiting flash devices for synchronous logging performance
Proceedings of the 2009 ACM SIGMOD International Conference on Management of data
FAWN: a fast array of wimpy nodes
Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles
Lazy-Adaptive Tree: an optimized index structure for flash devices
Proceedings of the VLDB Endowment
Cheap and large CAMs for high performance data-intensive networked systems
NSDI'10 Proceedings of the 7th USENIX conference on Networked systems design and implementation
ChunkStash: speeding up inline storage deduplication using flash memory
USENIXATC'10 Proceedings of the 2010 USENIX conference on USENIX annual technical conference
Tree indexing on solid state drives
Proceedings of the VLDB Endowment
FlashStore: high throughput persistent key-value store
Proceedings of the VLDB Endowment
SILT: a memory-efficient, high-performance key-value store
SOSP '11 Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles
Scalable real time data management for smart grid
Proceedings of the Middleware 2011 Industry Track Workshop
PBFilter: A flash-based indexing scheme for embedded systems
Information Systems
Workload analysis of a large-scale key-value store
Proceedings of the 12th ACM SIGMETRICS/PERFORMANCE joint international conference on Measurement and Modeling of Computer Systems
A study of space reclamation on flash-based append-only storage management
DASFAA'12 Proceedings of the 17th international conference on Database Systems for Advanced Applications
Using vector interfaces to deliver millions of IOPS from a networked key-value storage server
Proceedings of the Third ACM Symposium on Cloud Computing
A practical concurrent index for solid-state drives
Proceedings of the 21st ACM international conference on Information and knowledge management
MemC3: compact and concurrent MemCache with dumber caching and smarter hashing
nsdi'13 Proceedings of the 10th USENIX conference on Networked Systems Design and Implementation
Extending SSD lifetime in database applications with page overwrites
Proceedings of the 6th International Systems and Storage Conference
Hotness-aware buffer management for flash-based hybrid storage systems
Proceedings of the 22nd ACM international conference on Conference on information & knowledge management
Ranking and new database architectures
Proceedings of the 7th International Workshop on Ranking in Databases
Understanding and mitigating the impact of load imbalance in the memory caching tier
Proceedings of the 4th annual Symposium on Cloud Computing
Exploring storage class memory with key value stores
Proceedings of the 1st Workshop on Interactions of NVM/FLASH with Operating Systems and Workloads
LLAMA: a cache/storage subsystem for modern hardware
Proceedings of the VLDB Endowment
MILo-DB: a personal, secure and portable database machine
Distributed and Parallel Databases
Hi-index | 0.00 |
We present SkimpyStash, a RAM space skimpy key-value store on flash-based storage, designed for high throughput, low latency server applications. The distinguishing feature of SkimpyStash is the design goal of extremely low RAM footprint at about 1 (± 0.5) byte per key-value pair, which is more aggressive than earlier designs. SkimpyStash uses a hash table directory in RAM to index key-value pairs stored in a log-structured manner on flash. To break the barrier of a flash pointer (say, 4 bytes) worth of RAM overhead per key, it "moves" most of the pointers that locate each key-value pair from RAM to flash itself. This is realized by (i) resolving hash table collisions using linear chaining, where multiple keys that resolve (collide) to the same hash table bucket are chained in a linked list, and (ii) storing the linked lists on flash itself with a pointer in each hash table bucket in RAM pointing to the beginning record of the chain on flash, hence incurring multiple flash reads per lookup. Two further techniques are used to improve performance: (iii) two-choice based load balancing to reduce wide variation in bucket sizes (hence, chain lengths and associated lookup times), and a bloom filter in each hash table directory slot in RAM to disambiguate the choice during lookup, and (iv) compaction procedure to pack bucket chain records contiguously onto flash pages so as to reduce flash reads during lookup. The average bucket size is the critical design parameter that serves as a powerful knob for making a continuum of tradeoffs between low RAM usage and low lookup latencies. Our evaluations on commodity server platforms with real-world data center applications show that SkimpyStash provides throughputs from few 10,000s to upwards of 100,000 get-set operations/sec.