Removing Double-Logging with Passive Data Persistence in LSM-tree Based Relational Databases

Removing Double-Logging with Passive Data Persistence in LSM-tree Based Relational Databases

Explore a 15-minute conference talk from FAST '22 that addresses the critical double-logging problem in LSM-tree based relational databases. Discover a novel solution called Passive Data Persistence Scheme (PASV) that removes Write-ahead Log (WAL) in the storage engine layer. Learn about mechanisms such as passive memory buffer flushing policy, epoch-based data persistence scheme, and optimized partial data recovery process. Understand how these innovations improve system performance, increase throughput, reduce latency, save disk I/Os, and decrease recovery time. Gain insights into the implementation of a fully functional, open-sourced prototype based on Facebook's MyRocks and its evaluation results.