Understanding Performance Implications of Storage-Disaggregated Databases

Storage-compute disaggregation in databases is becoming increasingly important in cloud environments, with major companies like Amazon, Microsoft, Google, Alibaba, and Huawei adopting this architecture. This model separates compute from storage, enabling independent scaling and improved fault tolerance. A recent study presented at Sigmod 2024 explored the performance implications of storage-disaggregated databases, addressing critical performance questions that were previously obscured by the closed-source nature of these systems. The study introduced an open-source implementation of a storage-disaggregated database based on PostgreSQL and RocksDB.

The research identified three key design principles: Software-level Disaggregation, Log-as-the-Database, and Shared-Storage Design. The findings revealed that without buffering, storage disaggregation significantly hampers performance, with read and write speeds dropping drastically. However, buffering can mitigate some performance loss, particularly for reads. The Log-as-the-Database design showed improvements in performance under heavy workloads, while the introduction of multi-version pages enhanced write performance and eliminated high I/O issues during checkpointing.

Despite these advancements, challenges remain, particularly regarding the efficiency of write throughput, which only reached about 50% of single-node performance even with optimizations. The study highlighted the need for further investigation into the causes of this inefficiency, including potential issues with storage replication. Overall, the research underscores the potential benefits of disaggregated storage while acknowledging the technical challenges that need to be addressed.