SlateDB – An embedded database built on object storage
SlateDB is an embedded storage engine using object storage for high durability and scalability. It features a zero-disk architecture, tunable performance, and supports multiple readers with a single writer.
Read original article
SlateDB is an embedded storage engine designed to leverage object storage, distinguishing itself from traditional LSM-tree storage engines. It offers virtually limitless storage capacity, high durability, and simplified replication processes. By utilizing the durability of the underlying object store, SlateDB achieves an impressive durability rate of 99.999999999%. Its zero-disk architecture eliminates concerns related to disk failures and corruption, enhancing reliability. SlateDB allows for tunable performance, enabling users to optimize for low latency, cost efficiency, or high durability based on their needs. It supports a single writer and multiple readers, with mechanisms to detect and manage inactive writers. Built in Rust, SlateDB can be integrated easily with various programming languages, making it accessible for developers looking to implement an efficient embedded database solution.
- SlateDB utilizes object storage for enhanced durability and scalability.
- It features a zero-disk architecture, eliminating risks associated with disk failures.
- Users can configure performance settings to prioritize latency, cost, or durability.
- The engine supports multiple readers and a single writer, managing inactive writers effectively.
- SlateDB is built in Rust, allowing for easy integration with different programming languages.
Related
The Ultimate Database Platform
AverageDB, a database platform for developers, raised $50 million in funding. It offers speed, efficiency, serverless architecture, real-time data access, and customizable pricing. The platform prioritizes data privacy and caters to diverse user needs.
Show HN: SlateDB – An embedded storage engine built on object storage
SlateDB is an embedded storage engine for object storage systems, featuring batch writes and asynchronous operations. It is in early development, with basic functionalities and an open-source Apache 2.0 license.
Appropriate Uses for SQLite
SQLite is a lightweight, serverless database engine ideal for local data storage in embedded systems, supporting low to medium traffic websites, data analysis, and serving as a cache, but unsuitable for high concurrency.
SQLite is not a toy database
SQLite is a versatile, serverless database management system that supports advanced features, efficiently handles large datasets, and is suitable for small web applications, countering misconceptions about its limitations.
Zero-latency SQLite storage in every Durable Object
Cloudflare's Durable Objects now feature zero-latency SQLite storage, enabling synchronous SQL queries, enhancing performance, supporting scalability, allowing fast writes with data durability, and offering point-in-time recovery within 30 days.
- Many users question the "embedded" nature of SlateDB, suggesting it relies heavily on external object storage services.
- There are concerns about durability and data loss due to the in-memory write-ahead log, which could lead to lost writes if the writer fails.
- Several commenters feel that SlateDB is a thin abstraction over existing object storage solutions, lacking unique features that justify its existence.
- Users express confusion about the targeted use cases for SlateDB, questioning its advantages in various applications.
- There is a demand for additional language bindings, particularly for C++ or C, as current usage requires Rust knowledge.
15 commentsI don’t know if this was intended to be intentional funny, but there is a little ambiguity in the expression “great cost”, typically great cost means very expensive.
Very cool and useful shim otherwise :)
I've built something similar for low-cost storage of infrequently accessed data, but it uses our DBMS (MySQL) for the WAL (+ cache of hot reads), so you get proper durability guarantees.
The other cool trick to use is to use Bε-trees (a relatively recent innovation from Microsoft Research) for the object storage compaction to minimise the number of write operations needed when flushing the WAL.
I was excited at first and thought this was SQL atop S3 et al. I've jerryrigged a solution to this using SQLite with a customized VFS backend, and would suggest that as an alternative to this particular project. You get the benefit of ACID transactions across multiple tables and a distributed backend.
> NOTE
> Snapshot isolation and transactions are planned but not yet implemented.
I'm confused here, because Google says it's a db bundled with the application, but that's not really what I get from the landing page.
What problem does it solve?
Sounds like they just cancel each other out. Not sure what advantage embedding will yield here
Rust is needed to use SlateDB at the moment
Related
The Ultimate Database Platform
AverageDB, a database platform for developers, raised $50 million in funding. It offers speed, efficiency, serverless architecture, real-time data access, and customizable pricing. The platform prioritizes data privacy and caters to diverse user needs.
Show HN: SlateDB – An embedded storage engine built on object storage
SlateDB is an embedded storage engine for object storage systems, featuring batch writes and asynchronous operations. It is in early development, with basic functionalities and an open-source Apache 2.0 license.
Appropriate Uses for SQLite
SQLite is a lightweight, serverless database engine ideal for local data storage in embedded systems, supporting low to medium traffic websites, data analysis, and serving as a cache, but unsuitable for high concurrency.
SQLite is not a toy database
SQLite is a versatile, serverless database management system that supports advanced features, efficiently handles large datasets, and is suitable for small web applications, countering misconceptions about its limitations.
Zero-latency SQLite storage in every Durable Object
Cloudflare's Durable Objects now feature zero-latency SQLite storage, enabling synchronous SQL queries, enhancing performance, supporting scalability, allowing fast writes with data durability, and offering point-in-time recovery within 30 days.