Oxide: Control plane data storage requirements
The document specifies requirements for a control plane data storage system for Oxide, highlighting the need for high availability, scalability, security, and a thorough evaluation of NewSQL technologies.
Read original article
The document outlines the requirements for a control plane data storage system for Oxide, emphasizing the need for a highly available and horizontally scalable solution. It details the types of data that must be stored, including instances, virtual disks, networking resources, and user information. Key requirements include durable read/write/delete capabilities, strong consistency, high availability, and hands-off operation to minimize human intervention. The system should also support security measures such as mutual authentication and data encryption. The document discusses potential technologies for implementation, focusing on NewSQL systems like CockroachDB and Yugabyte, while ruling out traditional RDBMS and NoSQL systems due to their limitations in scalability and operational complexity. The evaluation process for selecting a suitable technology involves extensive testing and analysis of performance under various workloads and failure conditions. The document concludes with a call for a thorough investigation of existing technologies before considering building a custom solution.
- The control plane data storage system must support high availability and horizontal scalability.
- Key data types include instances, virtual disks, networking resources, and user information.
- Security features such as mutual authentication and data encryption are essential.
- NewSQL systems are preferred for their cloud-native capabilities and support for ACID transactions.
- A comprehensive evaluation process is necessary to ensure the selected technology meets operational requirements.
Related
Is an All-in-One Database the Future?
Specialized databases are emerging to tackle complex data challenges, leading to intricate infrastructures. A universal, all-in-one database remains unfulfilled due to optimization issues and unique challenges of different database types.
Making Machines Move
Fly.io has introduced "Clone-O-Mat," a system for managing stateful applications that allows asynchronous cloning of storage volumes, reducing downtime and improving data integrity during migrations on its cloud platform.
Whither CockroachDB?
CockroachDB is moving to a proprietary licensing model in November 2024, prompting Oxide to self-support on version 22.1, which will convert to Apache 2.0 by May 2025.
Just use Postgres
Ethan McCue recommends Postgres as the default database for web applications due to its reliability and ease of use, contrasting it with SQLite and various NoSQL and proprietary databases.
5 comments[1]: https://rfd.shared.oxide.computer/rfd/0110 [2]: https://rfd.shared.oxide.computer/rfd/0508
That's an hidden pearl of wisdom right there.
This CockroachDB issue is an example of their public decision making process.
RFDs described in detail here: https://rfd.shared.oxide.computer/rfd/0001
I was curious to know what type of tool this is built on. Sounds like it's a static site generator that can process AsciiDoc, but I don't have any more detail than that. The links to GitHub repos in section 5 were all broken - maybe it's private?
Related
Is an All-in-One Database the Future?
Specialized databases are emerging to tackle complex data challenges, leading to intricate infrastructures. A universal, all-in-one database remains unfulfilled due to optimization issues and unique challenges of different database types.
Making Machines Move
Fly.io has introduced "Clone-O-Mat," a system for managing stateful applications that allows asynchronous cloning of storage volumes, reducing downtime and improving data integrity during migrations on its cloud platform.
Whither CockroachDB?
CockroachDB is moving to a proprietary licensing model in November 2024, prompting Oxide to self-support on version 22.1, which will convert to Apache 2.0 by May 2025.
Just use Postgres
Ethan McCue recommends Postgres as the default database for web applications due to its reliability and ease of use, contrasting it with SQLite and various NoSQL and proprietary databases.