Pick Your Distributed Poison

The article discusses the complexities and challenges of distributed systems, particularly focusing on the concept of eventual consistency, which can lead to eventual inconsistency. The author emphasizes that rather than seeking to eliminate inconsistencies, one should accept and choose the type of inconsistencies they can tolerate. The piece explores various strategies for managing distributed systems, such as bootstrapping, which involves regularly destroying and recreating systems to avoid cycles, albeit at the cost of stability and resource management. The author expresses a preference for reproducible and bootstrappable systems, which, while minimizing the need for working memory, come with their own set of challenges, including the inability to detect leaks and manage long-lived properties. The discussion highlights the paradox of striving for safety and liveness in system design while often building systems that rely less on these properties. The author concludes with a personal reflection on their approach to system design, likening it to savoring a chosen "poison."

- Eventual consistency in distributed systems can lead to eventual inconsistency.

- Accepting and managing tolerable inconsistencies is crucial in system design.

- Bootstrapping can help avoid cycles but may introduce instability and resource leaks.

- Reproducible and bootstrappable systems are preferred for their simplicity but have limitations.

- The design of distributed systems often involves a trade-off between safety, liveness, and adaptability.