A Local-First Case Study

We built LegendKeeper using Yjs! (It's a mapping app as well, but fantasy). Ended up rolling our own sync server to handle large scale multiplexing, as we have D&D game-masters with 25,000+ documents to manage. (I don't know how they do it, tbh!)

We opt for the central server as a super-peer and use the Yjs differential update system to avoid loading docs in memory for too long. While there are many things about local-first that are a huge pain in the ass, the UX benefits are pretty huge. The DX can be nice too! Getting to focus on product and not on data transit (once you've got a robust sync system) is pretty sweet. The first 4 weeks of launching our Yjs-based system was rough though; lots of bugs that virally replicated between peers. It requires a really paranoid eye for defensive coding; after several years, we have multiple layers of self-healing and validation on the client.

It's funny how this newfangled "local first" thing is what us old fogies used to call just "applications."

I get that there is networking and integration that a modern application will typically need to do (depending on its core purpose), and syncing state to and from servers is a special concern (especially if conflict management is necessary) that native desktop applications rarely had to do in years past.

But at the end of the day, it sure does feel like we've come full circle. For a long time, every single application was "local first" by default. And now we're writing research papers and doing side POCs (I'm speaking generally, nothing to do with the author or their article) trying to figure out how to implement these things.

> Architecturally, Y-Sweet acts as a bus: clients connect to the Y-Sweet server rather than directly to each other. Whenever a client connects or makes changes, it syncs its local document with the Y-Sweet server. Y-Sweet merges the client’s document into its own copy, saves it to S3 and broadcasts updates to other clients. Since CRDTs are guaranteed to eventually converge on the same state, at the end of this process all clients have the same document.

I had thought that the advantage of CRDTs was you do not need a centralized server and that if you do have a central server Operational Transforms are easier. Am I missing why CRDTs are used here?

I love how the map automatically updates based on the places typed in the editor. A great visual aid to a text-based workflow.

I got confused by this comment though:

  > To determine when to re-render, “reactive” frameworks like Svelte and Solid track property access using Proxies, whereas “immutable” frameworks like React rely on object identity.

I thought React was just as reactive as all the other JS frameworks, and that the state/setState code would look similar.

For those interested in this stack,I have been working on an Obsidian.md plugin called Relay that makes it fully collaborative using yjs and y-sweet.

We also use a hub and spoke model, but we still rely on a central server (pocketbase) for management user flows like authorization and billing.

Obsidian is such a fantastic editor, and it fits so naturally with local-first collaboration.

Inability to scale with sustained usage (1+ person*year of data) is the fatal problem with this category in existing approaches. Root of this is primarily the “partial sync problem” - when dataset outgrows both the memory and compute resources available in the client device (which is unreliable, not under your control to make reliable, and resource constrained - and not everybody has the latest giga device), you have to somehow decide which working set to replicate. If the app structure is a graph and involves relational queries, this is undecidable without actually running the queries! If the app structure is constrained to topic documents, you still have to choose which topics to cache and it still falls over at document sizes easily seen in prolonged solo use let alone use at an enterprise which market is necessary to justify VC investment. All this in an increasingly connected world where the cloud is 10ms away from city/enterprise users (the $$$) and any offline periods (subway etc) resolve quickly, so a degraded offline mode (letting you edit whatever you were last looking at w/o global relational query etc) is often acceptable. Oh, and does your app need AI?

Change my view!

Ink & Switch is like the Medici family of the Internet era. Medici's funded the piano, and I&S is funding local-first. I love what they do.

Would be perfect is somehow it could work without S3. Would be awesome if the internet could just work in p2p mode out of the box, just some JS and HTML and you have 2 computers talking to each other collaborating on a doc without the need of a server (or S3)

This is great! I was quite excited to see Ink & Switch’s Embark and now this…

Jake makes creating a local-first multiplayer app seem so simple.

Is there some version of local-first that doesn't require a webserver, but does seamlessly sync state to a consumer cloud service like Google Drive? I'd love to write apps that have all the speed and portability of local apps, but the data isn't tied to a specific device. It seems like it would be feasible to have a large JSON blog background synced to a cloud file service after some threshold of accumulated change or time.

On an unrelated note, I'm getting local-first case studies everytime I leave a wifi space since my cell phone is still suffering from the Verizon outage (Los Angeles area). My conclusion has been I can read stuff like calendar invites (Google), I can save notes on stuff to do in Trello, but I cannot queue up IMs to send in GHC.

Awesome stuff. Reminds me of the offline-first movement from ~10 years ago.

I'm currently looking into TinyBase to make working with high latency decentralised services more bearable.

Would be cool if there were a better comparison of the different solutions for storage and sync, with features, good use-cases, etc.

The future is local-first. IF you haven't yet learned about it and how to break-out of the expensive cloud/serverless cage, this is a good start: https://localfirstweb.dev/

I've been hanging local-first circles but haven't made to switch to write anything with it yet. Is there a typical stack people recommend to get started? I'm not really sure where to start, especially in terms of backend.

This is awesome! Since you’re using CRDTs, do you have any plans to make it collaborative? I would find it useful to build an itinerary with multiple people

I'm excited by these explorations of dynamic components in rich text. Notion has popularized the idea of documents with rich blocks, where the blocks can provide dynamic behavior to traditional documents. And now we're seeing types of inline elements that also provide more structure to rich text. Those location routes seem something that I'd use myself.

Great write-up, I've been looking for a solution like this for adding syncing to my local-app!

I love that it's document stored in S3, and it's probably going to be way cheaper than if hosted elsewhere in a database. Can't wait to try it out soon

This is a gorgeous little app, and I'm really happy to see someone working on rich linking of map data and content - it feels like a really under-developed area of the whole modern maps ecosystem

I've been pretty impressed with https://roadtrippers.com for this purpose. (Not local-first, I guess)

That’s a nice font