Will I build a project with Litestream in 2022?
Jan 1, 2023
Litestream popped up on my radar again after  made the rounds -- it's a replication layer for sqlite that lets you replicate data to S3/GCS/etc. I've been aware of Litestream since at least May, 2021 (), and it's been on my shortlist of projects to take for a spin. Resolves to YES if I build "something" using Litestream, and get it to the point where I publish it publicly on Github and/or write a blog post about it. Reasons for YES: Litestream makes it easier for a project to scale-to-zero, so I'm more likely to build something that needs a persistent DB. Also, the idea of using sqlite in this way is aesthetically pleasing to me. Reasons for NO: I don't have a specific idea of anything to build picked out. : https://fly.io/blog/all-in-on-sqlite-litestream/ : https://benjamincongdon.me/blog/2021/05/31/Recently-May/#assorted-links
On the "hobby project" end of things, SQLite is simpler to setup and administer than Postgres since it runs in-process, and you don't have to configure/tune a DB node. Also, with Litestream you can continuously replicate writes from the SQLite write-ahead log to a cloud storage provider, so that if your application node goes down, you don't lose data. On the "real app" side of things, as Undox mentioned, there are some appealing performance characteristics for read-heavy workloads. Write-heavy workloads will fall over pretty quickly, IIUC. Postgres would still be my generic first choice of a RDBMS for something I'd like to productionize. ("Nobody gets fired for choosing Postgres"? 😆)
From what I see, the benefit is if you have a lot of reads and not too many writes it scale up much easier. You could have 1000 replica nodes for example, with very low latency read DB access on all those nodes. Might be good for MM since a lot of people look at markets, but trades are rarer (trades are limited by your funds!). Also means you can push everything to the edge, so you can serve me completely from my region.
Naive question: What's the benefit of SQLite over PostgresQL and MySQL? Obviously, we've been thinking a decent amount about our database https://manifold.markets/Austin/what-database-will-manifold-be-prim ; but we're mostly have only ever used Firestore, so curious to hear about the other options.