What database will this website be using on June 1st?
resolved Jun 1
Firestore (our current DB)
Selfhosted Postgres
Cloud Managed Postgres
Cloud hosted mysql
Ethereum Compatible Blockchain
We want to switch to a new database, because we've been experiencing some pain points with Firestore: - Our database bill has been getting quite expensive for a small startup ($1400 per month). - We have to duplicate data across documents to achieve performance goals. E.g. denormalizing user avatar URLs. - Queries are not very powerful. For example, you can't filter two fields simultaneously by the greater than operator '>'. - Related: Firebase cloud functions have extremely slow cold-start times (>5 seconds is common), and deploying new versions can take minutes. We are actively researching alternatives, but we could use your help! What database should we switch to? The rest of our tech stack is Nextjs, React, Typescript, and Tailwind. Submit your answer, or bet on which we will decide on! This is not a poll, this is a prediction market; this market will resolve to the database that is officially powering our site as of market close time (end of day PT on May 31st) (Our company is Manifold Markets, which runs this site!) Close date updated to 2022-05-31 11:59 pm
Get Ṁ600 play money

🏅 Top traders

#NameTotal profit
Sort by:
I think a simple sql database like postgres or mysql, hosted in the cloud, is probably right for you based on the small amount I know about your use case. (My job is building database engines and I'd be happy to chat about the topic!) I'm definitely betting on status quo for this market though, migrations are hard!
@jack yeah at this point, we don't have a migration already underway, so it's quite unlikely someone swoops in and subverts it. Would love to chat sometime about DB setup/switching/tradeoffs! Do you not believe in the "layer on top of postgres" stuff that eg Supabase provides? We're getting a lot of value from having auth & permissions handled by Firebase, I feel
@Austin Supabase looks like potentially a good option but I'm not familiar with it and the pros/cons of integrating things like auth into the database layer vs elsewhere in the stack. Mainly I agree with the previous comments about sql being generally better than nosql for this type of use case. And when you do need to scale, there are scalable sql database options that provide good dev productivity and good scalability - but it usually does come with tradeoffs like a less well developed ecosystem. Some are partly mysql/postgres compatible which helps.
@Austin In my experience, there are good middleware solutions that make it easy to handle authorization at the controller level. They essentially allow you to specify which route can be accessed by which user; that would be enough for 90% of the use cases. This approach is a bit more work for things like "users can only modify *their own* comments", where authorization depends not just on the route, but also on the resource that is accessed. This type of condition needs to be made explicit in the controller code. That said, there are usually already a bunch of data validation conditions in the controller code, so adding authorization isn't a very big deal. On the positive side, some things might get a bit easier, such as returning the correct HTTP status code for unauthorized accesses.
Supabase looks cool and is probably undervalued compared to the no-switching option.
Since @mqp advocates for Postgres this has gotten more likely IMO
Not a database, strictly speaking, but seems very cool: https://www.prisma.io/ h/t to Metaforecast: https://github.com/quantified-uncertainty/metaforecast/issues/33
Agree that it's cool. That said, this is essentially a performance optimization on top of Postgres. I think you don't need performance optimizations on top of Postgres. All you need is *basic vanilla as-simple-as-possible* Postgres, and then you can think about optimizations when there are a million users, and 20 developers in your team, and the basic Postgres starts to be a bit slow.
Thanks, I saw it mentioned on HN today
Woah, this is pretty cool! Reactive/streaming updates is one thing we'd love to have (one of the strengths of our current Firestore approach, incidentally). Plus their site design is pretty good. It has a gradient in it - my very unscientific heuristic on whether a site is "modern" haha
I'm honestly surprised by all the nosql options here. nosql usually means "drop some of the tried and trusted SQL features for increased performance". This means that you are effectively trading something that you urgently need (developer productivity) for something that you don't need at all (the ability to support 100M markets and millions of users). In my opinion, good old postgres or mysql would reduce your costs by 10x while providing all that you need. Also, don't underestimate the ecosystem around SQL. There are so many cool tools like ORMs, performance profilers, backup solutions, ... that the new contenders don't yet have. And if you ever have a problem, you can post to the sql tag on StackOverflow (625k questions) rather than the FaunaDB tag (224 questions). You'll get an instant answer.
vv Because it isn’t a great idea for your use case. Unless you host the chain to save fees and then in addition do the off chain magic to make it instant. But what a huge distraction it would be at this stage (later on maybe not).
Can't believe this didn't occur to us!
Intersects with Hasura. And more likely IMO.
/ Datalog
@mqp: I think that's fair; it's not as if our data model is especially fancy. I guess the mental model we've been operating under so far with Firestore is just "assume your client has everything a bunch of stuff memory; do your joins in client-side code" which is much simpler to think about, at the cost of reading much more data (and thus being expensive/slow)
I have to say, that's the most impressive list of users I've seen in a while. And the website's pretty slick!
I wrote a longer comment and lost it. Basically I don't really think that it's important to be working with a fancy OO model in most places in a program, so there is not really a "impedance mismatch". If I want to e.g. get all the data to display a page that shows a market and the ten most recent comments, I write a query that gets that data efficiently from the structures my database is storing, and then I hand it to a template or component. I don't need an intermediate step of "use an ORM to read the data into a canonical in-memory OO object model."
I'm curious if Postgres (or more broadly, RDBMS) supporters have any thoughts about the mismatch between tabular rows and client-side objects? https://en.wikipedia.org/wiki/Object%E2%80%93relational_impedance_mismatch My primary experience with relational DBs was with Spanner at Google; which was... tolerable? Probably not my first choice for dev happiness; I felt like it was a lot of plumbing work & mental overhead to translate from "what gets stored" to "what the user sees".
Honestly, if they even had good instructions on how to deploy this on GCP, there's a ~3% I would consider biting the bullet and managing the hosting ourselves -- or at least giving it a try and seeing what the developer experience is like.
I doubt it makes sense to stay with Firestore for long, but two months isn't very long and I expect you have better things to do.
I would use Postgres without thinking because I would assume it would be both easiest (feature-rich, tool support, people already know how to use it) and best (most optimization effort put into making it work well for the use case of "run a website that has a bunch of users posting data.")
migration hard. if anything, this is a hedge!
Thanks, I hadn't heard about this one before! Do you have any experience using AstraDB or Cassandra before? Both seem fairly niche, and I'm wondering how they stack up against more popular noSQL solutions.