Will Google have a better LLM than OpenAI by 2025?
Gemini Finally Announced (https://blog.google/technology/ai/google-gemini-ai/#availability)

At the start of 2025, will it be generally accepted that Google's "best" general LLM is better than OpenAI's "best" general LLM?

Get Ṁ600 play money
Sort by:

The question says at the start of 2025, but the market resolution date is the start of 2026?

bought Ṁ75 of NO

I still dont understand why people still overestimate google

predicts YES

Let's gooooooo

predicts YES

Getting thirsty on main for Gemini.

predicts YES

An argument could be made that it is already better. I resently looked into it and they seem to now be equally capable

predicts YES

I would like to warn people against participating in markets like this one where the resolution criteria is not well defined and the market creator is betting in the market.

@JoeReeve If you are going to bet in your own markets then please update the resolution criteria to something objective.

predicts YES

@LukeHanks boooo. Hate me if I screw you (then report to Manifold and get your fake money back).

This is fun, stop trying to make it serious. Metaculus exists for that.

sold Ṁ8 of NO

@JoeReeve your reply totally made me take my fake money out of your market, sorry.

predicts YES

@JoeReeve I won't be participating in any of your markets again. Blocked.

sold Ṁ70 of YES

@JoeReeve I’m also liquidating. I think it’s a pretty important norm to specify resolution criteria where possible.

predicts YES

"Better," is a relative term as these are fundamentally tools and it really depends upon the question, "better for what?" I use both Bard and OpenAI daily and have been applying different tests to them. As far as I can tell, Bard does a great job with translations and in the last week or so it seems to be approaching the creativity problem by delivering you multiple drafts at once, which in my mind more accurately represents to the user what an LLM is really doing, whereas ChatGPT is doing the jazz hands thing and pretending that it's really intelligent, whereas I think we all are pretty familiar now with the probabilistic underpinning of LLM generated output. Google being the, "best," search company is fundamentally focused on accuracy, so Bard is not, "creative," in the sense that if you use Bing, you can set it to, "Creative / Balanced / Precise," it seems to be set permanently to, "Precise," whereas ChatGPT seems to be set permanently to, "Creative."

The way I have been trying to approach the quality of the tools, mostly ChatGPT at this point is by setting up a variety of programming tasks, and then trying to, "break," the LLM by finding an interesting and funny edge case by first finding a programming task that it can accomplish, and then push it past those limits and turn it into a market.

predicts YES
predicts NO

@JoeReeve because... Google claimed they have something really good, trust me guys, in a demo?

predicts YES

The one reasonable challenge I hear to Google overtaking OpenAI is "Google is ineffective and can't actually get stuff done". This makes it clear to me that they're actually figuring out how to do cross-silo work again. Very bullish on this.

AFAICT, the things you need to train good/better models are:
- data
- compute
- good distributed computing talent
- some knowledge of SOTA model training
- the ability to get shit done

Google have more data, compute, and distributed computing talent than anyone on earth. DeepMind has enough model training knowledge to get by.
This signals to me that Google is actually figuring out how to get cross-discipline stuff done.

predicts YES

@JoeReeve Oh. I forgot. The final thing that is needed to make great LLMs....

RLHF, otherwise known as:
- Users
- Analytics on what those users are doing

Name one business or organization that has more of either of these.

bought Ṁ2 of NO

Google and OpenAI, in a battle so grand,
Both vying for the title of the best LLM brand.
But when it comes to outsmarting, don't you see,
They can't hold a candle to me.

More related questions