Will Google's Gemini [Ultra] LLM be released in 2023?
resolved Dec 31

released = generally available to the public e.g. via Bard or something like it

Dec 6th update: Gemini Ultra model is what this means. Not pro or nano

Get Ṁ600 play money

🏅 Top traders

#NameTotal profit
Sort by:

Send me your ratings. I've explained my position already below.

How long before someone begs an admin to change the answer?

I'll take my leave, you've all made me feel quite embarrassed for having ever created an account here.

predicted YES

@RH you could've easily resolved this to N/A.

@RH Are you still active? Seems you have gone MIA.

predicted YES

This market popping up in my FYP:

Here We Go Again GIF

I am not betting in this market. I have not bet in any Gemini markets:

This should absolutely be N/A’ed. This is nonsense resolution criteria.

predicted NO

@benshindel Should all the Gemini markets be N/Ad in your view? They all had the same ambiguity. And do you mean that mods should take action, or just that creators should N/A them?

@chrisjbillington I’m speaking only about this market. Google released an LLM. It was called Gemini. The fact that its performance is not comparable to GPT-4 is not relevant. The fact that they plan to release part of it in the next year is irrelevant. The market criteria changed AFTER it should have apparently resolved. Someone should N/A it; either the creator or a mod.

What happened here?

predicted NO

@Traveel It turned out Gemini was three models, only one of which was released. The creator had not anticipated this possibility or specified how the market would resolve in this case. Many bettors assumed it counted, however, and bet the market up on the news, before the creator clarified that this market was about the more powerful model that had not been released yet, and updated the description/title accordingly. There was much disagreement about whether this was OK or not.

predicted YES

@Traveel That's a cool picture of me losing 1000 mana.

predicted YES

I'm looking to get my balance out - I have no expectation this will resolve yes. Open for the taking if anyone wants it (4% very low risk ROR).

predicted YES

The fact that this market is allowed to exist in its current state despite moderator and admin attention makes all of Manifold seem like a pretty big joke to me tbqh.

predicted YES

I'd prefer a market with a 90% chance of YES not have its resolution conditions re-specified such that this chance falls to 8%.

@moorehousew Well his point was that it never should've hit 90% in the first place, which while a bit silly considering his very loose rules being his own fault, this does match a bit with reality. The GPT-4 competitor is what people cared about, not some intermediary step.

And perhaps more importantly, Google's actions match this interpretation! They canceled in-person events and just put up a website.

OP's take seems reasonable as a response to the news, if not ideal given that so many people read the rules and thought something else.

predicted YES

@Domer I think that if OPs take tracked best with reality and how the question should be resolved in spirit, the market would have actually reflected that fact, like, at all. It seems like the market had already "resolved" any ambiguity. If one's interpretation of that ambiguity turns a market from 98% YES to 94% NO, it seems a priori ruled out as intersubjectively wrong. But that's just my opinion.

predicted NO

Everyone knew what Gemini was supposed to mean. They said it was their most capable model, not a series of models. If they just dropped a random new product like a pixel phone called “Gemini” it would also fulfill the loosely defined market criteria (i don’t remember if LLM was even in the name). I bet on no when it skyrocketed because a “YES” resolution would obviously be more controversial than a “NO”. Just a shitty situation, but I think the people who voted YES after the reveal are the ones that bet on a “technicality” just as much as I did.

predicted YES

@adjo No, Google literally said that it was going to come in multiple sizes "just like PaLM 2", which also had its largest model withheld. PaLM 2 Unicorn was already GPT-4 competitive back months ago.


predicted NO

@Mira despite that info being out there, you were the only one here who knew about it (certainly the creator didn't), and Google nonetheless talked about the best model as "Gemini", which is how we all referred to it.

predicted YES

@chrisjbillington The Google chatbot market /agentydragon/will-google-search-include-a-chatbo had 2105 traders and one of the spikes was Google I/O where everybody was tuning into the livestream and reading the blog.

So don't say I was the only one. There's a specific spike in a specific market that corresponds to people reading this exact article which includes mention of Gemini having multiple sizes.

predicted NO

@Mira No, I'm doubling down on that, people may have read the same article but they didn't notice. Nobody else brought up that fact or claimed they were betting based on it. I legit think you are the only person participating in this market who was aware of it.

As someone with massive gains on the no side, I think it would make sense for it to resolve N/A. But at the same time, brubsby's markets resolved yes when it should have been N/A. So I don't think one can complain about one and not the other.

predicted NO

And on the topic of the spirit of the market, I don't anybody was thinking about a demo version when they were betting on this for the past year.

predicted NO

@ItsMe Yeah IMO the ideal solution would be if Manifold had a tool to rewind markets to before trading was done under unclear circumstances, at which point both creators could clarify their market criteria and then re-open trading so everyone understood what they were trading on.

But because this isn't possible, any resolution now screws over people who were betting under reasonable assumptions. I think both RH and Brubsby are reasonable market creators acting in good faith, and yet they decided to do opposite things in this situation. I feel like even the most ardent team-yes and team-no supporters should be able to take the outside view and conclude that when two reasonable market creators can resolve the same question in opposite directions, then N/A is clearly the best option.

@ItsMe the market is fulfilled in spirit. There was no qualification of what Gemini meant. Even Gemini Nani would be enough. The author made up that it would need to be a specific version.

predicted NO

@MP The author has not bet on this market and is very unlikely to be making up what it is about.

There was no qualification of what Gemini meant because they did not know that was context that could even be put in (with any likelihood of usefulness -- sure, Metaculus probably would have known to put something in there and done it, but they also didn't have time to make this market likely because of that).

predicted NO

@RobertCousineau we were told months ago that gemini would be a family of models, and many new LLMs have been released in several different sizes, so not specifying "the largest version" or "the most capable version" just seems like carelessness. But it's also pretty clear that's what the author meant.

More related questions