released = generally available to the public e.g. via Bard or something like it
Dec 6th update: Gemini Ultra model is what this means. Not pro or nano
1,000
3.00
@benshindel Should all the Gemini markets be N/Ad in your view? They all had the same ambiguity. And do you mean that mods should take action, or just that creators should N/A them?
@chrisjbillington I’m speaking only about this market. Google released an LLM. It was called Gemini. The fact that its performance is not comparable to GPT-4 is not relevant. The fact that they plan to release part of it in the next year is irrelevant. The market criteria changed AFTER it should have apparently resolved. Someone should N/A it; either the creator or a mod.
@Traveel It turned out Gemini was three models, only one of which was released. The creator had not anticipated this possibility or specified how the market would resolve in this case. Many bettors assumed it counted, however, and bet the market up on the news, before the creator clarified that this market was about the more powerful model that had not been released yet, and updated the description/title accordingly. There was much disagreement about whether this was OK or not.
@moorehousew Well his point was that it never should've hit 90% in the first place, which while a bit silly considering his very loose rules being his own fault, this does match a bit with reality. The GPT-4 competitor is what people cared about, not some intermediary step.
And perhaps more importantly, Google's actions match this interpretation! They canceled in-person events and just put up a website.
OP's take seems reasonable as a response to the news, if not ideal given that so many people read the rules and thought something else.
@Domer I think that if OPs take tracked best with reality and how the question should be resolved in spirit, the market would have actually reflected that fact, like, at all. It seems like the market had already "resolved" any ambiguity. If one's interpretation of that ambiguity turns a market from 98% YES to 94% NO, it seems a priori ruled out as intersubjectively wrong. But that's just my opinion.
Everyone knew what Gemini was supposed to mean. They said it was their most capable model, not a series of models. If they just dropped a random new product like a pixel phone called “Gemini” it would also fulfill the loosely defined market criteria (i don’t remember if LLM was even in the name). I bet on no when it skyrocketed because a “YES” resolution would obviously be more controversial than a “NO”. Just a shitty situation, but I think the people who voted YES after the reveal are the ones that bet on a “technicality” just as much as I did.
@adjo No, Google literally said that it was going to come in multiple sizes "just like PaLM 2", which also had its largest model withheld. PaLM 2 Unicorn was already GPT-4 competitive back months ago.
https://blog.google/technology/ai/google-io-2023-keynote-sundar-pichai/
@Mira despite that info being out there, you were the only one here who knew about it (certainly the creator didn't), and Google nonetheless talked about the best model as "Gemini", which is how we all referred to it.
@chrisjbillington The Google chatbot market /agentydragon/will-google-search-include-a-chatbo had 2105 traders and one of the spikes was Google I/O where everybody was tuning into the livestream and reading the blog.
So don't say I was the only one. There's a specific spike in a specific market that corresponds to people reading this exact article which includes mention of Gemini having multiple sizes.
@Mira No, I'm doubling down on that, people may have read the same article but they didn't notice. Nobody else brought up that fact or claimed they were betting based on it. I legit think you are the only person participating in this market who was aware of it.
As someone with massive gains on the no side, I think it would make sense for it to resolve N/A. But at the same time, brubsby's markets resolved yes when it should have been N/A. So I don't think one can complain about one and not the other.
@ItsMe Yeah IMO the ideal solution would be if Manifold had a tool to rewind markets to before trading was done under unclear circumstances, at which point both creators could clarify their market criteria and then re-open trading so everyone understood what they were trading on.
But because this isn't possible, any resolution now screws over people who were betting under reasonable assumptions. I think both RH and Brubsby are reasonable market creators acting in good faith, and yet they decided to do opposite things in this situation. I feel like even the most ardent team-yes and team-no supporters should be able to take the outside view and conclude that when two reasonable market creators can resolve the same question in opposite directions, then N/A is clearly the best option.
@ItsMe the market is fulfilled in spirit. There was no qualification of what Gemini meant. Even Gemini Nani would be enough. The author made up that it would need to be a specific version.
@MP The author has not bet on this market and is very unlikely to be making up what it is about.
There was no qualification of what Gemini meant because they did not know that was context that could even be put in (with any likelihood of usefulness -- sure, Metaculus probably would have known to put something in there and done it, but they also didn't have time to make this market likely because of that).
@RobertCousineau we were told months ago that gemini would be a family of models, and many new LLMs have been released in several different sizes, so not specifying "the largest version" or "the most capable version" just seems like carelessness. But it's also pretty clear that's what the author meant.