where'd all the liquidity go?

"Publicly available training data" as written disqualifies essentially all modern LLMs. The list of training data sources is not public, and even if it was I believe all modern training sets include non-public data. Taken literally the probability should be ~0%. Of course most traders disagree. As far as I can tell the first time anyone asked about that wording was on 9/8, nearly two years after the markets were opened. Was everyone else betting that a fully public-data frontier model would suddenly exist? For this reason I would vote against an N/A resolution, it seems clear that the intent was not to exclude frontier LLMs.
I'm broadly in favor of trusting OpenAI's claim that no human work was stolen, since it would be nearly impossible to disprove. However if that part is a crux still I would be in favor of letting the market remain open for another ~6 months to see if additional information comes out.
So at this point I'd lean towards resolving YES now or leaving the market open to wait for this solution to be confirmed, noting that the "publicly available training data" is nonsensical.
I don't have any stake in this market, or the other Millennium Prize markets by this creator.

@wasabipesto I basically agree with that on clause 2, which is the primary ambiguity. But note that it can't resolve yes yet because of clause 1 has not occurred yet, and it's a question of when

A peer‑reviewed paper in a recognised scientific journal or an officially accepted CMI submission

@jack You're right, I was jumping the gun there.

@wasabipesto This market was created in Oct 2024, barely a month after the release of o1-preview, the first model with CoT to go semi-public. The terms RLVR, synthetic data, and the like, had not made their way to public discourse. You may have forgotten, but back then models really were trained on public data.

Claiming that this market is "obviously 0%" is using the knowledge from the two intervening years to retcon the market. As I point out below, on the belief system of many on this site, there's no reason why an open-data model couldn't resolve a MP by 2030.

I have to say that disregarding a clear requirement of the market phrasing, while "trusting OpenAI's claim", is fundamentally unserious.

@pietrokc

back then models really were trained on public data

From the o1 system card:

The two models were pre-trained on diverse datasets, including a mix of publicly available data, proprietary data accessed through partnerships, and custom datasets developed in-house, which collectively contribute to the models’ robust reasoning and conversational capabilities.

And the Claude 3 system card from March 2024:

Claude 3 models are trained on a proprietary mix of publicly available information on the Internet as of August 2023, as well as non-public data from third parties, data provided by data labeling services and paid contractors, and data we generate internally.

I'm not sure about public discourse but I do recall that most system cards mention that both the list of sources and some of the sources themselves are proprietary.

I have to say that disregarding a clear requirement of the market phrasing, while "trusting OpenAI's claim", is fundamentally unserious.

I do think it's regrettable that we have to disregard that clause from the description, in a perfect world we would have been able to clarify this early with the creator before any major trades happened. As it stands I think this is the most reasonable path forward. Feel free to make your own market with clear rules about publicly available training data, I would be interested to see it!

As for trusting OpenAI it's not something I prefer to do, and I think there's a ~5% chance they're lying about zero human work being involved in the process, but the majority of the relevant information for this market relies on OpenAI being a reliable source. I'm not sure you could have a functional market on this topic without trusting OpenAI on at least some level. Would you prefer the market only accept this solution if a trusted third party is able to thoroughly review and confirm that no new mathematical insights were provided to the agents? I think the chances of that review happening are very low.

do you have opinion on how the other criteria: “no new mathematical insights may be added by people.” is affected by how the researchers told the models to use the new supporting theorem as the starting point for their investigation?

I'm trying to understand what is the status of this market. As @capybara points out, the clause

> Human assistance is limited to setting up the architecture,
> curating publicly available training data and verifying formatting

means that the Navier-Stokes solution doesn't count, even without getting into considerations of whether OpenAI stole people's work.

But it means more than that; it means that now Navier-Stokes can NEVER count for a positive resolution to this market, since the solution is now in the training data of every model.

So, now this market can only resolve positively if ANOTHER millennium problem is solved, and with a model trained only on publicly available data. But there do not exist any such models with a realistic chance of being the first to solve it. So on this read, this market should be close to 0%. What's going on?

I think people might be hoping that if they all collectively misinterpret the market it will resolve in their favour.

@pietrokc right, that condition is basically impossible to safisfy. There hasn't been any strong model trained only on public data since before this market opened, and probably never will be.

@mods, the market creator's account is deleted, so I guess you guys need to decide how to deal with this... either the criteria should be changed, or some big warning added to the description.

@pietrokc that clause is under the AI autonomy section, and if you read it as "what are humans allowed to contribute to training" then no LLM would count because humans have far more involvement in training LLMs than just setting up the architecture and curating training data.

I think the better read is that the intent is to match "no new mathematical insights may be added by people." Otherwise this market is nonsense

@jack training LLMs (or any NN model) is pretty much set things up and run the training. The clause seems reasonable. The requirement for public data could be understood from needing to ensure that part of the solution wasn’t in the training set.

@jack Do you wanna claim the "under review" rewards for this mod ticket? Or is this resolved?

@Quroe nah someone else should take this. Imo this clause is so bad a conflict between the intent and the literal text that my instinct is to N/A

@jack Should we close the market to deliberate so people aren't betting on the market's interpretation instead of the state of the world?

Has condition #1 even been met at time of comment?

@Quroe btw OP made 2 or three other markets just like this with different end times. It’s worth making the decision consistent across them, for example if you close as N/A or add details clarifying resolution.

@jack I disagree that there is a conflict between the intent and the text.

You just need to look around this website to see hundreds of people whose core belief is "AI to the moon", "infinite intelligence will be everywhere soon" and such. Under such a belief system, it's completely plausible that some open data model would resolve a millennium problem, with humans only "setting up the architecture and training data".

If I'm being honest I think almost everyone betting on this market, in either direction, including myself, was morally wrong. The thing that actually happened was weird and unexpected by all sides. We did not get "massive intelligence explosion solves MP easily from first principles", and we did not get "AI struggles to do more than trivial math, and in fact no MPs get solved by 2030 at all, by AI or otherwise", which I get the feeling were the two modes of opinion.

Unfortunately for the YES side, the market phrasing is very clear. The weird edge case that happened (so far) clearly lands on the NO side.

@pietrokc

everyone betting on this market, in either direction, including myself, was morally wrong

At the risk of getting lost in the weeds, my knee jerk reaction was to zero in on the wording here. The traders were morally wrong?

@Quroe Well, yes, I mean, isn't that the most common thing in the world? John Manifold bets yes or no with some theory in mind of what will happen, a theory with a lot more implicit details than just "yes" or "no". Then the market resolves John's way, but for completely different reasons than John expected. The intellectually honest position is that John was actually wrong in that case, even though he won.

I'm using the word "morally" to mean "in spirit", as is usual in math, if that's what's confusing.

@pietrokc Oh, you're talking about a Gettier Problem, right? Something about if knowledge is a "justified true belief", and how we can 'oops' into the right answer sometimes?

@pietrokc And yes, I was confused about your definition of "morally". If that's how you're using it, that clears things up.

Limit order up

Why does the bot fill my limit orders bit by bit 😔.

Each time I place a limit order I get spammed with notifications.

@Velaris hehe