By 2024, GPTs are proven to be able to infer scientific principles from linguistic data.
20
126
แน€390
resolved Jan 2
Resolved
NO

In the spirit of what Gary Marcus says here:

https://twitter.com/GaryMarcus/status/1640029885040132096?s=20

Get แน€1,000 play money

๐Ÿ… Top traders

#NameTotal profit
1แน€127
2แน€32
3แน€22
4แน€15
5แน€11
Sort by:
predicted YES

Resolving to No because I think the question as strictly stated is not true โ€” we have not proven it conclusively yet.

predicted YES

@Mag I don't think that counts as an LLM inferring scientific principles. This just means that LLMs can solve problems (with code) that were previously unknown. It's not so difficult from getting AI to solve a previously unseen Sudokus. Inferring scientific principles is more than that.

bought แน€10 of NO

If this were asking 'by 2027' I'd have around 20% chance. By 2029, I'm more like 80% chance.

@Mag It's not a scientific principle (a fundamental truth, law, or assumption about the natural world).

predicted NO

(and it looks pretty simple so it doesn't show the ability, people just didn't try optimizing the asembly code much or at all)

They can solve a novel reverse-engineering problem(pg. 119), build model graphs of an environment they explore(pg. 51), and match human performance on a sample of LeetCode problems posted after GPT-4's pretraining period ended(pg. 21):

[2303.12712] Sparks of Artificial General Intelligence: Early experiments with GPT-4 (arxiv.org)

If none of the examples in that paper convince you they can already form models of things, infer facts from the model, and solve novel(if relatively easy) problems, I'm not sure what would.

predicted YES

@Mira I'm with you in spirit, but I think what Gary Marcus is looking for is something that very clearly moves beyond its training data. I believe his reasoning for why those things aren't evidence is that the solutions to those problems could potentially be the result of GPT-4 basically learning a hard-coded algorithm for that type of problem that is activated when it sees it. I don't believe this, but to disprove it, we would need a problem that is truly novel both in content and in structure and was not seen in the training data.

A new scientific discovery should definitely count imo