
In the spirit of what Gary Marcus says here:
https://twitter.com/GaryMarcus/status/1640029885040132096?s=20
🏅 Top traders
# | Name | Total profit |
---|---|---|
1 | Ṁ127 | |
2 | Ṁ32 | |
3 | Ṁ22 | |
4 | Ṁ15 | |
5 | Ṁ11 |
People are also trading
Resolving to No because I think the question as strictly stated is not true — we have not proven it conclusively yet.
Two weeks left on this, I would argue these two are relatively strong evidence of this? What do you think?
https://www.technologyreview.com/2023/12/14/1085318/google-deepmind-large-language-model-solve-unsolvable-math-problem-cap-set/
Does this count? I don't think so personally
https://www.deepmind.com/blog/alphadev-discovers-faster-sorting-algorithms
(and it looks pretty simple so it doesn't show the ability, people just didn't try optimizing the asembly code much or at all)
They can solve a novel reverse-engineering problem(pg. 119), build model graphs of an environment they explore(pg. 51), and match human performance on a sample of LeetCode problems posted after GPT-4's pretraining period ended(pg. 21):
[2303.12712] Sparks of Artificial General Intelligence: Early experiments with GPT-4 (arxiv.org)
If none of the examples in that paper convince you they can already form models of things, infer facts from the model, and solve novel(if relatively easy) problems, I'm not sure what would.
@Mira I'm with you in spirit, but I think what Gary Marcus is looking for is something that very clearly moves beyond its training data. I believe his reasoning for why those things aren't evidence is that the solutions to those problems could potentially be the result of GPT-4 basically learning a hard-coded algorithm for that type of problem that is activated when it sees it. I don't believe this, but to disprove it, we would need a problem that is truly novel both in content and in structure and was not seen in the training data.
A new scientific discovery should definitely count imo