Will someone beat Victor Taelin's A::B prompting challenge this year?
21
220
515
resolved Apr 7
Resolved
YES
Get Ṁ200 play money

🏅 Top traders

#NameTotal profit
1Ṁ126
2Ṁ122
3Ṁ85
4Ṁ73
5Ṁ39
Sort by:
bought Ṁ50 YES

From the GitHub gist

GPTs aren't, and will never be able to perform long symbolic manipulations and, thus, long reasoning processes

Whilst I understand GPTs seem to be having trouble with this particular problem (though I have not much doubt someone will claim the prize), this seems like a really weird thing to say. GPTs are fantastic at continuing patterns, mechanically substituting tokens seems right up their alley.

I feel pretty confident that the failures are just because of a lack of this sort of thing in their training data. Not this problem specifically, but the idea of long chains of manipulating symbols according to rules.

@chrisjbillington Possible. Someone fine tuned gpt 3.5 to near perfection with 2000 examples.