This is a clone of Jack Clark's prediction.
Mintaka is a multilingual Q&A dataset recently released by Amazon. SOTA at the time of writing is 31% (finetuned T5).
1,000🏅 Top traders
| # | Name | Total profit |
|---|---|---|
| 1 | Ṁ65 | |
| 2 | Ṁ59 | |
| 3 | Ṁ40 | |
| 4 | Ṁ11 | |
| 5 | Ṁ3 |
People are also trading
I haven't been following this and there's no nice benchmark page on paperswithcode, so I'm gonna resolve based on the results sections of papers citing the original Mintaka paper that are listed on GScholar.
@JavierPrieto The most recent paper I found in that search gets 53.1% using ChatGPT (see table 1). They don't say whether that beats sota but, after a cursory glance at some of the other papers, I haven't seen anyone claim higher performance, so I'm gonna go with this one and reopen if someone finds a better one.
Good paper, very skeptical of the hand-wavy nature of his claim.
Retrieval transformers are the natural fit, but human agreement was only 82% and usually 90% is way harder than lower levels. (Sigmoid curves and label error/indeterminacy.)
Is there evidence he’s good at prediction or more of a promoter?
