'AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference' comes out on 2024/9/24. I will consider it out of date if >50% of the things it claims AI can't do can be done by then.
Protocol (rough draft, open to improvements):
Have LLM(s) extract concrete claims of what AI can't do from the book
Randomly sample a few (10-20?)
Have 2 people independently assess whether AI can do them (YES/NO) and give half credit in case of disagreement
1,000
3.00If anyone wants to do the work to resolve this market, I'll give them some mana – just post or dm me your price. Otherwise, I'll try to resolve it myself this weekend.
@JavierPrieto if your first reaction to the notion of reading a book (ostensibly within your fields of interest) is "what is the financial opportunity cost" then I genuinely, sincerely feel bad for you. but ok
@Radicalia this market isn't for "untitled median AI policy/governance book". digesting thousands of titles within an arbitrary field of work in order to extract common assumptions would be an excellent application for an LLM