The inverse scaling prize aims to find important tasks where larger language models do worse. The first round winners consisted of:

Note that all four of these tasks asked the model to pick between two predefined options.

) has argued that two out of these four tasks demonstrate U-shaped scaling:* as you scale your language model, while performance goes down at first, it eventually recovers with enough scale (that is, with the standard language modelling objectives and no further finetuning). 

The second round of the inverse scaling prize closed on October 22, and winners are expected to be announced soon. Will at least half of the winners demonstrate U-shaped scaling by 2025?

This question resolves yes if at least half of the round two winners (rounded up) demonstrate U-shaped scaling by Janaury 1st, 2025. If no winners are announced by Jan 1st 2023, this question resolves N/A. 