What AI advance in 2023 will freak me out the most?

AI•New Year's Resolutions 2024

Ṁ177

resolved Jan 3

ResolvedN/A

ALL

10%

Good generative video models

Transformer-variant being SOTA at multimodality benchmark

35%

Multimodal model showing increased positive transfer between modalities as scale increases.

New way of training models in a decentralized way were the compute can be divided without requiring a big GPU cluster with high interconnect.

New scaling laws that are significantly better than the chinchilla paper.

Turns out we were doing regularization wrong and either a new method or just a better rule of thumb to pick hyperparamenters like weight decay just gets you significat improvements in things like math and reasoning.

Better coding models end up significatly helping with AI research.

AI designs better AI architectures or edits the weights on a net to get SOTA results.

1.1%

We find out current models have a lot more capabilities most people think and we are just bad at prompt engineering/training them with goals aligned with what we want.

New or already existing architecture(maybe even RNN?) turns out to outperform transformers in most tasks when scaled to LLM size + a few tricks.

Deepmind uses AlphaTensor or something similar to get massive speedup in some aspect of training models.

1.1%

LLM with really big context length turns out to be able to "run" small snippets of code using CoT.

0.6%

We get AGI in 2023 somehow.

In 2022 the advance that freaked me out the most was CoT (I am writing this in September 2022).

If I end up working at an AI safety org in 2023 or there is otherwise a conflict of interest this market resolves N/A.

I will mark the top 3 narrowest correct answers as right. (So "something to do with language models" might end up being technically correct but I won't award points for it).

Get Ṁ200 play money