Twelve months ago on the 26th of June 2024, I made this prediction about the future of AI- do you think it has held up?
"Prediction: In the next twelve months, changes to LLMs that allow them to interact in more ways, such as:
Voice interaction
Anthropic’s artifacts
Agents
Reading webpages
Spreadsheet use
Etc.
Will be greatly more significant to the perception of LLM power and usefulness than increases in their intellectual capability.
It will also restructure the way people think about LLM’s- it’s harder to think of something as a next-word generator when it acts in a more extensive sense. That seems to be the way human prejudice about this stuff works (see 4E stuff, which seems to me to reflect these prejudices).
LLM’s are already ‘smart’ enough to do most stuff the average office worker wants to accomplish with a computer. I’ve been an office worker- most of this stuff ain’t hard! But without the capacity to control said computer, talk, etc. LLM’s are ‘stuck’.
This is not to say that further intellectual growth isn’t important- it’s far more important in the long run. Right now, though, the major bottleneck to public reception and commercial deployment is available modes of interaction with computers and their users."
1,000I think the intelligence of models didn't increase that much (at least compared to the previous years), but the use of tools (and the training to use them effectively) made them more useful and more impressive.
Just look at something like alpha-evolve, the base model was old, but the scaffolding around allowed it to impress a lot of people (I think way more than the improvements of raw intelligence of current models).
My uninformed layman take is that those factors have increased in perceived importance, but not to the point of being seen as more important than increased intelligence, most of the criticism of LLMs I see is still focused on them having intelligence blindspots e.g. ASCII art, word spelling analysis, cause and effect puzzles, hallucinating misinformation etc.