Will the next neural network architectural discovery explicitly incorporate some kind of self-reflection?
2
16
Ṁ25Ṁ140
2030
48%
chance
1D
1W
1M
ALL
LLMs use the Transformer architecture, which incorporates a self-attention mechanism. I am leaving it relatively open as to what should be considered a "self-reflection" type of mechanism, but note that traditional feed-forward networks would not be considered to incorporate anything that mimics "self-reflection."
If there is a new architectural breakthrough in neural networks that occurs before 2030, will that architecture utilize a new style of self-reflection in its design?
Get Ṁ200 play money
Mechanistic interpretability questions
By the end of 2026, will we have transparency into any useful internal pattern within a Large Language Model whose semantics would have been unfamiliar to AI and cognitive science in 2006?
50% chance
Do scaling laws happen because models experience a ton of tiny phase changes which average out to a smooth curve?
55% chance
Related questions
In 1 years time, what credence will John assign to the field of alignment converging toward primarlity working on decoding the internal language of neural nets?
44% chance
Will neurotechnology enable AI to predict and classify human decisions, along with their influencing factors, by 2030?
70% chance
In 2028, will an AI be able to play randomly-selected computer games at human level, given the chance to train via self-play?
63% chance
Will an AI produce encyclopedia-worthy philosophy by 2026?
20% chance
When will a language model be fine-tuned via self-play or expert iteration and achieve significant performance increase?
2025
Will geometric deep learning turn out to be as influential an idea as transformers?
24% chance
Will a Theory of Everything in physics be discovered by AI?
35% chance
Will Google Deepmind reveal their own autonomous agent in 2024?
38% chance
Will a major AI company acknowledge the possibility of conscious AIs by 2026?
53% chance