A recent paper proposes an operationalization of a "Scientific Theory of Deep Learning," and argues that the field should indeed eventually admit a scientific theory. Though not the only possible form of a scientific theory, the paper considers the possibility that we will understand the mechanics of the learning process. Some desiderata they list for such a theory:
"""
Learning mechanics should be fundamental, proceeding logically from a first-principles description of neural network training. Interim assumptions about network weights, dynamics, and performance will be useful tools, but they should ultimately be explained from first principles.
Learning mechanics should be mathematical, making unambiguous quantitative statements about important properties of neural networks. No mechanics is a qualitative science; neither will be learning mechanics.
Learning mechanics should be predictive, making claims supported by simple, repeatable empirical measurements. We have excellent experimental control of our system, and every major development should be unambiguously verified in experiment.
Learning mechanics should be comprehensive, describing aspects of neural networks’ training process, hidden representations, and final weights in a single picture. It is worth emphasizing that this theory will not — and should not — aim to describe everything...
Learning mechanics should be intuitive, being simple, illuminating, and satisfying in its demystification of deep learning. Like physics, learning mechanics should strive for simple insight over technical complexity.
Learning mechanics should be useful, serving as the scientific foundation for applied deep learning as physics does for other forms of engineering...
Finally, learning mechanics should be humble, being solid in what it describes and explicit about what it cannot...
"""
This paper resolves if, before 2032, there is a public and widely accepted body of work which comprises a Scientific Theory of Deep Learning, in the spirit with which the term is discussed in the paper. I will resolve based on my best judgment, consulting human and AI expertise where necessary.
1,000