See this paper for more info: https://arxiv.org/abs/2211.06738
Then we present our main open problem: is there a heuristic estimator that formalizes intuitively valid applications of the presumption of independence without also accepting spurious arguments?
The paper describes the concept of a "heuristic argument" and presents some work in this direction; however, it leaves as an open question whether a satisfactory formalization exists.
Resolves based on the opinion of Paul Christiano, Mark Xu, or other researchers familiar with the direction. Resolves N/A if there is significant controversy between researchers, and I deem myself unable to adjudicate with confidence.
Related questions
It might be worth distinguishing between three-ish kinds of things:
1. you get a heuristic estimator for "finite" objects, e.g. circuits, but the generalization to quantifiers is not done. I would consider heuristic arguments to "still be open" in this case.
2. you get a heuristic estimator that captures quantifiers, but the arguments are too long or the evaluation of arguments takes super-linear (or even super-polynomial) time. I would tentatively consider the question of whether or not there exists a heuristic estimator to be resolved "yes" in this case, but the question of whether or not there exists an efficient heuristic estimator, which is what one might need for the ambitious alignment schemes involving heuristic arguments, to still be open.
3. you get a heuristic estimator that captures quantifiers and is efficient. I would consider heuristic arguments to definitely be "solved" in this case.
@RyanGreenblatt I would consider the definition of open problem to be as long as there exists no answer to the posed question, regardless of whether anyone is interested in the answer. It might be worth making separate markets to capture "will a satisfactory heuristic estimator be found" or "will heuristic arguments still be considered a promising direction by Paul"
