Better, more-detailed, more-coherent, more-logically-compelling answers to this question: https://www.lesswrong.com/posts/maL9GejdnhrgGvEfc/build-knowledge-base-first-or-backchain

Will award to what I think are (by end-of-Friday) the best answers. Top 3, divvied approx by how useful I think they are. Prioritize logical-compellingness over uniqueness. Each winner will get at least 50 Mana. One answer-comment allowed per-person, but it can contain all your thoughts and be updated until end-of-Friday.

If no answer is satisfying, no bounty is awarded. (Same if it can't be tied to someone on LW, e.g. through a private-message after adding an answer to the LW question)

Object-level notes on where I'm coming from, with the question:

• Leaning towards conceptual/mathematical alignment, as eventually operationalizable in programming (especially w.r.t. ML).

  • Secondarily: Leaning towards "We should solve [which value to point to?] before we solve [how do we point AI at something at all?], for strategic reasons."

• My current "favorite" agendas are John Wentworth's "abstractions" and Tamsin Leake's "QACI", so I'm mostly asking about knowledge w.r.t. to both. Of course both those people have strong and opposite opinions on the "knowledge base or backchain" question lol.

  • I also think both those angles-on-alignment, if brought to fruition, would have at least some shared structure.