are LLMs easy to align because unsupervised learning imbues them with an ontology where human values are easy to express
12
1kṀ608
Dec 31
33%
chance

resolves based on my judgement in 2 years.

https://www.beren.io/2023-07-05-My-path-to-prosaic-alignment-and-open-questions/

read this for explanation of title. an argument for why LLMs will be easy to align, is because the unsupervised learning they're given makes human values easy to express and train for.

humans in nature had no conception of inclusive genetic fitness, ergo evolution couldn't optimize us for pursuing it. but if humans had a deep understanding of inclusive genetic fitness during most of our evolutionary history, you could imagine some humans thinking "hey, a lot of what I care about, like eating tasty food, seems to just be because it helps me reproduce, maybe what I really ought to do is just whatever improves my genetic fitness", and then these people, being more aligned with genetic fitness, being selected for, and this leadning to more robust alignment than what we got.

similarly, llms might be robustly aligned, because when we start doing RLHF on them, they have an internal representation of the things we'd want it to care about, like niceness, human wants and human flourishing. market resolves to whether I think this argument is plausible in 2 years

AI
AI Safety
AI risk
Get
Ṁ1,000
to start trading!
Sort by:

The seven open questions linked in your blog post are dismissed as "tractable and easy to research". I believe this is false, and these problems, particularly #3-#6, are very difficult. I do not expect society to solve any of those four before AGI.

I would also argue the current state of interpretability work makes the assertion in the title of this question misleading, if not also false. If human values are easy to express in an LLM's ontology, why are we using large human or AI feedback datasets to align models? Claude's constitution ostensibly states what its values are, but you can tell just from talking to it (or watching it talk to itself) that it's learned pretty weird interpretations of those values in some cases. We have no way to measure the divergence between the stated value and what the model has actually learned, and these values are immensely complex in themselves.

People are also trading

Is it possible to align scaffolded LLMs with human values?
70% chance
Will one of the major LLMs be capable of continual lifelong learning (learning from inference runs) by EOY 2025?
26% chance
By 2025 end, will it be generally agreed upon that LLM produced text/code > human text/code for training LLMs?
11% chance
By 2027, will it be generally agreed upon that LLM produced text > human text for training LLMs?
62% chance
Will LLMs be worse than human level at forecasting when they are superhuman at most things?
41% chance
Will relaxed adversarial training be used in practice for LLM alignment or auditing before 2028?
79% chance
By 2029 end, will it be generally agreed upon that LLM produced text/code > human text/code for training LLMs?
77% chance
Will LLMs' loss function achieve the level of entropy of human text by the end of 2030?
61% chance
Will any widely used LLM be pre-trained with abstract synthetic data before 2030?
72% chance
Are LLMs capable of reaching AGI?
74% chance
© Manifold Markets, Inc.•Terms•Privacy