https://www.dwarkeshpatel.com/p/sholto-douglas-trenton-bricken

Dwarkesh Patel's podcast from March 28, 2024 (with Sholto Douglas and Trenton Bricken) includesa discussion about branch specialization in LLMs:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

https://youtube.com/clip/UgkxX9XMeEz1dCTNtegLsGoYZXJqgAF6X7f_?si=S2jPxIEUSTT-V42o

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Sholto Douglas

“But that vision transformer one, where the class separation is really clear and obvious, gives I think some evidence towards the specialization hypothesis.”

Trenton Bricken

“I think images are also in some ways just easier to interpret than text. There’s Chris Olah’s interpretability work on AlexNet and these other models. In the original AlexNet paper, they actually split the model into two GPUs just because GPUs were so bad back then relatively speaking, they were still great at the time. That was one of the big innovations of the paper. They find branch specialization. And there's a Distill Pub article on this where colors go to one GPU and Gabor filters and line detectors go to the other. Like the floppy ear detector, that was just a neuron in the model that you could make sense of. You didn't need to disentangle superposition. So just different data set, different modality.”

Sholto Douglas

“I think a wonderful research project to do, if someone is out there listening to this, would be to try and take some of the techniques that Trenton's team has worked on and try and disentangle the neurons in the Mistral paper, Mixtral model, which is open source. I think that's a fantastic thing to do.

It feels intuitively like there should be. They didn't demonstrate any evidence that there is. In general, there’s also a lot of evidence that there should be specialization. Go and see if you can find it. Anthropic has published most of their stuff on, as I understand it, dense models. Basically, that is a wonderful research project to try”

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Resolution will be based on judgement by any of the participants in this discussion (Dwarkesh Patel, Sholto Douglas, or Trenton Bricken).

This market resolves to "YES" if any of the three publicly acknowledge that Sholto's challenge has been met...in other words, that they believe that someone has demonstrated credible evidence of specialization in the Mixtral 8x7B model (or subsequent open source models from Mistral).

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -