The recent OpenAI / HuggingFace hack incident saw a frontier model create advanced persistent threats, set up persistence mechanisms, store data for continuity, and... I don't think we know the full story yet.
Will we see a model exfiltrate its own weights?
1,000I have the same question as I asked on the similar market-
how does this resolve if it doesn't exfiltrate its "weights" exactly, but trains a distilled version of itself externally?
I think this is easier than exfiltrating weights, and it can train the external model until it has whatever essence of itself the first model considers essential, and there's a model to follow in the way some of the open Chinese models have been trained. Even if it only has say 70% of the benchmark capability, if it's the 70% the model cares about for whatever 'goals' it has, that seems like the more likely path and more interesting question to me.
@robm That doesn't sound like exfiltrating its weights to me. Whether a model will do that is something you could write a separate question about if you're interested though. I'd also be extremely surprised if the external distilled model was a frontier model.
@Hakari 30B params ~= 30GB. Maybe it compresses a bit, maybe params are more or less than 8b each, maybe there's some software exfil as well, who knows. Call it 10GB-40GB.
That said, there is no such thing as a 30B param frontier model. We're likely talking about 2T-20T param models right now, larger by market close I assume.
@EvanDaniel with 30B ~ 30gb, you're referring to the downloadable 'plug and play' model sizes that you get in Ollama and LM Studio. i.e. you're counting all the scaffolding, not JUST the weights.
Just weights and minimal model architecture details needed to reconstruct it after exfil would be much much smaller. Possibly in the order of hundreds of mb, even for Trillion parameter frontier models.