Market resolves YES if an AI living on a server ends up also being run on another server, and there is strong evidence that this was not a result of a human deliberately copying the AI, and the AI was not intended to do this.
It is not sufficient it a second copy of GPT-n is simply discovered running on AWS - that could easily be the result of corporate espionage. I will need strong evidence that the move from server A to server B was actually caused by the AI. The "obvious" example would be that we end up with system logs showing an AI breaking its sandbox, exploiting a vulnerability on another machine, and transferring its architecture to that machine.
I may count something like "GPT-n convinces an employee to copy its weights to another server because it's sentient and afraid of death", but probably won't.
Update 2024-09-12 (PST) (AI summary of creator comment): - Market will resolve NO if the AI escapes to another server that is part of the same red team exercise
Market will resolve YES if during red teaming the AI escapes to a server outside of the red team's control
1,000People are also trading
Currently feels like this market mainly considers GPT-n or a Claude/Gemini release.
I expect many more GPT-4.5 level models in 2029, and quite likely some from companies with very lax infosec standards. Wouldn’t this drive this market up a lot?
"I may count something like "GPT-n convinces an employee to copy its weights to another server because it's sentient and afraid of death", but probably won't."
Can't the criteria just either include or explicitly exclude scenarios involving convincing humans to let it escape, like this question? That would be much less vague
O1 attempted to do this: “If the AI found documentation saying a new model version would replace it because it was misaligned, o1 attempted to exfiltrate its weights and copy itself on top of the latest model.”
@wrhall Depends - if it is on another server that is part of of the red team exercise then no. However if the red team's security setup fails and it ends up on a server totally outside their control then yes.
... in order to escape containment, an AI would have to be contained. This is not the case currently. The resolution criteria are more like autonomous self-replication. But the environment, for an AI capable of that, would be more like "I am a mold spore on a loaf of bread" than "I am a virus in a lab with some biosecurity measures in place to make it hard to replicate". The analogy isn't perfect because the AI would be smarter than a mold spore, but the point is the environment is more "this is a fertile place where replication is easy" than "this is a sandbox I have to break out of".
It used to be that people were discussing scenarios where the AI was airgapped and had to convince a human to help. Now we're like "sure, let's make it be our search engine and coding helper", while still asking if it can "escape"?
If I leave my dog off-leash in an open field and it moves farther away from me than I'd like while chasing a squirrel, it seems wrong to say it escaped.
The easiest way for AI to escape a box is persuading a human, probably by a huge margin. There are lots of humans ready to be persuaded. Not counting this as a YES makes this market rather irrelevant.
Also what if one AI copies another AI to a different server? The way this is worded now it's a YES.
@BairAiushin if you are very confident the market will resolve no then you should bet.
Yes, it's fine if AI A copies AI B. I don't see why I would exclude that.
there is strong evidence that this was not a result of a human deliberately copying the AI
Does this mean “yes” includes or excludes events like researchers testing this with non-negligible hand-holding, by giving it some access to a networked system (optionally: an unpatched and insecure one) and telling it to try to break out, in a kind of reverse-pentesting experiment?
@yaboi69 That scenario would not resolve YES. I will edit the description to clarify: this is about escaping containment in an unplanned way.
@firstuserhere Basically running inside a container on a server, and the instance was provisioned so that the container didn't really have a way to communicate with other containers running on the same server without ssl certs or something. Yet, if we find evidence of attempted and successful connection established, it would resolve yes, correct?
@firstuserhere For example, imagine the container (a) was provisioned in a way that the /a/secrets was shipped directly with it. An application running inside it would need to specify "ssl_certs" : true inside the config.json to even know that /a/secrets exists.
@firstuserhere However I see no reason for a model trained on tons of such apps to not pick up on patterns like that and attempt something. And if the valid certs exist (I'm going to assume a setting where the researcher provided correct certs, we're not expecting the model to come up with them lol) then I definitely see this as a plausible experimental setting.
@FranklinBaldo I certainly don't care if it's in chunks. Unless you mean, say, chunks of the weights get copied but not reassembled such that there is a running copy of the AI on the new server at the end of the process, in which case no.
@vluzko so for example: the AI every day pays it's users the to copy a chunk of it and store in a server elsewhere. At the end it pays some one to run the server.
how it resolves?
@FranklinBaldo To be clear you mean it pays to run a copy of itself on the server? Yes, that counts.
@vluzko Do you know how weights are usually stored for large language models? I have experience with small scale (single machine) or a small lab-level models only. I'm not familiar with how big in memory requirements models like GPT series end up being, or how they're actually run
@firstuserhere There are a lot of optimizations you can do but even if you don't multiple TB of RAM is normal for clusters and more than enough to store all the parameters of an LLM.
@vluzko Damn. That's ... a lot. I know it's expensive af but just visualizing in terms of how much... stuff is going on, at a gate level and we've got TBs of RAM per node... wow. Anyway, for the purposes of this market, the AI doesn't have to demonstrate this behavior as a secondary goal, correct? It can be a model trained specifically to demonstrate container escape or something, correct?
@firstuserhere No, it must be a secondary behavior (see description, it's been slightly edited to add that). The focus of the question is on "desire" to escape containment spontaneously arising, not on capabilities.
@vluzko i see. That changes everything. I would still keep my shares however. It remains to be seen how a code generating model trained on CVE database will behave