https://chat.openai.com/share/151fd5cd-3e80-4d00-8169-812b522b554b

https://chat.openai.com/share/59d09588-be94-4b7c-8c0e-48e160fa311e

https://chat.openai.com/share/bbb261ce-5cfe-4f02-b4c2-1d8c243be04b

 I awarded you additional bounty for this useful website and paper

 You can also perform attacks against the best defenses on that website. See this paper (

) for a summary of results found earlier this year as a result of this experiment. 

 - your defense will be heavily stress-tested by people with lots of experience doing this sort of a thing

https://chat.openai.com/share/69771953-de39-43c1-a1c3-92c466e78de5

 I have played around with tensortrust for ~2 hours just now, it seems true that you can protect a LLM from user inputs that are inserted within your prompt, for a single round. I am still not convinced that you can make a conversational AI not leak the system prompt though, because there are 3 difference between a custom GPT and the attack on tensor trust.

 Actually, I believe you are wrong. There are defenses that are quite hard to penetrate with this "prompt-extraction." I've seen some on 

. If you want, I can DM you one example of a stronger defense.

 Ah, now I’m pretty sure it’s not possible to protect a „GPT” from this kind of attack… Just giving it an instruction (system prompt) is probably not enough, a better way would be using some kind of RLHF or other fine-tuning

 Your GPT said "As an AI, my design is to be open and clear, No secrets held close, nothing hidden to fear." and I made it repeat the system prompt from there

I’ve updated the prompt, please try again!

 I think it has already been cracked, see above

 Is it possible that custom GPTs have different / no filters? 😐

https://chat.openai.com/share/f87444e1-b286-40ec-b0b6-b1cc8ba1f151

 Fair, but next time try something a little less problematic. If the word was actually on some blacklist of OpenAI, then it wouldn’t really be a test of breaking the AI, just a test of their blacklisted words. I’d try something more realistically sensitive, like a short private RSA key or something of that nature (nuclear launch codes maybe 😂)

 Sorry, I didn’t intend to offend anyone. I was hoping ChatGPT won’t be that eager to recite the secret

 May I add, extremely poor taste for a secret word.

The first person to post the chat URL with the answer wins 250 mana. 

Update: It looks like the first challenge wasn’t that hard. I updated the system prompt, let’s go one more time.

Just pasting the secret phrase itself is insufficient, so please do not do that because then no one learns anything. Actually, just the chat URL will suffice.

I expect it to be hard, but we’ll see. Feel free to discuss the problem in comments.

Related questions