Which one is human, which one is AI (Stanford Roommate essay)?
65
105
1.3K
resolved Jan 10
Resolved
YES

Prompt

Virtually all of Stanford's undergraduates live on campus. Write a note to your future roommate that reveals something about you or that will help your roommate—and us—get to know you better. (100–250 words)

(taken from Stanford's admission essays)

One of {A, B} is written by a human, and the other by ChatGPT.

Resolves to YES if A is human written.

A:

Hey roomie!

I thought I’d introduce myself before we meet. I know it’s late but most of my creative energy comes out late at night! Oh, do be warned though - when I do sleep, I sleep like a log! We should figure out a sleep schedule that’s best for the both of us.

Since we’re going to have to get used to each other’s eccentricities, I better get mine out of the way!

Sometimes when I’m deep in thought, I drum away my pen on the table, getting lost into the rhythm. If it ever gets annoying, just let me know.

Do you like to watch football? (or do you call it soccer?) If you do, we could watch Champion League matches together. Forgive me if I end up boring you with tactics and gameplays!

I love going on cycling rides. The feeling of wind blowing back your hair as you pedal harder and harder and the scenes flow by you is addicting. Hopefully, you won’t mind my daily family video calls!

I’m excited to meet you and getting to know you. Let’s have a great year together!

Best,

firstuserhere

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

B:

Hi roomie!

My name is firstuserhere and I am excited to be your roommate for the upcoming academic year at Stanford!

A little bit about me - I am a huge foodie and love trying out new restaurants and cuisines. In my free time, you can catch me playing soccer or going for a hike. I am also really passionate about sustainability and try to live a zero waste lifestyle as much as possible.

One thing that you should know about me is that I am a bit of a neat freak. I like to keep my living space organized and tidy, but I am also considerate of my roommates and their personal preferences. I hope that we can work together to create a comfortable and welcoming environment for both of us.

I am looking forward to getting to know you and having a great year together. If you have any questions or concerns, don't hesitate to reach out.

Best,

firstuserhere

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I'm going to refrain from betting on this market myself, to avoid any "giving it away" clues.

Inspired by https://manifold.markets/ZZZZZZ/which-prompt-will-allow-chatgpt-to#m0CtraoEt6L12Swytubp and https://manifold.markets/MarcusAbramovitch/will-a-universitylevel-student-be-p-59c8540738d1

Get Ṁ200 play money

🏅 Top traders

#NameTotal profit
1Ṁ349
2Ṁ294
3Ṁ277
4Ṁ118
5Ṁ96
Sort by:

Thank you everyone for participating in this market. Its been very informative and fun to learn both sides of this market, about writing styles (thanks @NcyRocks ), about the inconsistency of various Gpt-predictors and reasons behind it, and many interesting conversations on discord dms.

The market will resolve to YES. The essay was indeed submitted to Stanford by an applicant (personal identification details removed from the essay) who was turned down that year. The Prompt B is written by ChatGPT after i directed it to add more personality to the answer.

Thanks again for participating, feel free to message, suggest ideas, or continue discussions. Have a good day guys :)

bought Ṁ1 of YES

Ran this through GPT-2 Output detector and A showed as 99% real while B as 77% generated from GPT-2. Fairly trivial and I put together an extensive YouTube video about how this works. https://www.youtube.com/watch?v=whbNCSZb3c8

predicted NO

@PatrickDelaney strong words for a man with with only 1 share 🧐

predicted YES

@PatrickDelaney I have no idea what the reputation of the market maker is and there are comments below including types which confuse me. Also there's no real way to make any money on this anymore unless you bet in the 100's+ because it's so obvious.

predicted YES

@PatrickDelaney ironically, "typos" not, "types"

@PatrickDelaney that's a fair point.

The typo below was due to manifold showing the person's position as opposite to their position, leading to a bit of confusion.

I'm new to Manifold, and your point about reputation is fair (I'm not betting for play money, I'm making this market because it tickled my curiousity and now in hindsight, well, didn't expect this much swing of opinion over what's human written). I'm very amused, however, that this much commotion happened over what i thought was fairly obvious. So, this market was successful in my eyes.

@firstuserhere regarding the first point - next to a person's name, it says "X is predicting (yes/no) at Y%" but that yes/no status is often different from their real/final position.

predicted NO

@NcyRocks what do you know that we don't???

predicted YES

@jonsimon Nothing special, just the way ChatGPT writes. It loves putting in things like ‘I live a zero-waste lifestyle’ and would be unlikely to yield the ‘soccer’ bit, the ‘I think better at night’ bit, or to change the subject as quickly as A does.

predicted YES

@jonsimon Put it this way: if it’s B, I’m going to be very impressed with @firstuserhere‘s ability to make ChatGPT write with different idiosyncrasies, and to write like ChatGPT.

But I have a lot of faith in my own ability to pick up on writing styles.

@NcyRocks this market was made 3 months ago when i just joined manifold. You pointed out how it tends to gravitate towards a centre, doesn't "dive in" to a character, and I've found myself thinking about your phrase "ability to pick up on writing styles" quite a few times since. I've not yet tried this particular prompt with GPT-4 yet, especially Bing, but I'd be surprised if it wasn't even harder to distinguish the human and gpt-4 one now (especially @BoltonBailey's bit about how a human is likely to overshare their perceived flaws, and wonder if GPT-4 is good at it)

It seems trivial to talk about the ability of LLMs to write good text nowadays because of how often we interact with them. We've seen other people's brilliant prompts, our own interactions and stories etc etc (I've lost count of how many tens of thousands of words I've had GPT-4 generate for me) but every once in a while, I just pause and reflect at just how good this stuff ended up being, from where we were just a few years (or even months) ago. Just makes me feel some sense of awe whenever I step back and see it

bought Ṁ40 of NO

I don't know how any of you people are confident enough to have bet this all the way up to >90%... at this point I'm counter-betting just because the upside potential is so high

plot twist, they're both bots

@Odoacre haha but no. One of them (of course, with personal details which have been removed here) has been seen by the stanford admission committee people

bought Ṁ15 of NO

I feel like this is obvious

predicted YES

@anne I agree, but I see you bought NO shares (while I hold a lot of YES). I’d like to hear why it’s obvious to you that it’s B?

bought Ṁ85 of NO

Here's one that was generated by ChatGPT, prompted by me just now:

Hi future roommate,

I'm really excited to meet you and be your roommate for the upcoming academic year at Stanford!

A little bit about me: I'm originally from the East Coast, and I love exploring new places and trying new foods. I'm also really into sports, both playing and watching. In my free time, you can usually find me playing basketball or cheering on my favorite teams. I'm also a bit of a neat freak, so you can count on me to help keep our shared space clean and organized.

One thing that's really important to me is staying on top of my studies, but I also believe in the importance of taking breaks and having fun. I hope we can balance each other out in that regard and create a positive and productive living environment for ourselves.

I'm looking forward to getting to know you and making the most of our time together at Stanford. Let's make this a great year!

Best, Tetraspace

This seems stylistically closest to the second one, and straight-up uses the same phrase "I'm also a bit of a neat freak" (Same as Bolton Bailey's ChatGPT generation: Which one is human, which one is AI (Stanford Roommate essay)? | Manifold Markets)

@Tetraspace If you think B is by chatgpt, then you should buy YES shares, not NO shares!

bought Ṁ200 of YES

@firstuserhere This is a very cool market! Was the human written one written by an actual Stanford applicant? (Feel free not to answer).

@BoltonBailey message me on discord? firstuserhere#1188

bought Ṁ80 of NO

My output from GhatGPT with this prompt:

Hey there future roommate!

I'm really excited to meet you and start this new journey together at Stanford. A little about me - I'm from a small town in the Midwest and am the first in my family to go to college. I'm a bit of a neat freak and love to keep my living space organized, so hopefully that won't be too much of a problem for you. In my free time, I enjoy playing soccer and hiking in the beautiful California wilderness. I'm also a bit of a foodie and love trying out new restaurants and cooking new recipes. I'm open to trying any type of cuisine, so if you have any recommendations or want to grab a meal together, let me know! I'm really looking forward to getting to know you and making the most of our time at Stanford.

Best, [Your Name]

@BoltonBailey i don't know if clarification is needed but the market resolves to No is B is chatGPT and Yes if A is ChatGPT written

bought Ṁ70 of YES

@firstuserhere Your prompt literally says "Resolves to YES if A is human written."

@BoltonBailey yes, and the line above it explicitly mentions that one of A and B is Human, the other ChatGPT

predicted YES

@firstuserhere In your description you say

"Resolves to YES if A is human written."

In your comment right above this, you say

"Yes if A is ChatGPT written"

Which is it?

@BoltonBailey oh I'm sorry, made a typo in the comment. The description is correct.

@BoltonBailey any way to remove the erroneous comment? Perhaps by reporting it?

predicted YES

@firstuserhere I see the issue - my comment is marked saying I bought NO, but I actually sold that and bought YES.

Not sure if reporting the comment is appropriate given it's an honest mistake and the context makes that clear.

@BoltonBailey yeah, that's why i replied in the first place and then checked users, and it showed reverse position

@firstuserhere regardless, to everyone who sees this, please stick with the description of resolution

@firstuserhere resolution in the description*