Will GPT-4 end up not as useful as most people think?
38
230
710
resolved Jan 13
Resolved
NO

Based on

https://at.tumblr.com/nostalgebraist/705192637617127424/4j5dlxpvwp5e

If GPT-4 is better than expectations at january (an improved chatGPT)-> then this resolves NO.

If it is underwhelming, and not as awesome and useful as most people expected -> this resolves YES.

Get Ṁ200 play money

🏅 Top traders

#NameTotal profit
1Ṁ132
2Ṁ79
3Ṁ79
4Ṁ58
5Ṁ39
Sort by:

@TheWiggleManRetired Can you please resolve? Ty!

How's gpt-4 going guys

bought Ṁ1,000 of NO

@TheWiggleManRetired

So, here is my real GPT-4 prediction: it won't be very useful, and won't see much practical use.

It gets so much practical use that OpenAI can't buy enough GPUs to meet demand. They were charging $1.6 million/year for GPT-4 dedicated instances, and can't sell them because of lack of GPUs.

Bing is running on GPT-4 now.

Everyone uses GPT-4 to generate data to train their models(RLAIF) as the "reference model".

GPT-4 has better reading comprehension than most of Manifold.

OpenAI expects $200 million in revenue after making $30 million last year. Microsoft put $10 billion in January, after seeing it and using it with Bing.

As of July 10, I chatted 274,865(me) + 607,556(GPT-4) tokens with ChatGPT's GPT-4. Plus some lesser unknown amount with Bing.

I use it all the time when coding to write small sections.

Seems like a lot of people find it practical and useful.

To anyone who's taken a good look at LLMs, they seem "obviously" good for something, indeed good for numerous things. But they are provably, reliably, repeatably good for very few things -- not so much (or not only) because of their limitations, but because we don't know how to use them yet.

  • Summarizing articles or long Manifold market descriptions

  • Translation

  • Reasoning about complex rules like the tax code or Magic the Gathering scenarios(but not remembering all the rules - you do have to provide it with the rules in-context)

  • Coding assistants

  • Discussion while reasoning through something: It can't be relied on like an oracle, but it can usually talk about something with you and point out the obvious.

  • LLMs in general are good for keeping a consistent personality when writing. If I write something myself, I can be identified from the writing style. So I often run things through GPT-4 or other language models if I don't want to be identifiable. GPT-4 is bad at writing as an arbitrary personality because OpenAI fixes the write style to always sound like ChatGPT - but it's useful for generating training data for finetuning using a character template.

  • Hedge funds are using GPT-4 to summarize earnings reports, replacing analysts that followed a bunch of stocks to pull out information to put in spreadsheets. Or to trade on news.

Also: The article explicitly mentions Morse Code and GPT-4 is able to do Morse Code: /yaboi69/will-gpt4-work-mostly-well-with-the

@Mira Mira! Good to see you, why were ppl so afraid to bring market down is over me. Did you use some LM for writing this comment too? (cheeky if u did :p)

https://mobile.twitter.com/ccanonne_/status/1639848150495301633

Often fails algorithm test questions in subtle ways, useful but will not replace programmer (in current state, on this task)

predicted NO

@TommyMorriss it failed one algorithm test question in exactly the same way I did!

More evidence of its limitations https://news.ycombinator.com/item?id=35297067

@TommyMorriss very interesting!

bought Ṁ10 of YES

https://tylerglaiel.substack.com/p/can-gpt-4-actually-write-code .

I am seeing people on twitter saying that it is over for programmers in general (obviously false, i just tried cursor.so and it failed miserably at a modification very in-scope). In january I heard rumours that would be ~~as good as a college senior at coding. I think that it is pretty far from that.

@TommyMorriss i wonder why this market is so high then!!! It should be low.

@Dreamingpast especially over time!

predicted YES

@Dreamingpast I don't know. I'm beginning to think it won't destroy programmers altogether, but it is "living up to the hype" in spirit I think. I am going to sell because I no longer feel strong enough about it either way

bought Ṁ20 of YES

@Dreamingpast Doesn't being high on this market mean you're low on GPT-4? If @TommyMorriss is right, being a skeptic of GPT-4, then the market should be high, right?

bought Ṁ200 of NO

@Dreamingpast I think you mean low

@Dreamingpast Because it's based on what most people thought, not what TommyMorriss thought. GPT-4 is a very big improvement over gpt-3.5-turbo.

@ShadowyZephyr ohhh yeaaa

@Dreamingpast Also, I thought you meant why is it so low. I didn't match the time of your comment with the probability at that time lol.

Btw, GPT-4 multimodal can solve Captchas. They are literally designed to prove you're human (and train other AI). That alone is already insane because it basically allows bots to pose as humans, and with GPT-4's other capabilities, I imagine it would be fairly easy for any programmer to hook up a social media account that is run entirely by a bot, and not have it detected.

I think with Visual mode in GPT-4 this might end up being more useful than expectations in Jan suggested.