Resolves N/A if it's not released by market close.
Nov 25, 11:11pm: Will GPT-4 be released with the ability to use a REPL, a calculator, or an internet search tool? → GPT-4 #4: Will GPT-4 be released with the ability to use a REPL, a calculator, or an internet search tool?
Related questions
I am inclined to resolve this either No or N/A. Everything I've seen from Microsoft has included something to suggest that it's not literally GPT-4 ("early version", "specialized for search", etc). I am inclined to no, since the actual answer is no. Does anyone have strong arguments for why I should choose N/A instead?
@vluzko My argument for N/A is that you previously said that you are going to resolve as YES unless OpenAI says that Bing is not actually GPT-4. I don't think they said and but now you don't even consider this possibility.
Early version doesn't mean not GPT-4. It is literally GPT-4, just another checkpoint.
Specialized for search does not indicate another model as well. There is an API for developers which has the "system window" which is used for prompts which are apparently handeled as a special case. One of the most obvious (and I believe it's the default way) is to write something "You are Sydney, a new chatbor from Microsoft which is a part of new Bing, your goal is ...". This alone is enough to call it specialized, as well as infrastructure which makes it access repl it. I would still call it GPT-4 nevertheless.
@ValeryCherepanov The first point is mostly fair, and the reason I'm reconsidering is that I think Microsoft has said several things to the effect of "not literally GPT-4" and has not said "literally GPT-4". I don't feel bound to the OpenAI part of it.
The checkpoint part is... obviously earlier checkpoints are different models? Like there's a range of similarities with the first checkpoint being totally different and the penultimate checkpoint being almost exactly the same, but I clearly stated literally GPT-4.
I agree that "specialized for search" does not necessarily imply any finetuning, but it's additional ambiguity.
@vluzko I don't think checkpoint are necessary different model tbh. For example, when ChatGPT just launched, they made a few updates to it, around 1 per 2 weeks and I think some of them uses different checkpoints of the same underlaying model, they used it for trying to fix specific jailbreaks for example.
Although it's hard to say definitely because often people take some model, fine-tune it and give it some completely different name.
I think N/A is appropriately here
@ValeryCherepanov Yeah I would not call those the same model, they're just being marketed under the same brand. I don't care about the marketing, I care about the underlying model. "Different checkpoint of the same underlying model" is just not meaningful. The model is its weights.
Damn, ran out of money so I couldn't buy more NO, and didn't see the closing date was so soon.
Anyways Bing is finetuned according to the guy in charge of Bing so not base GPT-4 Mikhail Parakhin on Twitter: "@zaptrem Multiple changes, including differently fine-tuned and RLHFed models, different prompts, etc." / Twitter
@vluzko This doesn't make it resolve YES as per your comment below?
https://blogs.bing.com/search/march_2023/Confirmed-the-new-Bing-runs-on-OpenAI%E2%80%99s-GPT-4
@IsaacKing I had not seen that, thank you for sharing. This page does say "early version" though, which is annoyingly ambiguous. If we get no additional information I will resolve YES based on this, but I am going to wait a few days in case OpenAI says something like "Bing is only running GPT-4 without RLHF" or "Bing is only running GPT-4 with half the training finished".
@VincentLuczkow Bing added additional stuff on top of GPT-4 to allow it to search the internet, so it's unclear whether that should count as "literally GPT-4". Seems like a larger change than finetuning, for example, which you said wouldn't count.
@IsaacKing 10 days ago Vincent said pretty firmly that if Bing was GPT-4, then he'd resolve yes. I'm voting by word-of-market-author here.
@IsaacKing They almost certainly have not changed the underlying model and have just written infrastructure to do communication between the model and the internet (probably with some prompting so the model can distinguish internet input and user input). If they did modify the model to use the internet directly (which would be crazy, that's a huge lift for basically zero gain), that would not count.
@vluzko Confirming: When you say "would not count", you mean it wouldn't count as Bing being GPT-4, since it's too different?
Bing runs on GPT-4 doesn't mean Bing is GPT-4. GPT-4 is the name of the LLM, Bing is a product that uses an LLM and integrate ability to use an internet search. I'm quite sure that the GPT-4 API will not provide internet search ability. However anybody will be able to integrate any internet search, calculator or whatever (as with GPT-3, someone even did integrate ability to run Python programs it writes, an obvious security nightmare of course)
@IsaacKing It is not even clear what it would mean to modify GPT-4 to directly access the internet (convert it into some kind of hierarchical model, I suppose??), but yeah if they did that the market resolves NO. But they didn't do that, and I think everyone reading this can safely forget about this possibility and just focus on whether the Bing version of GPT-4 is finetuned and/or missing additional training that OpenAI.