I've seen rumors that Apple might be working on local on-device LLMs for iPhones. I wonder if this is true. The market will resolve "yes" if Apple's new device features an on-device conversational LLM. It would suffice if the LLM is a combination of on-device and cloud-based processing.
1,000
1.00@JonathanMannhart @traders @Jacy @SimranRahman @alexlitz
Jonathan made a fair point: while the iPhone 16 is now available, it does not feature an on-device LLM at this moment. Although one could argue that the LLM will arrive via a future update, the market title uses the term “include” which reasonably implies that the feature should be present at launch.
I will allow 48 hours for any further arguments or objections from traders. If no compelling arguments are presented, I intend to resolve this market based on my current understanding, which leans toward a NO resolution.
I’m not sure what people thought this was going to be other than significant on-device LLM powered improvements to Siri.
Were we betting that Apple would release a different personal assistant? Or maybe that the interface would look more like ChatGPT?
I don’t have a big bet on the line, and it seems from the chart like the market has interpreted this differently, so I’m fine however it resolves. Looking at the launch page for the iPhone 16, though, I’m at least confident that what I thought I had bet on has happened.
Phrases like “The first iPhone built for Apple Intelligence” and “The all new chip that powers Apple Intelligence” seem to imply this is the on-device processing we saw touted at WWDC.
The entire section on Siri makes it pretty clear they’ve integrated LLM features like the ones we’ve seen in Claude*, ChatGPT, etc into conversations with Siri: “Richer language understanding and an enhanced voice make communicating with Siri even more natural. For example, if you stumble over your words, Siri still knows what you’re getting at.”
Ah, ok, I’ve read further down the comments and I see the issue is whether these features are available at launch. I see why this is contested. In fact, if it wasn’t for the entire focus of the device and launch messaging being the on-device LLM features I would probably be in the “No” camp.
EDIT: My best “steel man” for the “No” camp is, if the question was, “will the Tesla Model … have full self driving” I would expect it to resolve “No” regardless of the theoretical capabilities or any marketing promises if it wasn’t available at launch.
EDIT 2: And, my best argument for the “Yes” camp is, in the future, if there’s a trivia question, “Which Apple iPhone was the first to include an on-device LLM “ChatBot”?” I think clearly the correct answer will be “iPhone 16” and not, for example, “iPhone 17”. And, looking back at this market I think it will be very confusing and obviously incorrect to see that it was resolved “No”.
@MitchellButler no the right answer will be iPhone 15 Pro but i also had the same thought as you. It will be weird looking at this market in the future without understanding the context.
@Soli it should be pretty clear to everyone by now that you’re in a position with no perfect answer. Whichever way it resolves, you’ve made the effort to understand both sides and respond to a lot of the comments. I think that’s all the market can ask.
Some people saw this market and said, “no, they will not publicly (non-beta) release a new on-device LLM in time for the next iPhone event” and they were right. Others saw the market and said, “yes, the next iPhone release will focus on on-device LLM features” and they were right.
The market description asks whether the rumours of Apple working on an on-device LLM for the next iPhone release were true. Clearly they were.
--It’s unfortunate that people will try to turn market by profiting from ambiguity when the spirit of the market was pretty clear.-- (EDIT: Leaving this in for completeness. But I take it back. It was unfair.) Ultimately, that’s not your fault. In either resolution I think you’ve made a five star effort to at least understand both points of view and I think everyone should agree.
@MitchellButler “It’s unfortunate that people will try to turn market by profiting from ambiguity when the spirit of the market was pretty clear.“
Have to defend myself here. I did bet on YES on this market for the last few months! I was the biggest YES holder for a while.
What made me second-guess this was the word “include“. I stand by saying that this is pretty unambiguous. Of course the marketing still focuses on Apple Intelligence. That‘s because that‘s useful for Apple. They absolutely have to market it that way.
But that doesn‘t change the fact that Apple missed their own deadline here. And for a market that specifically asks “will the next iPhone include this feature“, and then Apple fails at doing this, that market should reflect that. Not the marketing messages that stayed the same (because ofc they‘re not going to change their whole marketing strategy because of this. They decide this half a year/a year in advance.)
@MitchellButler to your original point “The entire section on Siri makes it pretty clear they’ve integrated LLM features like the ones we’ve seen in Claude*, ChatGPT, etc into conversations with Siri: “Richer language understanding and an enhanced voice make communicating with Siri even more natural. For example, if you stumble over your words, Siri still knows what you’re getting at.”“
I disagree strongly that this should be resolved on marketing messages. Please don‘t look at the marketing, look at the reviews. Every single review points out that this feature is missing.
Something is built for XYZ ≠ something includes XYZ.
I agree with your Tesla example. And I don‘t agree that in the future, when people look back, they’ll think “oh yeah the iPhone 16 introduced Apple Intelligence“. The press is pretty clear right now that it‘s not there. And the iPhone 16 will get it at the exact same time as the iPhone 15 Pro line.
That they say “this is the first iPhone built for Apple Intelligence“ is really purely a marketing message, there is nothing behind this. The A18 Pro was designed long before they decided what computation needs the on-device LLM will need. Chip design takes years. It was the other way around: They built the LLM specifically so that it could run on the 3 nm chips and on 8GB RAM.
@MitchellButler thank you for the kind words, i really do appreciate it very very much. I will add a disclaimer to my future markets and profile that if there is disagreement I will go with the spirit of a market and in case someone detects an error in formulation they should give me atleast 48 hours to clarify before making trades that go against the understanding of market participants. I do have to defend @JonathanMannhart here though because I don’t think he acted in bad faith at all and I do think there are valid reasons why some people believe the wording of a market is more important than the spirit. Anyways, it was a very fun experience for me and I hope for others too (this is fake money haha). I will resolve tomorrow after 48 hours have passed as announced.
I also understand the spirit argument. But I do think, if we would ask MKBHD (not Apple) right now, he'd say "no, in spirit, the iPhone 16 didn't include the new Siri". Right? Or maybe we disagree on this? (Happy to consider arguments for why that wouldn't be the case.)
Like, right now it's not even certain that iOS 18.1 actually ships with the new Siri. That's just what they officially promise as of now. Which makes it likely, but it's still a promise. Apple has previously also delayed a launch further, from time to time. Or not delivered on the promise at all (see https://en.wikipedia.org/wiki/AirPower_(Apple) ).
@JonathanMannhart i think your argument about the definition of the word include is very strong and i am not making the case against it but honestly who cares if the iPhone has the marketed features on day one or not, i don’t think that would be an interesting question at all or one that would have attracted so many traders - it is obvious to me that the spirit of this market was not whether Apple will respect their marketing promises or deliver on day one and the reason the market was limited to Apple’s new devices was because it just made sense that if there is a hardware dependency then they would include it in the new iPhone. The description says I saw rumors they are working on local LLMs and wondered whether that was true and it was. I take fault in using the word include in the title and would have used support if I go back in time.
@Soli Fair! You seem exceptionally good faith when engaging with the arguments on here, I really respect this.
(E.g. I feel like even if you'd resolve this N/A or YES, you'd do it based on an argument that still would make sense. That would be fine by me.)
@JonathanMannhart i think for now No seems the be the right resolution, funny enough i had another market about macbooks but someone had asked whether it is ok if the llms support is added throufh an update later and i thankfully clarified it back then. I do appreciate the chaos you brought to my market and the educational experience. I updated the title of my other market thanks to you :))
/Soli/will-apples-next-macbook-2024-inclu
https://manifold.markets/Soli/will-apples-next-macbook-2024-inclu#dxeuplulc9g
@Soli @JonathanMannhart Sorry for the accusation. I've made an edit to keep it in my comment but retract it. That was unfair — I should've looked more deeply at the market history before saying something like that.
In defense of the folks with more "Yes" on the line, I do think there was a lot of value captured here from ambiguity. Had there been a separate market tracking whether these features would be available on release, I don't think it would've been sitting at 4%.
All that said, this is how prediction goes, and I'm more encouraged by seeing all the good-faith discussions than I would be by a resolution in my favor. At this point, I'll leave myself out of it and let you decide. I think Jonathan has made some very reasonable points.
@Guilhermesampaiodeoliveir yes you are right, i thought about it again and the use of the words “new device” actually doesn’t mean much. The argument with “include” still stands though.
@Choms are there any videos online showing this? i think this should indeed influence the resolution of the market if true, no? @JonathanMannhart
@Soli on their keynote, from minute 50 where they start talking about AI you can see several text-based UIs to interact with the thing https://www.youtube.com/watch?v=uarNiSl_uh4
@Choms The current Siri is not based on LLM tech, as far as we know. Apple definitely uses ML models to train it, but it‘s not a neural net itself. The current tech that is an LLM and that is on current iPhones is the new generation of auto-correction they introduced last year. But that‘s not a chat bot.
(I did research if the old Siri was in any way LLM-based before I placed my big bet. I didn‘t find anything. Obviously if anyone finds a good source saying this, then that should change the resolution to either N/A or YES, imo.)
https://www.macrumors.com/guide/apple-gpt/
https://techhq.com/2024/01/is-siri-ai-coming-in-2024/
Found all this in a few minutes, this is easy to research. Confused by the arguments here. (iOS 18.0 includes this old version of Siri, not the Apple Intelligence version. That ships with 18.1.)
@JonathanMannhart I get what you say, my thought is they will be releasing their on device models and looking at the keynote, you could write to it (they demo a couple email composition and image generation), again, just talking about the keynote, I wouldn't know which way to resolve this either at this point
@Choms I mean... That feature you speak of is just not included on an iPhone 16 if you buy one right now.