As of late, there is talk of an "AI war" brewing amongst the tech giants (mainly Google and Microsoft): https://www.youtube.com/watch?v=BdHaeczStRA&ab_channel=JomaTech
Currently, the consensus is that Microsoft (and OpenAI) have made the first move and are in a winning position against Google: https://www.youtube.com/watch?v=5X1O5AS4nTc&ab_channel=ColdFusion
However, some are of the opinion that Google will eventually gain back lost ground and defeat Microsoft: https://twitter.com/tibo_maker/status/1631613907583246342?lang=en
The market will resolve YES if by the end of 2024, Google appears to have an edge over Microsoft in the "AI war"; the market will resolve NO otherwise.
1,000🏅 Top traders
| # | Name | Total profit |
|---|---|---|
| 1 | Ṁ1,626 | |
| 2 | Ṁ1,485 | |
| 3 | Ṁ607 | |
| 4 | Ṁ344 | |
| 5 | Ṁ288 |
People are also trading
@mods Should resolve as NO. The market creator has not been active for almost half a year. This market has a subjective component, but the creator has promised to largely abide by the consensus opinion. According to the comments, the consensus opinion is that Google currently has caught up in many areas and even overtaken Microsoft / OpenAI in video generation, while still not being in a winning position overall.
@ChaosIsALadder I'd make the case that while capabilities are now similar with GDM and OA trading for the top spot in LLM benchmarks, context size, products the "winning" position should be more forward looking/holistic.
Microsoft doesn't have much to show on the hardware side - they are still buying vast numbers of GPUs (~485k in 2024 for ~$30bn) while TPUs are mature are ramping aggressively.
To the extent to which you think cost-effective compute is important I think there's a strong case to be made for Google having the edge, just in terms of raw FLOP/watt or FLOP/capex$
I wouldn't like to call this market because I don't think there's a clear winner/edge
@CameronHolmes You seem to be well-informed and largely accurate, but GDM (Google DeepMind) and OA (OpenAI/Microsoft) are not really trading for the top spot in LLM benchmarks. Instead, most benchmarks have become saturated, making the companies appear closer in performance than they actually are. OpenAI truly excels in the most challenging benchmarks, and no company, including Google, is currently in the same league:
The argument about Google possibly having better fundamentals in terms of raw FLOPs per watt or FLOPs per dollar of capital expenditure (FLOP/capex$) with their TPUs is a valid one. However, it did not convince the creator of this market. The market creator considered OpenAI to be in a winning position at the time he wrote the market description, even though the same argument about Google's advantages was already valid back then. Additionally, one could argue that Google is strategically behind because they have only purchased 169k GPUs (much less than Microsoft's 485k). While their TPUs might have advantages in FLOPs per watt and FLOPs per capex$, those benefits are less relevant if NVIDIA secures the majority of manufacturing capacity at TSMC. Google might simply be unable to produce enough TPUs in the short to mid-term to stay competitive, which likely explains why they still need to purchase GPUs.
Lastly, the market is about whether Google is in a winning position, not Microsoft. So, even if you believe there is no clear winner or edge, the market would still resolve as "NO" according to the rules set by the market creator.
@ChaosIsALadder When the market was created GDM trailed on LLM performance, now they are ~equal and it really comes down to how you cherry pick benchmarks/modalities/release status/ELO ratings. I think it's fair to say that a (largely theoretical at the time) hardware advantage wouldn't be enough to sway them when they were visibly behind on performance, but could be considered a tie-breaker. Also when the market was created there was more of an open question on GPUs/ASICs as other major labs were much further behind on the curve as well.
Google primarily buys GPUs for GCP not GDM, 169k GPUs is nothing - they likely did 1m+ H100-equivalent TPUs in 2024. I don't think TSMC allocation is a material concern at this point, CoWoS was the major bottleneck and even that's ramping now. The reason I think this is relevant is that I don't think the winner of this "war" will be paying a 10X markup on their compute so how far the various labs/hyperscalers are to bringing chip design in-house is a key parameter and something they seem broadly aligned on given they all have similar ambitions.
I'm not saying it's an easy call for Google to win this, I just think you are significantly overstating the consensus of the comments and that this is a pretty hard one to call.
@CameronHolmes If I'm not misunderstanding you, you haven't really argued that the consensus is that Google is in a winning position—just that there isn't a consensus that it's in a losing one. This is compatible with my original statement, so the market should still resolve as NO (see the creator's comment below "this market will inherently have a subjective element to it since i will be personally judging, but i will promise to largely abide by consensus opinion and am willing to discuss if there are disagreements").
However, I'll now additionally also make the stronger argument that Google is actually in a losing position:
Google and OpenAI are most certainly not equal in benchmarks. Let's examine how favorable the benchmarks cherry-picked by Google, when introducing their latest model (https://blog.google/technology/google-deepmind/google-gemini-ai-update-december-2024), are for the company. I had to augment them with OpenAI's results, which is already a significant indication that things aren't going well:
Out of its own cherry-picked benchmarks, Google only wins 3 out of 13 against OpenAI's weaker LLMs (excluding o1-pro and o3). Of those 3 benchmarks that Google wins, 2 are Google's own benchmarks. They win their own long-context benchmark, MRCR, because OpenAI chose not to support super-long 1M contexts (with more tokens than the entire Lord of the Rings series) for now. Google wins its own Factuality benchmark by a thin margin. It barely wins the independent Vibe-Eval (Reka) benchmark, likely because I was unable to determine the correct score for GPT-4o and simply used the lowest one I could find.
This is not what equality looks like. This is what clearly losing looks like. In contrast, the benchmarks chosen by OpenAI are all independent benchmarks, generally agreed to represent the most challenging available:
OpenAI wins every single one of them by a large margin. This is what clearly winning looks like.
I assume you are deriving the 10x markup from the simplistic calculation that Nvidia seems to pay about $3,000 to build a GPU and then sells them for around $30,000. Following that logic, Google could theoretically design and build equivalent chips for the same cost without incurring the 10x markup. There are various reasons, which I won't delve into here for brevity, why this calculation is incorrect. However, the easiest way to see the flaw is by looking at Nvidia's net margin:
55.69% is a very high net margin for a hardware company, but it is far from a 10x markup. It resembles something closer to a 1x markup. Moreover, it looks like a markup Nvidia could easily reduce by cutting prices if it encountered difficulties competing with Google's or others' in-house designs. For now, however, this markup is likely well justified; otherwise, Nvidia's total market capitalization wouldn't be valued 50% higher than Google's.
Google is clearly losing in the benchmarks right now. It also certainly doesn't appear to have an advantage with its in-house designs when Nvidia (whose fortune is mainly tied to Microsoft as its primary customer) has the higher market capitalization.
@VictorLi The resolution should be NO. There is considerable evidence that Microsoft/OpenAI remains firmly in a leading position, while Google still has less to show in comparison. Even in the LMArena (which has become a somewhat peculiar benchmark, not fully reflecting the capabilities of models like o1 and presumably o3), the models are effectively tied within the confidence interval, with both Google and Microsoft ranked first.
If you exclude o1(-pro) and o3 from OpenAI, which absolutely trounce Gemini and everyone else, including humans, on hard benchmarks, you might argue that Google is tied with Microsoft. However, it is untenable to claim that the consensus opinion is that Google is in a winning position against Microsoft at this moment.
@ChaosIsALadder You didn't mention the one area where Google is leading - the videos from Veo 2 make Sora look like a joke.
But overall, I agree this should still be a NO.
Am I misunderstanding something? How is this a measure of intelligence? It just seems like a random tasks an llm would be bad at
@Shai That was third party confirmation, but o3 also does well on Frontier Math. OpenAI is clearly the winner unless Google unveils something in 3 days.
@GeraldMonroe Gemini 2 was a nice improvement, but it's only sonnet3.5 level nice. And DeepWhatevs based on 1.5 and generally sucks.
@IsaacLiu Microsoft probably has more revenue from the Copilot tools they've built into practically every cloud service: https://www.datacenterdynamics.com/en/news/microsoft-on-track-for-10bn-in-annual-revenue-from-ai-inference/
(Full disclosure: I'm also a Microsoftie.)
@JamesRomeril this market will inherently have a subjective element to it since i will be personally judging, but i will promise to largely abide by consensus opinion and am willing to discuss if there are disagreements. an example of something that will be used as a metric to decide this would be things like the relative performance of their state of the art models, etc.
@VictorLJZ the state of the board right now seems to be that openAI has the most powerful top model with o1 pro. And the most users.
Google has a great faster model with 1206, better video generation with Veo2, cheaper cost, and a fraction of the userbase. As well as longer context, some of their fundamental technology seems to be better or different.
Assuming the situation doesn't change by the end of the month would that be an openAI win? It's close but the most important stats (the highest tier model being stronger and userbase) seem to be what would matter the most.
My only thought here is strategically, Google is in a better position. Their potential integration of Gemini into phones and Google docs is an advantage, they have strong internal AI teams, and they don't have to pay Nvidia. They are still losing battles right now but are potentially winning the overall campaign.
@VictorLJZ I feel like this should be a NO resolution. Along with other catastrophes where it becomes clear that the only winning move was not to play.