🏅 Top traders

Should resolve YES, at least based on Lmsys performance and probably also based on MMLU scores. xAI just announced Grok-2 including results on both benchmarks:

https://x.ai/blog/grok-2

Holy shit, I genuinely didn't believe the elongated muskrat had it in him to run a remotely competent organisation anymore, clearly he must be doing something right

Time to update priors

The lowest performing gpt-4 model is a fairly low bar now

they have sullied the GPT-4 name with these knockoffs

i put this at 70% because gpt-4-0613 had very weak performance

That's an inclusive or, right? It only has to beat any GPT-4 on either MMLU or LMSYS, not both?

@IsaacCarruthers Correct

the minimum of gpt-4 or maximum? is beating gpt-4-0613 enough?

@CampbellHutcheson "any GPT4 series variant"

beating any GPT-4 variant on lmsys isn't that hard these days. Claude Haiku beats gpt-4-0613