It has reached rank 3 as of yesterday, so I think the top 3 question can be resolved

done

All Llama 3. 1 models were released. This market should be resolved

i resolved some options but i think we have to wait till end of the year to resolve the rest because although unlikely meta can still releases a 3B parameter model

I distinctly remember Zuck talking about only three model sizes(8,70,405). And Llama 4 is already being trained.

Could you at least resolve the context windows market? There is no indication that Meta would extend it for new model(reminder, in April release they did say the new models would have extended model, but not now).

happy to resolve now if @traders don’t object

What did traders say? If they are not against, could you resolve questions about context window?

Additionally, when you will resolve questions regarding LMSYS ranks? Right after they first appear? Or you would wait a week or so(since earlier results are less reliable)?

No one objected till now, let’s just wait 48 more hours and then resolve the context window + model size markets, ok? Most traders receieved a notification when I mentioned them in my last comment. For LMSYS rank we will wait a bit more

Did anyone objected to resolution of context window market?

no will resolve now

@Soli presumably you're extending the close date to Dec 31?

corrrrect

LLAMA 3 Is out, no 3B version

@SarM they still might release one

@Soli So the market can never resolve no?

@Hazel The market resolves no if by end of the year the model is not released

What will be considered a 3 billion version? 2.8B? 3.7B?

@Sss19971997 Good question, maybe anything between 2.5 and 4.0?

https://huggingface.co/open-llm-leaderboard

I assume this refers to the "smaller versions" which are expected soon? https://analyticsindiamag.com/meta-set-to-release-two-small-versions-of-llama-3-next-week/

@HarrisonNathan i updated the text to say "some models will launch in April" and added another option for "all models" - thank you for pointing this out

Can you add the option of something like: (if the results are reported) gets 97% or more on TruthfulQA

@firstuserhere sure - i am not super knowledgable about this stuff, what do you suggest the exact wording be for this one?