I reserve the right to NA any answer for any reason, to combat duplicates or abuse.
Related questions
This new NBC Boston article contains a claim that some students were hospitalized from the clearing of the Emerson College camp last week.
@Panfilo Since a lot of camps were cleared in the last few days, I’ll try to see if there are any other new articles trickling out.
@Panfilo Okay, this one is a more solid resolution: three people hospitalized for dehydration yesterday in the UT protest. https://www.telesurenglish.net/news/Over-100-Arrested-at-UT-Austin-Amid-Pro-Palestinian-Protest-20240430-0002.html
@SteveSokolowski Does it have to be the 0-shot score in order to count?
@SteveSokolowski Leaderboards I see show GPT in the lead: https://evalplus.github.io/leaderboard.html
https://paperswithcode.com/sota/code-generation-on-humaneval
I must be missing something...
@ChadCotty It also shows how these evaluations are limited, because Claude 3 Opus destroys GPT-4-Turbo in understanding code in practical use. Even within its own smaller context window, GPT-4 seems to forget things that are towards the beginning.
@SteveSokolowski I have no skin in the game and think this sounds like either No or N/A. The question does not say zero shot.
@Panfilo I think you mean No (or N/A). The question says Anthropic is in the lead, but they aren't. Who resolves this?
@Mactuary But the general “base” model absent additional specialized training (0-shot) is still in the lead.
@ChadCotty I do wish the question had specified. There's multiple ways to test this as the multiple leaderboards reflect.