Man beats Machine in METR hackathon on September 6?
88
1kṀ45k
resolved Sep 7
Resolved
NO

There will be a hackathon competition between those using and not using AI tools. Who will win? Event is on September 6.

Resolves YES if Man (no AI tools) wins.

Resolves NO if Machine (man using AI tools) wins.

Resolves 50% if there is a tie somehow.

More details: https://x.com/FactoryAI/status/1958216269414662447

Get
Ṁ1,000
to start trading!

🏅 Top traders

#NameTotal profit
1Ṁ2,398
2Ṁ1,962
3Ṁ1,209
4Ṁ765
5Ṁ537
Sort by:

I really don't understand this contest. Of course using having the option to use AI versus someone handicapped to not would allow you to potentially be faster and more efficient and allow for more iteration.

It would be interesting if Claude or something was literally itself a hackathon competitor coming up with and executing ideas from scratch to see if that beat teams with humans on them.

@IsaacLiu you'd think so but there was this surprising result https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/. In this study, participants thought AI was making them faster, but it was actually slowing them down! That's what inspired this hackathon.

bought Ṁ100 YES

If machine won we would have probably heard about it by now. Simple hueristic

@bessarabov Damn. Though wondering if this just means the winner of the competition used machine and the humans did better on average,

Holding out hope this can still resolve yes

@JaundicedBaboon Seems pretty clear this should resolve NO based on criteria and tweet?

@Jasonb the market doesn’t have a clear criteria for what it means for man to beat machine. The sponsors of the event are all AI companies with incentive to want the ai side to win. I could imagine a scenario where an AI team won but humans did better on average, and in that case I feel it should resolve yes because that essentially constitutes man winning.

Even if technically They said “an ai team got first therefore ai won”

@JaundicedBaboon isn't it just whoever wins the contest?

@TimothyJohnson5c16 it doesn’t explicitly say that, but it’s kind of implied. Though I feel it would be kinda dumb if it resolves yes despite humans doing better on average, I’m biased lol

@JaundicedBaboon I get that, but I think that the default assumption here that winning in the context of a hackathon means coming first should still hold. Also, it feels the sense of "beat", is referring to the strongest team, and not the average. Otherwise you could just have some teams that performed pretty badly in a way that is not representative of the capabilities of man or machine bring either average down.

I think who performed best on average is still interesting, and perhaps this should've been a separate question, both with clear resolution criteria. I think in the event that one wins first place and the other wins average and there's significant controversy then the market should resolve N/A. I think resolving YES given the event organisers have given the opposite ruling would be a mis-resolution, assuming no actual serious shenanigans like them straight-out lying are involved.

@Jasonb https://manifold.markets/JaundicedBaboon/in-the-metrfactory-man-vs-machine-c Just made that market. Not too late. Especially given the results we saw with METR's open-source coding study I would not be surprised to see man take the win

@Jasonb Most of us would generally say it's a bad idea to resolve N/A due to predictable ambiguity in the resolution criteria. In this case I think the natural interpretation is "whoever the Hackathon declares as the official winner". But either way, it's incumbent on traders to clarify before trading. It's important not to resolve N/A purely due to controversy because that incentivizes people on the losing side of the market to drum up controversy.

opened a Ṁ200 YES at 28% order

I'm betting YES on some selection effects - the guy who doesn't use AI is an experienced curmudgeon and lethal. I do think he should try them though!

How do you know which group he will be in?

© Manifold Markets, Inc.TermsPrivacy