Benchmark Gap #7: Once 10% of the medical Grand Challenges are "solved", how many months will it be before AI are in common use in hospitals for analyzing medical images with minimal human oversight?
2
5
290
2050
64
expected
  • You can find a list of the grand challenges here.

  • Solved: the SOTA model is better than human expert performance. Challenges with no human benchmark available will be decided on a case by case basis.

  • Minimal human oversight - the amount of doctor time devoted to examining medical images is a small fraction of what it was pre-AI. (Of course this will depend heavily on the regulatory environment, but that's unavoidable).

  • If there is strong evidence that the AIs are being used but getting worse results than humans, that does not resolve the question YES. We are unlikely to have such evidence one way or another because measuring hospital efficacy is hard, but if it does happen that AI doesn't count.

Get Ṁ600 play money
Sort by:
bought Ṁ10 of HIGHER

Thank you for putting together the benchmark gap markets. I didn't see them until now. That being said, I bought, "lower," on all of your benchmarks except for this one. I believe that the schelling point for these markets is going to always be lower, as people seem to have mystical ideas about how soon things progress, and there's evidence that ML/AI researchers have in recent history over-estimated timelines (though that estimation capability could obviously change). On this market though, regarding medical stuff, I know for sure that there are legal barriers in place to prevent this from happening on a federal level. When you have legal barriers, that slows things down considerably. While I don't even think that surpassing radiology grand challenges are going to be an insurmountable task, I just think that once they are surpassed, it's going to be decades if not a century before humans in the loop are taken out of the equation. There is a century-long history of doctors having been put in place to put patient-care first, to make the human body inviolable to experimentation without patient will, and massive amounts of legislation in the most advanced country in this field, the US, so it's just a huge barrier to accelerate this, even if the technology is there -- like how Japan still uses fax machines, we will be stuck in time in medical technology (perhaps for good reasons?).

More related questions