šŸ• Will AI Achieve Significantly More, "Embodiment" by end of 2023?
22
closes Jan 1
42%
chance

Preface:

Please read the preface for this type of market and other similar third-party validated AI markets here.

Third-Party Validated, Predictive Markets: AI Theme

Market Description

AI2-THOR Rearrangement Challenge

This question pertains to the following AI challenge:

https://github.com/allenai/ai2thor-rearrangement#----2022-ai2-thor-rearrangement-challenge

  • The goal of this challenge is to build a model/agent that move objects in a room to restore them to a given initial configuration.

  • Example query:

task involves moving and modifying (i.e. opening/closing) randomly placed objects within a room to obtain a goal configuration. There are 2 phases:

  1. Walkthrough šŸ‘€. The agent walks around the room and observes the objects in their ideal goal state.

  2. Unshuffle šŸ‹. After the walkthrough phase, we randomly change between 1 to 5 objects in the room. The agent's goal is to identify which objects have changed and reset those objects to their state from the walkthrough phase. Changes to an object's state may include changes to its position, orientation, or openness.

The resolution for this market will be here:

https://leaderboard.allenai.org/ithor_rearrangement_1phase_2022/submissions/public

robot rearranging a room

Market Resolution Threshold:

  • If any 2022 AI2-THOR Rearrangement Challenge Submission Get a % Fixed Strict (Test) Score of >0.4 by end of 2023, this resolves as YES, otherwise NO.


Mar 22, 12:25pm: Will Any 2022 AI2-THOR Rearrangement Challenge Submission Get a % Fixed Strict (Test) Score of >0.4 by end of 2023? ā†’ Will AI Achieve Significantly More, "Embodiment" by end of 2023?

Get į¹€500 play money

Related questions

Will a large language model beat a super grandmaster playing chess by 2028?
MP avatarMP
48% chance
By the end of 2026, will we have transparency into any useful internal pattern within a Large Language Model whose semantics would have been unfamiliar to AI and cognitive science in 2006?
Will a 10B parameter multimodal RL model be trained by Deepmind in the next 12 months?
BionicD0LPH1N avatarBionic
66% chance
HumanEval 90% #1: Will pass@1 performance on the HumanEval benchmark be >= 90% by 2024?
vluzko avatarVincent Luczkow
69% chance
Will there be a Forward-Forward Algorithm based neural network with >65% Top 1 Accuracy on Papers With Code's ImageNet leaderboard by 2024?
l8doku avatarl8doku
28% chance
Will reinforcement learning overtake LMs on math before 2028?
JacobPfau avatarJacob Pfau
38% chance
Will there be a period of 12 contiguous months during which no new compute-SOTA LM is released, by Jan 1, 2033?
LeoGao avatarLeo Gao
71% chance
Short Term AI 2.3: By January 2024, will SOTA on MATH minival be >= 85%?
vluzko avatarVincent Luczkow
71% chance
Will a model costing >$30M be intentionally trained to be more mechanistically interpretable by end of 2027? (see desc)
NoaNabeshima avatarNoa Nabeshima
58% chance
When will tinygrad train its first MLPerf qualifying model?
Will we learn by EOY 2024 that large AI labs use something like activation addition on their best models?
JSD avatarJSD
32% chance
Will Transformer based architectures still be SOTA for language modelling by 2026?
LeoGao avatarLeo Gao
66% chance
Will a new best accuracy for ImageNet classification be achieved before the end of 2023?
Widden avatarWidden
49% chance
Will SOTA on MATH in Sep 2024 utilize a hard-coded search/amplification procedure?
EliLifland avatarEli Lifland
45% chance
Will anyone train a 50B parameter+ RetNet by the end of 2023?
vluzko avatarVincent Luczkow
32% chance
Will a news article containing the string "love in the time of large language models" be published in 2023
tftftftftftftftftftftftf avatarTessa B
26% chance
By 2025 end, a model exhibits action recognition (video) equivalent to human level accuracy on Something Something V2?
firstuserhere avatarfirstuserhere
58% chance
Will Metaculus have built-in support for reflective latent variables by 2025?
tailcalled avatartailcalled
45% chance
Will the FSRS algorithm be integrated into Anki by 2024?
brubsby avatarbrubsby
76% chance
Will language models be able to solve simple graphical mazes by the end of 2025?
Benx avatarBenx
48% chance
Sort by:
PatrickDelaney avatar
Patrick Delaneypredicts NO
PatrickDelaney avatar
Patrick Delaneypredicts NO

Another one, more of a cross section based on Google Big Bench:

PatrickDelaney avatar
Patrick Delaneypredicts NO

Other relevant markets: