Apart Research

This question will resolve to the minimum 

 submission. The calibrated random baseline is 85. The lower the better. See 

For true/false and multiple-choice questions, we evaluate models using 

, which is then divided by 2 to normalize between 0% and 100%. For numerical questions, we use L1 distance, bounded between 0% and 100%. We denote these question types as T/F, MCQ, and Numerical, respectively. To evaluate aggregate performance, we use a combined metric (T/F + MCQ + Numerical), which has a lower bound of 0%. A score of 0% indicates perfect prediction on all three question types. For more details, please check out the 