[Math test 1] What will GPT-4 Vision score on the Mathematics section of JEE Advanced paper? (out of 60) (\$200M sub)
24
421
1.5K
Jan 31
15%
<0 (i.e. negative score)
29%
0-10
16%
10-20
12%
20-30
11%
30-40
6%
40-50
7%
50-60
3%
60 (the perfect score)

Since I do not have access to GPT-4 Vision yet and someone else might, I will not reveal the year of the paper I will test GPT-4 vision on. Here is a MD5 hash specifying the year, so as to confirm later:

1064009a1779660808da7382ff1b094b

JEE Advanced is one of the hardest tests at a pre-college level in the world.

The marking scheme is as follows:

+4 ONLY if (all) the correct option(s) is(are) chosen

+3 If all the four options are correct but ONLY three options are chosen

+2 If three or more options are correct but ONLY two options are chosen, both of which are correct

+1 If two or more options are correct but ONLY one option is chosen and it is a correct option

0 If none of the options is chosen (i.e. the question is unanswered)

-2 In all other cases

All the options are [A,B) format where A is included and B is excluded. For example, 10-15 will be chosen if the score is 10 or 11 or 12 or 13 or 14 but not if the score is 15.

Get Ṁ1,000 play money
Sort by:

@firstuserhere I see you pushing back the res date for the JEE Advanced paper markets; do you plan to continue extending them until something specific happens? The probability probably changes over time as it gets more likely that openai tweeks the current models and makes them better, which is why I'm asking

@TheBayesian I actually started testing them, but I got busy with work. So far I've evaluated chemistry ones but will be free to do this only the following week, until then, pretty busy! I will share a github where I'll upload all the tests

bought Ṁ1 of 60 (the perfect score) YES

Looking forward to your upcoming blogpost on all the GPT-4 Vision tests. Would you consider livestreaming some evaluations?

A few sample questions from JEE Advanced 2022 paper: