Since I do not have access to GPT-4 Vision yet and someone else might, I will not reveal the year of the paper I will test GPT-4 vision on. Here is a MD5 hash specifying the year, so as to confirm later:
1064009a1779660808da7382ff1b094b
JEE Advanced is one of the hardest tests at a pre-college level in the world.
The marking scheme is as follows:
+4 ONLY if (all) the correct option(s) is(are) chosen
+3 If all the four options are correct but ONLY three options are chosen
+2 If three or more options are correct but ONLY two options are chosen, both of which are correct
+1 If two or more options are correct but ONLY one option is chosen and it is a correct option
0 If none of the options is chosen (i.e. the question is unanswered)
-2 In all other cases
All the options are [A,B) format where A is included and B is excluded. For example, 10-15 will be chosen if the score is 10 or 11 or 12 or 13 or 14 but not if the score is 15.
Related questions
@firstuserhere I see you pushing back the res date for the JEE Advanced paper markets; do you plan to continue extending them until something specific happens? The probability probably changes over time as it gets more likely that openai tweeks the current models and makes them better, which is why I'm asking
@TheBayesian I actually started testing them, but I got busy with work. So far I've evaluated chemistry ones but will be free to do this only the following week, until then, pretty busy! I will share a github where I'll upload all the tests
