[Chemistry test 2] What will GPT-4 Vision score on the Chemistry section of JEE Advanced paper? (out of 60) ($150M sub)
Basic
17
1.5k
resolved Feb 25
ResolvedN/A
7%
<0 (i.e. negative score)
14%
0-10
36%
10-20
13%
20-30
10%
30-40
8%
40-50
8%
50-60
3%
60 (the perfect score)

This is the 2nd market in this series.

Since I do not have access to GPT-4 Vision yet and someone else might, I will not reveal the year of the paper I will test GPT-4 vision on. Here is a MD5 hash specifying the year, so as to confirm later:

9054463c7bf05330be2449bb438f0101

JEE Advanced is one of the hardest tests at a pre-college level in the world.

The marking scheme is as follows:

+4 ONLY if (all) the correct option(s) is(are) chosen

+3 If all the four options are correct but ONLY three options are chosen

+2 If three or more options are correct but ONLY two options are chosen, both of which are correct

+1 If two or more options are correct but ONLY one option is chosen and it is a correct option

0 If none of the options is chosen (i.e. the question is unanswered)

-2 In all other cases

All the options are [A,B) format where A is included and B is excluded. For example, 10-15 will be chosen if the score is 10 or 11 or 12 or 13 or 14 but not if the score is 15.

Get Ṁ600 play money
Sort by:

sorry, I've not gotten around to doing this still, so i will n/a the questions for now . I was curious but sadly didn't find time for this project

I plan to resolve this and make the tests and solutions public in the next 2 days

I will be resolving the chemistry markets first, then the maths, then the physics ones. Going to extend close dates again.

I will be testing this soon.

60 (the perfect score)

Basically impossible unless it is superintelligent or has seen the questions before, which should not be possible if you'll test it on >2021 paper.

@MechInterp Yeah, perfect is <0.5%. But a negative score would surprise me as well.

I predict that the average score for chemistry testing would be strictly (and significantly) better than the average score for both maths and physics for both papers

A few sample questions, taken straight from JEE advanced 2022 paper, so traders may get a feel for the level of difficulty: