October market
https://manifold.markets/ChristopherRandles/global-average-temperature-october

@aenews Ah.. I should have checked https://data.giss.nasa.gov/pub/gistemp/

They did indeed use the very first GHCNm one that came out (which was the 10-17 one that came out on the 18th)

I suppose you were betting that they wouldn't revise it since they had already updated the data on their website with this file? This caused me to lose too much mana... 😭

      ghcnm.tavg.qcf.dat                                    2024-10-18 12:08  162M  
      ghcnm.tavg.qcu.dat                                    2024-10-18 12:08  163M  
      ghcnm.tavg.v4.0.1.20241017.qcf.dat                    2024-10-18 12:08  162M  
      ghcnm.tavg.v4.0.1.20241017.qcu.dat                    2024-10-18 12:08  163M  

@parhizj Naw it didn't update there until after they posted the update on Twitter.

@aenews Ah.

On one note some of their SSTs I get from the program I wrote are very different from the ERSSTv5 they just posted (I don't know what is going on).

Furthermore, they seem significantly different compared to the version they put out last month (is it because the data got revised or an error somewhere?).

Edit: Print the SBBX for this month and last month (they keep the last version at https://data.giss.nasa.gov/pub/gistemp/SBBX.ERSSTv5_traditional.gz - I checked it against my own copy that I pulled a few days ago and its the same), and look at the first output for instance from 1880, for the first gridbox compared to last month (completely different temps).

Last month's version:

Sea Surface Temperature anom (C) ERSSTv5 01/1880 - 08/2024 1880-2024 varying base period

latitude-rnge longitude-range year Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

64.16 65.51 -180.00 -171.00 1880 9999.00 9999.00 9999.00 9999.00 9999.00 9999.00 -1.79 -1.44 -1.52 9999.00 9999.00 9999.00

This months' version:

Monthly Sea Surface Temperature anom (C) ERSSTv5 01/1880 - 09/2024 1880-2024 varying base period

latitude-rnge longitude-range year Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

64.16 65.51 -180.00 -171.00 1880 9999.00 9999.00 9999.00 9999.00 9999.00 9999.00 5.62 6.99 5.47 9999.00 9999.00 9999.00

Edit: My own calculation from this months ersstv5 (they did change from last month but nowhere near the same amount as in their SBBX version):

Monthly Sea Surface Temperature anom (C) ERSSTv5 01/1880 - 09/2024 1880-2024 varying base period

latitude-rnge longitude-range year Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

64.16 65.51 -180.00 -171.00 1880 9999.00 9999.00 9999.00 9999.00 9999.00 9999.00 -1.06 -0.93 -0.68 9999.00 9999.00 9999.00

On another note:

I spent some time trying to get their programs for ERSST working that they put on their website. Even after modifying the programs so they do all compile there too many implementation details and problems remaining and I can't reproduce the same ERSST data (it's not even close RMSE wise so there is something wrong with what I'm doing):

1) They expect the (monthly separated) fortran binary files for the ssts as an input, but don't specify what should be the fill value for masked values. I've tried a couple different solutions (9999 or 0) and don't get the same results. Furthermore, it seems from the code for MaskRegrid.f they don't even have handling for masked data. After adding in code to skip those cells for a fill value (including weights/etc) I still don't get sensible results (I might be doing something wrong here though).

2) There is also the problem of the original script passing a tuning parameter of 0.5 to MaskRegrid.f (which is supposed to skip cells that are mostly ice by area in the subboxes); from their 2010 paper they say they weren't doing this any more from my read of their paper, so I don't know if its old code (this seems more likely, and it should be passed 0.0) or whether they have changed back since then.

3) Some other small pieces of code also needs to be changed to get it to work that I don't know if it is affecting its results. (i.e., trimSBBX.f line 66 needs to be changed to KM=12 to work properly).

Regardless, the combination of these three types of problems makes me unable to reproduce their ERSST data.

1.26

@ChristopherRandles Something weird is going on...

Did they use the very first monthly ghcnm and none of the updated ones? That seems to be the case...

@parhizj Hmm I always intended to resolve according to the first version published that I see, or someone posts, and later amendments would have no effect.

However if it is an error at the time it is published then there would certainly be an argument for it should be re-resolved.

Distinguishing between those might not be easy and perhaps I should stick to first published version that I see, or someone posts, in order to keep resolution simple.

Anyone want to argue for something different (preferably something that is reasonably easy to resolve)?

@ChristopherRandles I'm not really disputing the market resolution, just wondering what is going on. Right now trying to run their own programs to generate the ERSST data but its not as straightforward as they laid it out as there are some more odd hardcoded things it seems with what they store in the titles of the binary files and I dont have samples.

"News and Updates

October 18, 2024: The GISTEMP monthly update that had been planned for release on October 9 will be posted on October 21. The release was delayed as we waited for our primary source of input data, the NOAA National Centers for Environmental Information data center, to return to something like normal operations following the effects of Hurricane Helene on their home city, Asheville, N.C."

A new GHCNm came out ~ a couple hours ago, and from the run I just did it was 1.23 C again.

Still haven’t updated tables but the chart shows 1.24

something I didn’t notice, but they’ve also posted the scripts and code to generate the ersst data now.

“

October 20, 2024: The programs and the data file used to create SBBX.ERSSTv5 from NOAA/NCEI's data are now available on the Sources page in the Ocean data section..”

Would have saved me some time if they posted it two months ago 😭

@parhizj Any word from NOAA re when they'll release Sept?

@ScottSupak I looked through their website earlier and on twitter earlier and couldn't find anything.

New GHCNm came out an hour ago. Latest run holds the same as yesterday, at 1.23 C

GHCNm came out a little more than 5 hours ago.

I get 1.26 C for the run I just did.

A revised ghcnm came out early this morning (ghcnm.v4.0.1.20241018)

I redid the run and it has dropped substantially to 1.23 C

@aenews and other bettors on polymarket might want to consider hedging on the 1.17-1.22 bin ...

@parhizj Thanks, just bought some in case. I ran it a couple hours back and also got the same as you. Unclear which GHCN file NASA/GISTEMP will use, given the delays.

@aenews For some reason I think its likely they'll use the very last available one (the day before they release the official data). It doesn't take much time to run. Maybe they will wait up to 3-5 more days in case there are any more revisions...

@parhizj They used the first version, haha

Data delayed by hurricanes

@ChristopherRandles ERSSTv5 became available today. Waiting on GHCNM...

I'm still predicting about ~1.23 C (using a large downward correction though of -0.11). During July this (per-month) correction if I recall correctly showed to be too large (by about twice as much?) so perhaps a smaller correction might have been in order, however this is as best I can do ... if it ends up becoming ~1.28 I might have to reconsider cutting the per-month corrections in half. For now though it looks like 1.17-1.22 is very cheap on Polymarket ....

GIS TEMP anomaly projection (September 2024) (corrected, assuming -0.109 error, (absolute_corrected_era5: 16.063)):
  1.228 C +-0.075

@aenews Saw you bet up 1.195.

My best guess is at the moment 1.229 C +-0.075. Care to share yours?

I have a large CI though as usual .. (taking a 0.05 offset presently):

@parhizj My rough estimate from solely ERA5 is 127 +/- 6

Trying to use default bets on whether its higher or lower each day than my point estimate (as long as it doesn't move it past it) and limit orders on my Z=2 CI's for this month; will see how it works out...