Will anyone give me an easy way to programmatically fetch new markets in <5 seconds?
2
222
แน€90
resolved Sep 28
Resolved
YES

The /markets endpoint of the Manifold API only reflects new markets 15-60 seconds after they've been created, which is slower than I'd like. (I'm trying to make my search page update right away so it's more useful for live trading.)

Suggestions to get around this limitation of the API have included:

Those both seem like a pain. (I've never used Supabase, don't have an API key for it, don't know how its output is formatted, etc.)

Get แน€600 play money

๐Ÿ… Top traders

#NameTotal profit
1แน€42
2แน€15
Sort by:

Ok, ignoring the overcomplicated code and just using the URL that was in the github repo, here's what I'm looking at:

https://pxidrgkatumlvfqaxcll.supabase.co/rest/v1/contracts?order=created_time.desc&limit=100&apikey=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJzdXBhYmFzZSIsInJlZiI6InB4aWRyZ2thdHVtbHZmcWF4Y2xsIiwicm9sZSI6ImFub24iLCJpYXQiOjE2Njg5OTUzOTgsImV4cCI6MTk4NDU3MTM5OH0.d_yYtASLzAoIIGdXUBIgRAGLBnNow7JG2SoaNMQ8ySg

Seems pretty good, except that most of the markets are missing the url field? I guess I can recreate it using the creatorUsername + slug. Are there any other issues I should be aware of?

predicted YES

@IsaacKing Those complications and a little more (the code I shared was a simplified version and not what I actually do) will be required if you want to ensure you don't miss any markets and can catch on what you missed if you have downtime. Otherwise, seems fine to me.

@chrisjbillington Mind sharing what the other complications are?

predicted YES

@IsaacKing can share code later when I'm at a computer, but that you'll need to call it in a loop with a created_time < some_timestamp filter so that you can catch up on old markets, when you're behind by more than the limit you set as the max number of markets returned in a single query.

Or something like that. Will show code in a bit.

@chrisjbillington That's the same as on the Manifold API, no? They're both limited to a max of 1000 markets at a time.

predicted YES

@IsaacKing I gave you code, all indications are that it does what you asked, you've been active on the site since, can this resolve?

predicted YES

Resolution pls @IsaacKing?

predicted YES

Related, to see if it's worth my time engineering a workaraound vs. waiting for it to get fixed.

bought แน€100 of YES

I'll post some code when I'm out of bed, someone remind me if I forget

bought แน€50 of YES

@chrisjbillington Here you go @IsaacKing :

https://gist.github.com/chrisjbillington/e17a18acb53ce6c912530e25179cde1b

Is fast for me:

got 1000 new markets in 1.6s
got 0 new markets in 0.3s
got 0 new markets in 0.3s
bought แน€300 of YES

@chrisjbillington Oh and it's fast with respect to the market being created and you getting the data soon, which is probably what you meant. I created a market and saw it appear instantly upon the next call. So it looks to be as fast as the sleep duration you set.

note that we may deprecate the supabase client library in the future with little or no warning though. it's not a priority to. it may become more of a priority if flash bots start eating into our bills. (I don't have a good sense of how much usage is reasonable)

I do see that it's quite convenient tho, and am in favor of people doing things rather than not doing things

@Sinclair Any chance the Manifold API could be made to update faster? That seems like the ideal solution.

@chrisjbillington Do you know how to do this with bets instead of markets?

@EliLifland Figured it out, just need to replace โ€˜contractsโ€™ with โ€˜contract_betsโ€™

bought แน€10 of YES

here's a decent starting point for hacking on the supabase api that @chrisjbillington slapped together at my behest at some point: https://gist.github.com/chrisjbillington/243dba897d5cf7a71edc0fc486ef93b8

I assume at least part of this might be useful for listening to the live page, though I imagine it's socket based rather than a get request.

@brubsby I'm using Node, but this seems pretty straightforwards to translate over. I was worried it would be a more complicated process.

What specifically is this code fetching?

predicted YES

@IsaacKing That's an example of using the supabase API generally, but I've posted an example above that gets the latest markets (still in Python).

It's been more than five seconds and nobody has provided a way, resolve NO