 uses an LLM to read market descriptions and automatically trade on Manifold Markets. However, as of January 2024, it doesn't make a consistent profit:

There's probably a lot of room to improve a bot like this, even just with currently-available technology. For instance, existing data on past market resolutions on Manifold, Metaculus, and other prediction markets would likely be very useful for fine-tuning.

This is the main market for what I'm calling the Motley Bot Challenge (so-called because it rewards bots that are accurate on a very diverse range of questions). My hope is that this challenge encourages the creation of bots that can be scaled to increase Manifold's site-wide prediction accuracy and improve our understanding of the world!

On December 1, 2024, I will select 1,000 random YES/NO markets on Manifold that will resolve in about one month (see "Selecting markets" for more information). I will post a .txt file containing links to the 1,000 markets in the comments of this market.

For each market, a bot must do one of the following:

On January 8, 2025, I will determine which bot has the most profit across its 1,000 bets. Manifold bot fees do not count against profit. Unrealized profit from unresolved markets does count for profit.

I will resolve the market based on this maximum profit as follows:

Note that a bot can invest up to Ṁ1,000, so if it invested all of the mana it could, a profit of Ṁ100 would mean it made a 10% return.

If no bots enter the challenge, this market resolves NO.

Discord server for discussion and collaboration: 

https://manifold.markets/browse?topic=motley-bot-challenge

The script that will be used to select markets and a "Basic Bot" to start you off: 

https://github.com/CDBiddulph/motley-bot-resources/

I will use the Manifold API to choose random binary markets (

). I will search through as many markets as necessary to get 1,000 eligible markets. Markets will be filtered to fit the following criteria:

If there aren't enough markets that meet these criteria, I'll progressively broaden my filter as follows until I have enough markets:

If there still aren't enough markets, I'll post M<1000 markets (as many markets as I could find). Then I'll scale the actual profit by 1000/M. For instance, if I could only find 800 markets, and then the actual profit made by the highest-performing bot is Ṁ40, I would resolve the market to 40*(1000/800) = 50%.

Bots may use any information accessible via the Manifold API or on the Internet. Attempts to "time" individual markets are not allowed - you must make all of your trades at once, sometime before the end of December 1 (PST).

To formally enter your bot in this challenge, make a comment in this market with a link to the bot's Manifold page and another public link to its source code and add your bot's name as an option in 

. I would also highly recommend joining the 

. Each participant may only submit one bot, but teams working on the same bot are allowed and encouraged.

For ease of scoring, a bot shouldn't perform any trades other than the 1,000 trades that it makes in this challenge. You may want to make a separate bot for testing. You can trade on any markets you want with your testing bot, as long as they are ineligible for being selected for the challenge (i.e. they would not fulfill even the most relaxed set of requirements described in the "Selecting Markets" section).

Adapting other people's code for your bot is allowed. (Please give them credit though.) Right before you push code or release information about your bot, you can use your insider information to bet in this market or derivative markets. Assuming everyone accurately assesses the quality of their own work and trades accordingly, someone who builds off of someone else's work only profits on whatever additional value they created.

If your bot is based on someone else's code, it must be substantially different according to my judgement. Generally speaking, your change should add a substantial new strategy to the bot - simply changing a few parameters or prompts would not qualify.

Hypothetically, someone could use their bot's or someone else's bot's code to trade on a bunch of eligible markets ahead of time, wiping out whatever alpha the bot might have. I don't think this will be a problem, but please don't do this! I want everyone's bots to be open source so that people can build on each other's ideas, but if it becomes a concern, I may consider allowing code to be closed source until all bots have placed their bets.

As more bots participate in this challenge, it becomes more likely that one of them will achieve the highest score due to luck rather than true predictive accuracy, inflating the expected score of this market. To keep the number of bots from getting out of hand, I will only consider the bots that are at least partially included in the top 95% of probability mass in the market "

What will be the most profitable bot in the Motley Bot Challenge?

" on December 1, 2024. For a market that only considers a single bot, eliminating the effect of variance, see 

I may add/change rules as necessary to preserve the spirit of the market. For example, I might add to the list of tags that would disqualify a market from being selected. Please suggest any changes you think I should make!

I will personally fund the 4 qualifying bots that rank highest on 

 up to Ṁ1250 each. This should cover the maximum number of trades plus Manifold bot fees.

 Clarified what happens if I can't find 1,000 eligible markets. Suggested using a separate bot for testing. Added notes about the pros and cons of open source code. Added a rule that the bots must be substantially different from each other. Limited the bots that can participate to the top 95% of 

 Added a Discord server. Came up with the name "Motley Bot Challenge" and added it as a tag.

 Made clarifications in response to comments from 

 Clarified that close times will be based on the UTC time zone.

 Added a link to the GitHub repository with the script for selecting markets.

 Added a "Basic Bot" script and upgraded the market to Crystal.