Will I Be Able to Fine-Tune a 1B Parameter LLM on Prediction Market Data to Give Good Outputs by End of March?
28
178
490
resolved Apr 11
Resolved
NO
  • Must have a publicly accessible endpoint (though might be invite only to prevent crashing).

  • Will form resolution council of users to evaluate outputs.

  • This market is subjective, but we have to come up with some kind of scope of input parameters. The wider the scope of possible questions that could be asked, presumably the harder this will be to complete, the narrower the scope, presumably the easier this will be to complete.

This is a Continuation of Two Previous-Closing Markets:

End of January Market:

https://manifold.markets/PatrickDelaney/will-i-be-able-to-finetune-a-1b-par

End of February Market:

https://manifold.markets/PatrickDelaney/will-i-be-able-to-finetune-a-1b-par-1e2a645ec277

Q&A To Clarify Market (Updating from Comments in Previous Threads):

Q: What's the LLM being fine tuned to do exactly?

A: Answer questions about the likelihood of a basket previously asked questions, e.g. for a question, "Will A Happen?" which resolved as outcome YES, it needs to be able to give a positive answer.

Q: What's the testing methodology?

A: 1. I start with a massive sample set A. 2. I train on part of sample set A, call it B. 3. The output quality will be judged based upon applying it to C. which is B' selected from A.

Q: What are the Limitations? Isn't this all just irrelevant if you spend huge amounts of money on hardware?

A: I'm using an old consumer computer with a GeForce 1070 GPU, so there is a severe hardware capability constraint.

Q: What counts as performance having met the threshold?

A: TBD

Get Ṁ200 play money

🏅 Top traders

#NameTotal profit
1Ṁ116
2Ṁ43
3Ṁ35
4Ṁ25
5Ṁ17
Sort by:

Ready to resolve?

bought Ṁ5 of YES

You can use the script/outputs from https://github.com/CDBiddulph/motley-bot-resources/ to get your list of markets :)

More related questions