Resolves YES if a service or API exists before 2026 which does the following:
You upload text document(s) for fine-tuning
The service does full parameter fine-tuning on Llama 3.1 405B, without you having to rent GPUs
You can download the fine-tuned model
Extra Details:
I’ll accept a service which finetunes a model other than Llama 3.1 405B, so long as it is more capable (as judged by e.g. the lmsys leaderboard) and has at least 400B parameters. E.g. if Meta releases Llama 4 405B, that counts
The service must accept a large corpus of documents, >= 10B tokens in total
By full-parameter fine-tuning, I’m excluding methods like LoRA/QLoRA which do low rank updates. Similarly I’m excluding memory efficient optimizers (e.g. AdaFactor). The service should use AdamW or a similar optimizer without excessive quantization; all optimizer states, gradients, parameters, buffers etc. should be in at least 8 bit precision.
It should be generally available, e.g. like the OpenAI fine-tuning API. You shouldn’t have to consult with people for your specific fine-tuning job before using it.
If you think such a service exists, please post in the comments below. I’ll make a reasonable effort to confirm that it meets the requirements and resolve if it does.
1,000