Will my custom optimizer (Adalite) outperform Adam on evaluation loss in more than 1 of my tests? | Manifold

Will my custom optimizer (Adalite) outperform Adam on evaluation loss in more than 1 of my tests?

3

110Ṁ40

resolved Feb 25

Resolved

N/A

1H

6H

1D

1W

1M

ALL

Code at https://github.com/euclaise/SlimTrainer/

I plan to run small tests with CNNs (e.g. Fashion MNIST) and transformers (e.g. Wikitext), as well as a comparison on finetuning LLMs.

GitHub - euclaise/SlimTrainer: Full finetuning of large language models without large memory requirements

Full finetuning of large language models without large memory requirements - GitHub - euclaise/SlimTrainer: Full finetuning of large language models without large memory requirements

Get

1,000

to start trading!

People are also trading

Will Adam optimizer no longer be the default optimizer for training the best open source models by the end of 2026?

By 2027 will Adam be replaced by a novel optimization algorithm?

Related questions

Will Adam optimizer no longer be the default optimizer for training the best open source models by the end of 2026?

By 2027 will Adam be replaced by a novel optimization algorithm?

© Manifold Markets, Inc.•Terms + Mana-only Terms•Privacy•Rules