Finished training.
it should be competitive with SOTA of the time. Rough estimates ofc but not too far behind.
I will not trade in this market.
1,000People are also trading
@mods
State of the Art (SOTA) in AI and machine learning refers to the best-performing models or techniques for a particular task at a given time, typically measured by standard benchmarks or evaluations:
- Paper with specific details on BaGuaLu https://dl.acm.org/doi/10.1145/3503221.3508417
Other resources pointing in the direction with this model trained with above 10 trillion parameters:
https://www.nextbigfuture.com/2023/01/ai-model-trained-with-174-trillion-parameters.html
Kind regards
The evaluation shows that BaGuaLu can train 14.5-trillion-parameter models with a performance of over 1 EFLOPS using mixed-precision and has the capability to train 174-trillion-parameter models, which rivals the number of synapses in a human brain.
https://dl.acm.org/doi/10.1145/3503221.3508417
https://keg.cs.tsinghua.edu.cn/jietang/publications/PPOPP22-Ma%20et%20al.-BaGuaLu%20Targeting%20Brain%20Scale%20Pretrained%20Models%20w.pdf
https://www.nextbigfuture.com/2023/01/ai-model-trained-with-174-trillion-parameters.html
Y Combinator using this as a joke headline in November. https://www.youtube.com/watch?v=lbJilIQhHko
@Mira Facebook one year later:
Llama 4 Maverick, a 17 billion active parameter model with 128 experts...
@BarrDetwix yes you can :) it'll be a tad bit costly though. And I think i should ammend the description to say that it should be competitive with SOTA of the time
Apologies for the title change, I will compensate if you want me to. I've changed title to AI model instead of LLM because we soon will have VLMs (vision language models) become popular, etc.