Large meaning >= 20b parameters.
Competitive meaning the benchmark results are close, or better, than a model trained with only human text.
Computer generated text does not count, it has to be the output of a language model. For example, converting code to an intermediate representation of a compiler and training only on that would not count.
Processed text is valid, as long as it's sourced from the language model.
Multiple stages of training are fine, so if there's a training period on only human text, as long as the total AI training examples are > 50% of the total training examples over the entire training this will resolve yes.
Market resolves on March 1st 2025 to account for announcement of models trained in the last half of 2024.
A model extraction attack is enough to resolve this yes, right? Or any kind of distillation process where we train a model and use its output to train a model?
Does Constitutional AI count?
@MartinRandall It would have to be on text, not on logits, so like alpaca and friends are fine if they scaled it up to 750b tokens, but a traditional student/teacher setup is not.
Constitutional AI would count yeah, if it was >50% of the total.