How would you expect to prove this either way if the training data is at least partially a trade secret?

Useful: https://arxiv.org/abs/2101.00027 includes

youtube transcripts

The Pile: An 800GB Dataset of Diverse Text for Language Modeling

Recent work has demonstrated that increased training dataset diversity improves general cross-domain knowledge and downstream generalization capability for large-scale language models. With this in mind, we present \textit{the Pile}: an 825 GiB English text corpus targeted at training large-scale la…

@tftftftftftftftftftftftf True, but only 0.6% of the dataset.

Will more than 5% of GPT-4’s training data be YouTube transcripts?, 8k, beautiful, illustration, trending on art station, picture of the day, epic composition