Hey! I'd love to hear your thoughts, send me a voice note.

Modern AI was built on an extraordinary resource: human knowledge.

Books. Websites. Research papers. Code. Images. Video. Conversations. Billions upon billions of examples created by people.

But what happens when frontier models have already consumed most of the useful, accessible human-generated data?

In Episode 9 of Zero to Singularity, we investigate one of the biggest questions facing the next generation of artificial intelligence: Can AI keep improving if high-quality human training data becomes a bottleneck?

This episode explores:

• how pretraining data powers modern AI
• scaling laws and diminishing returns
• the difference between more data and better data
• synthetic training data
• self-play and automated curriculum generation
• reinforcement learning and verifiable tasks
• model-generated reasoning examples
• simulation and virtual environments
• multimodal data from video, audio, robotics, and sensors
• proprietary and expert-generated datasets
• inference-time scaling and test-time compute
• retrieval, tools, agents, and external memory
• model collapse and synthetic-data contamination
• why AI-generated internet content could become a training problem
• whether machines can eventually generate their own useful learning experiences

We also examine some of the biggest claims surrounding the future of AI training:

Is the internet running out of useful data? Can synthetic data replace human-created knowledge? Does training on AI-generated content inevitably cause model collapse? Can self-play generate effectively unlimited training material? And could future AI systems eventually design their own increasingly difficult curriculum?

The deeper question is no longer simply:

How much human knowledge can AI absorb?

It may become:

Can intelligence eventually create the experiences it needs to make itself smarter?