"The path to superintelligence - just train up the LLMs, train on more synthetic data, hire thousands of people to school your system in post-training, invent new tweaks on RL-I think is complete bullshit. It's just never going to work."

After 12 years at Meta, Turing Award winner Yann LeCun is betting his legacy on a radically different vision of AI. In this conversation, he explains why Silicon Valley's obsession with scaling language models is a dead end, why the hardest problem in AI is reaching dog-level intelligence (not human-level), and why his new company AMI is building world models that predict in abstract representation space rather than generating pixels.

(00:00:14) – Intro and welcome

(00:01:12) – AMI: Why start a company now?

(00:04:46) – Will AMI do research in the open?

(00:06:44) – World models vs LLMs

(00:09:44) – History of self-supervised learning

(00:16:55) – Siamese networks and contrastive learning

(00:25:14) – JEPA and learning in representation space

(00:30:14) – Abstraction hierarchies in physics and AI

(00:34:01) – World models as abstract simulators

(00:38:14) – Object permanence and learning basic physics

(00:40:35) – Game AI: Why NetHack is still impossible

(00:44:22) – Moravec's Paradox and chess

(00:55:14) – AI safety by construction, not fine-tuning

(01:02:52) – Constrained generation techniques

(01:04:20) – Meta's reorganization and FAIR's future

(01:07:31) – SSI, Physical Intelligence, and Wayve

(01:10:14) – Silicon Valley's "LLM-pilled" monoculture

(01:15:56) – China vs US: The open source paradox

(01:18:14) – Why start a company at 65?

(01:25:14) – The AGI hype cycle has happened 6 times before

(01:33:18) – Family and personal background

(01:36:13) – Career advice: Learn things with a long shelf life

(01:40:14) – Neuroscience and machine learning connections

(01:48:17) – Continual learning: Is catastrophic forgetting solved?

Music:

"Kid Kodi" — Blue Dot Sessions — via Free Music Archive — CC BY-NC 4.0.

"Palms Down" — Blue Dot Sessions — via Free Music Archive — CC BY-NC 4.0.

Changes: trimmed

The Information Bottleneck is hosted by Ravid Shwartz-Ziv and Allen Roush, featuring in-depth conversations with leading AI researchers about the ideas shaping the future of machine learning.

Atlas Wang (UT Austin faculty, XTX Research Director) joins us to explore two fascinating frontiers: the foundations of symbolic AI and the practical challenges of building AI systems for quantitative finance.

On the symbolic AI side, Atlas shares his recent work proving that neural networks can learn symbolic equations through gradient descent, a surprising result given that gradient descent is continuous while symbolic structures are discrete. We talked about why neural nets learn clean, compositional mathematical structures at all, what the mathematical tools involved are, and the broader implications for understanding reasoning in AI systems.

The conversation then turns to neuro-symbolic approaches in practice: agents that discover rules through continued learning, propose them symbolically, verify them against domain-specific checkers, and refine their understanding.

On the finance side, Atlas pulls back the curtain on what AI research looks like at a high-frequency trading firm. The core problem sounds simple (predict future prices from past data). Still, the challenge is extreme: markets are dominated by noise, predictions hover near zero correlation, and success means eking out tiny margins across astronomical numbers of trades. He explains why synthetic data techniques that work elsewhere don't translate easily, and why XTX is building time series foundation models rather than adapting language models.

We also discuss the convergence of hiring between frontier AI labs and quantitative finance, and why this is an exceptional moment for ML researchers to consider the finance industry.

Links:

• Why Neural Network Can Discover Symbolic Structures with Gradient-based Training: An Algebraic and Geometric Foundation for Neurosymbolic Reasoning - arxiv.org/abs/2506.21797
• Atlas website - https://www.vita-group.space/

Guest: Atlas Wang (UT Austin / XTX)

Hosts: Ravid Shwartz-Ziv & Allen Roush

Music: “Kid Kodi” — Blue Dot Sessions. Source: Free Music Archive. Licensed CC BY-NC 4.0.

In this episode, we hosted Judah Goldfeder, a PhD candidate at Columbia University and student researcher at Google, to discuss robotics, reproducibility in ML, and smart buildings.

Key topics covered:

Robotics challenges: We discussed why robotics remains harder than many expected, compared to LLMs. The real world is unpredictable and unforgiving, and mistakes have physical consequences. Sim-to-real transfer remains a major bottleneck because simulators are tedious to configure accurately for each robot and environment. Unlike text, robotics lacks foundation models, partly due to limited clean, annotated datasets and the difficulty of collecting diverse real-world data.

Reproducibility crisis: We discussed how self-reported benchmarks can lead to p-hacking and irreproducible results. Centralized evaluation systems (such as Kaggle or ImageNet challenges), where researchers submit algorithms for testing on hidden test sets, seem to drive faster progress.

Smart buildings: Judah's work at Google focuses on using ML to optimize HVAC systems, potentially reducing energy costs and carbon emissions significantly. The challenge is that every building is different. It makes the simulation configuration extremely labor-intensive. Generative AI could help by automating the process of converting floor plans or images into accurate building simulations.

Links:

• Judah website - https://judahgoldfeder.com/

Music:

"Kid Kodi" — Blue Dot Sessions — via Free Music Archive — CC BY-NC 4.0.

"Palms Down" — Blue Dot Sessions — via Free Music Archive — CC BY-NC 4.0.

Changes: trimmed