Latent Space: The AI Engineer Podcast · Latent.Space

METR’s Joel Becker on exponential Time Horizon Evals, Threat Models, and the Limits of AI Productivity

·56 min·5 clips

Joel Becker reveals how METR evaluates whether AI models pose catastrophic threats to society.

Joel Becker explains METR's mission: model evaluation measures current and near-term AI capabilities and propensities — what models will actually do in deployment, not just what they can do on benchmarks — and threat research connects those capabilities to specific threat models to assess whether AI poses catastrophic risks. He describes the framework for evaluating GPT-5 and GPT-5.1, working through a structured case for why these models do not yet pose catastrophic risks: their current capabilities fall short of what would be necessary to cause the specific large-scale harms METR most worries about, though the propensities side of the equation may become more important as capabilities increase. Becker discusses the time horizon evaluation work METR is known for — measuring how far into the future AI assistance can meaningfully extend a human's task completion — and the randomized controlled trial research on developer productivity. He explores the tension between theoretical LLM capability and observed capability in real-world deployment: many tasks that AI could do in principle remain undone because of workflow friction, trust gaps, or institutional barriers. The episode also touches on Manifold prediction market strategy, Becker's background, and what threat models METR thinks about that have not yet received sufficient attention in public AI safety discourse.

Listen to the show on

Apple Podcasts Spotify

Play clips from this episode More from Latent Space: The AI Engineer Podcast