Skip to content

Podcast

Open Source Wins, AGI Is Here, and Scorsese's AI Toolkit With CEOs of Cerebras & Black Forest Labs

All-In with Chamath, Jason, Sacks & Friedberg

Source ↗ ← All highlights
  • AI Data Center Buildout Outstrips Supply
    • The AI buildout is unprecedented with data centers using more power than the previous 50 years and customers pre-ordering capacity like OpenAI and Google.
    • Andrew Feldman cites a $25 billion Cerebras backlog and buildings with power like midsize cities being constructed worldwide. (Time 0:02:00)
  • Match Model Tier To Task Urgency
    • Run models with appropriate cost-tiering: use frontier models for hard problems and open-source or cheaper models for routine tasks.
    • Feldman compares this to using a Ferrari sometimes and a minivan other times to avoid wasting resources. (Time 0:05:52)
  • Models Now Understand Intent Not Just Prompts
    • Modern models increasingly infer user intent instead of requiring perfect prompts, reducing the need for ‘prompt whisperers’.
    • Feldman notes models like Fable and OpenAI 5.6 proactively offer alternate outputs (e.g., line and bar chart) that match user intent. (Time 0:07:14)
  • Unlimited Tokens Enable Deep Reasoning Loops
    • Unlimited tokens unlock extended reasoning: running long inference loops (24–48+ hours) produces qualitatively better outputs.
    • Feldman explains faster hardware (Cerebras) multiplies the effective reasoning you can run in a given clock time. (Time 0:10:20)
  • New Architectures Outpace Moore’s Law
    • Cerebras broke the traditional Moore’s Law trajectory for processors by achieving >2x improvements in the next 18 months through architecture and system-level optimizations.
    • Feldman contrasts newer architectures’ headroom versus 20-year GPU designs constrained by fab nodes. (Time 0:13:06)
  • Avoid Single-Vendor Dependence For Critical AI
    • Reduce vendor dependency by developing in-house or domestic open-source models when sovereignty matters.
    • Feldman points to hyperscalers’ lessons from x86/Intel and the rise of neoclouds making custom silicon to control destiny. (Time 0:16:10)
  • Sovereignty Drives Demand For Domestic OSS Models
    • Sovereignty and domestic open-source models are rising priorities for regulated industries and governments.
    • Feldman highlights OSS 120B, Chinese models, and the need for more US domestic open-source choices to give customers options. (Time 0:18:36)
  • Phased Releases And Red Teaming Are Reasonable
    • Incremental, government red-teaming and phased rollouts for powerful models are reasonable to patch systemic vulnerabilities.
    • Feldman compares staged AI releases to pharmaceutical-style safety checks and red teaming for national infrastructure. (Time 0:21:30)
  • Best Questions Reveal What You Didn’t Ask
    • Feldman recalls investors asking the smartest question he hadn’t covered, signaling curiosity and humility as markers of good questioning.
    • He parallels that with asking AIs to surface what you didn’t think to ask to broaden perspective. (Time 0:28:30)
  • Recursion Drives Rapid AI Improvement
    • Recursive loop maxing causes exponential improvements: iterate, learn, and re-run to produce vastly better results, and the stopping point is uncertain.
    • Feldman warns throwing more compute keeps improving answers until token or budget limits intervene. (Time 0:31:52)
  • AI Could Deliver Massive Abundance Despite Disruption
    • AI can deliver transformative societal benefits like eliminating cancer deaths, though economic dislocation will occur.
    • Feldman frames tradeoffs: massive abundance (education, housing, health) versus transitional job disruption similar to past technology shifts. (Time 0:38:15)
  • Latent Diffusion Origin Story
    • Robin Rombach described inventing latent diffusion during his PhD, which became the foundation for Stable Diffusion and modern generative image models.
    • He explains compressing images into efficient representations then training transformers on that latent space as the core mechanism. (Time 0:41:30)
  • Multimodal Pretraining Enables Action Prediction
    • Multimodal pretraining on images, video, and audio yields implicit physics and action prediction useful for robotics and real-world understanding.
    • Robin argues shared models can both generate media and predict actions, enabling robot deployment from the same backbone. (Time 0:42:51)
  • Expose Control Layers For Better Generation
    • Exposing manipulation layers (image-to-image, multi-image blending, text+image) makes generative tools useful and controllable for users.
    • Robin explains expanding modalities and exposing controls as the path to more precise creative outcomes, especially in video. (Time 0:45:33)
  • Keep Humans Central In Creative AI Workflows
    • Use generative AI as a creative medium with humans in the loop for best production outcomes rather than fully automated end-to-end film generation.
    • Robin emphasizes storyboarding, iterative vision work, and directors like Martin Scorsese using models to externalize ideas. (Time 0:48:10)
  • Scorsese Used AI To Externalize Visual Ideas
    • Robin sat with Martin Scorsese and iterated scenery outputs, helping Scorsese externalize mental images into visuals for potential scenes.
    • Scorsese used the tool to convey a village in Eastern Europe and described it as getting the mental picture out of his head. (Time 0:48:16)
  • Generative Sets Reduce Production Costs Dramatically
    • High-end film production is a demanding use case already using generative backgrounds and interactive sets to reduce costs and unlock productions that otherwise wouldn’t be greenlit.
    • Robin cites a Bitcoin movie using AI-generated scenery on a soundstage to cut budgets from $150M to $30M. (Time 0:53:18)
  • Fine-Tune Visually Pretrained Models For Robots
    • Fine-tune multimodal models with a small amount of in-context or task-specific data rather than retraining from scratch for each robot or factory setup.
    • Robin recommends few hours of fine-tuning on task-specific hardware mappings after large visual pretraining for deployable action prediction. (Time 0:57:01)