Skip to content

Podcast

The Government Knows AGI Is Coming

The Ezra Klein Show

Source ↗ ← All highlights
  • AGI’s Capabilities and Potential Impact
    • Artificial General Intelligence (AGI) aims to create systems capable of performing any cognitive task a human can.
    • These systems could potentially replace humans in demanding jobs, marking a significant shift in labor markets. Transcript: Ezra Klein Was coming. Ben Buchanan What’s coming? I think we are going to see extraordinarily capable AI systems. I don’t love the term artificial general intelligence, but I think that will fit in the next couple of years, quite likely during Donald Trump’s presidency. And I think there’s a view that this has always been something of corporate hype or speculation. And I think one of the things I saw in the White House when I was decidedly not in a corporate position was trend lines that looked very clear. And what we tried to do under the president’s leadership was get the U.S. Ezra Klein Government and our society ready for these systems. Before we get into what it would mean to get ready, what does it mean? Yeah. When you say extraordinarily capable systems, capable of what? Ben Buchanan The sort of canonical definition of AGI, which again is a term I don’t love, is a system. It’ll be good if every time you say AGI, you caveat that you dislike the term. It’ll sink in, right? Yeah, people really enjoy that. I’m trying to get it in the training data, Ezra. A canonical definition of AGI is a system capable of doing almost any cognitive task a human can do. I don’t know that we’ll quite see that in the next four years or so, but I do think we’ll see something like that, where the breadth of the system is remarkable, but also its depth, its capacity To, in some cases, exceed human capabilities, kind of regardless of the cognitive discipline. Systems that can replace human beings in cognitively demanding jobs. (Time 0:04:29)
  • AI’s Unique Development History
    • Unlike other revolutionary technologies, AI development isn’t primarily funded by the Department of Defense.
    • This lack of government involvement limits the DoD’s understanding and influence over AI’s trajectory. Transcript: Ben Buchanan And what’s fascinating to me is that this is the first revolutionary technology that is not funded by the Department of Defense, basically. And if you go back historically, last hundred years or so, nukes, space, early days of the internet, early days of the microprocessor, early days of large-scale aviation, radar, GPS, The list is very, very long. All of that tech is fundamentally comes from DOD money. And it’s the private sector inventing it, to be sure. But the central government role gave the Department of Defense and the U.S. Government an understanding of the technology that by default, it does not have an AI and also gave the U.S. Government a capacity to shape where that technology goes that by default, we (Time 0:07:36)
  • US-China AGI Race
    • The US aims to achieve AGI before China due to its economic and military implications.
    • Ben Buchanan emphasizes the importance of US leadership in shaping AI’s future and ensuring global security. Transcript: Ben Buchanan AGI, a term I’ve heard you don’t like, before China does. Why? I do think there are profound economic and military and intelligence capabilities that would be downstream of getting to AGI or transformative AI. And I do think it is fundamental for U.S. National security that we continue to lead AI. I think the quote that certainly I thought about a fair amount was actually from Kennedy in his famous Rice speech in 62, the we’re going to the moon speech. We choose to go to the moon in this decade and do the other things, not because they are easy, but because they are hard. Everyone remembers it because he’s saying we’re going to the moon, but actually I think he gives the better line when he talks about the importance of space. For space science, like nuclear science and all technology, has no conscience of its own. Whether it will become a force for good or ill depends on man. And only if the United States occupies a position of preeminence can we help decide whether this new ocean will be a sea of peace or a new terrifying theater of war? And I think that is true in AI, that there’s a lot of tremendous uncertainty about this technology. I am not an AI evangelist. I think there’s huge risks to this technology. But I do think there is a fundamental role for the United States in being able to shape where it goes, which is not to say we don’t want to work internationally, which is not to say we don’t Want to work with the Chinese. It’s worth noting that in the president’s executive order on AI, there’s a line in there saying we are willing to work even with our competitors on AI safety and the like. But it is worth saying that I think pretty deeply there’s a fundamental role for America here that we cannot abdicate. Paint the picture for me. (Time 0:08:29)
  • China’s Potential AI Advantage in Cyber Operations
    • China gaining AI dominance first poses significant risks in intelligence analysis and cyber operations.
    • Superior AI capabilities could allow China to perform better cyberattacks and defenses. Transcript: Ezra Klein Be great economic, national security, military risks if China got there first. Help me help the audience here imagine a world where China gets there first. Ben Buchanan So I think let’s look at just a narrow case of AI for intelligence analysis and cyber operations. This is, I think, pretty out in the open that if you had a much more powerful AI capability, that would probably enable you to do better cyber operations on offense and on defense. What is a cyber operation? Breaking into an adversary’s network to collect information, which if you’re collecting in a large enough volume, AI systems can help you analyze. And we actually did a whole big thing through DARPA, the Defense Advanced Research Project Agency, called the AI Cyber Challenge, to test out AI’s capabilities to do this. And I would not want to live in a world in which China has that capability on offense and defense and cyber, and the United States does not. And I think that is true in a bunch of different domains that are core to national (Time 0:10:20)
  • AI’s Data Analysis Advantage
    • AI’s ability to analyze large amounts of data, such as satellite imagery, presents significant national security implications.
    • Automating this analysis would give a substantial advantage to whichever country possesses this capability. Transcript: Ezra Klein That I find people bring up to me a lot, and it’s interesting, is that processing of information. So compared to, you know, spy games between the Soviet Union and the United States, we all just have a lot more data now. We have all the satellite data. We, I mean, obviously we will not eavesdrop on each other, but obviously we eavesdrop on each other and have all these kinds of things coming in. And I’m told by people who know this better than I do, that there’s just a huge choke point of human beings and their currently fairly rudimentary programs analyzing that data. And that there’s a view that what it would mean to have these truly intelligent systems that are able to inhale that and do pattern recognition is a much more significant change in the Balance of power than people outside this understand. Ben Buchanan Yeah, I think we were pretty public about this. And the president signed a national security memorandum, which is basically the national security equivalent of an executive order that says this is a fundamental area of importance For the United States. I don’t even know the amount of satellite images that the United States collects every single day, but it’s a huge amount. And we have been public about the fact that we simply do not have enough humans to go through all of this satellite imagery, and it would be a terrible job if we did. And there is a role for AI in going through these images of hotspots around the world, of shipping lines and all that, and analyzing them in an automated way and surfacing the most interesting And important ones for human review. And I think at one level, you can look at this and say, well, doesn’t software just do that? And I think that at some level, of course, is true. At another level, you could say, the more capable that software, the more capable the automation of that analysis, the more intelligent advantage you extract from that data. And that ultimately leads to a better position for the United States. I think the first (Time 0:14:59)
  • AI and the Surveillance State
    • AI’s potential to enhance surveillance and law enforcement raises concerns about privacy and civil liberties.
    • Ben Buchanan acknowledges the risks of bias, discrimination, and encroachment on rights. Transcript: Ezra Klein How frightened are you by the national security implications of all this? Which is to say that the possibilities for surveillance states. Yeah. So Sam Hammond, who’s an economist at the Foundation for American Innovation, he had this piece months back called 95 Theses on AI. And one of them that I think about a lot is he makes this point that a lot of laws right now, if we had the capacity for perfect enforcement, would be constricting, like extraordinarily Constricting. Laws are written knowing that human labor is scarce. And there’s this question of what happens when the surveillance state gets really good? What happens when AI makes the police state a very different kind of thing than it is now? What happens when we have like warfare of endless drones, right? I mean, the company Anderil has become like a big, you know, you hear about them a lot now. They have a relationship, I believe, with OpenAI. Palantir is in a relationship with Anthropic. We’re about to see a real change in this in a way that I think is, from the national security side, frightening. And there I very much get why we don’t want China way ahead of us. I get that entirely. But just in terms of the capacities it gives our own government, how do you think about that? (Time 0:17:38)
  • AI Development: A Dangerous Race
    • Klein expresses concern over the rapid development of AI amidst US-China competition for dominance.
    • He likens the situation to summoning an ally without understanding its implications. Transcript: Ezra Klein Find this all makes me incredibly uncomfortable. And one of the reasons is that there is a, it’s like we are summoning an ally, right? We are trying to build an alliance with another, like an almost interplanetary ally. And we are in a competition with China to make that alliance, but we don’t understand the ally. And we don’t understand what it will mean to let that ally into all of our systems and to all of our planning. As best I understand it, every company really working on this, every government really working on this, believes that in the not-too future, you are going to have much better and faster And more dominant decision-making loops by being able to make much more of this autonomous to the AI. Once you get to what we’re talking about as AGI, you want to turn over a fair amount of your decision-making to it. So we are rushing towards that because we don’t want the other guys to get there first without really understanding what that is or what that means. It seems like a potentially historically dangerous thing that AI reached maturation at the exact moment that the US and China are in this like Thucydides trap style race for superpower Dominance. That’s a pretty dangerous set of incentives in which to be creating the next turn in intelligence on (Time 0:20:49)
  • DeepSeek and the Future of AI Development
    • Some argue that DeepSeek’s success challenges the prevailing paradigm of AI development, which relies on massive compute.
    • They believe it demonstrates the possibility of decentralized, less resource-intensive AI development. Transcript: Ezra Klein That we were saying we did not need all this compute. We were saying we did not need these giant mega companies. That this was showing a way towards like a decentralized, almost solar punk version of AI development. And that in a sense, the American system and imagination had been captured by like these three big companies. But what we’re seeing from China was that that wasn’t necessarily needed. We could do this on less energy, fewer chips, less footprint. Do you buy that? I think two things are true here. Ben Buchanan The first is there will always be a frontier, or at least for the foreseeable future, there’ll be a frontier that is computationally and energy intensive. And our companies, we want to be at that frontier. Those companies have very strong incentive to look for efficiencies, and they all do. They all want to get every single last juice of insight from each squeeze of computation, But they will continue to need to push the frontier. And then in addition to that, there’ll be a kind of slower diffusion that lags the frontier, where algorithms get more efficient, fewer computer chips are required, less energy is Required. (Time 0:28:07)
  • Vance’s Accelerationist Stance
    • J.D. Vance advocates for less risk aversion in AI development, prioritizing speed and innovation.
    • He believes focusing too much on potential risks can stifle progress. Transcript: Ezra Klein There are warring cultures around how to prepare for AI. And I sort of mentioned AI safety and AI accelerationism. And J.D. Vance just went to the sort of big AI summit in Paris. And I’ll play a clip of what he said. When conferences like this convene to discuss a cutting-edge technology, oftentimes I think our response is to be too self-conscious, too risk-averse. But never have I encountered a breakthrough in tech that so clearly caused us to do precisely the opposite. What do you make of that? Ben Buchanan So I think he is setting up a dichotomy there that I don’t quite agree with. And the irony of that is, if you look at the rest of his speech, which I did watch, there’s actually a lot that I (Time 0:41:04)
  • Preparing for AI Opportunities
    • Klein suggests focusing on opportunities alongside AI safety, such as streamlining drug testing.
    • He argues that faster real-world implementation is crucial to translating AI benefits. Transcript: Ezra Klein Opportunities. But more than that, we are not preparing for opportunities. So if you imagine that AI will have the effects and possibilities that its backers and advocates hope, one thing that that implies is that we’re going to start having a much faster pace Of the discovery or proposal of novel drug molecules, a very high promise. The idea here from people I’ve spoken to is that I should be able to ingest an amount of information and build sort of modeling of diseases in the human body that could get us a much, much, Much better drug discovery pipeline. If that were true, then you can ask this question, well, what’s the choke point going to be? And our drug testing pipeline is incredibly cumbersome. It’s very hard to get the animals you need for trials, very hard to get the human beings you need for trials, right? You could do a lot to make that faster, to prepare it for a lot more coming in. You could think about human challenge trials, right? There are all kinds of things like this. And this is true in a lot of different domains, right? Education, et cetera. I think it’s pretty clear that the choke points will become the difficulty of doing things in the real world. And I don’t see society also preparing for that, right? We’re not doing that much on the safety side, maybe because we don’t know what we should do. But also on the opportunity side, this question of how could you actually make it possible to translate the benefits of this stuff very fast seems like a much richer conversation than I’ve seen anybody seriously having. (Time 0:47:09)
  • Government Structure and AI Adoption
    • Some argue that the US government’s structure hinders rapid AI adoption.
    • They believe streamlining processes is necessary to capitalize on AI’s potential. Transcript: Ezra Klein The federal government ready for this. I’ve been wandering around Washington, D.C. This week and talking to a lot of people involved in different ways in the Trump administration or advising the Trump administration, different people from different factions of What I think is the modern right. I’ve been surprised how many people understand either what Trump and Musk and Doge are doing, or at least what it will end up allowing as related to AI, including people I would not really Expect to hear that from, not TechRite people. In which the federal government, as constituted six months ago, moves at the speed needed to take advantage of this technology, either to integrate it into the way the government works Or for the government to take advantage of what it can do. That we are too cumbersome, endless interagency processes, too many rules, too many regulations. You have to go through too many people that if the whole point of AI is that it is this unfathomable acceleration of cognitive work, the government needs to be stripped down and rebuilt To take advantage of it. And like them, hate them, what they’re doing is stripping the government down and rebuilding it. And maybe they don’t even know what they’re doing it for, but one thing it will allow is a kind of creative destruction that you can then begin to insert AI into at a more ground level. Ben Buchanan Do buy them. It feels kind of orthogonal from what I’ve observed from Doge. I mean, I think Elon is someone who does understand what AI can do, but I don’t know how starting with USAID, for example, prepares the U.S. Government to make better AI policy. So I guess I don’t buy it that (Time 0:49:29)
  • US Stance on AI Regulation
    • Vance signals a shift in the US stance on AI regulation, rejecting multilateral agreements that might hinder progress.
    • He warns of retaliation against regulations penalizing US AI companies. Transcript: Ezra Klein President Vance’s speech was signaling to the Europeans that we are not going to sign on to complex multilateral negotiations and regulations that could slow us down, and that if they Passed such regulations anyway, in a way that we believe is penalizing our AI companies, we would retaliate. How do you think about the differing position the new administration is moving into vis-a Europe and its approach, its broad approach to tech regulation? (Time 0:51:58)
  • Pro-Worker AI Implementation
    • AI implementation should prioritize worker respect and avoid dehumanizing practices.
    • Focus on increasing worker agency and providing support during economic transitions caused by AI. Transcript: Ben Buchanan Other things that Vance talked about and that you said you agreed with is making AI pro-worker. What does that mean? It’s a vital question. I think we instantiate that in a couple of different principles. The first is that AI in the workplace needs to be implemented in a way that is respectful of workers and the like. And I think one of the things I know the president thought a lot about was it is possible for AI to make workplaces worse and in a way that is dehumanizing and degrading and ultimately destructive For workers. So that is sort of a first distinct piece of it that I don’t want to neglect. The second is, I think we want to have AI deployed across our economy in a way that increases workers’ agencies and capabilities. And I think we should be honest that there’s going to be a lot of transition in the economy as a result of AI. I don’t know what that will look like. You can find Nobel Prize winning economists who will say it won’t be much. You can find other folks who will say it’ll be a ton. I tend to lean towards the it’s going to be a lot side, but I’m not a labor economist. And the line that Vice President Vance used is the exact same phrase that President Biden used, which is give workers a seat at the table in that transition. And I think that is a fundamental part of what we’re trying to do here. And I (Time 0:53:17)
  • AI’s Impact on Labor Markets
    • Klein and Buchanan discuss the potential for widespread job displacement due to AI.
    • They express concern about the lack of effective policy solutions and the need for more creative thinking. Transcript: Ben Buchanan What they’re trying to do here. Ezra Klein So I’ve sort of heard you beg off on this question a little bit by saying you’re not a labor economist. I am not a labor economist. I will promise you the labor economists do not know what to do about AI. Yeah. You were the top advisor for AI. Yeah. You were at the nerve center of the government’s information about what is coming. If this is half as big as you seem to think it is, it’s going to be the single most disruptive thing to hit labor markets ever, given how compressed the time period in which it will arrive Is, right? It took a long time to lay down electricity. It took a long time to build railroads. I think that is basically true, but I want to push back a little bit. Ben Buchanan So I do think we are going to see a dynamic in which it will hit parts of the economy first, it will hit certain firms first, but it will be an uneven distribution across society. Ezra Klein Well, I think it will be uneven, and that’s, I think, what will be destabilizing about it, in part. If it were just even, then you might just come up with an even policy to do something about it. Sure. But precisely because it’s not even, and it’s not going to put, I don’t think, 42% of the labor force out of work overnight. Let me give you an example of the kind of thing I’m worried about, and I’ve heard other people worry about. So there are a lot of 19-year in college right now studying marketing. There are a lot of marketing jobs that AI, frankly, can do perfectly well right now. As we get better at knowing how to direct it, I mean, one of the things that can slow this down is simply firm adaptation. Yes. But the thing that will happen very quickly is you have firms that are built around AI. It’s going to be harder for the big firms to integrate it, but what you’re going to have is new entrants who are built from the ground up with their organization is built around, you know, One person overseeing these like, you know, seven systems. And so you might just begin to see triple the unemployment among marketing graduates. I’m not convinced you’ll see that in software engineers because I think AI is going to both take a lot of those jobs and also create a lot of those jobs because there’s going to be so much More demand for software. But you could see it happening somewhere there. There’s just a lot of jobs that are doing work behind a computer. And as companies absorb machines that can do work behind a computer for you, that will change their hiring. You must have heard somebody think about this. You guys must have talked about this. Ben Buchanan Yeah, we did talk to economists and try to texture this debate in 23 and 24. I think the trend line is even clearer now than it was then. I think we knew this was not going to be a 23 and 24 question. Frankly, to do anything robust about this, it’s going to require Congress. And that was just not in the cards at all. So it was more of an intellectual exercise than it was a policy exercise. Ezra Klein Policies begin as intellectual exercises. Sure, yeah. Ben Buchanan I think that’s fair. I think the advantage to AI that is in some ways a countervailing force here, though I hear you and I mostly agree with your side’s argument, is that it will increase the amount of agency For individual people. So I do think we will be in a world in which the 19-year or the 25-year will be able to use a system to do things they were not able to do before. And I think insofar as the thesis we’re batting around here is that intelligence will become little bit more commoditized. What will stand out more in that world is agency and the capacity to do things, or initiative and the like. And I think that could, in the aggregate, lead to a pretty dynamic economy. And the economy you’re talking about of small firms and dynamic ecosystem and robust competition, I think on balance, at an economy scale, is not in itself a bad thing. I think where I imagine you and I agree, and maybe Vice President Vance as well agree, is we need to make sure that for individual workers and classes of workers, they’re protected in That transition. I think we should be honest, that’s going to be very hard. (Time 0:54:28)