Skip to content

Podcast

Ben Thompson- Anthropic, the Pentagon, and the Limits of Private Power

The a16z Show

Source ↗ ← All highlights
  • AI As A Source Of Geopolitical Power
    • Politics will assert itself over powerful private AI when it becomes a source of real power.
    • Ben Thompson compares private super-powerful AI to private nuclear weapons, warning governments will intervene when strategic stakes rise. Transcript: Ben Thompson One of those ones, I don’t know if it’s good or bad, that it came out sort of at the end of the week. So I had a lot of time to think about it. Ultimately, I think it was good because I’m not sure anyone very, as explicitly made the point I did. And maybe it was bad because I feel there’s a lot of like caveats, maybe in retrospect, I should have put in the article that would have addressed a lot of the points that people are upset About. Yeah. Basically zooming out, this was not a normative article where I’m saying what’s happening is good or bad. And that’s really the one caveat I really wish I would have put on there. I mean, I’m being out there accused by like a Neil Apatow, like the full-throated fascist, endorsement of fascism or something like that. And it’s like, relax, okay? Can I get some credit for the last X number of years? Basically, there is a deep-rooted concern that I’ve had for a long time about, and I’m now hesitant to even use sort of EA as a term because it’s kind of now politicized thanks to the events Of the last week. But a failure to grapple with a world of guns is basically the long and short of it. And I actually think Eliezer has been the one guy who’s been honest about this, where he wrote that Time article about potentially bombing data center someday. And that’s actually a point worth bringing up, which is all this stuff is right now in the digital realm with robotics and potential other applications. And it’s obviously being used for military operations. It’s crossing over into the physical realm. But if AI is as powerful as people say it’s going to be, then there are going to be real world reactions to that. (Time 0:02:10)
  • Chip Dependency Shapes AI Risk Calculus
    • Strategic dependencies like Taiwan and TSMC change the calculus of AI export restrictions.
    • Thompson argues cutting China off from chips risks making conflict (e.g., bombing TSMC) a rational option for a state seeking AI dominance. Transcript: Ben Thompson There’s been a lack of awareness of this. That’s why I brought up the Taiwan-China thing. This has been a frustration I’ve had with Anthropik generally. They talk about, you know, Amadei’s been very outspoken in terms of opposing selling chips to China for, in a narrow, you know, aspect, very, very good reasons. My pushback has always been what happens if we get super powerful AI and China doesn’t? What are they going to do? John Coogan Sure. Ben Thompson The optimal thing would be to just bomb TSMC out of existence because suddenly that becomes optimal, even with all the costs that that does. And then what? Then what are we going to do? (Time 0:05:36)
  • Bob Noyce Lesson On Selling To Government
    • Ben Thompson recounts Bob Noyce’s Intel decision to sell to government but not design chips for it.
    • Noyce prioritized consumer markets to drive scale and capability, a precedent Thompson says matters for AI economics. Transcript: Ben Thompson And Bob Noyce made the decision that we will sell to the government, but we’re not going to design chips for the government. And the distinction there was you had guaranteed orders, which was great. The government would take your IP and there was, and in his mind, the more important thing is there was limited volume. And the way that he foresaw correctly that this was going to be a very upfront capital intensive process of designing, you have to design them, you have to do the equipment, all of which Is in the billions of dollars today, back then was in the tens of millions and hundreds of millions, is you need to find the largest possible market, which was the consumer slash business Market. (Time 0:16:10)
  • Recognize The Economics Prevent Purely Government AI
    • Do not assume governments can just build competing frontier AI models at scale.
    • Thompson notes model development needs massive CapEx and broad markets, so private firms will lead while governments remain customers. Transcript: Ben Thompson And Bob Noyce made the decision that we will sell to the government, but we’re not going to design chips for the government. And the distinction there was you had guaranteed orders, which was great. The government would take your IP and there was, and in his mind, the more important thing is there was limited volume. And the way that he foresaw correctly that this was going to be a very upfront capital intensive process of designing, you have to design them, you have to do the equipment, all of which Is in the billions of dollars today, back then was in the tens of millions and hundreds of millions, is you need to find the largest possible market, which was the consumer slash business Market. You design for that, that will accelerate your improvement and your capabilities so much that you will end up having better devices than the government could have ever requested or Made for itself. That is at stake on steroids with AI. People, like I was talking to someone, they’re like, why doesn’t the government just get someone to make their own model? It’s like, because it’s got, like you talk about government contracts, we’re like single digit billions. We’re talking about for the amount that’s going into CapEx, the cost of these models. We’re talking hundreds of millions of dollars for the models and hundreds of billions of dollars approaching a trillion dollars a year in CapEx. That is only sustainable and viable if you’re selling to everyone. But that introduces the entire new dynamics where the government built nuclear. It started there and it started with a lot of assumptions because it was a government program. We are necessarily for economic reasons, because of all the upfront costs entailed, starting with private companies of which the government is one of many customers. And that introduces the assumption that, well, it’s a private company with private property rights and all those sorts of things, all of which I want to be true. (Time 0:16:10)
  • Refusal To Cooperate Invites Political Countermeasures
    • Private companies resisting government demands create a power conflict because governments ultimately wield force.
    • Thompson warns that refusing to cooperate invites political measures aimed at denying a firm the ability to build a power base. Transcript: Ben Thompson I would like them to sell to the government and I would like Congress to pass a law addressing these digital surveillance issues. Yeah. And a lot of people are like, that’s unrealistic, which I’m amenable to. But at the end of the day, if you don’t have it’s legal or not legal as your guiding standard, the only alternative is someone has to decide. And the implication of that not being a sufficient justification is that means a private executive is deciding. And if AI is what it is, I think that’s going to be, I use this word intolerable. I didn’t mean intolerable to me. I meant intolerable to those with power to have a private executive making those decisions or not. And if you think about if power, if we’re going to have this very sort of brute analysis that power flows from or laws flow from power. AI is a source of power. Yeah. So it’s not just that. And I think this is where the supply chain again, which I’m not endorsing, but I think that’s where the motivation is coming from. The goal isn’t to find we just won’t use anthropic. I do think the goal is to hurt Anthropic. And if you’re not going to be subservient to us, you’re not going to be allowed to build a power base, period. (Time 0:19:50)
  • Lobby For Laws Not Just Company Policy
    • Work with government to push for legal reform rather than relying on private unilateral decisions.
    • Thompson suggests Congress should pass new laws addressing digital surveillance instead of informal company policy fights. Transcript: Ben Thompson Yeah, I mean, I wish the White House would come out and say, yeah, there’s a digital surveillance problem. Let’s work on a bit. Like I don’t, probably another regret I have is sort of putting this all on Anthropic. That was sort of the angle I was concerned about. And that left me, I think, fairly open to the critique that this is just like defending the White House’s approach. And that was, again, that was, I was trying to be a higher level that’s saying, look, this is what’s going to happen. But yeah, the, I think there should be a way to find a middle ground here. (Time 0:21:53)
  • Old Laws Fail Against Digital Scale
    • Existing legal frameworks are ill-suited for digital-scale capabilities and AI; new statutes are necessary.
    • Thompson compares antitrust mismatches to AI governance, arguing retrofitting old laws will fail and create bad trade-offs. Transcript: Ben Thompson And the reaction is always, that’s impossible. We can’t pass new laws. And okay, but realize the implications of what you’re saying. I mean, I saw a tweet. Again, I didn’t like it, so I lost it forever. It’s one of the most inferior things in the world. But someone was like, I would definitely rather have Dario Amadei make these decisions than, and he, to this tweeter’s credit, he wasn’t limited to Trump. Because to me, this isn’t a Trump issue. This is a, any politician issue. He said, I would rather have Amadei making these decisions than whoever comes out of our screwed up democratic process. And points for the honesty, because that’s the actual choice that is being put forward. And you could say Congress isn’t going to do anything. Therefore, Amadei should just appreciate that is giving up on the democratic process and saying we should have unelected, unaccountable individuals making weighty decisions. And again, I understand the sentiment. It’s hard to imagine Congress passing laws about anything, but just realize that’s like, that implication is quite fraught. (Time 0:23:46)
  • AI Amplifies Surveillance Loopholes
    • Digital surveillance loopholes expand when computing removes friction; AI amplifies that capacity massively.
    • Thompson stresses AI makes previously costly surveillance trivially scalable, undercutting old legal expectations and civil liberties. Transcript: Ben Thompson Scale infinitely. And AI, again, is going to be the idea that the NSA, by the way, this is my sort of like, I had to admit this in the article, I was so confused why the Pentagon was so obsessed with domestic Surveillance. I didn’t realize the NSA was part of the Pentagon. John and I had the same moment. Yeah, yeah. Yeah, I just sort of thought about it as like an independent agent, like the CIA, but that made a lot of this story make more sense. Right, no, exactly. John Coogan Yeah, I feel like like a lot of tech people are like reading the fourth amendment today and understanding like, uh, some of these like pretty basic processes. Ben Thompson Well, yeah, but like, it’s pretty, like the loopholes are massive. Like I’m not denying it. Like, like, and it, it, it’s similar to the chip thing with China. Like my prescription for anthropic to give in is to allow these massive loopholes to be exploited. And for the NSA to allegedly in the service of investigating foreign adversaries, but by, know, the process, basically surveilling the domestic population, I think is bad. And the reality is the nature of trade-offs is you’re choosing between multiple bad options. (Time 0:25:59)
  • Companies Will Split Over Military Cooperation
    • Different frontier labs will make divergent choices about military cooperation, creating public and talent tensions.
    • Thompson notes OpenAI agreed limits with the Pentagon while Anthropic resisted, leaving PR alignment misaligned with San Francisco talent views. Transcript: Ben Thompson The OpenAI angle is obviously interesting. I didn’t really get into OpenAI. It’s hard to parse exactly what’s going on. It seems to me they have agreed to the Pentagon that they will be, the Pentagon will be limited by lawful capabilities. Yep. And they make their own judgments about weapon usage. And as I understand it, OpenAI is like, we will on our side be free to stop the model from doing digital surveillance. Which sounds like you’re in sort of a jailbreak competition. It’s like, we’re going to agree to have a jailbreak competition with the U.S. Government. Which, again, it’s an example of how fraught this is that that’s probably the good place to come down on. Now, there’s obviously these dynamics of competing for the same talent base, being in San Francisco. You know, this is part of, I think, Anthropic has a local advantage in that most people, I think, in the industry are with them. And they have a national PR problem in that I think a lot of folks outside of tech don’t understand why tech companies always try to resist helping the U.S. Government. (Time 0:29:15)