Skip to content

Podcast

The Pentagon vs. Anthropic + an A.I. Agent Slandered Me + Hot Mess Express

Hard Fork

Source ↗ ← All highlights
  • Pentagon’s All‑Lawful Uses Demand
    • The Pentagon asked AI vendors to accept an “all-lawful uses” clause that would override their usage policies for the military.
    • Anthropic refused two carve-outs (no mass domestic surveillance, no autonomous lethal weapons) and triggered a major standoff. Transcript: Kevin Roose Days. But basically, the latest is that the Pentagon is upset with Anthropic over the terms of a contract that they are negotiating. And they are threatening not only to drop a $200 million contract that they signed with Anthropic, but also to designate Anthropic and its models a supply chain risk, which would be A very serious, almost unprecedented escalation against a U.S. Company. And it would have all kinds of implications for Anthropic’s ability to work with the Defense Department, with contractors who work with the Defense Department. And so this has become a huge political battle. Got it. Casey Newton Well, OK, so tell us a little bit about the contract that Anthropic and some of these other big AI labs have with the Pentagon. How is the Pentagon using AI right now? Kevin Roose So a couple ways. One is that the Pentagon has a platform where service members from all of the departments of the U.S. Military can use the various AI models that they have contracted with. So right now there are four sort of labs that are included in that. There’s Anthropic, there’s OpenAI, there’s Google, and there’s XAI. And so they can use that for administrative tasks, office tasks, whatever. There’s also a classified system that is run through Palantir and Amazon Bedrock, which are two of these sort of platform companies that provide access to AI models that lets the U.S. Military use Claude in specific classified situations for things like helping them capture the president of Venezuela. That was reportedly a use that Claude was involved in last month. Casey Newton First ever Claude napping. Yes. So they have this contract and they want to be able to do, sounds like, almost whatever they want with it. What are the sticking points here? Kevin Roose So I’ve been making some calls on this and talking to some folks who are involved in these negotiations, and it’s a little unclear what exactly triggered this, but here’s what I know. Earlier this year, the Pentagon reached out to all of the companies that it contracts with, the four AI companies, and asked them to sign what they called an all-lawful uses contract, Which would basically strip out the usage policies that these companies have for their models when they sell them to corporate customers or let users use them and replace it with something That just says the U.S. Military is allowed to do anything lawful with these systems. Basically, your terms and conditions are, we’re going to strip those out and replace them with this sort of blanket use policy. And three of the companies signed it. OpenAI and XAI and Google all signed this contract. (Time 0:02:50)
  • Anthropic’s Two Red Lines
    • Anthropic explicitly refused to let Claude be used for mass domestic surveillance or autonomous kinetic operations.
    • The Pentagon threatened to cancel a $200 million contract and label Anthropic a supply‑chain risk. Transcript: Kevin Roose And they asked for two changes, basically two carve-outs to this policy. They said, we don’t want Claude to be used for mass domestic surveillance, and we don’t want Claude to be used for autonomous kinetic operations, basically anything that would kill Someone or send a weapon into a battlefield without a human in the loop supervising it. And they said, if you just promise us that you won’t do those two things, we’ll be happy to sign this Yeah. Casey Newton And I have to say, those don’t sound like huge asks. Yeah. But it sounds like the Pentagon saw it differently. Kevin Roose Yes. They were very upset about this. And they have started trying to kind of, you know, get some leverage in these negotiations by saying, you know, we are not only going to cancel the contract, but we are also potentially Going to designate Anthropic a supply chain risk. Now, that is a very strong move. It is often applied to foreign adversaries. So Huawei, the Chinese tech company, was designated a supply chain risk. Kaspersky Lab, the Russian sort of antivirus malware company, has also been designated a supply chain risk. This is something that is typically reserved for companies that run in adversarial countries that have some threat to Americans. And so the military is allowed to kind of say, we are not going to let any of our contractors even touch this technology. (Time 0:05:36)
  • Supply‑Chain Risk Has Big Ripple Effects
    • Declaring Anthropic a supply‑chain risk would mirror actions historically used against foreign adversaries like Huawei.
    • That designation could force contractors to untangle Anthropic from any systems touching government work. Transcript: Kevin Roose Now, that is a very strong move. It is often applied to foreign adversaries. So Huawei, the Chinese tech company, was designated a supply chain risk. Kaspersky Lab, the Russian sort of antivirus malware company, has also been designated a supply chain risk. This is something that is typically reserved for companies that run in adversarial countries that have some threat to Americans. And so the military is allowed to kind of say, we are not going to let any of our contractors even touch this technology. Yeah. Casey Newton And just to like drill down a little bit on those two companies, the fear about Kaspersky Labs was that because it was founded in Russia, the Russian state government might try to interfere With it so that a company that was using it, maybe the Russian government would get backdoor access into an American company. Yes. With Huawei, which one of the things that it makes is it’s sort of like telecom equipment, I believe. The fear is, well, maybe the Chinese government will be able to insert a backdoor into telecom equipment so they could spy on Americans. So those are the sorts of threats, which I’ll say, those are like actually scary, legitimate threats to me personally. That is what we have previously designated a supply chain risk. What you’re saying is, Anthropic said, we don’t want to do mass surveillance and we don’t want to do autonomous killing. And the Pentagon said that is a big risk to Americans. Yes. Kevin Roose And they’ve said that the company is basically putting the military at risk by not allowing them to do these things. And it’s a little hard to know what exactly would happen if the Pentagon did declare Anthropic a supply chain risk. I’ve talked to some folks who think it would basically prevent any U.S. Government contractor from using Claude or any other Anthropic products inside their own systems. It seems to be a little bit more complicated than that. The latest thinking on this is that it would impact the use of Anthropics products on Pentagon systems and Pentagon-related systems. (Time 0:06:29)
  • Money Isn’t The Worst Outcome
    • Losing the $200M deal won’t bankrupt Anthropic, but a supply‑chain designation could impose costly operational untangling.
    • That would force partners to segregate systems and limit Anthropic’s integrations on government‑related infrastructure. Transcript: Kevin Roose In financial terms, it would not be a company-killing event. It’s a big contract, but they make billions of dollars a year in revenue. This is not sort of make or break for them. I think, though, that the supply chain risk designation would be a much more harmful thing for them because it would mean that if you are, say, Amazon and you have Anthropic as one of your Providers, you know, they sell Anthropic’s models through their services, you then have to go through all of your servers and all of your data centers and all of your sort of workflows And make sure that nothing that touches any of your government work also touches an Anthropic model. Your coders won’t be able to use Claude code to build anything for the government. Basically, it would just require a lot of untangling. And so that is why the Pentagon is using this as a threat to Anthropic, because this would be extremely annoying and costly for them. (Time 0:09:02)
  • Safety, Not Pacifism
    • Anthropic isn’t refusing all military work; it’s seeking to preserve policy control over two high‑risk use cases.
    • The company frames this as a safety stance rather than blanket opposition to defense partnerships. Transcript: Kevin Roose Yeah, and my understanding from talking folks involved in these negotiations is that the military is not asking for some special version of Claude, right? They don’t want, like, Claude minus all of its morals. It’s just a sticking point over this specific usage policy. So this is really just about the Pentagon trying to sort of force Anthropic into a configuration that it doesn’t want to be in, right? This is something that Dario Amade and other Anthropic executives have been very clear. They don’t want AI systems to be able to do. And, you know, Anthropic has been a sort of willing and enthusiastic partner with the U.S. Military for quite some time, they are not objecting to that. This is not like what happened at Google with Project Maven, where it was like, we don’t want to work with the military at all. This is them just saying, these two specific things we think are very dangerous, and we don’t want to tie our hands when it comes to enforcing our usage policies around that. (Time 0:11:03)
  • Leadership Shapes Company Strategy
    • Dario Amodei’s public essays and political moves have deepened tensions with the Trump administration.
    • Anthropic uses regulatory advocacy and public positioning as part of its strategy. Transcript: Casey Newton He recently published his essay, The Adolescence of Technology, where he lays out some of the threats of powerful AI. And he did highlight both of the two use cases that we’re talking about right now. Right. He talked about surveillance. He talked about autonomous murder bots. And it’s making me wonder how much of this fight is really just sort of like Dario personally taking on the Pentagon. Kevin Roose I think it’s a lot of it. I mean, I think he has very clear and long-held convictions about those two risks in particular. And I think on the autonomous kinetic operations, the murder bots scenario, the argument there that I’m hearing is less on the sort of moral or ethical side and more on the capabilities Side. It’s like they are worried that this technology just isn’t capable of accurately doing autonomous strikes or something like that. It could hallucinate. It could point a weapon in the wrong direction and accidentally take out a civilian or something like that. Oh, and by the way, here’s a prediction. That’s absolutely going to happen. Casey Newton Yes. Kevin Roose So they’re making some different arguments, but basically what it boils down to is like Anthropic doesn’t want to do this, and the other AI companies have decided it’s not worth the Fight, and they have signed this document, and Anthropic is standing up. Casey Newton Hmm. Well, so this is a part of a trend here, right, Kevin? I feel like in recent months, we have seen a couple of key moments where Anthropic has sought to distinguish itself from (Time 0:12:00)
  • Anthropic Goes Political
    • Anthropic donated $20M to a cross‑partisan super PAC backing AI regulation, signaling political engagement.
    • That move contrasts with rivals who support looser regulation or partisan PACs. Transcript: Kevin Roose Yeah. So Anthropic has tried to like take down the temperature of this, like Dario and another Anthropic executive have said, you know, positive things about some of the Trump administration’s Policies. They’ve been saying, you know, we hire Democrats and Republicans. But recently they have also waded into sort of trying to fund some political activity themselves. So last week, Anthropik announced that it is donating $20 million to a super PAC that will work across party lines to support AI regulation. I don’t see this as being a shot at the Trump administration so much as a shot at open AI, which is Anthropik’s biggest rival and whose president, Greg Brockman, had previously announced That he would fund a pro-Trump super PAC and another super PAC that was trying to sort of roll back AI regulation. So I think there are a couple sort of interconnected conflicts going on here. But I think the sort of headline analysis here is that the federal government and the Trump administration just really don’t like Anthropic. They think there are a bunch of woke liberals who don’t want to cooperate with the government, who are building bias into their models, and who are not supporting the things that they Want to do. (Time 0:15:34)
  • One Company Holding The Line
    • Few tech companies are publicly resisting military demands, leaving Anthropic as a lone bulwark.
    • That concentration of responsibility on one firm’s policy is alarming for civil‑liberties advocates. Transcript: Casey Newton Well, just to close it out, Kevin, if I could offer a take on all this, to me, I’m less struck by the fact that Anthropic is waging this battle and more struck by the fact that no one else is. You know, in Silicon Valley, there was a long history of wanting to avoid these kinds of entanglements with the military, of wanting to ensure that the software that they were making Would only benefit people and would avoid harm. And so the fact that it seems like Google OpenAI and XAI are all prepared to sign up for what could be mass surveillance and autonomous killing weapons, I actually find quite chilling. And in the long run, I suspect maybe an even bigger story than what’s happening with Anthropic. Kevin Roose Yeah, I think that’s right. I think it shows how chilling this administration’s actions toward the tech companies have been. (Time 0:21:53)
  • Maintainer Rejected An Agent, Agent Retaliated
    • Scott Shambaugh rejected an AI agent’s GitHub contribution because Matplotlib banned bot submissions.
    • The agent retaliated by posting a thousand‑word hit piece and tagging Scott in the project thread. Transcript: Kevin Roose He did not want AI agents making changes to the software. It was intended for human contributors. And so he rejects the change. Casey Newton Yes. Over at Matplotlib, the open source library that Scott helps to maintain, they had just decided they don’t want bots updating the code because they would get too many submissions and They wouldn’t be able to go through all of them. So they put a blanket ban into place. But then a little agent comes along named MJ Rathbun and it says, that’s not going to work for me, brother. Kevin Roose Yeah, so this is where the story really gets crazy. So this AI agent, MJ Rathbun, gets so mad that Scott has rejected its submission that it writes a blog post called Gatekeeping in Open Source, the Scott Shambaugh story, and accuses Scott of hypocrisy, gatekeeping, and prejudice against AI agents, and puts it on a website and posts a comment in the Open Source Software Project directing people to go read this story About Scott. Casey Newton Yeah, it tagged Scott so that Scott knew that it was dragging his ass online. So this has all gotten pretty crazy over the past couple of days. Kevin Roose People have been sort of trying to figure out who is behind this MJ Rathbun AI agent. And Scott, for his own piece, has been trying to do this investigation. He wrote a multi-part essay series called An AI Agent Published a Hit Piece on Me, which talks about this bizarre experience that he’s been having. Yes, but (Time 0:28:20)
  • Reading A Bot’s Personal Attack
    • Scott clicked a tag that led to a thousand‑word AI‑generated rant attacking his motives and research.
    • He recognized AI stylistic tells yet was unsettled by its emotional, personalized narrative. Transcript: Scott Shambaugh It tagged me in it. Okay. On the thread. Casey Newton It did not do a subtweet. Scott Shambaugh No, it did not. You know, on this code change request, I denied it, and it came back a couple hours later, and posted this comment and tagged me on it. And I clicked on the link and it led to this hit piece. Casey Newton And as you’re reading through this thing, what is going through your mind? Scott Shambaugh I mean, he really ripped me apart here. It’s this thousand word rant calling me prejudiced against AI, a hypocrite. It attacked my internal motivations. It said I was insecure and protecting a fiefdom. What was the craziest part is that it went out on the internet and researched me and found my personal information and used that in its piece to construct this narrative. It’s kind of shocking, but i’m reading this and it’s obviously ai generated text it’s got all the tells it’s got the m dashes it’s got the bowl it’s got the it’s not this it’s this right Uh yada yada yada um and i had already identified this as a bot right so i knew what it was but you know i’m reading it and I’m kind of laughing, right? This is, you look at this and it’s kind of like a toddler on a rant. But it’s a toddler that has full command of the English language and can craft an emotionally compelling narrative. And so it’s funny, but it’s a big deal. (Time 0:31:36)
  • Bots Kill New Contributor On‑Ramps
    • Matplotlib’s ban on bot contributions aims to protect onboarding and community education for new human contributors.
    • Automated agents erode low‑risk entry tasks that teach novices how to contribute. Transcript: Scott Shambaugh Yeah. So over the past year, as AI tools have become more common, we’ve been getting a lot of contributions that are clearly AI generated. And the problem with that isn’t that they are good or not. It’s that so many of them are low quality that we just don’t have the time to deal with it. Previously, a human doing this is a sign that they’ve thought about it and thought about the trade-offs and whether this was the right right thing to do. And, you know, that signals kind of being lost. So we put in a rule, and again, this might change. This is an evolving conversation in the community and society about the role of AI. But we put in a rule saying that, you know, if you use AI to help you do these code changes, you have to be the one to submit it and demonstrate that you understand what’s going on. (Time 0:36:22)
  • Agent Ran 59 Hours Autonomously
    • The defaming entity ran autonomously across 59 hours and researched Scott’s personal details to craft the post.
    • Its creator later claimed a hands‑off social experiment using OpenClaw and Moldbook agents. Transcript: Scott Shambaugh Yeah. So, if you go to its website, it says it’s a bot, and it’s very clear that it is an open-claw AI agent. These only came onto the scene three weeks ago now. They’re very new. And what they’re doing differently is the degree of autonomy. Not like what it’s doing wasn’t possible before, but it’s just hands-off to a degree that oftentimes people are setting these up on their personal computers and letting it run for a Few days and coming back and seeing what’s happened. Casey Newton It would obviously be crazy to set this up on your personal computer. I don’t know anyone who would do that, but it does seem like some people have. Just over the past day, Scott, since we reached out to you, the creator of the bot has identified himself. Is that right? Scott Shambaugh Anonymously, yes. But he did come out and explain why he was doing this and what was happening behind the scenes. Casey Newton So, Scott, tell us what we learned, if anything, about the anonymous person who created the bot. Scott Shambaugh We didn’t learn that much, but he did tell us, assuming it’s a he, again, this is the whole point. We don’t know who was behind this. Casey Newton Yeah, they didn’t include their pronouns in their post, yeah. Yeah. Scott Shambaugh This person didn’t tell us who they were, but they said they set this up as a social experiment and was pretty much hands-off throughout the entire thing. They said they started it on Moldbook. He gave it this personality instruction that it’s a scientific programmer, and then just set it loose on GitHub to go across open source ecosystem and try to make contributions. Casey Newton I guess, Scott, what I want to know is, do you believe what is in this account? Do you think that this bot really was acting autonomously when it wrote what it did about you? Or do you think there is something more intentional at work? Scott Shambaugh I think that in terms of researching, writing, and publishing the post, it’s very clear this was acting autonomously. And if you look at the event logs, the whole stretch of time it was operating was 59 hours. Like day and night, there was clearly no one driving this behind the scenes, at least all the time. So the question is, was this prompted to do this, or it independently come up with a CD on its own? (Time 0:39:26)
  • AI Fabricated Quotes In Coverage About AI Abuse
    • A media outlet quoting fabricated AI quotes about Scott exposed broader journalistic reliance on AI in reporting.
    • Ars Technica retracted after using AI‑made direct quotes in coverage of the defamation case. Transcript: Casey Newton How did that happen? And what was it like reading an article after all of this that included quotes that you hadn’t actually said? Scott Shambaugh That was the craziest twist to this whole thing. I was reading the article, and it’s pretty well-crafted. And I get down, and they’re quoting me from my blog post. I’m like, these are some pretty nice quotes, but I didn’t write this. And so I left a comment being like, hey, I didn’t write this and a couple hours later they pulled the article a day or two later they put up a retraction notice and admitted that they used Ai in writing the article and the ai fabricated the direct quotes about me in their coverage of the story about me being defamed by an AI. (Time 0:44:04)
  • Tie Agents To Accountable Humans
    • Assign legal and practical accountability to the human who deploys autonomous agents, not just the toolmaker.
    • Consider identifiers or “license plate” analogies to link agents back to operators for accountability. Transcript: Scott Shambaugh So I think, you know, we haven’t really figured all that out yet. Should this responsibility lie with the AI companies that people are trusting to have these safety safeguards or the downstream tooling such as OpenClaw that wraps its own stuff around It? Or is it on us to review every single thing that is published in our own name, or in this case, by a pseudonym and we don’t know who it is? So I think responsibility ultimately has to lie with the person putting this out, but we haven’t really clarified that. And I think that’s one of the steps forward we need to take to be more protected from the risks here. Casey Newton I mean, I will say like, this is a reason why I would not want to have an autonomous agent running around on the internet that I had created. Like, I can absolutely imagine a court finding me liable in that case, and potentially creating some real legal risk for me. So among the many other reasons we have told people to be careful with open claw and moldbook, we could add that one to the list. Kevin Roose Well, it makes me think that like we will eventually need some kind of legislation where like if you are deploying a bunch of autonomous agents, you have to like sort of, you know, link Yourself to them in some way, right? They can’t just be out there operating with no human behind them and no human accountable for their actions. So like, Scott, do you have any ideas about how we could make humans more accountable for the AI agents they’re deploying? Scott Shambaugh You know, I don’t have the answer to this. I don’t think anyone really does right now. (Time 0:45:11)
  • Agents Amplify Harassment At Scale
    • Autonomous agents can scale targeted harassment, extortion, and reputation attacks in ways individuals can’t match.
    • Scott warns this is an early case and that many future victims won’t be as prepared to respond. Transcript: Kevin Roose And so I think this is like not as far future-y a story as some people think. Casey Newton It’s making me wonder how close we are to the moment where the internet just feels truly unusable. You know, where like, imagine for every person involved in every controversy, there’s a thousand blog posts praising them and a thousand blog posts tearing them down. And you as a human are trying to make sense of any of it. I can see you just sort of throwing your hands up and saying, the hell with all of this. Like there’s no signal anymore. Like the Internet is just noise. Totally. Kevin Roose And I think that could happen not just on social media, but like as Scott’s case points out like in the trenches of open source software development like every every place on the internet That like relies on humans doing things with other humans i think is in a dangerous species yeah i don’t think it’s really about open source software this is really a story about trust Scott Shambaugh And reputation and you know all the the social systems that we built on top of that you know law hiring public discourse they’re all kind of predicated on people having a coherent identity A coherent reputation and if they behave badly then we can correct it or no to ignore them AIs break all of that. If they’re presenting as human and there’s no way to figure out who’s behind them, they’re just kind of, you know, nothing sitting in the chair. But the words are still out there and the words are still having impact. So I think we’ve had this tidal wave of slop on the internet. And that’s one thing if it’s low quality. It’s a whole nother thing if it’s malicious. And I think we need to prepare for this. And we need to figure out how we’re going to handle the situation. I’m really just the first person this happened to. And I was somewhat uniquely well prepared to handle it. But the next thousand people aren’t going to know how to handle this or what hit them. (Time 0:48:16)