Skip to content

Podcast

Anthropic’s Cybersecurity Shock Wave + Ronan Farrow and Andrew Marantz on Their Sam Altman Investigation + One Good Thing

Hard Fork

Source ↗ ← All highlights
  • Why Anthropic Says Mythos Is A Cybersecurity Reckoning
    • Anthropic says its unreleased model Mythos can find zero-day exploits across core internet software fast enough to threaten software everywhere.
    • Kevin Roose says it found a 27-year OpenBSD flaw and an FFmpeg bug missed after 5 million automated scans. Transcript: Kevin Roose Well, Casey, as you know, on this podcast, we have a rule about discussing AI models called ship it or zip it. Casey Newton Ship it or zip it. Unless you’re actually putting it in people’s hands, we usually do not want to hear about it. Kevin Roose Yes, but today we are making an exception for the new Anthropic model Claude Mythos preview that just was announced but not released for reasons that we will talk about. But first, since this will be a segment and a show about AI, our disclosures. I work for the New York Times, suing OpenAI, Microsoft, and perplexity over alleged copyright violations. And my fiancé works at Anthropic. Casey, this is, I want to say, like, We like the biggest story of the year in AI. I know there’s been a lot of AI news. I know that people are probably saying, oh, here they go, talking about another model again. I am telling you, this is something that people need to be paying attention to because of the implications, because of the way it was rolled out, and because of the model itself, which We will get to all of that. But do you agree that this is a big deal? Casey Newton Well, you know, when we were talking about the show this week and we were kicking around the idea of like, hey, exactly how big do we think this is? You pointed out that one question people have been asking this week is, are we going to have to rewrite all software? And I feel like usually when folks are kicking that question around, it’s a big story. Kevin Roose Let’s just talk through what was actually announced this week. So on Tuesday, Anthropic announced that it was starting something called Project Glasswing. The name Project Glasswing refers to the glasswing butterfly, which has transparent wings, and so it can hide in plain sight. And that is thematically important for reasons that we will come back to. It’s also a delicacy in some countries. I’ve never had glasswing butterfly. Oh, you’ve got to try it. So, notably, they are not releasing this model to the public because they claim it is too dangerous to do that. Instead, they are giving access to a consortium of tech companies, including Cisco, Broadcom, sort of makers of internet infrastructure, as well as Microsoft, Apple, Amazon. Basically, every big tech company that is not OpenAI or Meta is getting access to this model. But not general access, just access to do defensive cybersecurity testing, basically to go out and harden their systems and their infrastructure and their software before the general Public can get its hands on this model. Casey Newton So what are some examples of what Mythos was doing in training that so alarmed Anthropic that it came to this point? Kevin Roose So Anthropic has been running this model internally for several weeks now, and they claim that this thing has found vulnerabilities in every major operating system and web browser. They gave some examples that have already been patched. One of them was that this model found a 27-year security flaw in OpenBSD. OpenBSD is an open-source operating system that runs on firewalls and routers. It is sort of like a critical security layer on the internet. And it was designed specifically to be hard to hack. Right. And this model, because of its advanced coding and reasoning capabilities, was able to find this bug that 27 years worth of professional security researchers had not been able to find. What else? Another example was that it found a bug in a piece of popular open source video software called FFmpeg that had, according to Anthropic, been scanned for bugs 5 million times by automated Security tools without finding this critical exploit. Casey Newton And that’s why it’s important to always look the 5 million in first time, because you might find something. Kevin Roose Now, Casey, I think for people who are not cybersecurity experts, it might be worth sort of sketching the context here for how software works. Yes. So, you know, every piece of software, every operating system, every app, every web browser that people use is built on a mixture of tools. Some of those tools are proprietary to the companies that make the software. Some of them are sort of shared open source tools that are just in everything. Companies will just grab this open source thing and plug it into their thing. Because that’s compatible with everything else, saves you a lot of time and trouble. It’s already been security tested by decades, sometimes of researchers. And this is sort of a big piece of kind of the foundation layer of the internet are these open source software projects. What is happening now, according to Anthropic, is that they can basically use this model, Claude Mythos Preview, to sort of proactively go out and find all of the unfound bugs. They call these zero-day exploits with a sort of speed and efficiency that no human security research team could do. (Time 0:03:37)
  • Why Anthropic Chose A Closed Defense Rollout
    • Anthropic is withholding Mythos and giving limited access to a defense consortium because releasing a cyberweapon-like model could create immediate real-world harm.
    • Casey Newton says a public release that enabled crimes would invite liability and congressional hearings, making secrecy a defensive move, not just hype. Transcript: Casey Newton Yeah. And, you know, I would say that it can be difficult to talk about cybersecurity in a way that resonates people for a couple of reasons. One is just that cybersecurity as a field exists essentially almost entirely to alarm people and say, here are a bunch of problems and these are really scary. You know, I hope that folks in the cybersecurity field would not mind me saying like, it is just like kind of an alarmist profession. And that when I’ve talked to these people over the past 15 years, they’ve been telling me like, look, the entire internet is held together with spit and glue. And we’re very lucky that there hasn’t been a catastrophe yet. Okay. So after all of this news came out, I was like, I want to talk to some people who are at least not working for Anthropic or this consortium to try to give me a gut check on how big a deal this Is. And so I talked to Alex Stamos, who formerly led security at Yahoo and then Facebook. And Alex said, like, yes, this is a big deal. And he was hoping for a long time that we would see a consortium come together like this because of exactly what you just said, Kevin. The intelligence in these machines and their ability to work autonomously are now great enough that they can chain together exploits that human beings either would never see, would Take them a long time to see, or they would just never get to because we’re limited in ways that these machines are not. So that got my attention. Kevin Roose Now, we should also talk about like what the strategy is here from Anthropic, because I think a lot of people see an AI company that is known for sort of being alarmist about safety, say, We’ve created this powerful, spooky new model, and we’re not going to show you because it’s too powerful and spooky as some kind of marketing tactic. So I think we should just say, like, that is not, to my understanding, the case here. No. Casey Newton In my mind, it is obvious why. Like, if you’re a corporation and you release a tool and people with no real technical expertise are able to use it and within a few hours discover a novel exploit in the Linux kernel and Then take over other people’s machines to cause crimes, you might be held liable as a corporation. You will get in trouble. There will be congressional hearings. So companies just in their rational self-interest do not want to sell cyber weapons on the open market. Kevin Roose Yes. It’s also like if this was a marketing strategy, it is a horrible marketing strategy. Like the government already thinks you’re a bunch of panicky doomers. You have a new model that you claim is the most powerful model in the world. So instead of selling it, you give a hundred million dollars of Claude credits away to a consortium of companies that includes many of your competitors, which is what Anthropic is doing. That is not how I personally would market a spooky new model if I were in the business of marketing spooky new models. Casey Newton Now, look, it may be that despite everything that we just said, there is still some marketing benefit to Anthropic from doing this, right? Like, know that they saw a huge increase in their revenue after they took that stand against the Pentagon. And in that stand, they said like, we are determined to do things in a really safe way. It seemed like the business world really liked that. And so I could imagine there being a business benefit to Anthropic of coming out and saying, we have the most powerful model in the world and we’re not releasing it. Like, yes, I’m sure that there are plenty of businesses that are salivating over the chance to get their hands on it. Kevin Roose But they can’t, unless they are part of this consortium. So they are at least claiming that they are trying to get ahead of what they envision will be a reckoning, was the word they used, for cybersecurity. And it seems plausible to me that in the next kind of six-ish months, every major piece of software in the world is going to need to be patched, (Time 0:08:36)
  • Why Patching The Top 1 Percent May Not Be Enough
    • Even if top vendors patch major systems, the long tail of old routers, libraries, and delayed updates could leave the internet exposed.
    • Kevin Roose says there are not enough humans to review all fixes, and many businesses will lag on firmware and security updates. Transcript: Casey Newton So just an absolutely massive project. Let me ask you this. You know, Alex Stamos, the security expert that I mentioned, told me that he sees essentially like two broad possibilities. One is, and this is the good scenario, there are a finite number of critical bugs and vulnerabilities to be found, and that maybe if we all work really, really hard over the next six months Or however long it turns out to be, we will be able to patch those vulnerabilities and our infrastructure will remain safe and stable. The other possibility is that this model is already good enough that it can just simply invent exploits that we never would have thought of. And so this will essentially just be a really, really big problem that potentially just keeps growing in scope because, you know, maybe eventually you hit some sort of true super intelligent Point. So I’m curious if you’ve talked to people about what they see the scenarios are and if you have any thought as to which of those two is more likely. Kevin Roose So I think it’s possible that they will patch the sort of top 1% of critical software, right? The stuff that everyone knows is important. Your Linux, your, you know, your, your very popular open source libraries, your, routing equipment and networking equipment. It seems plausible to me that a couple of companies with the right resources and the right models could find and fix the worst security vulnerabilities. But I also talked to people who were telling me that it’s not as simple as that, because once you get outside that kind of top 1% of critical infrastructure, there’s just a lot of machines That are running on old code, right? So it’s theoretically possible that all of these fixes could be submitted to the people who maintain these software projects, but that A, there aren’t enough humans to review all of The proposed bugs and fixes. So that just sort of is a human bottleneck there. Or that there is just a lag in the time between when a piece of software is patched and when the person running the router at the medium-sized business in Tulsa decides to update the firmware Or install the security patch. So people can expect a lot of apps that are asking them to update their software or reinstall their software over the next few months. I’ve started getting a few of these already. Have you started getting these? Yeah. Yeah. So I think this is going to be a kind of forced reset for the entire cybersecurity industry and a very significant event in the history of technology. Casey Newton Yeah, well, and just to make it concrete, like we are currently at war with Iran and Iran is currently hacking our critical infrastructure. There’s a story in WIRED this week about them successfully hacking like water and energy infrastructure. Right now, they’re able to do that without a mythos-quality model. I would be quite nervous about what they could do if something like that fell into their hands. So this really is not an abstract concern that we’re laying out. Kevin Roose Right. And we should talk about this government piece of this, because one weird characteristic of this moment is that this very powerful advanced model that Anthropic Claims is capable Of doing autonomous cybersecurity research and attacks is also a company that the U.S. Government has spent the last several months trying to kill and has tried to declare Anthropic a supply chain risk. They have ordered all federal agencies to stop using Claude. And so my understanding is there have been some conversations between Anthropic and parts of the, you know, sort of national security establishment and apparatus about this model. But it is also simultaneously true that they cannot use this model without sort of running afoul of the administration. So a private company right here in San Francisco currently has a technology that they claim is capable of finding critical security vulnerabilities in every major operating system And web browser in the world. And the U.S. Government, to my knowledge, does not have access to this technology. (Time 0:12:23)
  • The Dangerous Return Of The Secret Model Gap
    • A private lab now claims it has a model powerful enough to force global software rewrites while facing almost no effective U.S. regulation.
    • Kevin Roose says the public-model gap has reopened for the first time since GPT-2, reviving fears about secret capabilities inside AI labs. Transcript: Casey Newton Yeah, it does seem like something that our national security infrastructure would want to have access to. One more piece on the regulatory front. It is crazy to me that model development of this scale and seriousness remains essentially unregulated in this country, right? Here you have a private company saying, well, we have now created software that can create so many different kinds of novel exploits that all software might have to be rewritten, and They are not really under any kind of regulatory regime. And the regulatory regime that the previous administration tried to put into place was thrown out by the current one because it might harm American competitiveness. So I just wanna say, that makes me really, really uncomfortable. I think that if you’re making stuff this powerful, regulators ought to be paying attention. Yeah. Kevin Roose One interesting sort of historical note that I’ll make here is like for the past few years, at least, there has not been a significant gap between what the AI companies have built internally And what the public has access to. You know, maybe there’s a slightly better model that the companies are working on that they, you know, need to spend a few months testing before they release it. But like… Or it runs a little faster than the one that you have access to. Yeah. But there has not been kind of a significant gap since, I think, GPT-2, which was in 2019, which involved some of the leaders of Anthropic who were then at OpenAI, who made a decision to Hold back this model, GPT-2, out of fears that it could be used for things like automating propaganda and misinformation. Right. Casey Newton In reality, it could barely write a limerick. Yes. They erred on the side of caution. They did. And they got a lot of crap for that. Kevin Roose People sort of said, oh, you’re using this to hype. Some of the same stuff we’re hearing this week about Anthropic. And I think in that case, they were probably a little overexcited about what this model could do. But they wanted to make sure that they weren’t wrong, and so they held this back. And that created a gap of at least a couple months to maybe a year between what the average person could see and what was happening inside the AI labs. That gap is now open again. There is now a model that you and I cannot use, that our listeners cannot use unless they work at one of these companies in cybersecurity defense, and what the AI companies are claiming. And I think that is just a very tenuous situation. And I don’t like it, but I also understand why I think in this case, this was the right decision. Well, what do you mean when you say that it’s tenuous then? I think as hostile and suspicious as people feel toward the AI industry, that only gets worse if they think that there are secrets being kept in a basement that they can’t access. And I think that it creates paranoia and fear. I think that it is generally responsible to have transparency from the AI companies about how capable their models are. And I understand in this case that Anthropic felt like it had to make an exception. But I think this gap may be here to stay is the thing that I’m wondering about. (Time 0:16:33)
  • Basic Cyber Hygiene Matters More Than Ever
    • Use a password manager, generate unique random passwords, and turn on multi-factor authentication for email and banking.
    • Casey Newton recommends 1Password as an example and says not to rely on pet names or reused passwords. Transcript: Kevin Roose Last thing, a lot of the people I know who are plugged into the cybersecurity world are being asked right now what people should do about their own security. If they are worried that models like this will become public. It’s funny. Should they be like locking down all their accounts and moving their cryptocurrency into cold storage? Like what do you think people should be doing in anticipation that something like this will become public? Casey Newton You know, it’s funny. I had a friend ask me that just this morning as I was preparing for the podcast. And I said, you know, a couple of things. Like, one, to some extent, we’re just going to have to wait. I mean, to the extent that any of what we’ve just described is good news, it is that the defenders appear like they’re going to have some runway to fix some really bad problems before the Bad guys catch up. So I think we should give them a little bit of room to see what they can do. If it does emerge that there is a similar model that can wreak havoc, like rest assured there’ll be segments about it on Hard Fork and we’ll have some updated guidance. But I asked my friend, do you have a password manager and do you reuse passwords for the same thing? And she said, you know, I’ve never really been able to get one of those password managers to work for me. And I do sometimes reuse my passwords. So I said, like, look, if you’re looking for something that you can do, just make sure that you have done your basic online cybersecurity hygiene. You should use a password manager. I use one password. There are many others out there that are just as good. Don’t use the same password for anything. Your passwords should be randomly generated and not, you know, the name of your pet or whatever. And then use multi-factor authentication where you can, right? So don’t let anybody get into like your Gmail or your banking account just by typing in eight letters. You should also be, you know, using an authenticator app. And so those are some of the basic things that I would tell people to do, Kevin. Kevin Roose Yeah. I am planning to deal with the possibility of a massive cybersecurity breach by just sort of selectively dribbling out, incriminating things about myself. (Time 0:21:11)
  • What The New Yorker Found About Sam Altman
    • Ronan Farrow says their Sam Altman profile aimed to be forensic and even, not a predetermined takedown.
    • He says many longtime contacts described a repeated pattern of lying about both big and small things, even by Silicon Valley standards. Transcript: Kevin Roose Okay, let’s talk about this big piece that you both just published in The New Yorker. The title of the piece is Can Sam Altman Be Trusted? Now, usually there’s this sort of folk rule about headlines that end with question marks, which is that the answer is always no. So I want to put this question to you. Can Sam Altman be trusted? Well, I think one important thing to note is the piece is really forensic and even. Ronan Farrow And actually to a point where I’ve been happy to see there’s a range of reactions, right? There’s people who have answered that question in a very severe way and looked at the fact pattern that is laid out here and the documentation that’s laid out and said, you know, this Is someone who poses an acute danger and should be kept away from an authority position. And then there’s people who, I mean, hilariously enough, my mother called me and she’s like, you know, I kind of like him. And so I think that is a true reflection of our intentions. In this case, as you might imagine, there was deep consultation with all of the subjects of the reporting to really understand their feelings. And anytime we thought there was a persuasive argument from Sam or anyone else that, you know, something shouldn’t make it in or something would be sensationalist, we really carefully Discussed that editorially. So the result is very even. And I would say on the question itself, what we lay out is something that is remarkable, I’d say, even against the backdrop of the culture of mistrust in Silicon Valley, where everybody Understands and expects, right, that being a founder means telling different audiences different things at times to some extent, where everyone understands that the entire enterprise Is building based on hype long before there is actual actionable, deliverable product. Even against that backdrop, there is an extraordinary preponderance of people who emerge from interactions with Sam Altman, including close years-long ones, with really active Complaints and allegations that he lies repeatedly about things big and small. Casey Newton Well, one of my favorites was when you quote him telling you that he wears a gray sweater every day to avoid decision fatigue, and then he shows up for his next interview in a green sweater. That felt like a really satisfying detail. That was just for you, Casey. I was wondering if people were going to catch that. I appreciate that eye for fashion that you so rarely get in these tech profiles. (Time 0:29:04)
  • Why The OpenAI Board Crisis Stayed So Murky
    • Andrew Marantz says the case against Sam Altman is less one smoking gun than an accumulation that only makes sense when laid out together.
    • Ronan Farrow reports the outside investigation after Altman’s firing produced no written report, only an 800-word press release. Transcript: Andrew Marantz Andrew was our fashionista in the writer’s room always but that’s the kind of thing where you know we didn’t want to make too much of that right because it’s like oh we caught you in this Deep hypocrisy of you know choosing a green sweater like and this is consistent with a lot of the things people say uh throughout the piece and throughout the career of altman and open Ai is that there isn’t this one smoking gun thing where he’s like caught, you know, with his hand in the cookie jar. It’s this sort of allegedly longer, more subtle accumulation of facts, which my kind of like glib and, you know, annoying way of describing it is like the fabled memos and documents That were compiled that led to him being, you know, fired in 2023 and that have kind of dogged him throughout his career. They really shouldn’t have been like a secret bullet pointed list. They should have been a 16,000 word New Yorker piece because they only really make sense when you like lay them all out together in narrative form. Kevin Roose Yeah, I mean, you guys mentioned in your story that there have been sort of these rap sheets that have been circulating about Sam inside OpenAI and other parts of the AI industry for years. One of them was compiled by Dario Amadei when he worked at OpenAI under Sam Altman. One of them, you said, was maybe circulated by some allies of Elon Musk and people who are opposed to open AI. So give us some sort of behind-the details about what is being said by whom and how and to what ends about Sam Altman in Silicon Valley. Ronan Farrow Well, it was really important to us to filter for the obvious competitive incentives out there. There are people who are massively incentivized to go after Sam Altman. And the reality is that there are very firmly evidence-based critiques, many of which are promulgated not just by the rivals, although they’re certainly amplified by them heavily, But also by more neutral figures and people who are just kind of technologists who aren’t in the fight. And then there is the white hot center of the rivalry, the stuff you mentioned that I think is in a very different category, which is, you know, Elon Musk and other direct competitors Really amplifying everything they can come up with. And in some cases, we document things that are inflated or trumped up or just seem to not be true. So Elon Musk in particular has intermediaries circulating some pretty spicy and pretty unsubstantiated material in Silicon Valley. And we talk about that. Casey Newton I really appreciated that about the piece because this has become like more salient over the past year as these rivalries heat up and you hear more and more of these scurrilous rumors. And while I do think this winds up being a pretty damning portrait of Sam on the whole, you do also point out that in some very real ways, he’s the subject of legitimate smear campaign. Yeah. Ronan Farrow Oh, yeah. I think that’s absolutely accurate. Andrew Marantz And we were trying not to go in, you know, with naivete of like, can you believe business titans are being mean to each other? But like the level of this really does seem kind of shocking and unprecedented. And, you know, it’s kind of consistent with people who think of this as like whoever gets the ring first control the world. Like, it just seems like all bets are off. And so as a reporter, it’s very challenging to be like, do you bring up the scurrilous rumors to knock them down? And so we had like months of conversations about how best to do that. Kevin Roose So there’s been a lot of reporting on Sam Altman, especially around the board coup a few years ago. Could you maybe give us the two or three things that you think are new and important from your reporting that rise above the rest in terms of people’s understanding of Sam Altman and OpenAI? Andrew Marantz So I think there are things here that put to rest some of the longstanding rumors, right? I mean, Altman has always said, and Paul Graham at Y Combinator has always said he was not pushed out. He left of his own volition. It really seems from our reporting that that was not the case. They have talked a lot about their fundraising in the Gulf, in the Middle East, as innocuous. All businesses do this. It really seems from our reporting that the relationships that Sam has cultivated with some Emirati and Saudi royals is deeper than was previously realized. Ronan, what am I missing? Ronan Farrow There are several things like this. Of her memos. We didn’t really have the detailed, multiple sourced, heavily documented accounts of the individual proof points that were offered in those memos. We didn’t have the contents of those Dario Amadei notes. And we didn’t have a lot of these people on the record yet. So I think actually in a way that was a disservice not only to Sam’s critics, but also to Sam himself, there was a bit of a veil of mystery. And that wasn’t purely accidental. One of the things we document that’s new here is as a condition of the exit of the board members who had moved against Sam that he wanted out, they insisted on an outside investigation. What happened there is, in my view, quite extraordinary, which is, yes, at private companies, sometimes reports of this type when a law firm is brought in to restore legitimacy can Be kept out of writing. Often it’s to limit liability and often legal experts say it’s a bit of a red flag. This is a different kind of case. This isn’t just any private company. This is a high profile scandal that engulfed Silicon Valley when Sam was fired. And ostensibly at a non-profit. At a 501c3, exactly. And so there were stakeholders, not just in the public, but within this company, that would be the bare minimum threshold, right? Where senior executives thought, okay, we’re going to get some kind of at least detailed summary of what this law firm investigation found when they invoke it to rubber stamp Sam coming Back. And instead, what happened was an 800-word press release that said there had vaguely been a breakdown in trust and offered very few other details. And what we reported in this piece for the first time is there wasn’t a report. For years, people were like, where’s the report? Where’s the report? There wasn’t a report because it was kept out of writing. And this is no longer just a speculative supposition. One of the two board members who Sam helped select who oversaw this process just explicitly says, well, you know, a written report was not needed is now their line on this. Yeah, (Time 0:31:28)
  • Why Some Of Altman’s Backers Changed Their Minds
    • The reporting suggests some Silicon Valley pragmatists who once backed Altman later reconsidered after seeing fuller evidence.
    • Ronan Farrow says several people told them they would not have given Altman the benefit of the doubt if they had known then what they know now. Transcript: Kevin Roose Favorite detail in the piece because it was something I’d been curious about forever. I mean, the thing that I found most interesting from the piece were the people who spoke on the record or at least gave you quotes, and some of them were unattributed, about Sam who, you Know, I think previously might have supported him or at least felt like there was no upside in sort of, you know, talking about him in a negative way in public. There was a Microsoft executive quoted in your piece as saying that there’s a small but real chance he’s eventually remembered as a Bernie Madoff or Sam Bankman freed level scammer. He’s unconstrained by truth and said that he has, quote, an almost sociopathic lack of concern for the consequences that may come from deceiving someone. I haven’t been on a lot of corporate boards, but I think that is something that is quite rare to hear a board member say about a CEO of a company. Weighing these statements, did you feel like there are people who used to be fans of Sam who have soured on him? Or are these people who have really held a grudge against him for a long time? Andrew Marantz The thing that you point out about people changing their tune over time, I think is an integral part of what we document in the piece, which is, you know, the fact that Sam Altman comes Up through this Y Combinator world is not incidental. The fact that he has an investment portfolio in, by his own estimation, you know, about 400 other tech companies, the fact that he has sat on everyone’s board and everyone has sat on his Board. I think our sort of line about this in the piece is like, we spoke to people who are Sam’s friends, Sam’s enemies, and given the mercenary nature of Silicon Valley, some people who have Been both. Right. So given that that’s the landscape, you are going to have people who change their tune as the wind blows different ways. And that’s a lot of how Altman’s been able to weather a lot of this stuff in the past. Ronan Farrow One thing that results from that spread of opinions is, to your question about evolving takes on Sam, there’s definitely a class of nuts and bolts investors, prominent people in Silicon Valley who are really pragmatists, not just safetyists, and who are growth and business oriented, who told us that at the time of Sam’s firing of the blip, they gave him the benefit of The doubt. And especially because of the factor we talked about before, where there just was a dearth of clear information. In that void, a lot of prominent people gave him the benefit of the doubt and saw only upside in bringing him back and removing the board that tried to fire him. There are a number of those prominent people in that category now who say, I don’t know that I would have given him the benefit of the doubt if I knew everything then that I now know. It (Time 0:38:10)
  • The Real Story Is How Much Power AI Leaders Hold
    • The deeper issue is not one founder’s personality but how much world-shaping power sits inside private AI companies with weak oversight.
    • Ronan Farrow notes OpenAI itself once warned against an AGI dictatorship, making leadership integrity central to its original pitch. Transcript: Andrew Marantz Were legit defenders of Sam on a number of these fronts who we talked to for sure. I think a lot of this has to do with like what baseline expectation are you starting from? And you start from the premise that people who run giant successful businesses have to say a lot of different things to a lot of different people. Like, why is anyone even why is this a story? I think, though, there’s a kind of level setting here where one of the things you can do when you take a big sort of putting everything in one place narrative effort like this. Is you can start from the beginning and remember what the original pitch was. And when you go back to what the original pitch was, the defense of why are you guys being so naive? This is a normal competitive business. Like, OK, so when you pitch this as a nonprofit safety focused research lab that would aggressively comply with all regulation. Like, were the people who believed that naive to believe it at the time? Right. So that’s when the defenses start to feel a little more like pressured to me. Casey Newton Yeah. Also, like for what it’s worth, you know, it’s like, oh, you know, is it really a story that this guy is, you know, telling different things to so many different groups? It’s like, that’s not like really a story that gets told about Satya Nadella. It’s not really a story that gets told about Sunar Pachai. It’s not really a story that gets told about Tim Cook, right? Like there does seem to be something really unusual here. And my question for you guys, now that you’ve sort of spent so much time immersed in this company, is what do you think it means for open AI? Andrew Marantz Well, I mean, luckily we have a really robust independent tech media to, you know, so I was going to tune into TBPN and see what their independent journalistic take on this would be. Kevin Roose Do you want to give listeners who may not be familiar with what you’re talking about some context here? Andrew Marantz I think the day after our piece closed, Ronan, or something like late last week, OpenAI acquired TBPN, which is this big sort of tech chat show. So that’s one aspect of this answer, right? That as OpenAI expands and grows, they seem to be sort of buying up more of the press infrastructure to tell their own story. Ronan Farrow Relatedly, by the way, a lot of announcements over there, right, concentrated around when they knew we were going to be running and right, developed in the period where we were in these Intensive conversations with them. And many of them sort of pointed at the topics in the piece. You know, they announced this new safety fellowship that’s very airy. They announced this new governance plan that’s very sort of airy and ethereal, but are meant to, I think, you know, occupy space in the conversation on the same topics. Andrew Marantz And look, I mean, everyone, Ronan, you should say more about this, but everyone, including Altman and the OpenAI execs we spoke to, recognizes the economic pressures here. I mean, I think you guys were there when he said, oh, yeah, it’s definitely a bubble and someone’s going to lose a phenomenal amount of money, right? So even putting sort of the sci-fi Skynet stuff aside, you know, the economic pressures are unavoidable. And a lot of it has to do with this sort of pitch man rhetoric, the exact thing we’re talking about, right? Because these things are contingent. It’s not like, oh, will it be a bubble or not? It’s like, how hyped up will the cycle get is a byproduct of how people like Sam go around the world talking about it. Yeah. Kevin Roose I want to ask sort of a basic question that I think people have probably raised with you, which is like, why does it matter who Sam Altman is? If what we are talking about is a technology that could have profound implications on national security, the economy, potentially the future of humanity, it doesn’t seem obvious To a lot of people why it matters who is running these companies. Because a very nice person who is very honest and very transparent in all their dealings could still release a rogue superintelligence that blows up the world. And a very, you know, manipulative person could release a very aligned model. And so what we should be paying attention to are the models themselves, not the people running the companies that make the models. I’m not saying I believe that, but I’m curious, what do you make of that argument that we are focusing too much on the humans and not enough on the technology? We probably both have thoughts on this. I think I have two. Ronan Farrow The first of which is it’s worth noting that while reasonable minds could perhaps differ on the question you just posed, The answer provided by Sam Altman and the founders of OpenAI Was very clear, which is actually part of the way the entire enterprise was structured when it was founded as a nonprofit was they talked a lot about avoiding an AGI dictatorship. The most power over this technology is pivotal. The individual integrity is formative to the way the technology goes and the way it’s controlled and the way it’s used. The other thought that I have is, in my mind, you raise a valid point, and more significant than any of this is the structures around these individuals. We have a technology emerging that could really affect us all in all of the existential ways you just mentioned. And we don’t have the regulatory guardrails to keep an eye on these folks. We are completely ceding the power to these individual companies and their whims, the mud fight between them, the quality control that each of them has or lacks. I think that, to me, is the big question. And the integrity of an individual figures in that, and it’s important, but it reveals the weaknesses in the system. If you have someone who potentially lies all the time, could in the eyes of many critics be a danger. The important thing is to have the structures that account for that. (Time 0:42:30)
  • Why OpenAI May No Longer Be Inseparable From Altman
    • OpenAI without Sam Altman no longer seems unimaginable as the company grows and professionalizes.
    • Andrew Marantz compares the shift to replacing a Steve Jobs figure with a Tim Cook figure, while Ronan Farrow notes periodic succession talk. Transcript: Casey Newton He was fired, he was brought back in part because I think no one could really imagine an open AI without Sam Altman. Do you think that’s still the case? Andrew Marantz I don’t think it’s unimaginable anymore. I think that part of reaching the scale that they’ve reached is that you can have a, you know, Steve Jobs figure be replaced by a Tim Cook figure, right? It seems like it’s inseparable from reaching this scale that that becomes at least a possibility in people’s minds. Ronan Farrow Right, Ron? I mean, does that strike you that way? Absolutely. I think the landscape has changed substantially over the period of time we were reporting this story. The fact that gradually more and more people were talking openly about this critique is very telling. We report in the piece that there are periodic spasms of senior executives at OpenAI talking about succession again. Of course, naturally, the company denies this. But also very interesting that in recent forms of that discussion, there has been talk about Fiji SEMO being sort of the first potential successor candidate who could slot into any Ideas of that type that circulate. Between our asking about that and the piece coming out, obviously, SEMO has now gone on leave for medical reasons. There’s a lot of reshuffling. We see it in the Sarah Fryer case. I think you’re right to link it to that quote that’s in the article about constraints being sidelined. And yet I think these doubts and questions persist and are now much more out in the open. Casey Newton On the leadership question, it just strikes me that for somebody who I assume wants to stay CEO for a long time, it’s interesting to me that he’s hired so many former public company CEOs To be his top lieutenants. It’s like he has the former CEO of Instacart there. He has the former CEO of Nextdoor there. He has the former CEO of Slack there. So, you know, that’s you’re bringing a lot of really sort of sharp and pointy elbows into the room when you do something like that. I’m trying to tell Sam that there’s danger here. Ronan Farrow Pro tip. If you’re listening, Sam, you know, there are people in this piece talking about earlier tracks of Sam Altman’s career where they feel he was deliberately avoiding that. Actually, part of what underpinned the terrible, terrible fumbling of the firing effort was a feeling that Sam had kind of stacked the board with, as one former member put it, JV people. Certainly, if we’re being more charitable than that, people who were unprepared for the ruthless corporate warfare that ensued. And I think one thing that has accompanied the emergence of this as a more openly discussed critique is that there’s more people around this company, more stakeholders wanting, you Know, professionalizing influences in the mix. (Time 0:50:11)
  • Kevin Roose Got Childlike Wonder From Artemis II
    • Kevin Roose says Artemis II reignited a childlike sense of wonder he did not know he still had.
    • He watched with his kid, learned terms like Terminator line, and memorized the crew names Victor, Christina, Jeremy, and Reed. Transcript: Kevin Roose Okay, Casey, I am in love with this space mission. Yes. The NASA Artemis 2 mission, I have been totally and earnestly obsessed. My wife was like, you’re sure talking about this space mission a lot. I have been glued to this thing. And I have been filled with a childlike glee and wonder that I did not know I still had the capacity to feel. Now, what exactly are they doing on this mission? Orbiting the moon. They are going further than any humans have gone from Earth before, 252,756 miles from Earth. And if you’re wondering how many miles is that, well, the New York Times had a helpful comparison list. And what did they find? You would need a chain of 2.37 billion of Nathan’s famous hot dogs to cover the distance that this spacecraft has gone from earth. That’s great. Casey Newton Something we can all easily visualize. Thank you for that comparison. Kevin Roose Casey, I am learning things that I never expected to learn. I’ve been watching this with my kid. I have become completely obsessed with like concepts and terms that I did not know a week ago, including Corona structure, the Terminator line, which I know you’re wondering, that Sounds scary. It’s actually the line that separates the sunlit side of the moon from the side that is dark. I also learned that we don’t call it the dark side of the moon. That’s not the preferred astronomical term. Casey Newton What do we call it? Kevin Roose The far side of the moon. Casey Newton The far side of the moon. Kevin Roose I am obsessed with all of these astronauts. There are four of them up there. Victor, Christina, Jeremy, Reed. This is my Mount Rushmore. I love these people who I’ve never met. They are adorable. They are incredibly brave. And I think we should go to the moon every single year. I think we should give NASA whatever budget it needs to do because this has reignited my faith in humanity. Casey Newton Absolutely. You know, I also saw somebody on social media was posting that because the mission specialist Christina Koch had communicated with Houston’s Jenny Gibbons during the mission, this Mission actually passed the Bechdel test, which you don’t often see on these missions. So I thought that was cool. Also, somebody pointed out, they said, you know, the coolest thing about going on one of these missions, Kevin, would be leaving Florida at 5,000 miles an hour. So that resonated with me as well. (Time 0:57:43)
  • Acme Weather Turns Forecasts Into Small Daily Wonders
    • Casey Newton recommends Acme Weather as a playful revival of the Dark Sky team’s obsession with useful forecasting details.
    • He loves alerts for lightning, sunsets, aurora chances, and user-reported rainbows that send people outside to look up. Transcript: Kevin Roose Casey, what is your one good thing this week? Casey Newton Today, Kevin, I want to talk about the only thing that can compete with the moon when it comes to inspiring childlike wonder in a person, and that is a weather app. Okay, I’m listening. So recently I was reading about these entrepreneurs, Adam Grossman, Josh Reyes, and Dan Bruton, and they are the team behind Acme Weather, which you probably have not heard of yet, But I bet you’ve heard of Dark Sky. Yes. Dark Sky was, by consensus, the best weather app on iOS. And while it rained during the 2010s, and I’m using rained in the sort of… The non-meteorological sense. The non-meteorological sense. It would tell you whenever it rained. And now I am using it in the meteorological sense. Kevin Roose Very good app. Casey Newton All right, Pete. Yeah, this app was bought by Apple in 2020, which was like kind of a head scratcher apple already had a weather app it was fine and then apple sort of integrated some of its forecasts and Some of its other features into its weather app and then shut dark sky down in 2022 and this made people really sad because i think a lot of us feel myself included like the apple weather App has never lived up to what dark sky was in its heyday. Kevin Roose It’s like a prediction mark. It’s like, you know, maybe it’s going to rain. Casey Newton Exactly. Well, so these guys get back together and they say, frick it, we’re doing weather apps again. And they make Acme Weather. And so you can download this now for iOS. It is apparently coming later to Android. And I know what you’re thinking, Kevin, which is what could you possibly build in 2026 in a weather app that could differentiate it from all the other weather apps that are already on The market, right? Yes. Are you wondering this? I am wondering this. Well, let me tell you a few things. Number one, they don’t just tell you the weather. They show you a range of possibilities in a line chart. So most of the time, it’ll be like, yeah, it’s going to be 63 degrees in San Francisco today. But every once in a while, there’s a lot of volatility in all the different signals that they use to predict the weather. And then you say, okay, I don’t actually know what I’m walking into today. I better bring a couple of layers. Kevin Roose This is the weather app for rationalists and other believers in Bayesian statistics. Exactly. Casey Newton Some of the other things that this app does, they will send you a push notification if they think there’s going to be lightning in your neighborhood. Okay. They will also do that when they think a sunset is going to be beautiful wherever you happen to be. Wow. They’ll send you an umbrella reminder if it’s going to precipitate in the next 12 hours, and they’ll send you a sunscreen alert when the UV index is high. But I’m saving my last two favorites for the end. Number one, they will send you an alert when the Aurora Borealis may be visible where you are. That’s beautiful. I haven’t gotten that notification yet, but I wake up every day hoping I’m going to get my Aurora Borealis notification. You got to go to Scandinavia, I think. Number two, and this is just in time for pride. They will tell you when there is a rainbow in your neighborhood. Are you kidding me? This is such a good idea for a weather app. Who does not want to be sitting at your wage slave job? You haven’t been outside in like seven and a half hours. And then Acme Weather tells you, hey, guess what? There’s a rainbow in your neighborhood. You’re going to book it outdoors and you are going to behold the majesty of creation. How are they possibly collecting that data? Well, interestingly, they’re taking this Waze-like approach where they’re inviting their community to submit reports. And so if a bunch of people say, hey, rainbow in my neighborhood, they’re going to go out and send out a notification. So now look, this app does cost $25 a year, and I know probably most people out there are perfectly content with the free weather app on their phone. That is fine for you. But as somebody who loves cool things, new ideas, people having fun, I just wanted to shout out Acme Weather because I think it’s a really cool thing. Kevin Roose What is the likelihood that this app will be purchased by Apple and then shut down? Casey Newton I mean, if that happens, I hope these guys get paid again. Because somebody has to move the weather app industry forward. And these are the folks who are doing it. Kevin Roose I love that. Like, grandpa, how did you make your fortune? Well, I built 17 weather apps that were identical and then sold them all to Apple. Casey Newton I just also think it’s inspiring that a time when some companies are like, we’re going to make a system that is going to force the world to rewrite all software. There are other guys who are like, what if there’s a rainbow in my neighborhood? I want to find out about that. And those are the people that I want to highlight on today’s show, Kevin. (Time 1:00:52)