Skip to content

Podcast

The Tech Behind Signalgate + Dwarkesh Patel's "Scaling Era" + Is A.I. Making Our Listeners Dumb?

Hard Fork

Source ↗ ← All highlights
  • Surprise Plumbing Charge
    • Casey Newton discovered an $18,000 charge on his credit card from Kara Swisher’s former heating and plumbing company.
    • This was accidentally charged due to his card being on file, highlighting the importance of managing stored card information. Transcript: Casey Newton Former landlord of me. And when I investigated, it turned out that Kara Swisher had charged my credit card for $18,000. For what? What costs $18,000? I don’t know what is going on, but it costs $18,000 to fix. And until I made a few phone calls yesterday, that was going to be my problem. So here’s what I want to say to the people of America. You need to watch these landlords. You might think that you’re out from underneath their thumb, but they will still come for you, and they will put $18,000 on your credit card if you do not watch them. Kevin Roose Now, this is slightly terrifying to me, the idea that Kara has access to your credit card in some way, shape, or form. Well, and I should say, basically, it was on file with the heating and plumbing company. Casey Newton So I’m not sure that I could actually blame Kara for this, but I did have to talk to her about it. Oh, (Time 0:01:15)
  • SignalGate Incident
    • Jeffrey Goldberg, editor-in-chief of The Atlantic, was accidentally added to a Signal group chat with senior US national security leaders.
    • This chat, “Houthi PC Small Group”, discussed sensitive military plans, sparking the SignalGate controversy. Transcript: Kevin Roose I feel smarter already. Well, Casey, the big story of the week is SignalGate. Yes. What I would say, Kevin, is the group chats are popping off at the highest levels of government. Yes. And if you have been hiding under a rock or on a silent meditation retreat or something for the last few days, let’s just quickly catch you up on what has been going on. Yeah. So on Monday, The Atlantic, and specifically Jeffrey Goldberg, the editor-in of The Atlantic, published an article titled, The Trump Administration Accidentally Texted Me Its War Plans. To a Signal group chat with 18 of the U.S.’s most senior national security leaders, including Secretary of State Marco Rubio, Tulsi Gabbard, the Director of National Intelligence, Pete Hegseth, the Secretary of Defense, and even Vice President J.D. Vance. The chat was called Houthi PC Small Group, PC presumably standing for Principal’s Committee and not Personal Computer. And I would say this story lit the internet on fire. Absolutely. Casey Newton You know, we have secure communications channels that we use in this country, Kevin, to sort of organize and plan for military operations. They were not used in this case. That is kind of a big deal in its own right. But to accidentally add one of the more prominent journalists in all of America to this group chat as you’re planning it is truly unprecedented in the history of this country. Kevin Roose Yeah, and unprecedented in my life, too. Like, I never get invited to any secret classified group chats. But I also feel like there is an etiquette and a procedure around the mid-sized group chat. So I’m sure you’ve had the experience of being added to a group chat. In my case, it’s usually like planning a birthday party or something. And there’s always like a number or two on this group chat that you don’t have stored in your phone, right? That’s right. The unfamiliar area code pops up along with the named accounts of everyone who you do know who’s in this group chat. Absolutely. And the first thing that I do when that happens to me is I try to figure out who the unnamed people in the group chat are. Yes. And until you figure that out, you can’t be giving your A material to the group chat. Casey Newton This is so true. You know, I saw someone say on social media this week that gay group chats have so much better operational security than the national security advisor does. And this is the exact reason. If you’re going to be in a group with seven or eight people and there’s one number that you don’t recognize, you’re going to be very tight lipped until you find out who this interloper Is. Exactly. Kevin Roose And maybe someone will even say, hey, who’s, you know, three, four, seven? There’s a protocol for this is what I’m saying. Casey Newton Yes. And as you’re pointing out, it’s a protocol that most people take extremely seriously. Yes. Even when they are talking about things like planning birthday parties. Yes. And not military strikes. Exactly. Yeah. Kevin Roose So before we get into the tech piece, let’s just sort of say what has been happening since then. So this story comes out on Monday in the Atlantic. Everyone freaks out about this. The government officials involved in this group chat are sort of asked to respond to it. There’s actually a hearing in Congress where several of the members of this group chat are questioned about how a reporter got access to these sensitive conversations. And basically, the posture of the Trump officials implicated in this has been to deny that this was a secret at all. There have been various officials saying nothing classified was discussed in here. This wasn’t an unapproved use of a private messaging app. Basically nothing to see here, folks. Casey Newton Yes, and on Wednesday, The Atlantic actually published the full text message exchanges so that people can just go read these things for themselves and see just how detailed the plans Shared were. Yes. Kevin Roose And let’s just say it does look like some of this information was, in fact, classified. It included details about the specific timing of various airstrikes that were being ordered in Yemen against the Houthis, which are a sort of rebel terrorist militia. Like, this was not a party planning group chat. Yeah. Casey Newton Here’s a good test for you. When you read these chats, imagine you’re a Houthi in Yemen. Would this information be useful to you to avoid being struck by a missile? I think it would be. To me, that’s the test here, Kevin. Totally. Yeah. So let’s (Time 0:02:45)
  • Secure Communication Practices
    • While Signal is secure, personal devices are vulnerable to exploits.
    • Use designated, non-public channels for sensitive communications. Transcript: Kevin Roose Dive into the tech of it all, because I think there is actually an important and interesting tech story kind of beyond the headlines here. So, Casey, what is Signal and how does it work? Casey Newton Yeah, so Signal, as you well know as a frequent user, is an open source, end-to encrypted messaging service that has been with us since July 2014. It has been growing in popularity over the past several years. A lot of people like the fact that unlike something like an iMessage or a WhatsApp, this is something that is built by a nonprofit. Actually, it’s funded by a nonprofit organization, and it is fully open source. It’s built on an open source protocol so anyone can look and see how it is built. They can poke holes in it, try to make it more secure. And, you know, as the world has evolved, more and more people have found reasons to have both end-to encrypted chats and to have disappearing chats. And so Signal has been sort of part of this move away from permanent chats stored forever to more ephemeral, more private communications. Kevin Roose Yeah. And I think we should add that among the people who think about cybersecurity, Signal is seen as kind of the gold standard of encrypted communications apps. It is not perfect. No communications platform is ever perfectly secure because it is used by humans on devices that are not perfectly secure, but it is widely regarded as the most secure place to have Private conversations. Casey Newton Yeah. I mean, and if you want to know why is that, we could go into some level of detail here. Signal makes a priority to collect as little metadata as possible. And so, for example, the government went to them and they said, hey, we have like Kevin’s signal number, tell us all of the contacts that Kevin has. They don’t actually know that. They don’t store that. They also do not store the chats themselves, right? Those are on your devices. So if the government says, hey, give us all of Kevin’s chats, they don’t have those. And there is some pretty good encryption and privacy practices in some of the other apps that I think a lot of our listeners use on a daily basis. WhatsApp has pretty good protection. IMessage has pretty good protection. But there are a bunch of asterisks around that. And so if security is super, super important to you, then I think many of us would actually recommend Signal as the best place to do your communicating. (Time 0:07:10)
  • Convenience vs. Security
    • High-ranking officials likely use Signal due to its ease of use compared to official channels.
    • Secure communication is often inconvenient, but necessary for protecting sensitive information. Transcript: Kevin Roose Yeah. And you and I both use Signal. Most reporters I know use Signal to have sensitive conversations with sources. I know that Signal has been used by government officials in both Democratic and Republican administrations for years now. So Casey, I guess my first question is like, why is this a big deal that these high-ranking government officials were using Signal, if it is sort of the gold standard of security? Casey Newton Sure. So I would put it in maybe two sentences, Kevin, that sums this whole thing up. Signal is a secure app, but using Signal alone does not make your messages secure. So what do I mean by that? Well, despite the fact that Signal is secure, your device is vulnerable, particularly if it’s your personal device, if it is your iPhone that you bought from the Apple store. There is a huge industry of hackers out there developing what are called zero-day exploits. And a zero-day exploit is essentially an undiscovered hack. They are available for sale on the black market. They often cost millions of dollars. And criminals and more often state governments will purchase these attacks because they say, hey, it is so important to me to get into Kevin’s phone. I have to know what he’s planning for hard fork this week. So I’m going to spend $3 million. I’m going to find a way to get onto his personal device. And if I have done that, even if you’re using Signal, it doesn’t matter because I’m on your device now. I can read all of your messages, right? So this is the concern. So wait, what are American military officials supposed to do instead? Well, we have special designated channels for them to use. We have networks that are not the public internet, right? We have messaging tools that are not commercially available, and we have set up protocols to make them use those protocols to avoid the scenario that I just described. Kevin Roose Yeah, so let’s go into that a little bit because, as you mentioned, there are sort of designated communications platforms and channels that high-ranking government officials, including Those with access to classified information, are supposed to use, right? There are these things called SCIFs, the sensitive compartmented information facilities. Those are like the physical rooms that you can go into to receive like classified briefings. Usually you have to like keep your phone out of those rooms for security. Casey Newton Yeah, I keep all of my feelings in a sensitive compartmentalized information facility. Kevin Roose But you’re working on that. Casey Newton I’m working on that. I’m working on it. (Time 0:09:32)
  • Government Tech Needs Improvement
    • Government tech is often outdated, leading officials to use convenient but less secure private options.
    • This highlights the need for better government technology that balances security and usability. Transcript: Kevin Roose And so I think the place where I land on this is that this is, I would say, obviously a dumb, probably unforgivably dumb mistake on the part of a high ranking national security official. My favorite sort of like cover up attempt on this was the national security advisor, Michael Waltz, was asked sort of how this happened because he was the person, according to these Screenshots of this chat, who added Jeffrey Goldberg from the Atlantic to this chat. And he basically gave the statement that was like, we’re all trying to figure out what happened here. We saw the screenshot, Michael. You added him. Yeah. And I think there are obvious questions that raises about, you know, whether he had mistaken him with someone else named Jeffrey Goldberg, maybe a national security official of some Casey Newton Kind. Oh, I bet. I bet. I bet the words Jeffrey Goldberg never even appeared on Michael Waltz’s screen. Okay. This is like the realm of pure speculation. But let me just tell you, as somebody who is routinely contacted by people anonymously on Signal, usually their full name is not in the message request. It’s like, you have a new message request from JG, so just those initials. And so I will look through my Signal chats, and I’ll be trying to, I want to ask that one person about that one thing. What was their Signal name? And I’m looking through a soup of initials. So I actually understand why that happened, which is yet one more reason why you might not want to use Signal to do your war planning. Yes, exactly. Kevin Roose I think the most obvious sort of Occam’s razor explanation for why all these high-ranking officials are on Signal is that it’s just a better and easier and more intuitive product than Anything the government is supposed to be using for this stuff. It’s more convenient. Yes, and I find this totally plausible having spoken with people who have been involved with government technology in the past. Like it is just not the place where like cutting edge software is developed and deployed. You know, there famously was this sort of struggle between President Obama and some of his security advisors when he wanted to use a BlackBerry in the Oval Office. And there was sort of like no precedent for how to do that securely. And so he fought them until they sort of made him a special BlackBerry that he could use. Like, this is a time-honored struggle between politicians who want to use the stuff that they used when they were civilians while in political office and are told again and again, like, You can’t do that. You have to use this clunkier, older, worse thing instead. (Time 0:15:58)
  • Dwarkesh Patel’s podcasting journey
    • Dwarkesh Patel started his podcast as a side project and it unexpectedly became his career.
    • His technical, in-depth interview style resonated with audiences, leading to its success. Transcript: Kevin Roose Dwarakish Patel, welcome to Hard Fork. Thanks for having me. I want to start with the Dwarakish origin story. You are 24 years old, correct? You graduated from UT Austin. You majored in computer science. I’m sure a lot of your classmates and people with your interest in tech and AI chose the more traditional path of going to a tech company, starting to work on this stuff directly. Presumably, that was a path that was available to you. Why did you decide to start a podcast instead? Dwarkesh Patel So it was never my intention for this to become my career. I was doing this podcast basically in my free time. I was interested in these economists and historians, and it was just cool that I could cold email them and get them to come on my podcast and then pepper them with questions for a few hours. And then when I graduated, I didn’t really know what I wanted to do next. So the podcast was almost a gap year experience of, let me do this. It’ll help me figure out what kind of startup I want to launch or where I can get hired. And then the podcast just went well enough that dot, dot, dot, I’m like, yeah, this could actually be a career. This is a more fun startup than whatever CodeMonkey, you know, third setting and Android kind of. So basically just kept it up and it’s grown ever since. Kevin Roose And it’s been a fun time. Yeah. I mean, I’m curious how you describe what you do. Do you consider yourself a journalist? I guess so. Dwarkesh Patel I don’t know if there’s a good word. I mean, there’s like journalists, there’s content creator, there’s blogger, podcaster. Sure. Journalists. Yes. Humanitarian. Kevin Roose I asked because I started listening to your podcast a while ago, back when it was called the Lunar Society. And the thing that I noticed right away was that you were not doing a ton of explanation and translation. Like I often think of our job as journalists as one primarily of translation, of taking things that insiders and experts are talking about and like making them legible to a broader and Less specialized audience. But your podcast was so interesting to me because you weren’t really doing that. You were kind of not afraid to stay in the sort of wonky insider zone. You were having conversations with these very technical experts in their native language, even if it got pretty insidery and wonky at times. Was there a theory behind that choice? Dwarkesh Patel No, honestly, it never occurred to me because nobody was listening in the beginning. Right. So I think it was a bad use of my guest time to have said yes in the first place. But now that they’ve said yes, like, let’s just have fun with this. Right. Like who is listening to this? It’s me. It’s for me. And then what I realized is that people appreciated that style of because with a lot of these people, they’ve done so many interviews. You’ve heard their, you know, what is your book about kind of thing before the intuition I always go for is pretend like you’re at dinner with this person. And if you’re at dinner with them, you just ask them about your main cruxes. Like, here’s why, here’s, you know, what’s going on here. Here’s why I disagree with you. You tease them about like their big ideas or something, but initially it was just an accident. I (Time 0:26:40)
  • Why Scaling Works
    • There’s no single, universally accepted answer for why scaling AI models works.
    • Dwarkesh Patel suggests intelligence might be a complex combination of circuits and pattern-matching abilities. Transcript: Kevin Roose Asked at least a couple of people in your book, Some Version Of, was basically what’s their best guess at why scaling works? Why pouring more compute and more data into these models tends to yield something like intelligence? I’m curious what your answer for that is. What’s your current best guess of why scaling works? Dwarkesh Patel I honestly don’t think there’s a good answer anybody has. The best one I’ve heard is this idea that intelligence is just this hodgepodge of different kinds of circuits and programs. And this is so hand-wavy, and I acknowledge this is hand-wavy, but you’ve got to come up with some answer. And that fundamentally what intelligence is is this pattern-matching thing, this ability to see how different ideas connect and so forth. And as you make this bucket bigger, you can start off with noticing, does this look like a cat or not? And then you get to higher and higher levels of abstraction, like what is the structure of time and the so-called ether and the speed of light and so forth. Again, so hand-wavy, but I think it ultimately will just be this hodgepodge. It (Time 0:32:20)
  • Materialism vs. Non-materialism
    • Dwarkesh Patel believes intelligence is primarily based on materialism, or raw processing power and data.
    • This contrasts with views that intelligence involves something beyond the physical. Transcript: Kevin Roose Among the AGI believers and the AGI skeptics over the question of whether there is something other than just materialism in intelligence, whether it is just like, intelligence is Just a function of having the right number of neurons and synapses firing at the right times and sort of pattern matching and doing next token prediction. I’m thinking of this like famous Sam Altman tweet where he posted, I am a stochastic parrot and so are you. Basically sort of dealing with rebutting the sort of common attack on large language models, which was that they were just stochastic parrots. They’re just learning to sort of regurgitate their training data and predict the next token. And among a lot of the sort of AGI true believers that I know, there is this feeling that we are just essentially doing what these language models are doing in predicting the next or synthesizing Things that we’ve heard from other places and regurgitating them. That’s a hard pill for a lot of people to swallow, including me. I’m not quite a full materialist. Are you? Do you believe that there is something about intelligence that is not just raw processing power and data and pattern matching? I don’t. I mean, it’s hard for me to think about what that would be. Dwarkesh Patel There’s obviously religious ideas about there’s maybe a soul or something like that, but separate from that, something we could like sort of have a debate about or analyze. Kevin Roose Yeah. And actually I’m curious about like what, what kind of thing could it be? Ethics? I don’t know. Like, that sounds very fuzzy and non-scientific, but, like, I do think there is something essential about intelligence and being situationally intelligent that requires, like, Something outside of your immediate experience, like knowing what is right and what is wrong. Casey Newton Well, I think one reason why this question might be a bit challenging is that there are still many areas where the AI we have today is just less than human in its quality level, right? Like these machines don’t really have common sense. Their memories are not great. They don’t seem to be great at acquiring new skills, right? If it’s not in the training data, sometimes it’s hard for them to get there. And so it does raise the question, well, is the kind of intelligence we have kind of categorically different than whatever this other kind of intelligence is that we’re inventing? Yeah, (Time 0:33:40)
  • AI Leaders’ Blind Spots
    • AI industry leaders often lack concrete visions of a positive future with advanced AI.
    • They focus on short-term advancements rather than addressing complex societal implications. Transcript: Kevin Roose Yeah. Yeah. In your conversations with all these AI researchers and industry leaders, are there any blind spots that you feel they have consistently or places where they are not paying enough Attention to the consequences of developing AI? Dwarkesh Patel I think they do not, with a few notable exceptions, they don’t have a concrete sense of what things going well looks like and what stands in the way. If you just ask them what the year 2040 looks like, they’ll say things like, oh, we’ll cure cancer, we’ll cure these diseases. But what is our relationship to billions of advanced intelligences? How do we do redistribution such that the, I mean, it’s not your or my fault that we’ll be out of a job, right? There’s no in principle reason why everybody couldn’t be better off, but there shouldn’t be the zero something where we should make sure the AIs don’t take over and we should also make Sure we don’t treat them terribly. (Time 0:39:05)
  • Learning in the Age of AI
    • In an age of AI, focus on deep understanding of specific industries to leverage AI’s capabilities.
    • Broad learning remains valuable, so pursue knowledge for personal enjoyment and exploration. Transcript: Kevin Roose Think that you will be podcasting? Casey Newton Will you still hang out with us? Kevin Roose It’s funny because we have our, I mean, we have our post-AGI careers already, right? Dwarkesh Patel Even after the AGI comes, they might automate everybody else in this office, but you and I will just get in front of a camera and… Casey Newton That there will still be value in sort of like having a personality, being able to talk, explain, being somebody that people relate to on a human level. That’s right. I think so. Kevin Roose I am curious though, because a thing that I know about you from our brief interactions and just, you know, reading things that have been written about you is that you believe in learning Broadly. You have been described as a person who’s being on a quest to learn everything. Sounds exhausting. Casey’s on a quest to learn nothing. Casey Newton I’m on a quest to learn what I need to learn. Kevin Roose Just-in manufacturing. I think a lot of people right now, especially students and younger people, are questioning the value of accumulating knowledge. We all have these like pocket oracles now that we can consult on basically anything. And sometimes I think I was at a school last week talking with some college students. And one of them basically said they felt like they were a little bit like the taxi drivers in London who still had to like memorize all the streets, even after Google Maps was invented. And that was sort of like obsolete. Like they felt like they were just sort of doing it for the sake of doing it. I’m curious what for you, the value of broad knowledge accumulation is in an age of powerful AI. Dwarkesh Patel The thing I would say to somebody who is incredibly dismayed is like, why am I going to college? Why is any of this worth it? Is if you believe AGI, ASI is going to be here in two years, that’s fine. I don’t think that’s particularly likely. And if it is, what are you going to do about it anyways? So might as well focus on the other worlds. And in the other worlds, what’s going to happen before the fully automated robot that’s automating the entire economy is these models will be able to help you at certain kinds of tasks, But they will fundamentally just give you more leverage on the world. My friend, Asholto Douglas, put it this way. Just imagine you’re going to have 100x the amount of leverage on the future. And the kinds of things that you will be in a good position to do is if you have deep understanding of a particular industry, the relevant problems in it. And it’s hard to give advice in the abstract like this, because I don’t know about these industries, so you’ll have to figure it out. But this is probably the time to be the most ambitious, to have the most amount of agency, to actually, these models currently aren’t really good at actually doing things in the real World, or even the digital world. If you can do that and use these as leverage, this is probably the most exciting time to be around. Here’s my answer for that. You don’t want to be in a world where you just have to ask ChatGPT everything. Casey Newton Do you know what I mean? Like, there’s a lot of effort involved in just sitting down, writing the prompt, reading the report that comes out of it, internalizing it, synthesizing it, relating it. Like you’d be better off actually just getting an education and then checking in with the chatbot for the things that chatbot is good at, at least for, you know, I don’t know, next few Years. Kevin Roose Yeah, I don’t know. I believe that. And I want to believe that the thing I’ve spent my life doing is not going to be obsolete, trying to be smarter and learn things. My sort of guiding principle on this is like, learning is fun. And if you can just do it for your own enjoyment, like, I don’t think learning the streets of London is that fun. But I think learning broadly about the world is fun. Dwarkesh Patel And so you should do it if it’s exciting and fun to you. I think that’s totally correct. I also, if I’m like actually talking to a younger version of myself. Who would be six years old, to be clear? Who’s the young man we’re talking to today? Hey, little buddy. Just advice on careers in general is so bad. And for, especially with how much the world’s going to be changing it’s going to get even worse and so i mean who would have told me who what kind of reasonable person would have told me Four years ago man this computer science stuff just stop that focus more time on the podcast right um so yeah it’s going to change a lot i think but see that’s not helpful like what are you Going to do with this idea that like all advice is wrong? It was even a worse position. Just this idea that like, yeah, be a little bit skeptical of advice in general. Really trust your own intuition, your own interest. Don’t be delusional about things, obviously. But yeah, explore, try to get a better handle on the world and do more things and run more experiments than just this is the thing that’s going to be high leverage in the eye. And that’s where I’m going to do this based on this first principles argument. Yeah. I think run more experiments is just really great underused advice. Is that why you built a meth lab in your house? Casey Newton Yeah, it’s going great for me. Bought me that hot tub. (Time 0:44:30)
  • AI for Sermon Prep
    • Nathan Bourne, an Episcopal priest, uses AI tools like Readwise and Claude to enhance his sermon preparation.
    • These tools help him find relevant material and think more deeply, demonstrating AI’s benefit for research. Transcript: Kevin Roose Okay, so first up, we’re going to hear from some listeners who felt strongly that AI was not making them dumber or worse at critical thinking, who believe that it is enhancing their ability To engage critically with new material and new subjects. So let’s play one from a perspective that we haven’t really engaged with a lot on this show so far, which is people of the cloth. My name is Nathan Bourne, and I’m an Episcopal priest. A big part of my work is putting things in conversation with one another. I’m constantly finding stories, news articles, chapters of books, little bits of story that people have shared with me and interpreting them alongside scripture. I’ve long struggled to find a good system to keep track of all those little bits I’ve found. Over the last year, I’ve turned to AI to help. I’ve used the Readwise app to better store, index, and query pieces that I’ve saved. I’ve also used Claw to help me find material that I would never encounter otherwise. These tools have expanded my ability to find and access relevant material that’s helped me think more deeply about what I’ll preach and in less time than I used to spend sifting through Google results and the recesses of my own hazy memory. Religion-related projects. I wrote a book about going to Christian college many years ago, and I spent a lot of time in church services over the years. And so much of what the church services that I’ve been in have done has tried to, like, sort of find a modern spin or a modern take or some modern insights on this very old book, the Bible. And I can imagine AI being very useful for that. Casey Newton Oh, yeah, absolutely. I mean, this feels like a case where Nathan is almost setting aside the question of AI and critical thinking and just focusing on ways that AI make his researching and writing that he Has to do every week much easier, right? Solid uses of the technology as it exists. And they’re still leaving plenty of room to bring his own human perspective to the work, which I really appreciate. And, you know, of course, always love to hear about a man of the cloth sort of clasping his hands together and saying, Claude, help me. All right, let’s hear the next one. This is from a software engineer named Jessica Mock, who told us about how she’s taking a restrained approach to asking AI for help with coding. Speaker 4 When I was being trained, my mentor told me that I should avoid using autocomplete. And he said that was because I needed to train my brain to actually learn the coding. And I took that to heart. I do that now with AI. I do use Copilot, but I use it for floating theories, asking about things that I don’t know. But if it’s something that I know how to do, I put it in myself and then I ask Copilot for a code review. And I found that to be pretty effective. My favorite use of Copilot though is what does this error mean when I’m debugging? I love asking that because you get more context into what’s happening. And then I start to understand what’s actually going on. Is it making me dumber? I don’t think so. I think it’s making me learn a lot. I’m jumping into languages that I was never trained in, and I’m trying things that I normally would have shied away from. So I think it really depends on how you use it. So (Time 0:52:05)
  • AI-Assisted Problem-Solving
    • Anna, a self-employed professional, used ChatGPT to solve a complex Airtable problem.
    • While ChatGPT’s instructions were incorrect, they sparked an idea that led Anna to the correct solution. Transcript: Kevin Roose Do one more in this camp of people who don’t think that AI is making them dumber or worse at critical thinking. Speaker 4 My name is Anna and I live in a suburb of Chicago. Casey Newton I wanted to share a recent experience I had with AI and how it made me think harder about solving a problem. I’m self-employed and don’t have the benefit of a team to help me if I get stuck on something. I was using an app called Airtable, which is a database product. I consider myself an advanced user, but not an expert. I was trying to set up something relatively complex, couldn’t figure it out, and couldn’t find an answer in Airtable forums. Finally, I asked ChatGPT. I explained what I was trying to do in a lot of detail and asked ChatGPT to tell me how I should configure Airtable to get what I was looking for. Speaker 4 ChatGPT gave me step-by instructions, but they were incorrect. I prompted ChatGPT again and said, Airtable doesn’t work that way. And ChatGPT replied, you’re right. Here are some additional steps you should take. Casey Newton The resulting instructions were also incorrect, but they were enough to give me an idea, and my idea worked. In this example, the back and forth with ChatGPT was enough to help me stretch the skills I already had into a new use case. I (Time 0:59:50)
  • Over-Reliance on AI
    • Katia, a master’s student, over-relied on AI coding tools due to deadline pressure.
    • She prioritized speed over understanding, raising concerns about AI-induced skill degradation. Transcript: Kevin Roose Which is why I plan if anyone objects to any segments that we do on the show today or in the future, I do plan on blaming ChatGPT. That was the ChatGPT’s idea. Yeah. Interesting. If it’s a good segment, I did it. If not, it was Claude. All right, let’s move to another listener message. This one’s from a listener named Katia, who’s from Switzerland. She told us about how looming deadline pressure caused her to maybe over-defer to AI outputs. She wrote, quote, last semester, I basically did an experiment on this myself. I was working on a thesis during my master’s studies and decided to use some help. My choice fell on Cursor, which is one of these AI coding products. She writes, initially, I intended using it for small tasks only just to be a bit faster, but then the deadline was getting closer. Panic was setting in, and I started using it more and more. The speed was intoxicating. I went from checking every line of code to running rounds of automatic bug fixing without understanding what the problems were or what was being done. So (Time 1:04:31)
  • The Pressure of AI in the Workplace
    • Workplace AI adoption raises expectations, potentially forcing individuals to prioritize speed over understanding.
    • This pressure can lead to less critical thinking and reduced skill development. Transcript: Casey Newton I actually think this is the most important email that we’ve gotten so far because it highlights a dynamic that I think a lot of people are going to start feeling over the next couple of Years, which is my bosses have woken up to the fact that AI exists. They’re gradually raising their expectations for how much I can get done. If I am not using the AI tools that all my coworkers are now using, I will be behind my coworkers, and I will be putting my career at risk, right? And so I think we’re going to see more and more people do exactly what Katja did here and just use these tools like Cursor. And while, you know, to some certain level, I think that’s okay. We’ve always used productivity tools to make ourselves more productive at work. There is a moment where you actually just stop understanding what is happening. And that is a recipe for human disempowerment, right? At that point, you’re just sort of barely supervising a machine, and the machine is now doing most of your job. So this is kind of like a small story that I think contains a dark warning about what the future might look like. Kevin Roose Yeah, I think that kind of mental outsourcing does worry me, the sort of autopilot of human cognition. An analogy I’ve been thinking about recently in trying to distinguish between tasks that we should outsource to AI and tasks that we probably shouldn’t is forklifting versus weightlifting. Okay, tell me about this. So there are two reasons that you might want to lift heavy things. One of them is to get them from point A to point B for some like, you know, purpose. Maybe you work in a warehouse. Obviously you should use a forklift for that, right? There’s no like salutary benefit to carrying heavy things across a warehouse by yourself. And that’s very slow. It’s very inefficient. And the point of what you’re doing is to try to get the thing from point A to point B. Use a forklift for that. Weightlifting is about self-improvement. Weightlifting is, yes, you could use a machine to lift this heavy object, but it’s not going to make you stronger in any way. The point of weightlifting is to improve yourself and your own capabilities. So I think when you’re in a situation where you have the opportunity or the choice of using AI to help you do some task, I think you should ask yourself whether that task is more like forklifting Casey Newton Or more like weightlifting and choose accordingly. I think it is a really good analogy and people should draw from that. I want to offer one last thought of my own, Kevin, which is that while I think it is important to continue this conversation of how is AI affecting my critical thinking, I think in this Last anecdote, we see this other fear being raised, which is what if the issue isn’t do I still have my critical thinking skills? And what if the actual question is, do I have time to do critical thinking? Because I think that one effect of these AI systems is that everybody is going to feel like they have less time. The expectations on them have gone up at work. They’re expected to get more done because people know that they have access to these productivity tools. And so you might say, you know what, I actually really want to take some time on this. And I don’t want to turn to the LLM. And I want to bring my own human perspective to this. And you’re going to see all your co-workers not doing that. And it is just going to drag you into doing less and less of that critical thinking over time. So while I think, you know, is AI making me dumber is a really like interesting and funny question that we should keep asking. I think, am I going to have the time that I need to do critical thinking might actually be the more important question. (Time 1:05:31)