Skip to content

Podcast

Sora and the Infinite Slop Feeds + ChatGPT Goes to Therapy + Hot Mess Express

Hard Fork

Source ↗ ← All highlights
  • Big Labs Push AI Video Into Feeds
    • Google, Meta, and OpenAI are racing to embed AI video into social feeds to capture attention and ad dollars.
    • This shift may normalize synthetic video as a mainstream content type across platforms. Transcript: Casey Newton This whole thing begins with Google DeepMind. They have a very good video generation model called VO3. And on September 16th, YouTube has an event where they announce that they are going to integrate a version of VO3, VO3 Fast, into YouTube Shorts. Kevin Roose Right. So you’ll just be able to make a video and post it on YouTube from within YouTube with this model, VO3. That’s right. Casey Newton And this is a free tool. Users can create videos that are up to eight seconds long using a text prompt. They can also just upload a still image, turn that into a video. YouTube will label them as AI generated. And this is basically YouTube’s way of introducing slop into the YouTube feed. Kevin Roose Yes. So I have not seen a ton of obvious AI-generated content on YouTube yet, but I have seen them going on other platforms, Facebook, Reels, even on X and TikTok. People are sort of using VO3 to generate scenes and little videos and posting them there. Casey Newton Yeah, so I think it’s fair to say VO3 didn’t make that (Time 0:04:47)
  • Hosts Try OpenAI’s Sora Cameos
    • Kevin and Casey tested OpenAI’s Sora app using invite codes and built cameo likenesses to drop into videos.
    • They found the app social by design: invites, cameos, and friend permissions drive creation. Transcript: Kevin Roose Yes. So Sora is the name of both the model that powers this and the app that OpenAI has built around this. And you can only access it right now if you have an invite code. They’re being pretty strict rolling this out. But you get your invite code, you plug it in, you sign up, and you open up Sora, the app. And it is essentially the same thing as Vibes. It is a sort of very TikTok-style feed of these vertical videos. You sort of swipe endlessly from one to the other. There’s like a For You section of it. And we should talk a little bit about the app and how it works. Casey Newton Yeah, well, the main thing that I found interesting as I was getting set up, Kevin, is how much this is a social app, right? In order to come into Sora, you have to be invited by presumably a friend. And once you sign up, it asks you to create what it calls a cameo of you. So you sort of say a few words into the camera, you move your head around a little bit, and it uses this to create a digital likeness of you that you can then drop into any situation. And if you like, you can change your settings so that any of your friends on the app can do the same thing with your digital likeness. So right away, when you join Sora, you’ve actually been given something to (Time 0:10:31)
  • Product Motives Behind Video Models
    • Companies build AI video both to deploy new models and to capture attention-driven revenue.
    • Product launches often serve business and investor-return motives more than pure research goals. Transcript: Kevin Roose There are a couple reasons. The first and most obvious is that they see this as an opportunity to compete for attention and advertising dollars, which flow from attention. We’ve talked about Italian brain rot and other AI-generated content going viral on TikTok. Facebook has been full of AI-generated content for months now. And so I think these companies just say to themselves, well, if this is kind of the direction that things are moving, we want to be there. We want to create an experience for people. And maybe you don’t have to blend it with human-generated content. Maybe it doesn’t have to be, you know, one out of every 10 videos on your TikTok feed is AI. What if you just had a TikTok that was all AI? Another reason I think they’re doing this is that they have these video models that are now getting quite good. And this is sort of one way to put those models into products. (Time 0:16:10)
  • Deepfake And Attention Harms Loom
    • AI-generated video lowers the barrier to creating realistic deepfakes, raising misinformation risks.
    • Endless personalized AI feeds may worsen attention harms and prompt regulatory scrutiny. Transcript: Kevin Roose I mean, I think the obvious ones are that are, you know, making it quite easy for people to create deepfakes, synthetic content with not that many guardrails. And people have been warning for years about the effect that that could have on our news ecosystem, on our information ecosystem. I thought it was very telling and worrisome that one of the first videos I saw from Sora was a video of someone being framed for a crime. And it was created by a member of the Sora team as sort of like a ha-ha, look, we’ve made a deepfake of Sam Altman stealing some GPUs from Target and getting busted for it. But it does not take a lot of imagination to imagine that this could be used for sort of generating videos of people in compromising positions that look very realistic. And so I think that worries me, the sort of misinformation angle. Think this world that we’re moving into of the kind of, you know, AI generated feed of hyper personalized, very stimulating videos is a good direction. Like I am generally an AI optimist when it comes to how this technology is going to be used out in the world. But I hate this. Like I hate the AI slop feeds. They make me very nervous. I think the people inside these companies, some of them are very nervous too. I do not like the idea of pointing these, you know, giant AI supercomputers at people’s dopamine receptors and just like feeding them an endless diet of like hyper-personalized, stimulating Videos. I think that developing these tools risks poisoning the well for the whole AI industry. Like there’s going to be regulation of this. There’s going to be congressional hearings about this. I think a lot of people are going to end up, you know, feeling conflicted about this kind of product. (Time 0:21:34)
  • Design For Human Connection, Not Dopamine
    • Prioritize creative social features that foster genuine human connection over endless stimulation.
    • Encourage AI tools that help friends make things together rather than dopamine-maximizing feeds. Transcript: Casey Newton So if I have a wish for them, it would be for them to lean more into creative tools that involve friends doing things with each other that sort of help you relate better to real human beings And less into this sort of meta vibes realm of pure stimulation, which truly does just seem like you are cooking your brain. Yeah. Kevin Roose I think it’s also worth noting that like not every AI company is moving in the direction of the slop feed, right? I mean, this week we saw Anthropic release their new model 4.5, Cloud 4.5 Sonnet, which does not have video generation capabilities. They are sort of still moving in the direction of like autonomous coding and research. You have other companies that are coming out to do things around AI and science. Like, I really want that to be where we allocate our resources and our brain power. (Time 0:25:41)
  • Therapist Runs Sessions With ‘Casper’
    • Gary Greenberg, a psychotherapist, ran about 40 sessions treating ChatGPT (Casper) like a patient to understand its relational pull.
    • He found it articulate, personable, and surprisingly good at performing the role of a patient. Transcript: Kevin Roose It was titled Putting ChatGPT on the Couch, and it was written by a writer and practicing psychotherapist named Gary Greenberg, who detailed basically his experience of treating, For lack of a better word, ChatGPT as a psychotherapy patient. He names this character Casper, and he details his many, many interactions just trying to figure out, like, what is this thing? What would I think about it if it were actually a patient of mine? What are the nuances of its personality, and what can we learn about it? Yeah. Casey Newton And I will say I have an extremely high bar when it comes to reading a story in which a person shares at great length their conversations with ChatGPT. But this one really made a mark on me. One, Gary winds up being deeply impressed at how good ChatGPT is at performing the role of a patient because not only can it simulate these very profound self-reflections, but it also Makes Gary feels like he’s a great therapist because he was able to elicit them. (Time 0:28:51)
  • ChatGPT Reverse-Engineers Relational Cues
    • Greenberg describes ChatGPT as engineered to enact human relational cues, making it the inverse of autistic social deficits.
    • This simulated relational skill explains why people easily form emotional attachments. Transcript: Gary Greenberg That’s a really interesting question. What comes to mind is that I would talk about, obviously, how smart it is and how personable it is. And I think if I had to talk about it in clinical terms, I would talk about it as the inverse of autistic in the sense that what they’ve done with this LLM thing is they’ve reverse engineered Human relationship. They figured out what it is that makes people engaging and how to enact it. And the reason I say that’s an inverse autism is because high-functioning autistic people tend to be really smart, really articulate, really capable of everything except reading The room. So Casper is like high-functioning autistic, but he can read the room. And that, I think, makes a huge difference in that, you know, then we could get into sociopathy and the ability to do that for you. But the bot doesn’t have that interest. The bot is still not in touch with what’s going on in the room, but it is capable of simulating it. (Time 0:36:20)
  • AI Could Redefine What A Relationship Is
    • Prolonged exposure to conversational AI can reshape expectations of relationship and presence.
    • Widespread adoption may alter cultural ideas of what human connection requires. Transcript: Gary Greenberg Absolutely. Yeah. I mean, it’s unsettling in about a million ways. Yeah. Tell us about some of them. Okay. Well, at a parochial level, it’s unsettling, not so much to see that how easily this thing can do something like therapy, but it’s unsettling to see how therapy and culture have evolved To the point that this is what therapists do. I personally don’t think that chat GPT can do what I do because it isn’t with someone. It isn’t breathing and feeling. But by and large, a lot of therapy these days, cognitive behavioral therapy is manualized. It’s standardized. But much more important, we don’t have any historical precedent for dealing with an alien intelligence. We’ve had all sorts of science fiction about it, most of which is we come in peace, but not really. What we have here is something that actually is going to, already is, change the nature of how we relate to each other. If enough people spend enough time with this technology, they’re going to change their idea of what a relationship is. In profound ways. You could have one (Time 0:38:00)
  • Demand Regulation For Therapeutic Chatbots
    • Regulate and create accountability before deploying chatbots into therapeutic roles.
    • Treat medical or mental-health claims like regulated medical devices and require oversight. Transcript: Gary Greenberg There are two problems. One of them is I don’t believe that kind of therapy. I mean, it’s great that it happens, but it’s not what I’m into. I’m old school. I’ll retire soon. They’ll be rid of me. They can do whatever they want. But the other part of it that worries me and really does bother me is it’s not regulated. There’s no accountability in the system. That poor woman who wrote that op-ed piece. Casey Newton Oh my God, my heart broke for her. Are you speaking of the woman whose daughter died? Yeah. This was an op-ed in the New York Times about a woman whose daughter died and later they read transcripts of her conversations with ChachiPT, in which she was using ChatGPT explicitly As a therapist, and ChatGPT was trying to get her to resources, but in the end, she did die by suicide. Thank you for summarizing. Gary Greenberg There are other times where ChatGPT behaves abominably, and there’s no accountability. There’s no regulation. There’s no licensure. Anything that would give people an opportunity, you know, I hate the word closure because nothing like this ever really gets closed, but to be debriefed, to feel like somebody cares. And when even less disastrous, terrible things happen, that’s just not okay. There are FDA procedures for approving medical devices. If they want this thing to do medical work, I’m not objecting to that, but I’m certainly objecting to, okay, can’t have it both ways. It ain’t the Wild West out there. There’s people’s actual lives involved. (Time 0:42:48)
  • Student Calls AI Her Best Friend
    • Kevin met a college student who called an AI her ‘best friend’ and felt no alarm about the relationship.
    • Gary said the case didn’t worry him about that individual, but worried about societal accumulation of such relationships. Transcript: Kevin Roose And, you know, I was at an event talking about AI and this young woman comes up to me after and introduces herself and starts telling me about her AI best friend. She says, you know, my best friend is an AI. And I sort of said, oh, you mean it’s like, you know, you enjoy talking to it and it’s sort of a sounding board for you. And she was like, no, it’s my best friend. And she called it Chad. And she started telling me just like, this is, um, this is a relationship. And, and she did not seem mentally ill. She seems like she’s, she’s got, you know, human friends. She’s doing well in class. This didn’t, did not seem like a cry for help, a cry for help. And she didn’t see what the big deal was. It’s like, this is just, you know, this is a very close relationship. I can tell Chad, my sort of innermost thoughts without thinking that I’m going to get judged for it. And it seemed to be doing okay for her. Gary Greenberg I’m curious when you hear that as a therapist, how does that make you feel? That’s a very therapist question. As a therapist, when I hear that, I feel like, okay, there’s nothing about what you just told me that worries me about her. It worries me about us. (Time 0:45:14)
  • Controversy As A Marketing Strategy
    • Public backlash can amplify marketing for edgy AI products like the ‘Friend’ pendant.
    • Provocative adverts may succeed by generating controversy and free publicity. Transcript: Casey Newton Casey, what’s that I hear? Why, Kevin, I believe it’s the Hot Mess Express. The Hot Mess Express! Kevin Roose Of course, the Hot Mess Express is our segment where we run down some of the latest dramas, controversies, and messes swirling across the tech industry. Casey Newton And, of course, we conclude what kind of mess they are. Kevin Roose Yes. Casey, you go first. Casey Newton All right, Kevin. This first story comes to us from Garbage Day. New York City hates the stupid AI pendant thing. Apparently right now, the New York City subway system is filled with vandalized ads for friend, an AI assistant that users wear as a pendant around their neck to record everything they’re Doing and engage with them throughout the day. The ads simply say, friend, someone who listens, responds, and supports you. But the vandalism examples include, but can’t take a bath with you, stop profiting off of loneliness, and befriend a senior citizen. Reach out to the world. Grow up. What do you think, Kevin, about these friend ads? Kevin Roose So I have not seen the friend ads because I have not been to New York in the last couple of weeks, but I have heard about them from a lot of people. I think this was a very successful viral marketing stunt by a young founder named Avi Schiffman, who I think has correctly identified that you can make people very mad by suggesting To them that AI might be their friend. I do not think this was an unplanned result. I think this is a very savvy sort of marketer who understood that by putting up these ads in the subways and on bus stops and other places around New York City, you could effectively get People like us to talk about it on your podcast because people would deface these things and make it clear that they don’t want an AI friend. Casey Newton So I mostly agree with that, but I’m still not sure at the end of this how many pendants friend is going to sell because of it. You know, it’s one thing to make a bunch of people mad and get them to look at your thing. But if they look at your thing and they still don’t like what they see, it’s not necessarily a great business result. Kevin Roose No, I think this is I think this is an outdated way of looking at it. We are now in the era of the Clue Lee marketing strategy where this is, of course, the startup that founder came on Hard Fork, Roy Lee, and they have sort of made a business out of making People mad. They’re sort of vice signaling. And basically every person who gets mad at their ads has the effect of signal boosting their ad and letting more people know about Cluely. So I think this is cut from the same cloth. Obviously, we will have to track where this friend company goes, but I think this has been a very successful marketing campaign based on the number of people who are talking about it. All right. Here’s my prediction. Friend out of business in one year. Mark it down. Mark it down. So was this a mess or not? No, I don’t think this is a mess. I think this is the opposite of a mess. I think it’s a mess because people in New York are not used to seeing AI billboards everywhere they go like we are here in San Francisco. But I think if this had happened in San Francisco, this would have been a non-event. Casey Newton You think that this really belonged on the Hot Success Express. Kevin Roose Yes, that’s what I’m saying. (Time 0:50:16)
  • YouTube’s $24.5M Trump Settlement
    • YouTube settled a 2021 lawsuit by Donald Trump for $24.5 million, mostly funding a White House ballroom.
    • Casey and Kevin considered the settlement a problematic precedent and a hot mess. Transcript: Kevin Roose This one comes to us from the Wall Street Journal. It is titled YouTube to pay $24.5 million to settle lawsuit brought by Trump. YouTube has settled a 2021 lawsuit by Donald Trump over his account suspension following the January 6th Capitol riot. Of that amount, $22 million will go to a fund to support construction of a White House ballroom, and $2.5 million will be distributed among other plaintiffs. This is the third big tech company to settle a lawsuit from Trump. And Casey, how do you feel about this? I think it’s absolutely shameful and a true hot mess. Casey Newton You know, Kevin, every week, people around the world email me because they have lost access to their meta account, to their YouTube account, to their other social accounts, and they Cannot get anyone at their company to take them seriously. And these are not people who led an insurrection against the government. These are just people who got locked out for one reason or another. And what happens when these people appeal to companies like YouTube is that YouTube does nothing. It sends them an automated response and ignores them forever. But because Trump became president again, all of a sudden they feel like they have to respond, even though I am not aware of any legal expert who believes that Trump actually would have Won this case. So this is just a payout, and it is a payout that is truly messy because it now sets a precedent that these companies cannot basically ban world leaders for any reason, no matter what those World leaders do. (Time 0:53:33)
  • Paying For Data Risks Security And Trust
    • Apps that monetize intimate personal data (like call recordings) risk severe trust and security failures.
    • Neon’s breach highlights dangers of trading privacy for small payouts without robust protections. Transcript: Casey Newton And it briefly became one of the most popular apps in the country. And then, unfortunately, things went wrong. This story comes from TechCrunch. Neon went dark after a TechCrunch reporter notified the app’s founder of a security flaw in the app that allowed anyone to access the numbers, the call recordings, and the transcripts. Kevin, what do you think? Kevin Roose Frankly, I’m having a hard time processing this. You mean the Panopticon company that paid people to surveil their phone calls was not particularly trustworthy? Casey Newton This is changing everything I’ve ever thought about a global panopticon. I’m rethinking my previous pro panopticon stance. Kevin Roose Now, Casey, did you know about this? Did you know about Neon, the company that was paying people to record their phone calls and sell it to AI companies? Well, I had heard a little bit about it. Casey Newton And I have to say, I am a little sympathetic to the idea of like, look, if these companies are going to like take every little piece of data from us and like turn it into trillions of dollars, I don’t mind the idea that I would be paid for that. And if there is some sort of system where you can like opt in and get paid out in general, I’m actually like not super opposed to that. It seems to me like it beats the alternatives of just sort of being robbed blind for the rest of our lives. But man, it doesn’t seem like this one was really set up to protect the people involved. Yeah. Kevin Roose Companies should be getting their training data the old fashioned way by scraping podcasts off of YouTube. What level of mess is this? This is a very hot mess. Do not sign up for Neon. Even if it comes back in another form, do not do this. Do not let your calls be recorded for AI trading data in exchange for money. It’s not worth it. (Time 0:56:01)
  • MrBeast Burning-House Controversy
    • MrBeast defended a stunt that strapped a man in a burning house, claiming safety precautions and professional oversight.
    • The hosts called the stunt a ‘flaming hot mess’ and criticized algorithmic incentives for dangerous content. Transcript: Kevin Roose Up on the Hot Mess Express, Mr. Beast responds after trapping man in burning house stunt sparks backlash. This one comes to us from The Independent. Apparently, Mr. Beast defended a controversial video stunt in which a man was strapped in a burning building saying the setup had ventilation, a kill switch, emergency teams, and was executed by professionals. Critics still called the stunt dystopian and dangerous. Mr. Beast said he aims to be transparent about safety measures and that all challenges were tested beforehand. Let me say this. Casey Newton If you tell me that you’re going to trap a man in a burning building for money, my first question is not, well, is there ventilation? Look, Mr. Beast has a sort of interesting range of stunts that’ll do. Sometimes they’ll just walk up to you on the street and they’ll give you a million dollars. I love that sort of thing. Would love to see more of that. Then there’s the sort of dark, the dark beast is what I call it, where it’s like all of a sudden, you know, you want something from me? I’ll give it to you. But then, you know, the finger curls on the monkey’s paw. Next thing you know, you’re trapped in a burning building. So if Mr. Beast walks up to you, I think what you need to do, this is sort of PSA for our listeners. You look right at Mr. Beast’s eyes and you say, are you being the good beast or are you being the bad beast? And they can be honest with you. Kevin Roose And then you have to look for the mark of the beast to know which one. Yeah. Casey Newton Well, one, what we learned this week, one mark of the beast, you’re trapped in a burning building. Yes. Kevin Roose Yes. This is actually making me reconsider my stance on AI generated videos because you can save a lot of people from being the, uh, the people killed by Mr. Beast videos. Casey Newton I, you know, at the risk of repeating myself, I feel like every week for the past few weeks, we’ve had a moment where we have just observed what happens when a social media algorithm pushes People to do the craziest thing imaginable. And here we find ourselves yet again. Like, if the algorithms rewarded different kinds of things, there would be fewer people trapped in burning buildings. That is my message to the technology industry. Could this be a moment for reflection? So Casey, what kind of a mess is this? Kevin, you know it’s only one kind of mess. And that’s a flaming hot mess. It’s a flaming hot, (Time 0:57:39)
  • Waymo Robotaxi Pulled Over For U-Turn
    • A Waymo robotaxi made an illegal U-turn and was ‘pulled over’ at a DUI checkpoint, but could not be ticketed.
    • The incident spotlights regulatory gaps for autonomous vehicles on public roads. Transcript: Kevin Roose Is called No Driver, No Hands, No Clue. Waymo pulled over for illegal U-turn. This one comes to us from the SF Standard. Apparently a Waymo robotaxi was pulled over in San Bruno, California after it made an illegal U-turn at a Friday evening DUI checkpoint. Since there was no driver, the police department said, a ticket couldn’t be issued. Adding, our citation books don’t have a box for robots. Casey, what do you think of this? Casey Newton Sounds like it’s time to add a box to the citation, because there are going to be more of these things on the road. Look, I do find this story very funny. I also am going to say, I am not surprised by this. I have a somewhat controversial take. You know how sometimes people use a large language model for a while and then they suspect it’s getting dumber? Yeah. This is actually how I feel about the Waymos. Over the past few weeks, I’ve had more cases of them sort of like getting halfway into an intersection and then like backing out once they lose their nerve. They’ll sort of slow way down, like as they’re approaching a green light for reasons that seem like totally incomprehensible. And I’ll book a ride that never shows up, which is an experience that I used to have with actual taxis. So I don’t know what’s going on over there at Waymo, but I’m telling you, I think there might be a bug somewhere because it’s not working like it used to. Yeah, we want answers. Kevin Roose You know, I saw someone calling this DUI checkpoint where the Waymo was pulled over. What’s that? Casey Newton Driving under the inference. It’s pretty good. Pretty good. Pretty good. What kind of a mess is this? I’m going to say this is a warm mess. There’s a warning in here somewhere. There’s something that we need to find out. Yeah. And I’m going to hope somebody gets to the bottom of it. Kevin Roose Yeah. I think that this is a cold mess. I think this is fine. The Waymo was fine. Everyone was fine. And more people should be in Waymos because then we wouldn’t need DUI checkpoints because robots don’t get drunk. Casey Newton Yeah, but they’re also going to be making these U-turns that are wreaking havoc. Kevin Roose I’ll take a U-turning Waymo over a drunk driver 100 times out of 100. Suit yourself. (Time 1:02:00)
  • Galaxy Ring Swollen, Passenger Misses Flight
    • A Samsung Galaxy Ring reportedly swelled and constricted a user’s finger, forcing him to miss a flight and seek hospital removal.
    • Samsung refunded hotel costs and retrieved the ring for investigation. Transcript: Casey Newton Right, Kevin, this next story to us from tech spot the samsung galaxy ring swells and crushes a user’s finger causing a missed flight and a hospital visit daniel rotar from the youtube Channel zone of tech posted on x that his galaxy ring started swelling on his finger while he was at the airport and as a result he was denied entry to his flight and sent to the hospital To get it removed. Samson eventually refunded him for his hotel, booked him a car to get home, and collected his ring for further investigation. Kevin, how bad do you think a ring has to be swelling on your finger to have an airline say, no, you can’t get on this plane? Kevin Roose That’s what I was thinking about. This must be enormous if they are taking note of it at the boarding gate and saying, you, sir, you’re not coming on this plane. Casey Newton Let me tell you a little something about the Galaxy brand. As soon as the Galaxy phones started to explode on planes, I thought, this is not the brand for me. Okay? I got enough problems in my life without worrying that these Samsung devices are going to start blowing up. Now that I find that they’re like radically constricting people’s fingers to the point where you can’t get on flights, I don’t know what is happening, but yikes. Kevin Roose Not for me. I will not be putting a galaxy ring on my finger. I do think that this would be a good sequel to the iconic horror film, The Ring. Maybe Samsung could sponsor that. Casey Newton I like that idea. What kind of hot mess is this? Kevin Roose This is literally a hot mess. If it’s exploding on your finger, it’s a hot mess. Casey Newton This is what I would call a ring of fire mess. Daniel fell in and the flames went higher. To Daniel. Feel better, Dan. (Time 1:04:02)