Skip to content

Podcast

Is AI Slowing Down? Nathan Labenz Says We're Asking the Wrong Question

The a16z Show

Source ↗ ← All highlights
  • Separate Impact From Capability
    • AI progress and its societal impact are distinct questions and should be evaluated separately.
    • Nathan argues GPT-5’s improvements were real but harder to perceive because expectations shifted and interim releases blurred leaps. Transcript: Nathan Labenz Yeah i mean i think for one thing it’s really important to separate a couple different questions i think with respect to ai one would be is it good for us right now even and is it going to Be good for us in the big picture and then i think that is a very distinct question from are the capabilities that we’re seeing continuing to advance and at a pretty healthy clip. So I actually found a lot of agreement with the Cal Newport podcast that you shared with me when it comes to some of the worries about the impact that AI might be having even already on people. You know, he looks over students’ shoulders and watches how they’re working and finds that basically he thinks that they are using AI to be lazy, which is no big revelation. I think a lot of teachers would tell you that. Shocker. He puts that in, yeah, puts that in maybe more dressed up terms that people are not even necessarily moving faster, but they’re able to reduce the strain that the work that they’re doing Places on their own brains by kind of trying to get AI to do it. And, you know, that continues. And I think he’s been, I think, a very valuable commenter on the impact of social media. Certainly, I think we all should be mindful of how is my attention span evolving over time? And am I getting weak or averse to hard work? Those are not good trends if they are showing up in oneself. So I think he’s really right to watch out for that sort of stuff. And then as we’ve covered in many conversations in the past, I’ve got a lot of questions about what the ultimate impact of AI is going to be. And I think he probably does too. But then when it comes to, it’s a strange move from my perspective to go from, there’s all these sort of problems today and maybe in the big picture too, but don’t worry, it’s flatlining, Like kind of worry, but don’t worry because it’s not really going anywhere further than this or it’s scaling has kind of petered out or we’re not going to get better AI than we have right Now. Or even maybe the most easily refutable claim from my perspective is GPT-5 wasn’t that much better than GPT-4. And that I think is where I really was like, what, wait a second. I was with you on a lot of things. And some of the behaviors that he observes in the students, I would cop to having exhibited myself. When I’m trying to code something these days, a lot of times I’m like, oh man, can’t the AI just figure it out? I really don’t want to have to sit here and read this code and figure out what’s going on. It’s not even about typing the code anymore. You know, I’m way too lazy for that. But it’s even about figuring out how the code is working. Can’t you just make it work? Try again, you know, and just try again. And I do find myself at times falling into those traps. But I would say a big part of the reason I can fall into those traps is because the AIs are getting better and better. And increasingly, it’s not crazy for me to think they might be able to figure it out. So that’s my kind of first slice at the takes that I’m hearing. (Time 0:01:40)
  • Post-Training And Context Matter
    • Progress can come from larger models or smarter post-training and context handling, not just raw scale.
    • Nathan highlights longer context windows and post-training as a major source of recent gains. Transcript: Nathan Labenz So they did train a much bigger model, which was GPT 4.5, and that did get released. And there are a number of interesting, of course, there’s a million benchmarks, whatever. The one that I zero in on the most in terms of understanding how GPT 4.5 relates to both O3 and GPT 5, and OpenAI obviously famously terrible at naming. We can all agree on that. I think a decent amount of this confusion or sort of disagreement actually does stem from unsuccessful naming decisions. 4.5 on this one benchmark called SimpleQA, which is really just a super long tail trivia benchmark. It really just measures, do you know a ton of esoteric facts? And they’re not things you can really reason about. You either just have to know or don’t know these particular facts. The 03 class of models got about a 50% on that benchmark, and GPT 4.5 popped up to like 65%. So in other words, it basically, of the things that were not known to the previous generation of models, it picked up a third of them. Now, there’s obviously still two thirds more to go, but I would say that’s a pretty significant leap, right? These are super long tail questions. I would say most people would get like close to a zero. You’d be like the person sitting there at the trivia night who like maybe gets one a night is kind of what I would expect most people to do on simple QA. And that checks out, right? Like obviously the models know a lot more than we do in terms of facts and just general information about the world. So at a minimum, you can say that GBT 4.5 knows a lot more. A bigger model is able to absorb a lot more facts. Qualitatively, people also said in some ways, maybe it’s better for creative writing. It was never really trained with the same power of post-training that GPT-5 has had. And so we don’t really have an apples-to comparison, but people did still find some utility in it. I think maybe the way to understand why they’ve taken that offline and gone all in on GPT-5 is just that model’s really big. It’s expensive to run. The price was like way higher. It was a full order of magnitude plus higher than GPT-5 is. And it’s maybe just not worth it for them to consume all the compute that it would take to serve that. And maybe they just find that people are happy enough with the somewhat smaller models for now. I don’t think that means that we will never see a bigger GPT 4.5 model with all that reasoning ability. And I would expect that that would deliver more value, especially if you’re really going out and trying to do esoteric stuff that’s, you know, pushing the frontier of science or what Have you. But in the meantime, the current models are really smart, and you can also feed them a lot of context. That’s one of the big things that has improved so much over the last generation. When GPT-4 came out, at least the version that we had as public users was only 8,000 tokens of context, which is like 15 pages of text. So you were limited. You couldn’t even put in like a couple papers. You would be overflowing the context. And this is where prompt engineering initially kind of became a thing. It was like, man, I’ve really only got such a little bit of information that I can provide. I got to be really careful about what information to provide lest I overflow the thing and it just can’t handle it. There were also, as context windows got extended, there were also versions of models where they could nominally accept a lot more, but they couldn’t really functionally use them. You know, they sort of could fit them, you know, at the API call level, but the models would lose recall or they’d sort of unravel as they got into longer and longer context. Now you have obviously much longer context and the command of it is really, really good. So you can take dozens of papers on the longest context windows with Gemini, and it will not only accept them, but it will do pretty intensive reasoning over them and with really high Fidelity to those inputs. So that skill, I think, does kind of substitute for the model knowing facts itself. You could say, geez, let’s try to train all these facts into the model. We’re going to need, you know, a trillion or who knows, five trillion, however many trillion parameters to fit all these super long tail facts. Or you could say, well, a smaller thing that’s really good at working over provided context can, if people take the time or, you know, go to the trouble of providing the necessary information, I can kind of access the same facts that way. So you have a kind of, do I want to push on this size and do I want to bake everything into the model? Or do I want to just try to get as much performance out of a smaller, tighter model that I have? And it seems like they’ve gone that way. And I think basically just because they’re seeing faster progress on that gradient. You know, in the same way that the models themselves are always kind of in the training process, taking a little step toward improvement, you know, the outer loop of the model architecture And the nature of the training runs and where they’re going to invest their compute is also kind of going that direction. And they’re always looking at like, well, we could scale up over here, maybe get this kind of benefit a little bit, or we could do more post-training here and get this kind of benefit. And it just seems like we’re getting more benefit from the post-training and the reasoning paradigm than scaling. But I don’t think either one is, I definitely don’t think either one is dead. We haven’t seen yet what 4.5 with all that post-training would look like. (Time 0:07:55)
  • Reasoning Reached New Frontiers
    • Reasoning abilities have advanced from high-school math to IMO-level performance in months.
    • Models are starting to contribute to scientific discovery and hypothesis generation that GPT-4 couldn’t achieve. Transcript: Nathan Labenz I mean, a big one from just the last few weeks was that we had an IMO gold medal with pure reasoning models with no access to tools from multiple companies. And, you know, that is night and day compared to what GPT-4 could do with math, right? And these things are really weird. Like, it’s nothing I say here should be intended to suggest that people won’t be able to find weaknesses in the models. I still use a tic-tac puzzle to this day where I take a picture of a tic-tac board where one of the players has made a wrong move that is not optimal and thus allows the other player to force A win. And I ask the models if somebody can force a win from this position. Only very recently, only the last generation of models are starting to get that right some of the time. Almost always before they were like, tic-tac is a solved game. You know, you can always get a draw. There’s, and they would wrongly assess my board position as the player can still get a draw. So there’s a lot of weird stuff, right? The jagged capabilities frontier remains a real issue and people are going to find, you know, peaks and valleys for sure. But GPT-4, when it first came out, couldn’t do anything approaching IMO gold problems. It was still struggling on, like, high school math. And since then, we’ve seen this high school math progression all the way up through the IMO gold. Now we’ve got the frontier math benchmark that is, I think, now like up to 25%. It was 2% about a year ago or even a little less than a year ago, I think. And we also just today saw something where, and I haven’t absorbed this one yet, but somebody just came out and said that they had solved a canonical, super challenging problem that No less than Terrence Tao had put out. And it was like this, you know, this thing happened in, I think, days or weeks of the model running versus it was 18 months, you know, that it took professional, not just any professional Mathematicians, but like really, you know, the leading minds in the world to make progress on these problems. So, yeah, I think that’s really, you know, that’s really hard jumping capabilities to miss. I also think a lot about the Google AI co-scientist, which we did an episode with. You can check out the full story on that if you want to. But, you know, they basically just broke down the scientific method into a schematic. You know, and this is a lot of what happens when people, there’s one thing to say, the model will respond with thinking and it’ll go through a reasoning process. And, you know, the more tokens it spends at runtime, the better your answer will be. That’s true. Then you can also build this scaffolding on top of that and say, okay, well, let me take something as broad and, you know, aspirational as the scientific method. And let me break that down into parts. Okay, there’s hypothesis generation, then there’s hypothesis evaluation, then there’s, you know, experiment design, there’s literature review, there’s all these parts of the Scientific method. What the team at Google did is created a pretty elaborate schematic that represented their best breakdown of the scientific method, optimized prompts for each of those steps, and Then gave this resulting system, which is scaling inference now kind of two ways. It’s both the chain of thought, but it’s also all these different angles of attack structured by the team. And they gave it legitimately unsolved problems in science. And in one particularly famous kind of notorious case, it came up with a hypothesis, which it wasn’t able to verify because it doesn’t have direct access to actually run the experiments In the lab. But it came up with a hypothesis to some open problem in virology that had stumped scientists for years. And it just so happened that they had also recently figured out the answer, but not yet published their results. And so there was this confluence where the scientists had experimentally verified, and Gemini, in the form of this AI co-scientist, came up with exactly the right answer. And these are things that, like, literally nobody knew before. And GPT-4 just wasn’t doing that. You know, I mean, these are qualitatively new capabilities. That thing, I think, ran for days. You know, it probably cost hundreds of dollars, maybe into the thousands of dollars to run the inference. You know, that’s not nothing, but it’s also, like, very much cheaper than, you know, years of grad students. And if you can get to those caliber of problems and actually get good solutions to them, like, you know, what would you be willing to pay, right, for that kind of thing? So, yeah, I don’t know. That’s probably not a full appreciation. We could go on for a long time. But I would say, in summary, GPT-4 was not able to push the actual frontier of human knowledge. To my knowledge, I don’t know that ever discovered anything new. It’s still not easy to get that kind of output from a GPT-5 or a Gemini 2.5 or, you know, a Claude Opus 4 or whatever. But it’s starting to happen sometimes. (Time 0:13:37)
  • Launch Glitches Hurt Perception
    • Nathan says GPT-5’s launch suffered from a broken router that routed queries to a weaker model, ruining first impressions.
    • He notes initial negative reactions softened as fixes showed GPT-5 is the best model available. Transcript: Nathan Labenz I mean, I think a decent amount of it was that they kind of fucked up the launch, you know, simply put, right? They, like, were tweeting Death Star images, which Sam Altman later came back and said, no, you’re the Death Star, I’m not the Death Star. But I think people thought that the Death Star was supposed to be the model. That was generally the, you know, the expectations were set extremely high. The actual launch itself was just technically broken. So a lot of people’s first experiences of GPT-5, they’ve got this model router concept now where, I think another way to understand what they’re doing here is they’re trying to own the Consumer use case. And to own that, they need to simplify the product experience relative to what we had in the past, which was like, okay, you got GBD4 and 4.0 and 4.0 Mini and O3 and O4 Mini and other things, You know, 4.5 was in there at one point. You got all these different models. Which one should I use for which? It’s like very confusing to most people who aren’t obsessed with this. And so one of the big things they wanted to do was just shrink that down to just ask your question and you’ll get a good answer. And we’ll take that complexity on our side as the product owners to do that. Interestingly, and I don’t have a great account of this, but one thing you might want to do is kind of merge the models and figure out, just have the model itself decide how much to think. Or maybe even, you know, have the model itself decide how many of its experts, if it’s a mixture of experts, architecture it needs to use. Or maybe, you know, there’s been a bunch of different research projects on like skipping layers of the model. If the task is easy enough, you could like skip a bunch of layers. So you might have hoped that you could genuinely on the back end merge all these different models into one model that would dynamically use the right amount of compute for the level of Challenge that a given user query presented. It seems like they found that harder to do than they expected. And so the solution that they came up with instead was to have a router where the router’s job is to pick, is this an easy query? In which case we’ll send you to this model. Is it a medium? Is it a hard? And I think they just have two really models behind the scenes. So I think it’s just really easy or hard. Certainly the graphs that they showed, you know, basically showed the kind of with and without thinking. The problem at launch was that that router was broken. So all of the queries were going to the dumb model. And so a lot of people literally just got bad outputs, which were worse than 03, because they were getting non-thinking responses. And so the initial reaction of like, okay, this is dumb, and that sort of traveled really fast, I think that kind set the tone. My sense now is that as the dust has settled, most people do think that it is the best model available. And, you know, things like the meter, the infamous meter task length chart, it is the best. (Time 0:18:58)
  • Scrutinize AI Study Contexts
    • When evaluating AI studies, inspect task choice, tool familiarity, and context window limits before generalizing results.
    • Nathan warns Meter tested cursor where models were least helpful and users were novices with the tool. Transcript: Nathan Labenz For one thing, I think that meter paper is worth unpacking a little bit more because this was one of those things that was, and I, I’m a big fan of meter and I have no, um, you know, no shade On them. Cause I do think do science publish your results. Like that’s good. You don’t have to, uh, make every experimental result and everything you put out conform to a narrative. But I do think it was a little bit, was a little bit too easy for people who wanted to say that, oh, this is all nonsense to latch onto that. And, you know, again, there’s something there that I would kind of put in the Cal Newport category too, where for me, maybe the most interesting thing was the users thought that they Were faster when, in fact, they seemed to be slower. So that sort of misperception of oneself, I think, is really interesting. Personally, I think there’s some explanations for that that include, like, hitting go on the agent, going to social media and scrolling around for a while and then coming back. The thing might have been done for quite a while by the time I get back. So honestly, one like really simple, and we’re starting to see this in products, one really simple thing that the products can do to address those concerns is just provide notifications. Like the thing is done now. So, you know, stop scrolling and come back and check its work. That in terms of just clock time, you know, it would be interesting to know like what applications did they have open? Maybe they took a little longer with Cursor than doing it on their own. But how much of the time was Cursor the active window and how much of it was, you know, some other random distraction while they were waiting? But I think a more fundamental issue with that study, which, again, wasn’t really about the study design, but just in the sort of, you know, interpretation and kind of digestion of it, Some of these details got lost. The, they basically tested the models or the, you know, the product cursor in the area where it was known to be least able to help. This study was done early this year. So it was done with, you know, kind of one, depending on how you want to count, right, a couple releases ago with code bases that are large, which again, strains the context window. And, you know, that’s one of the frontiers that has been moving. Very mature code bases with like high standards for coding, and developers who really know their codebases super well, who’ve made a lot of commits to these particular codebases. So I would say that’s basically the hardest situation that you could set up for an AI, because the people know their stuff really well, The AI doesn’t. The context is huge. People have already absorbed that through working on it for a long time. The AI doesn’t have that knowledge. And again, a couple generations ago, models. And then a big thing, too, is that the people were not very well versed in the tools. Why? Because the tools weren’t really able to help them yet. I think the sort of mindset of the people that came into the study in many cases was like, well, I haven’t used this all that much because it hasn’t really seemed to be super helpful. They weren’t wrong in that assessment, given the, you know, the limitations. And you could see that in terms of the, um, of the instructions and the help that the meter team gave to people one of the things that is in the paper that they would if they noticed that you Weren’t like weren’t using cursor super well they would give you some feedback on how to use it better one of the things that they were telling people to do is make sure you at tag a particular File to bring that into context for the model so that the model has, you know, the right context. And that’s literally like the most basic thing that you would do in Cursor. You know, that’s like the thing you would learn on your, in your first hour, your first day of using it. So it really does suggest that these were, you know, while very capable programmers, like basically mostly novices when it came to using the AI tools. So I think the result is real, but I would be very cautious about generalizing too much there. (Time 0:26:12)
  • Code Enables Fast AI Flywheels
    • Code is ideal for agent development because you can run and validate outputs quickly, creating a fast improvement loop.
    • Companies aim for automated research workflows to multiply engineering capacity via AI-assisted PRs and agents. Transcript: Nathan Labenz Utopia or dystopia is really the big question there, I think. I mean, is maybe one part technical, two parts social in terms of why code has been so focal? The technical part is that it’s really easy to validate code. You generate it, you can run it. If you get a runtime error, you can get the feedback immediately. It’s, you know, somewhat harder to do functional testing. Replit recently, just in the last like 48 hours, released their v3 of their agent. And it now, in addition to, you know, code, code, code, try to make your app work, v2 of the agent would do that. And it could go for minutes and, you know, in some cases generate dozens of files. And I’ve had some magical experiences with that, where I was like, wow, you just did that whole thing in one prompt and it like worked amazing. Other times it will sort of code for a while and hand it off to you and say, okay, does it look good? Is it working? And you’re like, no, it’s not. I’m not sure why, you know, you get into a back and forth with it. But the difference between V2 and V3 is that instead of handing the baton back to you, it now uses a browser and the vision aspect of the models to go try to do the QA itself. So it doesn’t just say, okay, hey, I tried my best, wrote a bunch of code, like, let me know if it’s working or not. It takes that first pass at figuring out if it’s working. And, you know, again, that really improves the flywheel, just how much you can do, how much you can validate, how quickly you can validate it. The speed of that loop is really key to the pace of improvement. So it’s a problem space that’s pretty amenable to the sorts of rapid flywheel techniques. Second, of course, they’re all coders at these places, so they want to solve their own problems. That’s very natural. And third, I do think on the social vision competition, who knows where this is all going, they do want to create the automated AI researcher. That’s another data point, by the way, this was from the 03 system card. They showed a jump from like low to mid single digits to roughly 40% of PRs actually checked in by research engineers at OpenAI that the model could do. So prior to 03, not much at all, you know, low to mid single digits as of 03, 40%. I’m sure those are the easier 40% or whatever. Again, there will be, you know, caveats to that. But that’s, you’re entering maybe the steep part of the S-curve there. And that’s presumably pretty high end, you know, I don’t know how many easy problems they have at OpenAI, but presumably, you know, not that many relative to the rest of us that are out Here making generic web apps all the time. So, you know, at 40%, you got to be starting to, I would think, get into some pretty hard tasks, some pretty high value stuff. At what point does that ratio really start to tip where the AI is doing the bulk of the work? GBD5 notably wasn’t a big update over 03 on that particular measure. It also wasn’t going back to the simple QA thing. GBD5 is generally understood to not be a scale-up relative to 4.0 and 03. And you can see that in the simple QA measure, it basically scores the same on these long tail trivia questions. It’s not a bigger model that has absorbed like lots more world knowledge. It is, it is, you know, Cal is right. I think it is analysis that it’s, it’s post-training, but that post-training, you know, is potentially entering the steep part of the S-curve when it comes to the ability to do even The kind of hard problems that are happening at OpenAI on the research engineering front. And, you know, yikes. I’m a little worried about that, honestly. The idea that we could go from these companies having a few hundred research engineer people to having, you know, unlimited overnight. And like, what would that mean in terms of how much things could change? And also just our ability to steer that overall process. I’m not super comfortable with the idea of the companies tipping into a recursive self-improvement regime, especially given the level of control and the level of unpredictability That we currently see in the models. But that does seem to be what they are going for. So in terms of like why, I think this has been the plan for quite some time. Even you remember that leaked Anthropic fundraising deck from maybe two years ago, where they said that in 2025 and 2026, the companies that train the best models will get so far ahead That nobody else will be able to catch up. I think that’s kind of what they meant. I think that they were projecting then that in the 25, 26 timeframe, they’d get this like automated researcher. And once you have that, how’s anybody, you know, who doesn’t have that going to catch up with you? Obviously, some of that remains to be validated. (Time 0:36:16)