Skip to content

Podcast

Inside OpenAI’s Agentic Browser, Atlas

AI & I

Source ↗ ← All highlights
  • Context-Aware Browsing Unlocks More Questions
    • Atlas embeds ChatGPT directly into browsing so the model sees page context without copy-paste.
    • This reduces friction and lets users ask more questions while staying on the page. Transcript: Ben Goodger Yeah. So I think that, you know, maybe the future will get to a place where like more and more of your workload can be can be automated. And I think we’re making progress in that direction. But but today we wanted to design Atlas with this idea that you could bring ChatGPT with you wherever you go on the web. And so, yeah, I mean, I think the thing that you note of like, what do I do with this? This is something that we hear a lot from people, but then also we hear some aha moments as they go on the same journey that you have and begin to figure out some use cases for it. This is something that we actually want to take some of that learning that we have from how people are using it and help offer more proactive advice to people like in product to help them Figure out how to optimize use of the tool. But I think today, like one of the things that I noticed when I use Atlas versus when I go back and use a sort of pre AI browsing environment, I find myself just able to ask just a lot more questions And just be more knowledgeable about a topic. If I’m doing online shopping, I can feel confident that I’m getting the best deal or I have the right coupon code or I have all that sort of stuff. If I’m like researching a topic that’s of interest to me, I can sort of brainstorm different viewpoints on it. I can just sort of have this sort of friend or advisor that sort of comes with me and I can just like have this conversation with it. And that’s just made the web a lot richer and more dynamic. Can you make that more concrete for me? Dan Shipper Because I think some of those things, someone might be listening and being like, well, yeah, I could do that with ChatGPT now. That’s what ChatGPT does for me. So what does it mean to have that in the context of your browser? Ben Goodger It just means that you don’t need to go. I think for anyone that’s had ChatGPT in a tab, you probably have the experience of going and taking some content from another tab and pasting it in and asking a question about it perhaps. Whereas when you have a browser that’s built with this at the core of it, that context is provided directly to the model. (Time 0:04:22)
  • Never Open Settings Again
    • Dan stopped opening settings panels after using Atlas because the agent handles rare tasks.
    • He uses it to manage tedious admin flows like AWS permissions and seat changes. Transcript: Dan Shipper Got to tell you, like, and hopefully maybe this can be like a little bit of a user research session, too, because I feel like I’m doing something with this that I’m very excited about. And I’m curious if you guys are doing it, if you’re seeing other people doing it, how you’re building for this. So the big unlock that I had with Atlas is I realized I never need to look at a settings panel ever again. You’re not alone. Yeah. And that is such a refreshing feeling. It’s, I think it’s both refreshing for users and for software developers. I think it’s refreshing for software developers because you don’t have to worry about adding another knob because someone like the agent’s going to do that. So you can make software more customizable, more easily. But for users, like, I think the canonical example for me is looking at the AWS dashboard. I don’t know if you have… I assume you guys have both logged into that. And it’s like 50 different services. And then the settings, the permissioning system is like… It’s like launching a nuclear missile in order to like do anything and i run a company and we have like 20 people and so i’m i’m sort of constantly being asked hey can we like add a seat to This or like can you change the permission on this thing or and it’s like some account that we set up five years ago that i don’t even remember you don’t do these things so frequently and Ben Goodger So yeah it’s like not top of mind how to do it again yeah my example of this that i’ve been using was um uh i used it to help me create google forms to do user research uh and you know google form Builder i think is maybe less complicated than the aws control panel but uh still it’s not something i use every day and so i think for me to be able to ask the agent to go off and do that and Have it do that in a few minutes and come back and I can just submit, you know, certainly allowed me to get to the meat of the problem much quicker. Dan Shipper Yeah. It’s one of those tasks where like, there’s a certain amount of activation energy and you don’t have to spend the activation energy anymore. Darren, what were you going to say? Darin Fisher I was just saying it’s that time of year to go into work day and figure out how to get my year end-to pay stub so I can share that with my tax advisor. And I’m like, where do I go again? They moved it again, you know, and I don’t go there often enough. (Time 0:07:54)
  • Introduce Features Gradually
    • Expose advanced features gradually using progressive disclosure during onboarding.
    • Balance familiarity with discoverable power so users get rewarded as they explore further. Transcript: Darin Fisher One of the features that is pretty powerful, but, or relatively, or I should say we struggled with how to expose it, is this feature called cursor chat. If you’re interacting with a form field in the browser, you’ll see a little icon, a little chat to be icon, and you can hover over and then interact with the model and interact with the Model in the context of that specific form field. We struggle with how in your face to make this, right? We want people to be aware of this power. It’s actually really powerful. The people who use it are people who rave about this, helping them compose and that sort of thing. But actually, a lot of people don’t discover it, even though we have this little hint. And so it’s always a question, how big do you make that hint? How do you introduce this to people? Certainly during onboarding, we already have a lot of things we try to tell people about, because this is an AI browser. There’s new things to learn. Fundamental things like web memories and capabilities like side chat and so on. (Time 0:12:28)
  • Use Inline Helpers For Form Tasks
    • Add inline helpers like cursor chat on form fields to speed composition.
    • Make these helpers visible but not overwhelming so discoverability improves without clutter. Transcript: Darin Fisher If you’re interacting with a form field in the browser, you’ll see a little icon, a little chat to be icon, and you can hover over and then interact with the model and interact with the Model in the context of that specific form field. We struggle with how in your face to make this, right? We want people to be aware of this power. It’s actually really powerful. The people who use it are people who rave about this, helping them compose and that sort of thing. But actually, a lot of people don’t discover it, even though we have this little hint. And so it’s always a question, how big do you make that hint? How do you introduce this to people? Certainly during onboarding, we already have a lot of things we try to tell people about, because this is an AI browser. There’s new things to learn. Fundamental things like web memories and capabilities like side chat and so on. But we can only tell you about so many things at once. So that’s been a challenge for us from a design perspective, for sure. Dan Shipper That makes sense. I’ve seen that little icon. I have not clicked it. So now I feel like I need to click it. It’s (Time 0:12:39)
  • Agents Differ From Crawlers
    • Agentic browsers act on behalf of a personalized human intent, unlike public web crawlers.
    • The web’s declarative HTML structure makes it possible for agents to reliably interact with forms and elements. Transcript: Dan Shipper Because right now, it’s really designed for this bifurcation between bots and humans. And there’s a human experience and there’s a bot experience that you’re presumed to be crawling. And there’s bots that text and robots that text and all that kind of stuff. And this is sort of this sort of in-between thing where it’s personal and it’s driven by you, but it is not you. And like, yeah, how do you think the web should evolve for that kind of thing? Yeah. Ben Goodger So, I mean, this is a super interesting one. And I do think over time, like, there’ll have to be some like notion of, you know, maybe a non-human operator that is acting nonetheless on behalf of a human, like for a specific request, Because I see these things as quite different to example, to, for example, web crawlers, you know, web crawlers are out there traversing websites, and, you know, sort of synthesizing Across that for the benefit of many, whereas like, you know, this is, do the same thing, admittedly, much more painfully if you were to write a local shell script that would go off and Obtain the content of a website, maybe issue the direct HTTP requests to the resources that you wanted and so on. And this is much closer to that, where there is like your own personalized intent behind it. So I think just from how we think about these things conceptually, that’s how I look at it. In terms of how things evolve, I think one of the most interesting things about it is, at some level, like stuff doesn’t need to evolve because we have, you know, computer use models that Can just go off and like read the screen and click and do all that sort of thing. I think a lot of the evolution here will come from, are there ways to make that more seamless? Are there ways to make that higher performance so that we can do many things at once, just basically support scaling this up? Because I think what we really want to do is have something that can do many things on your behalf simultaneously over the course of time. And that will just require like a lot more sort of interesting sort of evolution of the platform. And I think there are probably a variety of different ways to do that. One of the wonderful things about the web is that it’s a very declarative medium. (Time 0:15:26)
  • AI Makes The Web Machine-Understandable
    • Models can understand non-semantic websites by interacting like humans do.
    • That reduces the need for a fully semantic web while still enabling machine understanding. Transcript: Ben Goodger It’s just it’s very early for us to figure out what that looks like. Darin Fisher But your original question about how the web might change is a really interesting one. I think as more and more of the user agents are perhaps driven by agents or models, you know, ought that not end up having some bearing or impact on how developers create their content. I think at some point, maybe there is an inflection point there. It’ll be interesting to see how the ecosystem evolves, right? People create content for human consumption. In the past, we’ve had moments when we were pushing heavily semantic web make a web that’s more understandable look at all the benefits that come from that screen readers will work better Websites will be more machine understandable what’s happened in this now with these ai models is they’re able to make sense of the websites that aren’t very ordinarily machine understandable But because these models are interacting with it in the way humans do, they’re able to glean the information just as humans do. And that’s kind of a big unlock for the computer to help you because it can understand these websites, right? But as that unlocks more and more computer models, based models, driving these systems, these websites, maybe, who knows, maybe the websites start changing as well. (Time 0:20:17)
  • Explore Agent-Native Interfaces
    • Experiment with multiple interaction protocols (apps, agent APIs) to find efficient agent-native methods.
    • Use the ChatGPT app ecosystem to encourage developers to build toward agent-driven workflows. Transcript: Ben Goodger We’re thinking through a whole host of different technologies to help us drive web browsing. I think as well, beyond the Atlas team, just to think broadly about what ChatGPT is doing. We’ve also launched this app ecosystem around the product, and I think that’s a very direct way in which we’re encouraging developers to build for a more dynamically composed world. But that’s, you know, that’s, of course, in the browser, but it’s, you know, maybe not like part of Atlas in particular. So I think some of these things are, we’re going to try a few things and see how it works out. Darin Fisher You know a lot of the technology that’s powering the chat gpt atlas agent now you know has its roots in the original operator tech preview that chats that open ai put out if you rewind the Clock back to then and compare the performance then to now you start to see you know sort of the rate of improvement there’s been like leaps and bounds improvements for the quality and Performance. And we’re kind of on that curve of figuring out how to optimize and make these things work a lot better. (Time 0:22:58)
  • Build Browsers Like Operating Systems
    • Treat a browser like an embedded operating system when designing features and runtime.
    • Prioritize shipping expected functionality before layering new AI experiences. Transcript: Darin Fisher If it’s keeping you busy, what are the sorts of things that are keeping you busy? Well, I mean, if you think about it, browsers have definitely evolved over the years, right? Know, rewind back to Netscape, and then think about Firefox, then think about Chrome, and think about when Chrome first launched, and then think about all the features that have been Added since. And, you know, not everybody uses all of those features, but some people use them, and we hear from those people. And Atlas has a significant subset of those features from the get-go because we knew they were important. And building on top of Chromium meant that some of them we were able to expose, but many things we had to reimagine, rebuild, figure out how to build in a new way. And some things we have not yet done. So we’re in the, for example, one of the things we heard about early on when we launched Atlas was, where’s my tab groups? And that’s a feature that Chrome added a few years back, but it certainly wasn’t there in the initial version of Chrome. And I know that when we first launched into Chrome, not that many people were excited about it or used it. It was sort of a small feature until eventually it’s become something that maybe a good number of people actually do care about. And we hear about those. We hear from those people because they want to carry their workflows over, you know. So one way to think about a browser is that it’s kind of like an embedded operating system. Ben Goodger And so in that sense, you might think of a browser as an app, but I think that’s maybe not the right way to look at it. A browser is closer in complexity to an operating system. It has an app runtime. It has a window manager. It has various notification surfaces and launchers and other stuff. And so there’s just a lot of complexity in building all of that stuff out. Now, you can short circuit a bunch of that. I think Darren says maybe it’s a solved problem. I think for a lot of browsers it is, including Atlas, part of it is solved because of Chromium, like the fact that Chromium is open source. It presents this just amazing, incredible baseline upon which to build. And you could stand up a browser very quickly that looks more or less like Chrome. I think our product ambition ran a bit deeper than that. I think we wanted to differentiate a bit more in our product UX. And so that caused us to take a different path, which we’ve written about. But that does mean that there’s a bit more of this legwork for us to go and make sure all of this functionality that people expect works in the way that they expect. But we think that at the end of the day, that will give us a lot more ability to sort of shape the product in new and interesting ways. (Time 0:33:30)
  • AI Enables Fast Prototyping
    • Codex accelerated prototyping and decision speed, letting the team try ideas quickly.
    • Rapid prototyping shifts focus to deciding which experiments are worth investing in. Transcript: Dan Shipper What percentage of your code is written by AI? Darin Fisher Oh, man, I don’t even have stats on that, but I know everybody’s leveraging Codex and ChatGPT heavily as part of this project. Dan Shipper I would say just like finger in the wind, if you had to guess. Ben Goodger Majority of it, I would say. I can’t pick the precise amount. It wouldn’t surprise me if it was north of 75%, just that like most people’s PRs start with Codex. Maybe there’s some like dialing in that you do through the process, but that just means that in terms of raw volume, Codex is probably authored well over, more than half, safely more Than half of the NetNew code that we have at this point. Dan Shipper You guys have been building browsers for many, many years. You started at Netscape, you worked together at Chrome or on Chrome. How does it compare being able to build a browser with, you know, codex at your side in terms of team size, velocity, all that kind of stuff? Like what is, give me a sense for what’s different or maybe it’s, maybe it’s very similar, but yeah. How does it, how does it compare? Ben Goodger Yeah. I was going to say we have a very small team, although we continue to grow to take on a bunch more possibilities. I think one of the things that has excited me about this world is it’s not just the pace of development, because I think to get a feature to work right, it’s always going to take a few iterations. It’s how quickly you can decide that something is worth pursuing. And so there’ll be an idea that I’ll have in my head even as like a team manager, where I want to see if the juice is worth the squeeze, as it were. And I will just run off and do that in codex. And I’ll have a build and I’ll see if I like the thing or not. And if I do like it, then it makes sense to go and invest in that area. And sometimes we spend a long time, like in the pre-codex world, sort of wondering about if you should do this or that, because it takes so long even to prototype. (Time 0:38:44)
  • Use AI To Manage Legacy Code
    • Use Codex to inspect and query large legacy codebases to surface context and patterns.
    • Leverage AI to generate unit tests and repetitive refactors to improve velocity and reliability. Transcript: Dan Shipper Guys have been building browsers for many, many years. You started at Netscape, you worked together at Chrome or on Chrome. How does it compare being able to build a browser with, you know, codex at your side in terms of team size, velocity, all that kind of stuff? Like what is, give me a sense for what’s different or maybe it’s, maybe it’s very similar, but yeah. How does it, how does it compare? Ben Goodger Yeah. I was going to say we have a very small team, although we continue to grow to take on a bunch more possibilities. I think one of the things that has excited me about this world is it’s not just the pace of development, because I think to get a feature to work right, it’s always going to take a few iterations. It’s how quickly you can decide that something is worth pursuing. And so there’ll be an idea that I’ll have in my head even as like a team manager, where I want to see if the juice is worth the squeeze, as it were. And I will just run off and do that in codex. And I’ll have a build and I’ll see if I like the thing or not. And if I do like it, then it makes sense to go and invest in that area. And sometimes we spend a long time, like in the pre-codex world, sort of wondering about if you should do this or that, because it takes so long even to prototype. Whereas Codex just makes prototyping a matter of minutes or hours for a lot of things. Darin Fisher And, you know, for as long as we’ve spent in the Chromium code base across our careers, man, that thing’s complicated and it’s grown. And so being able to ask Codex questions about Chromium is just invaluable. And, you know, any kind of very large legacy code base is going to have so much complexity and layers to it. And so, you know, the ability to ask these agents questions about it is just unbelievably useful. But same thing goes for, you know, figuring out how to build certain kinds of UI effects, constantly probing ChatGPT for what’s the right way to set this thing up so I’ll get a good animation Or something like that, just trying to learn some new strategies with core animation or something like this. So we have, like Ben said, a lot of our code is able to be created by Codex because there’s a lot of straightforward aspects to what we’re doing. But there’s also very delicate aspects that we’re doing. We have to get in there and really study it. But these tools can be tremendous companions as we’re trying to figure out, well, exactly what’s the right strategy here to kind of explore the solution space. (Time 0:39:27)
  • Quick Tab Stability Prototype
    • Ben prototyped a tab-scroll stability fix in about an hour using Codex.
    • The fix preserves tab strip scroll position when briefly checking other tabs. Transcript: Ben Goodger Us? Oh, yeah. Let me tell you about something I’ve been working on. Um, um, yeah, just in the process of, uh, so I’m like a heavy tab user and I nerd out on, on like the little details of, of how tabs work. So like the Chrome tab strip, like a lot of the way it behaves around like where tabs get inserted, what gets selected after you close them, how like the tab strip like reflows, animates When you move your mouse out of the way. Like I worked on that like years and years ago, like, you know, it’s almost 20 years ago at this point. And although like I have not have had less of a direct engineering role in Atlas myself, I do like to poke at different things. And so one of the things I’ve been playing with, you know, as Darren and the team work on tab groups, I have been exploring ways to just help make sure that the tab layout and scroll position Remains stable as you switch back and forth between tasks. And you might be deeply buried down on a task. You might have like lots of tabs, lots of tab groups open. You might have scrolled your sidebar of tabs down to a certain position. And then I have this moment where I want to go back and check my Gmail and I like get a like a tracking link or something and I open it up. And all of a sudden my tab strip is flung back to the top, like it gets scrolled back to the top and so this is this is what happens today in atlas and so i was able to go off and prototype a solution To that in codex in about an hour uh where i’m actually able to go and check on something without messing with the scroll position and it’s just like a transient uh a transient world where I can go and like look on something quickly so that’s the kind of thing where like if you’re interested in just making the app better, you can go off and just do a really quick exploration And determine that something makes sense. (Time 0:42:20)
  • Human Judgment Still Matters
    • Engineers retain craft because agents need human judgment for subtle context.
    • AI handles mundane patterns but experienced developers guide decisions and validation. Transcript: Dan Shipper Like I know a lot of professional programmers, even people that work at every, even people who are super psyched about AI who are also like, it also is kind of a bummer that, you know, a Lot of code isn’t being written by hand anymore. And there’s a certain craft to it that is maybe, you you just sort of like writing code how do you guys feel about it i like writing code i but i think i would i i like the sort of crafting aspect Darin Fisher There’s some something they’re almost like therapeutic about it you know just sort of it’s like art or something you know, but I still feel like there’s a lot of elements of that. What I, the way I really view this is it’s like, it’s a tool that will accelerate like the mundane parts of the work. Example, I tediously did a refactoring across the code base that was a little bit tedious because each time each part was different and I didn’t really quite know how to prompt it through All of that. But then once I had done it, I needed to do another one. I was like, Codex, just do that for me. Do the other one. And it was a similar scale and it knocked it out within an hour, right? And it was because it could follow my pattern for all the times when I worked through all the quirks, it could just follow those quirks, those patterns. I thought that was amazing. And then, like I said, if I’m crafting some animation or something like this, Codex is going to be really useful to give me ideas. But I got to get in there and try it and see. And sometimes that’s just how I work. But I find that it still is accelerating me quite a bit. And I still get that satisfaction of getting in there and crafting. (Time 0:44:28)
  • Always Generate Tests With AI
    • Prompt Codex to generate unit tests alongside feature code to reduce overhead.
    • More automated tests increase confidence and catch edge cases the developer might miss. Transcript: Darin Fisher But thanks to Codex, actually, we have a lot more unit tests because the overhead of creating a unit test is greatly reduced when you can just prompt for what you want to have tested. And even the model is able to go and consider cases I didn’t prompt for. Because really, I’m saying, can you unit test this API for me? I’ve been really impressed with this because that’s a mundane task, creating unit tests for crafting the API. It’s an interesting task. I’ll work on that. And then once I have it, hey, Codex, can you create a bunch of tests for me? It’s been a fabulous friend in that regard. And I think we’ve seen a lot of benefit from that. And tests are, of course, super valuable. (Time 0:47:55)
  • Agent Laziness Hits Complex Apps
    • Dan asked Atlas to copy-edit a Google Doc but hit the agent’s “laziness” limit on long, dextrous tasks.
    • The team acknowledges Google Docs is complex and is improving persistence and scrolling behavior. Transcript: Ben Goodger And is that something that you could mix? Yeah. To do that? Is that what? Yeah. There’s been a known issue with our agent. We call it laziness, where sometimes you’ll see it say things like, oh, this task is too time consuming. I, you know, basically I give up and it’s not just Google Docs, but it will, you know, for a variety of sites where the task might take like very many steps or an extremely long time to run. And especially if it’s like having to like scroll multiple times to get through like you know if you’re imagine you could be tens hundreds of pages even it may give up under those conditions That’s something that the team has been working on as improvements to that but you’re also right that Google Docs is a it’s a fairly complex web app it’s not something that it’s a bit different To a lot of web content. I talked before about declarative web where there’s just a, you know, a tag that you can read through and see everything. Whereas Google Docs is much more like a traditional app. It uses a canvas. It just renders text directly when you scroll. It is the one drawing, not not the web runtime. And that makes it a bit more challenging to get all of the context out. And so, yeah, I think the agent is maybe the right way to do complex things there, but the sort of laziness fixes should eventually help with that kind of thing. (Time 0:50:10)