Skip to content

Podcast

Harness Engineering 101

The AI Daily Brief: Artificial Intelligence News and Analysis

Source ↗ ← All highlights
  • Harness Engineering Is The Next Layer Beyond Prompts
    • Harness engineering extends prompt and context engineering by focusing on everything around the model that lets it actually do work.
    • Nathaniel Whittemore contrasts prompts, context access, memory, and state to show why model performance depends on the surrounding system, not just wording. Transcript: Nathaniel Whittemore And you might notice that there is kind of a lineage of engineerings that we focus on that have changed over the years in AI. In 2023 and 2024, we talked a lot about prompt engineering, the art and the science of finding the right ways to prompt the model to get the results that you wanted. There was so much in prompt engineering that people spent so much time on. Think about the things that everyone used to recommend, like getting the model to adopt a persona, or later on, the whole idea of JSON engineering, where people hyper-structured their Prompts in the way that an engineer might. Now, last year in 2025, we started to talk a lot more about context engineering. The idea of context engineering was that it turned out that what mattered for AI performance was not just the way you spoke to the model, but what set of information or context that model Had access to. Take the example of asking ChatGPT to help you create a marketing campaign. One part of getting good results, sure, might be what you prompt it for and how you ask it, but obviously it’s kind of intuitive that if ChatGPT had access to information about the performance Of all your past marketing campaigns, it might be able to be more informed in how it helped you. So context engineering was all about the way that we brought together different context and gave AI access to it. Now, context engineering actually kind of has had divergent meetings for different people. For engineers and developers, context engineering has often been about designing the systems that surround AI and agents in order to better interact with and use context, dealing With problems like persistence in memory and state. And in a way, this is kind of a part of what we’ll talk about with harness engineering. For laypeople, for non-technical users, context engineering has been much more about what’s the best way to give AI access to the information it needs to help me do its job. Now, it’s important to note that while prompt engineering might have decreased a little bit in its importance scale, context engineering is still very much alive and important. In fact, I did that entire episode about a week ago about how to build a personal context portfolio so that you could transport your personal context from LLM to LLM or agent to agent without Having to repeat yourself every time. But the term du jour right now is harness engineering, which is effectively about everything you put around a model, the systems, the tooling, the access that help it do what it’s meant To do. (Time 0:01:31)
  • Why AI Products Are Converging On The Same Shape
    • Products like Cursor 3 and Anthropic Managed Agents embody harness engineering by organizing agents, tools, and execution into one workspace.
    • Nathaniel Whittemore says features like parallel agents, local-cloud handoff, and production infrastructure are the harness turned into product UX. Transcript: Nathaniel Whittemore At the beginning of April, Cursor launched its newest version, Cursor 3. In their announcement post, they wrote, software development is changing and so is Cursor. In the last year, we moved from manually editing files to working with agents that write most of our code. How we create software will continue to evolve as we enter the third era of software development, where fleets of agents work autonomously to ship improvements. We’re building towards this future, but there is a lot of work left to make it happen. Engineers are still micromanaging individual agents, trying to keep track of different conversations, and jumping between multiple terminals, tools, and windows. We’re introducing Cursor 3, a unified workspace for building software with agents. The new cursor interface brings clarity to the work agents produce, pulling you up to a higher level of abstraction with the ability to dig deeper when you want. It’s faster, cleaner, and more powerful with a multi-repo layout, seamless handoff between local and cloud agents, and the option to switch back to the cursor IDE at any time. So all of the features that then go on to announce, having all of your agents in one place, the ability to run many agents in parallel, new UX for handoff between local and cloud, all of This is the instantiation of harness engineering into a product. Even more recently, we got Claude Managed Agents. In their announcement post, they said explicitly it pairs an agent harness tuned for performance with production infrastructure. And in the accompanying blog post, they basically say this is kind of all about harnesses. The post was called Scaling Managed Agents, Decoupling the Brain from the Hands. Now, of course, in this metaphor, the brain is the model and the hands are the harness. Harnesses, they write, encode assumptions that go stale as models improved. Managed agents, then, is built around interfaces that stay stable as harnesses change. (Time 0:03:52)
  • The Big Model Versus Big Harness Debate
    • The core debate in agent building is whether better models replace scaffolding or whether better harnesses unlock more value from blank-slate models.
    • Nathaniel Whittemore contrasts Claude Code and OpenAI arguing for minimal wrappers with Jerry Liu arguing users still need heavy workflow and context engineering. Transcript: Nathaniel Whittemore At the beginning of March, Latent Space dropped a post called Is Harness Engineering Real? And to provide another analogy, their team references back to when they worked in finance. It doesn’t say for sure, but I assume this is Sean slash Swick’s writing because this was part of his experience set. But whoever it was wrote, a common debate in my finance days was about the value of the human versus the value of the seat. If a trader made 3 million in profits, how much of it was because of her skills and how much was because of the position, institution, and brand she is in, and any generally competent human Could have made the same results. They continue, the same debate is currently raging in harness engineering, the system subset of agent engineering and the main job of agent labs. Agent labs, by the way, are how the latent space team refers to everyone like cursor, cognition, etc. The central tension, they continue, is between big model and big harness. An AI framework founder you all know once confided in me at an open AI event, I’m not sure these guys even want me to exist. To define harness, they write, in every engineering discipline, a harness is the same thing, the layer that connects, protects, and orchestrates components without doing the work Itself. They continue, talking with the big model guys, you really see it. Every podcast with Boris Cherney and Kat Wu, the creators of Claude Code, emphasize how minimal the harness of Claude Code is, meaning their job is mostly letting the model express Its full power in the way that only the model maker knows best. In one interview, Boris said, I would like to say there’s nothing that’s secret in the sauce. Generally, our approach is, all the secret sauce, it’s all in the model. And this is the thinnest possible wrapper over the model, They continue, We literally could not build anything more minimal. Kat added, it is very much the simplest thing, I think, by design. Nome Brown from OpenAI seems to agree. They quote him as saying, Before the reasoning models emerged, there was like all of this work that went into engineering agentic systems that made a lot of calls to GPT-4 or these non-reasoning Models to get reasoning behavior. And then it turns out we just created reasoning models and you don’t need this complex behavior. In fact, in many ways, it makes it worse. You just give the reasoning model the same question without any sort of scaffolding and it just does it. And so people are building scaffolding on top of the reasoning models right now, but I think in many ways, these scaffolds will just be replaced by the reasoning models and models in General becoming more capable. On the other side, says Latent Space, are the big harness guys. Jerry Liu from Llama Index wrote a post on this on X that he titled The Model Harnesses Everything. He added a picture that sums up his point as saying, agent reasoning is exponentially improving, but models are blank slates. The biggest barrier to AI value is the user’s own ability to context and workflow engineer the models. The more complex the business process, the more complex the prompt that users need to define. Now where Latent Space comes out is that while they might have some bias towards the big model thesis, actually referencing the bitter lesson that we talked about in episodes a couple Of weeks ago, they also acknowledge that harness engineering has real value. (Time 0:05:38)
  • Coding Agent Failures Are Usually Configuration Problems
    • Kyle argues coding agents fail less from weak models than from weak configuration, so harness design matters now, not just after future model upgrades.
    • He cites ignored instructions, dangerous commands, and looping failures, then reframes skills, MCPs, memory, and agents.md as one configuration surface. Transcript: Nathaniel Whittemore And for part of our guide, we’re going to use a post from humanlayer.dev from the middle of March called Skill Issue, Harness Engineering for Coding Agents. Author Kyle writes, We’ve spent the last year watching coding agents fail in every conceivable way, ignoring instructions, executing dangerous commands unprompted, and going In circles on the simplest of tasks. Every time, the instinct was the same. We just need better models, GPT-6 will fix it, we just need better instruction following. It’ll work when niche library I’m using is in the training data. But over the course of dozens of projects and hundreds of agent sessions, we kept arriving at the same conclusion. It’s not a model problem, it’s a configuration problem. Yes, models will get smarter, and yes, some existing failure modes will disappear. And then because they are smarter, we will give them new problems which are bigger and harder, and they will continue to fail in unexpected ways. Unexpected failure modes are a fundamental problem for non-deterministic systems. So instead of praying for GPT-64 codex ultra-high extended to save us all, what if we focused instead on answering the question, how do we get the most out of today’s models? And the next point that Kyle makes is the one that I was saying before, which is that most of us who have been dabbling in these systems, be it OpenClaw or CloudCode or Codex, have been doing Harness engineering whether we realize it or not. He continues, there are lots of ways to get better performance out of your coding agent. If you use coding agents for moderately hard tasks, you’ve probably configured your coding agent a bit. Have you used skills, MCP servers, subagents, memory, agents.md files? A coding agent equals AI models plus a harness. These are all technically separate concepts, but they are all part of the coding agent’s configuration surface. Basically, what does the model use to interact with its environment? Harness engineering, they write, describes the practice of leveraging these configuration points to customize and improve your coding agent’s output quality and reliability. They continue by arguing that harness engineering is the subset of context engineering which primarily involves leveraging harness configuration points to carefully manage the Context window of coding agents. It answers how do we give our coding agents new capabilities? How do we teach it things about our code base that aren’t in the training data? How do we increase task success rates beyond magic prompts? (Time 0:08:31)
  • Harnesses Backfill What Models Cannot Do Natively
    • A harness adds missing capabilities by mapping desired behaviors to concrete components like bash, sandboxes, memory, web search, and long-horizon loops.
    • Nathaniel Whittemore uses LangChain and OpenAI examples to show progressive disclosure gives agents more context without overfilling the context window. Transcript: Nathaniel Whittemore To some extent, harnesses work backwards from what models can’t do natively to create some component to solve for that. In another post from Viv from Langchain, called The Anatomy of an Agent Harness, Viv added a chart that showed the desired agent behavior versus what the agent adds. For example, the simple one that’s a part of every Claude code session. If the desired agent behavior is to write and execute code, the harness adds bash and code execution. If the desired agent behavior is safe execution and default tooling, the harness adds sandboxed environments and tooling. If the desired agent behavior is remembering and accessing new knowledge, the harness is going to need to provide memory files, web search, and MCPs. And importantly, when you’ve heard about all of these techniques, like Carpathie’s auto research or the Ralph Wiggum loops, those are harness additions to get to the desired agent Behavior of completing long horizon work. They also point out that this is something that the big labs are talking about quite a bit now, too. Back in February, OpenAI dropped a post called Harness Engineering, Leveraging Codex in an Agent First World. The place that they start from in this post is the goal of building and shipping an internal beta of a software product with zero lines of manually written code. That has been the context through which they have had to figure out what needed to be part of the harness that they were designing. One of the big experiments that they found was effectively that in this new approach to engineering, they had to uncover new ways of giving the agent progressively more context. This is this idea which you might have heard me talk about before called progressive disclosure, which is a key part of the way that agent skills have been designed, where skills that Provide context effectively unfold with the agent being able to access the minimum amount of information to know if it needs to go deeper into that skill without having to crowd out Its context window with all sorts of unnecessary information. The key part of the story, though, is in some of the last lines in the post. They conclude, Our most difficult challenges now center on designing environments, feedback loops, and control systems that help agents accomplish our goal, building and maintaining Complex, reliable software at scale. (Time 0:10:42)
  • Harnesses Work As Information Execution And Feedback Layers
    • Several teams now describe harnesses as a stack of information, execution, and feedback layers that govern what agents see, do, and learn from.
    • Nathaniel Whittemore highlights memory and tools, orchestration and guardrails, plus evaluation and observability as the three-layer architecture. Transcript: Nathaniel Whittemore Now, as we get more discourse about harness engineering, we also get more maps and guides for what it actually means. In an Aetna Labs post, for example, they describe the key components of harness as a three-layer architecture. The information layer, which determines what information an agent can see and what capability it can invoke, i.e. Memory and context management and tools and skills. An execution layer that determines how work is decomposed, how agents collaborate, and how to recover in the case of failure, which is basically orchestration and coordination and Infrastructure and guardrails. And a feedback layer, which determines how the system can improve over time, whether the results of the execution are verified, and whether each failure is recorded and transformed. So that’s evaluation and verification and tracing and observability. And there is increasing evidence out there of the power of harnesses. Blitzy, who has over the last year been a frequent sponsor and collaborator on the show, recently released a 66.5% performance on Sweebench Pro. That is much higher than, for example, GPT 5.4’s 57.7. Now, effectively, Blitzy’s whole thesis could be reframed as being that the harness layer, the agent scaffolding, the orchestration, and the context infrastructure wrapped around The foundation models, can unlock bigger performance gains than the model themselves. One of the key things that they found when auditing their performance versus GPT 5.4 is that in many cases, GPT 5.4’s failures weren’t catastrophic. It got close on every problem, but missed intricate details in corner cases. When Blitzy succeeded on those same tasks, it succeeded because its knowledge graph gave its agents deep code-based context that a raw model doing a single pass couldn’t match. Langchain has also recently been writing about how they’ve been improving agent performance with harness engineering as well. Nicolas Charrier thinks that there is some amount of consensus around the power of the harness that is starting to happen. (Time 0:16:27)
  • Blitzy Beat GPT 5.4 With Better Harnessing
    • Blitzy outperformed GPT 5.4 on SWE-bench Pro by wrapping models in stronger orchestration and context infrastructure.
    • Nathaniel Whittemore says GPT 5.4 often got close but missed corner cases, while Blitzy’s knowledge graph supplied deeper codebase context for those same tasks. Transcript: Nathaniel Whittemore Blitzy, who has over the last year been a frequent sponsor and collaborator on the show, recently released a 66.5% performance on Sweebench Pro. That is much higher than, for example, GPT 5.4’s 57.7. Now, effectively, Blitzy’s whole thesis could be reframed as being that the harness layer, the agent scaffolding, the orchestration, and the context infrastructure wrapped around The foundation models, can unlock bigger performance gains than the model themselves. One of the key things that they found when auditing their performance versus GPT 5.4 is that in many cases, GPT 5.4’s failures weren’t catastrophic. It got close on every problem, but missed intricate details in corner cases. When Blitzy succeeded on those same tasks, it succeeded because its knowledge graph gave its agents deep code-based context that a raw model doing a single pass couldn’t match. Langchain has also recently been writing about how they’ve been improving agent performance with harness engineering as well. Nicolas Charrier thinks that there is some amount of consensus around the power of the harness that is starting to happen. (Time 0:17:14)
  • The General Harness Explains Product Convergence
    • Nathaniel Whittemore argues the general looping harness explains why Linear, Notion, OpenAI, and Anthropic all resemble each other more over time.
    • The shared pattern is simple: a goal, a model, tools, context management, and repeated tool-calling until a business outcome appears. Transcript: Nathaniel Whittemore The AI entrepreneur recently wrote a post on X called The Great Convergence, and it plums through some similar themes as my episode, Every AI Product is Becoming Every Other AI Product, But puts it in the context of the harness. Nicolas writes, over the last year, a strange thing has happened in tech. Very different companies have started moving towards the same product shape, and it feels like everyone is building the same thing. Linear announced last week that they’re building coding agents. OpenAI is deprecating Sora and focusing entirely on Codex. Anthropic is obviously all in on Cloud Code and Cowork. Notion is building agents for work. Sora, Google, Microsoft, Meta, Lovable, Retool, and many others. What changed, he writes, is not just that models got better, although that’s a major part of it working. He argues that the important shift is the invention of what he calls the general harness. The simple harness architecture diagram that he describes, a user input hits context engineering, which moves to the model, which calls in tools, which access context engineering In a loop, until the task result comes out on the other side. Nicholas writes, Clawed code was a massive breakthrough. Although initially invented for coding use cases, it turns out that a smart looping agent generalizes incredibly well towards any computer-based task if you give it the right tools. So this new technique emerges and turns out to be a general problem-solving machine. It also scales on a very unique dimension. It can keep running for a long time. It takes the shape of a model harness, a goal, and a set of tools. It runs in a loop calling tools until it stops and produces a result. He points out that so many of the new agents that we’re seeing all come back to just a harness-looping agent architecture with the right tools and context management. Ultimately, he predicts that by the end of 2026, many software companies will look like they are selling the same thing. He writes that’s not because the industry lost imagination, but because the architecture and economics are pushing everyone towards the same destination, self-improving software Systems that can take a goal, use tools, and produce business outcomes. The harness, he writes, explains the convergence. The self-improvement explains the acceleration. Once agents can be monitored, evaluated, orchestrated, and improved by changing their own code and context, the companies that own more of the loop will improve faster and their progress Will compound. The winners, he says, will not just have better models. They will have distribution, trusted workflow positioning, proprietary context, and the shortest path from observation to improvement. (Time 0:18:17)
  • Managed Agents Treat Harnesses As Disposable Layers
    • Anthropic’s managed agents suggest the lasting advantage is not one perfect harness but infrastructure that can swap harnesses as models improve.
    • Nathaniel Whittemore says context resets solved Sonnet 4.5’s context anxiety but became dead weight on Opus 4.5, proving assumptions go stale. Transcript: Nathaniel Whittemore Which brings us back to Anthropix’s managed agents. Because what they show is that we’re now at the point where we’re not only recognizing harness engineering, we’re starting to build towards inevitable changes in what harness engineering Is. Remember, the subtitle from their accompanying blog post is, Harnesses encode assumptions that go stale as models improve. Managed agents is built around interfaces that stay stable as harnesses change. Here’s how they set it up. A running topic on the engineering blog is how to build effective agents and design harnesses for long-running work. A common thread across this work is that harnesses encode assumptions about what Claude can’t do on its own. This then goes back to that idea from the Langchain blog that the harness is about adding things that address a certain desired agent behavior that aren’t in the model natively. Continuing Anthropic writes, however, those assumptions need to be frequently questioned because they can go stale as models improve. As just one example, in prior work we found that Claude Sonnet 4.5 would wrap up tasks prematurely as it sensed its context limit approaching, a behavior sometimes called context anxiety. We addressed this by adding context resets to the harness, but when we used the same harness on Claude Opus 4.5, we found that the behavior was gone. The resets had become dead weight. We expect harnesses to continue evolving, so we built managed agents, a hosted service in the Claude platform that runs Long Horizon agents on your behalf, through a small set of interfaces Meant to outlast any particular implementation, including the ones we run today. Basically, they say, building managed agents meant solving an old problem in computing of how to design a system for programs as yet unthought of. Another way to put it is that Anthropic is building a meta-harness, a system that’s deliberately unopinionated about what any specific harness should look like because they expect Harnesses to keep changing as models improve. Going back to that brains versus hands metaphor, effectively, Anthropic separated the agent loop, the brain, from the execution environment, the hands or sandbox, and even separated It from the event log, which is the session. Each can fail or be replaced independently. And I think that this has big implications for the debate that we started on of big harness versus big model in the way that Latent Space framed it. It sort of a little bit obliterates that debate. Anthropic is saying effectively, yes, harness engineering matters. In fact, it matters so much that we’re building infrastructure to make harnesses disposable. The whole point is that any given harness is temporary. The discipline is permanent, the specific implementation is not. (Time 0:20:31)
  • Enterprises Need To Design The Agent Environment
    • Enterprise AI success depends less on choosing the best model than on designing the environment where agents and people can work effectively together.
    • Nathaniel Whittemore says user-built outer harnesses like agents.md and repo structure shape outcomes, then extends that logic to organizational design itself. Transcript: Nathaniel Whittemore But let’s talk about why any of us should care. First of all, if you use Clawed Code, Cursor, Codex, or OpenClaw, which is by far the most successful open source harness we’ve had so far, you are already doing harness engineering, Whether you call it that or not. Every time you write an agents.md file, structure your repo so the agent can navigate it, anything like that, you’re building an outer harness. Brigitte Bocular actually distinguishes between the user harness and the outer harness. Basically, there’s the inner harness, which is built by the builders of the coding agent, i.e. Anthropic or OpenAI, and the outer harness, which is built by you, the user. The outer harness is what’s going to determine whether the agent produces good work around your specific code base or your specific goals. But I would argue that that is not the only reason to care about harness engineering. If you’re an enterprise leader, the mental model matters because it reframes AI adoption from pick the best model to pick the best environment for agents to work in. It is a technical capstone on the larger truth which all enterprises are realizing, that AI success is not about dropping in a tool and hoping it works, but about designing a new system In which the capability set that AI has and that AI enables among your people can thrive. One could torture the harness engineering frame of reference and frankly extrapolate it out to the entire organizational design need. Simply put, the model and the tools are necessary but insufficient. The environment you put them in is going to determine the output quality. And lastly, if you’re just a consumer watching this, hopefully understanding harness engineering a little bit better will go some way to explain why every product seems to be turning Into every other product. When we understand that the core loop of models calling tools in a loop until they are done actually turns out to be general purpose, you understand why Linear is building coding agents And why Notion is building work agents and why they’re all heading towards the same place. The harness enables them to use the models to accomplish whatever goal they were set out to do with a process and a pattern that starts to look really familiar. Anyways, guys, hopefully this was a valuable primer in a term that you’re going to be hearing a lot more of. But for now, that’s going to do it for today’s AI Daily Brief. Appreciate you listening or watching as always. (Time 0:22:59)