Tuomas Piippo
CTO, Founding Partner
Rework
“Philosophy is about not letting the imaginary smug guy win.” — Aristotle, probably
Lately, I’ve been trying to put something about AI and software engineering into words, but there’s this imaginary guy who keeps getting in the way.
This is not the first time, and at least for me, that is how difficult thinking often goes. You try to formulate something, and there he is, pointing out the obvious objection before you’ve even finished. And annoyingly, he’s often right. You discard the idea and try again. Hopefully, you gradually discover that the thing you were attempting to say is slightly different from what you originally thought.
The guy I’m arguing with this time is a big proponent of agentic software development. So am I. I sometimes find his confidence in what agents can handle a little naive, but I’m having a surprisingly hard time showing where he’s wrong. We even agree that software engineering hasn’t been solved just because agents have become so good at implementing things. Making coding much faster just moves the bottleneck elsewhere. But why couldn’t we use agents to address that bottleneck too? And the next one?
If software engineering isn’t solved, what exactly are we missing?
The (internal) argument heats up when I suggest there must be some part of the process agents just can’t do. And that’s where I keep getting stuck. There’s obviously more to developing software than getting the next change right, however quickly we can do it. But whenever I try to put my finger on what that more is, and why current agentic best practices couldn’t already cover it, he comes up with the same answer.
“Agents can do that.”
Let’s try it. Something more than getting the next change right? Ah, it’s architecture! If agents implement one perfectly sensible change after another, who’s keeping an eye on what kind of system those changes are adding up to? Perhaps responsibilities are gradually shifting between services. Maybe this is what the agents are missing, at least long term?
The imaginary smug guy doesn’t even need to think about it.
“Agents can do that.”
Of course they can. Give an agent the repository and the architecture decisions. It can check what each service actually does against what was decided, trace how responsibilities have shifted across a series of changes, and flag the drift, every night if we want. Modern coding harnesses are already quite good at maintaining plans, running checks and recording findings. So much for that argument.
All right, what about the business requirements? One service considers an order cancelled when the cancellation request is accepted, another only when the warehouse has released the reservation. Both pass their tests. Was this difference intentional? Surely somebody needs to understand what the software is actually supposed to do.
“Agents can do that.”
Yes. They can examine the implementations, trace the requirements and decisions, and ask the right people when something remains unclear. They can even keep the question open while development continues. Fair enough.
Maybe it’s what we’ve learned about the system along the way. Last month, an investigation established that inventory figures could be fifteen minutes out of date, but concluded this was acceptable because they were only used in a daily report. Since then, a new feature has started using those same figures to tell customers whether something is in stock. The delay hasn’t changed, but the earlier conclusion may no longer hold. Surely somebody needs to keep track of that?
“Agents can do that.”
Yes, they can. They can reconstruct the investigation, reconsider the conclusions, and keep the records up to date. We can even give an agent a standing responsibility to do exactly that.
Maybe the guy was right
Let’s face it: agents are pretty impressive in what they do. They can do an astonishing amount already, and they keep getting better. The imaginary smug guy doesn’t even have to be particularly imaginative. Almost anything I propose can be turned into a perfectly reasonable assignment for an agent.
And I think that’s part of why I’ve been getting stuck. Every example has been another assignment: inspect the architecture, clarify the requirements, reconsider an earlier finding. We can assign all of them, even on a continuing basis. But the ability to perform those investigations doesn’t, by itself, establish that anyone is developing an understanding of the software through them.
Maybe I’ve been looking for a missing task when I should be asking what it would actually mean for software engineering to be solved.
Viewed from the outside, software development can look like an automaton. Requirements go in, working software comes out. And for sufficiently well-defined requirements, that automaton is becoming remarkably good. I can ask an agent to build me a tic-tac-toe game and get a perfectly functioning one. I don’t need to hire a software engineering team for that, and increasingly, the same is true of much more substantial software.
But that’s not quite how software engineering works from the inside. In a real software project, the requirements aren’t simply waiting to be implemented. They need to be understood, interpreted, negotiated and revised. From what I’ve seen, building the software is what reveals that we misunderstood the problem in the first place, sending us back to requirements and decisions we thought we’d already settled. The undertaking develops, and so does our understanding of both the problem and the software we’re building.
The other thing software engineering produces
Software engineering produces two rather different things. One is the software itself, together with its tests, specifications, documentation and other artifacts. The other is an understanding of that particular software, developed by the people working on it. They have encountered problems, investigated questions, made mistakes, revised their explanations and learned things about the system that weren’t apparent when they started.
In his 1985 paper Programming as Theory Building, Peter Naur went further: developing that understanding is the primary result of programming. He called it a theory of how a part of the world is handled by software. Having that theory means understanding not just what the code does, but why it is structured as it is, how it relates to the problem, and how it can be changed without undermining the design. The code and documentation reflect that theory, but they cannot spell out every situation in which it will matter. Having the theory also means recognizing how it applies when something new happens.
There’s an interesting connection to agents here. Part of what makes them useful is precisely their ability to deal with situations we haven’t reduced to a predetermined procedure. But that same open-endedness is also why developing a theory of the particular system matters.
An experienced engineer approaches a new change from somewhere. She knows what she has investigated, what she has reason to believe, where her understanding is uncertain, and which earlier findings might now deserve reconsideration. Her experience affects what she notices and what she considers worth investigating next.
And there’s something important about that somewhere. To know a system from somewhere is also to have things outside your view. An investigation establishes something about particular circumstances, not necessarily about everything the software might do. A transcript can record what happened without preserving what those events actually established, or where the evidence stops. A positioned understanding has those boundaries. It includes not just conclusions, but the grounds on which they rest and the things that remain unknown. Much of that can be put into words and carried forward, including to another agent. But the written account isn’t the theory itself. The connections that make it meaningful have to become part of how the next participant understands the system.
What a position is for
Positionedness is, I’d argue, one of the foundations for taking responsibility for an undertaking. Responsibility isn’t just the ability to perform whatever task happens to be assigned. It involves maintaining a relationship to the particular thing you’re responsible for, understanding how it has developed, and exercising judgment about what needs attention. An LLM may give excellent answers about a system without having a position of its own. Getting to know that system and being responsible for what happens to it requires one.
This isn’t a claim about consciousness or some private inner experience. I’m talking about a particular, historically developed point of view. It can be incomplete or mistaken. It should remain open to correction.
That viewpoint doesn’t have to belong to a human. An AI that has one is what I’ve elsewhere called subjective AI: not subjective as in biased or merely personal, but as in having a position inside the work. The question is how an AI could develop and exercise one as part of its participation in software engineering.
Think back to those delayed inventory figures. An observer that had investigated the issue initially would know its conclusion rested on the figures being used only in a daily report. When it sees a new feature using those same figures to show customers whether something is in stock, that is a reason to reopen the question. The earlier finding isn’t necessarily false, but the observer has reason to find out whether the delay is still acceptable. And it could recognize that reason before anyone asks.
There’s also a more mundane consideration. We’ve been giving the imaginary smug guy quite a lot of credit. Whenever agents can do something, we’ve conceded the point. Fair enough. But anyone who has done serious work with agents knows how impressive they can be, and how confidently they can sometimes miss something important. Being capable of investigating a problem doesn’t guarantee that an agent will recognize the need to investigate it.
And making the agents better doesn’t automatically tell us who is keeping track of what they may have misunderstood. A continuing observer would be fallible too, but its accumulated experience could give it reasons to question things that an otherwise capable agent might simply take for granted. Its understanding would also need to remain inspectable and contestable, against the actual code, versions and tests.
The case for a biographical harness
A coding harness gives an LLM the repository, the tools to change it, and ways to check its work against the actual system. It can also keep plans, findings and decisions across tasks. An agent can develop an impressive working understanding while doing the work. But each LLM invocation ends, and that understanding isn’t simply what gets written down. The next invocation may understand the system just as well, but repeatedly doing so for individual tasks isn’t yet the same as getting to know the software through an ongoing undertaking.
An engineer who knows the system recognizes when a new change calls an earlier assumption into question, or when something previously learned no longer applies. The records help investigate those questions, but their mere existence doesn’t ensure that anyone recognizes the need to ask them. As long as that remains the programmer’s job, the harness hasn’t taken on that part of engineering.
For an agent to take it on, it would need more than access to records. It would need a position built through its own investigations: what it has established and on what grounds, what it assumed, what it left open, where it turned out to be wrong. In effect, a biography with the project, as the human participants do.
And it would need to use that position without being asked, recognizing when a new change or fact gives it reason to reconsider an earlier finding.
A coding harness lets an agent work on the software. I’d call the counterpart that lets it develop a history with the software a biographical harness.
Biographical harness (noun). A software infrastructure and orchestration layer that enables an artificial participant to develop, carry forward and revise a theory of a particular software system through its own investigations, and use that theory to guide what it notices, questions and does next.
Faster coding brings another bottleneck into view: understanding the system, keeping track of what we’ve learned, and deciding what deserves attention next. When that responsibility remains with a programmer, progress depends on their attention. If an agent could develop some of that understanding, the undertaking, and not just the code, could keep developing while the programmer’s attention is elsewhere. That wouldn’t solve software engineering, but it would let more of it proceed without one person having to hold it all.
I can already hear the imaginary smug guy, and this time I hope he’s right.
“Agents can do that.”