Dave Didn’t Write This
When the production system fails at 3am, someone gets called in to fix it. That has always been true. What has changed is whether they understand what they are looking at.
Picture Dave. The production system has gone down at three in the morning and Dave is the engineer who gets the call. He drags himself to a laptop, logs in, and starts working the problem. It is miserable, but it is manageable, because Dave wrote this code. He knows it. He remembers the decisions he made, the shortcuts he took, the part of the system that always felt fragile. He has a mental map of the thing he is debugging because he built it line by line. Within an hour or two he has found the fault and the system is back up. This is the 3am call as it has existed for the entire history of software.
Now picture Dave today. Same engineer, same 3am call, same dead production system. Except this time Dave did not write the code. An AI wrote it, from his instructions, faster than he could have ever typed it. Dave reviewed some of it. Most of it he approved in batches because there was too much to read line by line and the deadline was real. Now he is staring at a fault in a system he directed but does not actually know. There is no mental map. There is just a lot of code that looks reasonable, does mostly what it should, and has failed somewhere in a way he cannot immediately locate because he was never inside it the way he used to be.
That difference; the gap between code you wrote and code you merely commissioned, is one of the biggest risks I see in the way we are building software now. And to understand how we got here, it helps to look at how the relationship between specifications and code has changed over the last twenty-five years.
I studied software engineering and graduated in 2002, right as agile was becoming the hot new thing. The world I trained in was the world before it. Serious engineering work began with a strong specification. You wrote the spec, and from the spec you derived your test cases and your code. The spec was the contract. It told you exactly what to build, and the discipline of writing it forced you to think through the problem before you wrote a single line. It was slow. It was also clear.
Agile changed that, and mostly for the better. Specs got lighter. Documentation got leaner. Work moved in fast iterations rather than long upfront design phases, and software got built faster as a result. Here is the thing that made it work: the developers were human. When a lightweight user story did not specify exactly what should happen in some edge case, a competent engineer filled the gap with judgment. They understood the intent behind the request. They knew what the product was trying to do and they made sensible decisions about the parts nobody had spelled out. Agile worked because it ran on human understanding and trust. The lightness of the spec was not a problem because the developer carried the missing context in their head.
Now the developer is an AI, and the missing context is exactly what it does not have.
This is the part people get wrong about AI coding failures. When AI produces buggy or wrong code, it is usually not because the model cannot code — modern models are genuinely excellent at writing code. It is because the instructions were not clear enough and the model had to guess at the intent. A human engineer guessing at intent draws on years of context and usually guesses well. An AI guessing at intent draws on the words in the prompt and the statistical patterns in its training, and when those run out, it fills the gap with something plausible rather than something correct. The lighter the spec, the more guessing it has to do, and the more surface area there is for a guess to be wrong in a way nobody notices until 3am.
Give the same AI a genuinely strong specification, a detailed, well-thought-through PRD that actually says what the software is supposed to do and why. And the output is frequently outstanding. Clean code that passes the tests, handles the edge cases, does what was asked. The model is not the bottleneck. The clarity of the instruction is. We spent two decades training ourselves out of writing strong specs because human developers did not need them. We are now handing the work to a developer that does.
But the spec problem, as real as it is, is the solvable half. You can teach teams to write better PRDs. You can build the discipline back up. The harder problem is the one Dave is sitting with at 3am.
The volume of AI-generated code going into production is growing faster than anyone’s ability to review it. That is not a hypothetical, it is the current reality on a lot of teams. The code is produced so quickly that meaningful line-by-line human review has quietly stopped happening, because the economics of it no longer work. You cannot have a senior engineer carefully read every line when the AI is generating ten times more code than the team could ever review at that depth. So the review gets shallower. The approval gets faster. And the human understanding of the system, the thing that used to live in the heads of the people who wrote it, gradually thins out until it is barely there at all.
Now, anyone who has worked on a large legacy product in a real enterprise is already objecting, and they are right to. The honest truth is that the pristine version of Dave never fully existed. On any system of real size, Dave left three years ago. The code was written by twenty different people, half of whom are no longer at the company. The documentation never stayed as good as anyone wanted. And nobody ever really understood the whole thing perfectly anyway, there was always some module that everyone was slightly afraid to touch. The 3am call into a system nobody fully comprehends is not new. It predates AI by decades. I am not pretending otherwise.
But here is the point. AI did not create this problem. It amplifies it, and that is exactly the point. The comprehension gap used to accumulate slowly, over years, as people left and code aged and context understanding faded one resignation at a time. It was a chronic condition that teams managed. What AI does is compress that timeline dramatically. The gap that used to take a decade to open up can now appear in a single sprint, because the code is being produced faster than understanding can possibly keep pace with it. We have taken a known, manageable, slow-moving problem and put it on fast-forward. The disease is the same. The speed of progression is entirely different.
For as long as everything works, this is fine. Better than fine, it is fast and cheap and productive. The problem only shows up when something breaks, and something always eventually breaks. And when it does, the people called in to fix it are increasingly debugging code they did not write, did not deeply review, and do not carry a mental model of. They are reverse-engineering their own production system under pressure, in the dark, at the worst possible hour. The speed that felt like such an advantage during development becomes a liability at exactly the moment when understanding matters most.
I do not think this means we should stop using AI to write code. That ship has sailed and the productivity is real. But I think we are systematically underweighting the cost of the comprehension gap, because it is invisible right up until the moment it is catastrophic. The teams that will weather this well are the ones that treat understanding as something to be deliberately maintained, not assumed, strong specs going in, meaningful review on the parts that matter, and someone who actually knows how the critical systems work, even when a machine wrote them.
Because Dave is still going to get the call at 3am. The only question is whether he recognises what he is looking at when he does.
