Code Got Cheap. Understanding Didn't

We’ve made writing software incredibly cheap. A developer can describe what they want, get a reasonable implementation in minutes, make a few changes, and move on. And for the most part, the output is good. It compiles, the tests pass, and it does what the ticket asked. But I keep coming back to a question that is much harder to measure: do we actually know what we’ve built? That might sound like an odd question when everything is working, but anyone who has spent any time in a large codebase knows that writing the code is only a small part of the job. The expensive part is building a mental model of the system: knowing how a request moves through it, why a decision was made, what depends on what, and what is likely to happen when you change something. AI has dramatically changed how quickly we can add to a system. It hasn’t made the system itself any easier to reason about.

You can see this happen in small, almost harmless ways. One service handles retries with exponential backoff. Another uses a fixed delay. A third has a circuit breaker layered on top. There may be perfectly good reasons for each decision, and none of them would necessarily raise a red flag in a pull request. But six months later, an engineer is debugging an issue and discovers one of these paths for the first time. They can read the code. They can see what it does. But they don't necessarily know why it does that, whether it is intentional, or whether it is different from the other services for a reason. Now imagine the same thing happening across authentication, error handling, configuration, state management, logging, and data access. Nothing has to be catastrophically wrong for a system to become difficult to reason about. You just end up accumulating decisions that made sense individually, but are increasingly difficult to understand as a whole.

But that cost still comes due—and you end up paying it in production. Something breaks at 2 a.m. A request is failing somewhere deep in a service you've never touched before. You start tracing it and land on a block of code that looks perfectly reasonable. Then you ask the questions that matter: Why is this happening here? Why does this service behave differently? What calls this? What assumptions were made when this was introduced? Was this a deliberate design decision or simply the result of how the code happened to be generated? You end up reconstructing the history of a piece of the system while you're also trying to fix the incident. The problem isn't necessarily that the code is wrong. The problem is that you are meeting part of your own system for the first time while it is on fire.

This is also why I don't think simply adding more code review is going to solve it. Review is good at finding bugs and bad decisions when the reviewer has enough context to recognize them. But a clean diff can be surprisingly difficult to challenge. If the names make sense, the tests pass, and the implementation follows a familiar pattern, what exactly are you looking for? The reviewer can confirm that the code does what it appears to do without ever discovering why this was the right thing to do in the first place. We have traditionally relied on the person writing the code to carry that context into the review. When a model is doing more of the implementation, that assumption becomes less reliable. The result is a strange situation where the amount of code we can produce grows much faster than the amount of context any individual engineer can absorb.

Alternatively, I don't think the answer is to use intelligent systems less. Quite the opposite. We should take advantage of it. But if the bottleneck has moved, our engineering practices need to move with it. We need better ways of preserving the reasoning behind changes, seeing how new work fits into the existing system, and getting an engineer up to speed on unfamiliar parts of a codebase without making them rediscover everything from scratch. A pull request should explain why a decision was made. Tests should capture intended behavior. Design should happen before hundreds of lines of implementation exist. We can keep changes small and clean up duplicated approaches instead of continually adding new ones. None of this is particularly revolutionary. What has changed is how important it becomes when producing another thousand lines of code is no longer the expensive part.

There is a bigger question underneath all of this for engineering teams. We are entering a world where an individual engineer can produce an enormous amount of software. That is an incredible force multiplier, but software systems still have to be operated, debugged, changed, and eventually handed to someone else. The ability to generate an implementation is not the same thing as having the context to safely operate it. And there is a risk that we optimize so heavily for output that we forget where engineering judgment comes from. Some of it comes from writing the code ourselves. Some comes from debugging the thing we built. Some comes from making a bad decision, living with it, and understanding exactly why it was bad. If AI removes too much of that experience, we may end up with engineers who are exceptionally good at getting software to the user but have less practice making sense of the systems they've produced.

That leaves me with a question I don't think we've figured out yet. If AI is going to make building software faster and easier, how do we get better at everything that comes after that? How do we make sure that when something breaks in production, an engineer can quickly understand what happened instead of starting from zero? How do we preserve the reasoning behind a system as it grows? How do we give engineers the context they need without expecting them to read an entire codebase before they can safely change it? I don't think the answer is to slow down. I think the answer is to get much better at helping people see and understand the systems they are building.

And that's the question I want to explore next: in an age where AI can build so much of our software for us, how do we get better at understanding what we've built?