What actually changed
The shift is not that models got better at writing code, though they did. The shift is that they got better at using tools in a loop. An agent that can read files, run a command, and see the output can verify its own work. That closes a feedback loop that a pure text generator never had.
Practically, this turns three things from research demos into daily tools:
- Multi-file changes. Renaming a concept across a codebase, threading a new parameter through several layers, or migrating a deprecated API call in forty places.
- Test-driven repair. Give the agent a failing test and it can iterate until the test passes, which is a far more reliable target than "make this better".
- Codebase questions. "Where does the billing webhook get validated?" answered by actually searching the repository rather than guessing from a snippet.
None of this requires the model to be brilliant. It requires it to be persistent and to have honest feedback.
Where agents earn their keep
The work that goes well shares a pattern: the goal is checkable by a machine, and the blast radius is small. On our engagements the consistent wins have been:
Migration and mechanical refactors
Framework upgrades, dependency swaps, and lint-rule adoptions are tedious, well-specified, and verifiable. An agent that can run the build after every batch will grind through these far longer than a person will.
Test backfill
Legacy modules with no coverage are a natural fit. The agent reads the implementation, writes cases, and you review the assertions. You are reviewing intent rather than syntax, which is fast.
First-draft scaffolding
New endpoints, form validators, data-access layers, and boilerplate that follows an existing pattern in the repo. The agent copies your conventions more faithfully than a new joiner would in week one.
Investigation
Tracing a bug across services, summarising what a subsystem does, or producing a change-impact list before an estimate. This is often the highest-value use and the least discussed.
Where they still fail
Agents fail in ways that are quiet, which is what makes them dangerous. The recurring categories:
- Plausible-but-wrong architecture. Asked to add a feature, an agent will happily introduce a fifth way of doing caching rather than find the existing one. It optimises for a working diff, not a coherent system.
- Tests that assert the bug. If the agent writes both the code and the test, it will sometimes encode the current behaviour as correct. Coverage goes up, confidence should not.
- Silent scope creep. "Fix the date parsing" becomes a diff that also reformats a file, bumps a dependency, and renames a variable. Each change is defensible; the review burden is not.
- Confident failure on ambiguity. When the requirement is underspecified, a person asks. An agent picks an interpretation and builds on it for twenty minutes.
Notice that none of these are fixed by a larger model. They are fixed by narrower tasks and stronger verification.
The practices that make it work
The teams getting real leverage are not the ones with the best model access. They are the ones who did the unglamorous work first.
Make the repository legible
Agents perform in direct proportion to how discoverable your conventions are. A written architecture note, consistent folder structure, and a single obvious way to run tests are worth more than any prompt engineering. This is the same investment that helps human onboarding, which is a useful sanity check — if a new engineer would struggle, so will the agent.
Give it a real verification loop
An agent without a test suite is a very fast intern with no supervisor. Fast, reliable tests are the single highest-leverage input. If your suite takes twenty minutes, the agent's iteration loop is twenty minutes, and it will do less useful work per hour than a person.
Bound the task
"Improve error handling in the payments module" is a bad task. "Make every handler in payments/ return the shared AppError type, and keep the existing tests green" is a good one. The best predictor of a clean agent run is whether you could have written the acceptance criteria before starting.
Review the diff, not the transcript
It is tempting to read the agent's reasoning and feel reassured. Do not. Review the change exactly as you would a colleague's pull request, with the same standards. The reasoning is marketing; the diff is the product.
A realistic adoption path
When we introduce agents into a client's delivery process, we sequence it deliberately rather than switching on everything at once.
- Read-only first. Codebase questions, change-impact analysis, and review assistance. No writes, no risk, and the team builds intuition for where the model is reliable.
- Tests and tooling. Let the agent write tests and build scripts. Mistakes here are cheap and visible.
- Bounded implementation. Well-specified tickets in modules with good coverage, still behind normal code review.
- Routine multi-file work. Migrations and refactors, with a human owning the merge.
Most teams find their honest ceiling somewhere in step three or four, and that is a good outcome. The goal is not autonomy. The goal is removing the work that was never interesting.
What this means for how you staff a team
The most common question we get from founders is whether agents let them hire fewer engineers. In our experience the answer is no, but the shape of the work changes.
Time spent on mechanical implementation goes down. Time spent on specification, review, and architecture goes up — because the bottleneck moves there. A team that could previously absorb sloppy requirements now feels every ambiguity immediately, because the agent will act on it at speed.
That is why our engagements still start with discovery and a clear specification even when we expect heavy agent use. The clearer the target, the more of the work you can hand off. Teams that skip that step do not get a faster project; they get a faster mess.
Where to start
If you want a concrete first move, pick one module with decent test coverage and one recurring, boring task. Run it through an agent under normal review for two weeks and measure two things: how much review time it cost you, and how much implementation time it saved.
If the second number is bigger, expand the boundary. If it is not, the problem is almost always the specification or the test suite, not the model — and fixing either of those is worth doing regardless.