AI Coding Agents in Production: What Actually Changes

Maciej Lewandowski · · AI-assisted engineering

I run agents against real codebases most working days. Not toy repos, not greenfield demos: existing systems with history, invariants nobody wrote down, and money moving through them. After enough of that, the interesting question stops being "can it write the code" and becomes "what did this actually change about the job".

The honest answer is smaller and stranger than either the marketing or the panic suggests. One thing moved, and almost everything else is a consequence of that one thing.

1. The bottleneck moved

Writing code was never the expensive part, but it was the visible part, and for most of a career it was the part that limited throughput. It is not the limit any more. Producing a plausible 600-line change now costs a few minutes. Deciding whether those 600 lines are correct costs what it always cost, and possibly more.

More, because plausible code defeats the machinery you normally read with. Skimming a diff works because you are not really reading it, you are pattern matching against how a person writes when they are tired, rushed, or half-informed. Suspicious variable names. A copy-pasted block with one thing not renamed. A comment that no longer matches the line under it. Generated code has none of those signals. It is uniformly well-shaped, correctly named, plausibly commented, and wrong in the middle in a way that looks exactly like the parts that are right.

So reading 600 lines of agent output is genuinely harder than writing 150 yourself. Not slower necessarily, harder. Different muscle, more expensive per line, and the tell you relied on is gone.

the bottleneck intent + context agent diff VERIFY merge rejected throughput is set by the gate, not by the generator

Every practice below is downstream of that picture. If a technique does not either improve what goes into the loop or make the gate cheaper, it is not helping.

2. Context is the actual product

The single biggest predictor of output quality is not prompt wording. It is what the agent can see. A vague request with rich context beats a beautifully engineered prompt with none, reliably enough that I stopped optimising prompts and started optimising context instead.

Rich context, concretely, means: the repo conventions written down somewhere the agent reads by default (a project instructions file, and yes it needs maintaining like code); the failing test rather than a description of the failure; the exact file rather than a guess at the module; the list of constraints that are non-negotiable in this codebase and would otherwise be invisible. "We never allocate in this hot path." "This crate must stay no_std." "That column is append-only." None of that is inferable from source alone, and all of it is what separates a change that lands from a change that reads well.

The corollary is uncomfortable and worth saying plainly: the time you used to spend writing code, you now spend curating what the agent knows. That is not overhead around the work. In this loop, it is the work.

3. Decomposition is the skill

Given real context, the next question is whether the task splits at all, and that is where most of the leverage lives.

Independent work fans out beautifully. Audit forty files for one pattern. Write test coverage per module. Perform a mechanical migration across every call site. Draft docs for N components. Each unit needs no result from any other unit, so N agents finish in roughly the time one takes, and you review N diffs.

Dependent work does not fan out at all. If step two needs the shape that step one chose, running them concurrently produces two incompatible half-solutions and a merge you will regret. Knowing which kind of work you are holding is the difference between a six-times speedup and six conflicting attempts.

independent work: fan out task agent 1 agent 2 agent 3 agent 4 review separate worktrees, no shared files dependent work: chain step 1 step 2 step 3 step 4 no parallelism here

One mechanical detail matters more than it sounds. Parallel agents editing the same files is a merge disaster, and it is the default failure of a first attempt at fan-out. Isolation fixes it: give each unit its own worktree or its own branch, let them finish, then integrate deliberately. Concurrency in an agent runner obeys the same rules as concurrency anywhere else, and shared mutable state is still the enemy.

4. Review it like a PR from a fast junior with no memory

The most useful mental model I have found is not "assistant" and definitely not "pair". It is a very strong junior engineer who is tireless, unusually well read, has no memory of yesterday, and has no stake in whether this works in production next month.

That model tells you exactly what review posture to adopt, because it is the one you already use for that person:

  • Read the diff, not the summary. The summary is generated from intent, not from what the code does. It is confident either way.
  • Ask what happens at the edges. Empty input, concurrent callers, the retry, the partial failure. Breadth is the strength; the edges are where breadth runs out.
  • Be suspicious of fluent prose about code that was never executed. "This correctly handles the reorg case" means the sentence was generated, not that the case was tested. Run it or do not believe it.

5. Where they genuinely win

The pattern in every real win is the same: breadth over depth. Work that is wide, shallow and verifiable.

  • Sweeping an entire codebase for a pattern, an anti-pattern or a missing guard.
  • Writing the fortieth test that looks like the previous thirty-nine.
  • Mechanical refactors: rename across call sites, thread a parameter through, split a module, migrate an API version.
  • First drafts of documentation, where the raw material exists and only assembly is missing.
  • Exploring unfamiliar code to answer "where does X actually happen", which used to cost an afternoon and now costs a coffee.
  • Scaffolding: the boring 80% of a new component that has to exist before the interesting 20% can start.

Note what all of these share. Each result is cheap to check. That is not a coincidence, it is the whole selection criterion.

6. Where they lose

Novel architecture is the clearest loss. Models average over what exists, and the part of your problem worth designing is precisely the part that does not exist yet. You will get a competent version of the standard answer, which is useful as a baseline and actively harmful if the standard answer is why you are here. Gnarly concurrency is the second: the failure modes are interleavings, they do not appear in the diff, and a confident explanation of why the lock ordering is safe is not evidence.

Above both of those sits the real disqualifier: anything where a mistake is silent. If a wrong change crashes, CI catches it and you have lost minutes. If a wrong change produces a plausible number, nothing fires, and you find out during reconciliation weeks later. In that regime the review cost is unbounded, and unbounded review cost eats any speed gain you thought you had.

This is the same argument as floats in money, which I keep making because it keeps being the same argument. The dangerous bug is not the loud one. It is the quiet one that looks like a result.

7. Guardrails are the multiplier, and they are all boring

Nothing on this list is new, which is the point. Types. A fast test suite. Property based tests for the laws your types cannot state. Linters. CI that actually blocks rather than politely warning. All of it existed before and was worth having before. All of it just got more valuable, because it converts an expensive human gate into a cheap automated one.

The compiler is the cheapest reviewer you will ever hire, and it does not get tired at 11pm. This produces a visible split in practice. A codebase with strong static guarantees gets far more out of agents than a dynamically typed one sitting at 40% coverage. The agent's error rate is roughly the same in both. The difference is entirely in whether your side of the loop catches it before merge or after deploy.

If you want a lever, that is the lever. Not a better prompt. A test suite that runs in ninety seconds and a type system that refuses to compile the mistake.

8. Non-determinism is a workflow property, not a bug to fix

Ask twice, get different code. Both versions may pass. Both versions may be reasonable. They will not be the same, and no amount of prompt discipline makes them the same.

Stop treating that as a defect to engineer away and start treating it as a property the workflow has to tolerate. Practically: pin the intent in tests, because tests are the only artifact that survives regeneration. Review the diff every time, including the fifth time you asked for something similar. And never let "it worked last time" be the evidence, because last time is not this time and there is no shared state between them.

9. What it actually costs

An honest accounting, because the wins above are real and so is this side of the ledger.

  • Context and prompt time is real time. Some days you spend forty minutes assembling constraints for a change you could have typed in twenty five. That trade is worth it when the change is wide and terrible when it is narrow, and telling the difference in advance takes practice.
  • Plausible-but-wrong cleanup is the expensive failure. Not the obviously broken output, which you discard in seconds. The one that is 90% right, so you patch it, and the remaining 10% turns out to be an assumption baked through the whole structure.
  • Skill atrophy is a genuine risk. If you stop reading code closely you stop being able to, and close reading is now the load-bearing skill. Reviewing well is the practice that keeps the ability alive; accepting diffs is the practice that kills it.
  • Sunk cost applies to generated code, and it should not. This is the subtlest trap. An approach arrives already written, complete, syntactically confident, and that existence exerts a pull that a whiteboard sketch never did. But it cost nothing to produce. Deleting 600 lines and asking again is close to free, and the fact that it feels expensive is a bug in you, not a fact about the code.

The checklist

  1. Optimise context, not prompts. Conventions written down, the failing test, the exact file, the constraints that are invisible in source.
  2. Classify before you dispatch. Independent work fans out, dependent work runs as a chain, and guessing wrong costs more than the work.
  3. Isolate parallel work in separate worktrees or branches. Never let two agents edit the same file.
  4. Read the diff, never the summary, and treat confident prose about unexecuted code as marketing.
  5. Refuse the silent-failure domains. If a mistake produces a wrong number instead of a crash, the review cost eats the gain.
  6. Invest in boring guardrails. Types, fast tests, property tests, blocking CI. They are what makes your side of the loop cheap.
  7. Pin intent in tests, because regeneration is non-deterministic and tests are what survives it.
  8. Throw away freely. Generated code owes you nothing and you owe it nothing back.
Bringing coding agents into a production engineering workflow? I help teams establish the repository context, review boundaries and verification gates that make the speed useful without lowering the standard. See engineering services and engagement options.

The job did not get easier and it did not get automated. It got rebalanced: less time producing, far more time deciding what is true. Which is, if you think about it, what senior engineering was always mostly made of. The typing was just hiding it.