Stop Blaming Code Review for AI’s Delivery Problem
Most engineering teams I talk to who use AI coding tools say they’ve never been more productive. Some tell me they’re building two to three times as many features as they used to. But pushing those features into production – that’s a different story.
And to be totally honest – we have the same challenge at Testkube. Our engineers are building features faster than ever before but we’re struggling to get them into releases as quickly as we’d like. From what I hear at conferences and in customer calls, we’re far from the only ones.
Not surprisingly, one common culprit is code review. Faros AI looked at more than 10,000 developers and found that teams with high AI adoption merged 98% more pull requests, while the time spent reviewing them went up 91%. That’s a lot more code landing on the same number of reviewers.
But that isn’t the whole story. Review is often where teams first notice the slowdown, but in my experience, it isn’t where the slowdown starts (or ends). You can speed up review with better tooling, or by having AI help with it, but every approved change still has to go through a (preferably automated) testing process that teams set up for far less code than they produce today, so the changes end up waiting there instead. And if you have come as far as using AI to help you write more tests – running them becomes the next hurdle to overcome for achieving AI velocity through the entire SDLC.
In most organizations, testing runs inside the CI/CD pipeline. A developer pushes a change, the pipeline builds it and runs the tests, and if they pass, the change moves on toward production. CI/CD tools were built to build and deploy software. Teams added testing as one more step because that’s where the code already was, and when a team shipped a handful of changes a day, that worked well enough.
I’ve been frustrated with this setup since well before AI came along. From my experience, running automated tests at scale through CI has never worked very well; Tests only run when a pipeline runs. If one flaky test fails, you often end up rerunning the whole pipeline to find out whether the failure was real. Results end up scattered across hundreds of pipeline runs, so it’s hard for anyone to say what actually passed for a given release. Teams tell me their pipelines take 45 minutes and developers go get coffee while they wait – every build chugging through hundreds or even thousands of tests that deserve better!
Now AI is pushing a lot more code through the same setup. Every extra change waits for the same builds and runs the same suites in the same queue. A recent National Bureau of Economic Research working paper that followed more than 500,000 developers on GitHub found that autonomous coding agents raised commits by about 240%, but releases by only about 30%. That matches what I hear from teams: plenty of work in progress, and not much more of it getting pushed out the door.
What I’d like to see is teams stop treating testing as a step in the build and start running it as its own system. The details will look different from one team to the next, and some teams have already built parts of this themselves. But the basic idea is that tests shouldn’t have to wait for a build. A team, (or an autonomous coding agent for that matter, should be able to run required tests when an API changes, on a schedule or whenever they need to, and run many of them at once when a lot of changes come in. They should preferably run against real environments, so a passing result tells you something about how the software will behave in production. And the results should end up in one place instead of scattered across pipeline logs, agent sandboxes and local harnesses.
I’m not saying teams should get rid of CI/CD. Pipelines are fantastic at building and deploying software, and they should keep doing that. They can still kick off tests. They just shouldn’t be the only place testing can happen.
Now AI is pushing a lot more code through the same setup. Every extra change waits for the same builds and runs the same suites in the same queue. A recent National Bureau of Economic Research working paper that followed more than 500,000 developers on GitHub found that autonomous coding agents raised commits by about 240%, but releases by only about 30%. That matches what I hear from teams: plenty of work in progress, and not much more of it getting pushed out the door.
It matters more as teams move toward agentic development, where AI agents handle more of the work between a spec and production. If agents are writing a growing share of the code, checking that code has to keep up with them. Tests that only run when a change reaches the front of a pipeline queue will have a hard time doing that – agents should be able to trigger those tests, and act on their results immediately without having to wait for an already overwhelmed build to give them the green (or red) light.
I’m super-optimistic about AI (agreeably, I’m super-optimistic about most things). The models we have now already write a lot of useful code, and I expect them to keep getting better. But in my experience, writing the code has rarely been the slowest part of shipping software. Knowing whether it works, and being confident enough to release it, has always taken longer, and AI has made that much easier to see.
So when someone tells me testing and release confidence is what’s holding their AI-powered teams back, I start by asking two questions: how are you running your tests — and where do they run? In my experience, the answers tell you more about why features aren’t shipping than the review queue does.
SD TIMES Q&A
Why is AI-generated code not making it to production faster?
Teams are producing far more code with AI coding tools, but testing and CI/CD pipelines were built for much lower code volumes. The bottleneck shifts from writing code to validating it, since tests are still run as a single step inside the build pipeline instead of as an independent, scalable system.
Is code review really the bottleneck slowing down AI-assisted development?
Code review is often where teams first notice delays, but it is not the root cause. Once a pull request is approved, it still has to pass through a CI/CD testing process that was not designed to handle the volume of code AI tools now generate, so the backlog just moves downstream.
How much do AI coding tools actually increase developer output and releases?
According to a National Bureau of Economic Research working paper covering more than 500,000 GitHub developers, autonomous coding agents raised commits by about 240% but increased releases by only about 30%. This gap suggests more code is being written without a proportional increase in shipped software.
Should teams get rid of CI/CD pipelines to speed up AI-driven software delivery?
No, the recommendation is not to eliminate CI/CD, since pipelines remain effective for building and deploying software. Instead, testing should be decoupled so it is not solely dependent on pipeline runs, allowing CI/CD to still trigger tests without being the only place testing can happen.
The post Stop Blaming Code Review for AI’s Delivery Problem appeared first on SD Times.
Tech Developers
No comments