Harvard study finds AI coding boosts output but slows final software delivery
Research shows AI increases code volume by 30 percent, yet human review delays negate productivity gains and extend merge times.
Key points
- AI usage correlates with a 30 percent increase in lines of code produced.
- Average pull request review time rose from 7.03 to 10.48 days.
- Researchers found no statistically significant improvement in completed software output.
A new study from Harvard University researchers indicates that while artificial intelligence tools significantly boost the volume of code developers produce, they do not accelerate the delivery of finished software. The research identifies human code review as the new bottleneck that negates the initial productivity gains provided by AI coding assistants.
According to the paper by Fiona Chen and James Stratton, organizations using AI saw a 30 percent increase in lines of code and a 20 percent rise in code commits. However, these metrics did not translate into a proportional increase in completed software products, suggesting that downstream processes are absorbing the initial speed advantages.
The shifting bottleneck in development
The study reveals that the surge in AI-generated code has shifted the workload to the review phase, causing substantial delays. The average time required to review and merge pull requests increased by 49 percent, rising from 7.03 days to 10.48 days. This extension highlights how the rapid generation of code creates a backlog for human reviewers who must validate the output before it can be integrated.
Reviewers are also engaging more deeply with the code, with comments per pull request increasing by 35 percent. The proportion of employees participating in code reviews also rose by 14 percent, indicating that teams are allocating more human resources to catch errors or issues in the AI-generated submissions rather than writing new code from scratch.
Implications for workforce and efficiency
Despite widespread concerns that AI would lead to immediate job losses, the research suggests that human workers remain vital to the software development lifecycle. The findings imply that companies are not replacing developers but are instead shifting their roles toward oversight and quality assurance. The increased need for corrections in AI-generated code underscores the continued necessity of skilled human intervention in the development process.
The authors question whether AI is truly helping if the generated code requires substantial corrections, potentially offsetting the time saved during initial creation. This dynamic suggests that the current integration of AI tools may be adding to the overall workload rather than streamlining it, as teams struggle to manage the higher volume of code that requires rigorous human scrutiny.
What should you do?
Security and technology professionals should monitor their internal development metrics to identify if AI adoption is creating similar bottlenecks in their code review pipelines. Teams should assess whether the increase in code volume is matching with adequate review resources to prevent delays in software delivery.
What to do and how to stay safe: AI
- Track the ratio of AI-generated code to human review time to identify potential bottlenecks in your development pipeline.
- Ensure that code review processes are scaled appropriately to handle the increased volume of pull requests generated by AI tools.
- Monitor the quality of AI-generated code by analyzing the number of corrections and comments required per pull request.
- Evaluate whether current staffing levels for code review are sufficient to maintain delivery timelines without excessive delays.
Step-by-step guide: Prompt Injection Warning Signs: How to Spot AI Manipulation
General security guidance from the Threat Times newsroom. It is not confirmed advice from the organisations named in this story.
Frequently asked questions
Does AI coding improve the speed of software delivery?
No, the Harvard study found no statistically significant improvement in completed software despite a 30 percent increase in code production.
How has AI affected code review times?
The average review process increased from 7.03 days to 10.48 days, with a 49 percent rise in time taken to merge pull requests.
Are companies replacing developers with AI?
The research indicates that human workers remain vital, with the proportion of employees doing code reviews increasing by 14 percent.
