Harvard study finds AI coding agents boost code volume but fail to increase software output

A new study by Harvard University researchers reveals that while AI coding agents generate significantly more code, they do not increase the overall software output of firms. The research indicates that human code review processes act as a bottleneck, absorbing the efficiency gains from automated code generation.

The study, published in October 2026, analyzed data from over 700 software development firms. Researchers Fiona Chen and James Stratton found that the introduction of AI tools did not lead to increased software production or reduced employment. Instead, the extra code generated required more extensive review, slowing down the overall development process according to the analysis of aggregated analytics data.

Methodology and Data Sources

The researchers utilized aggregated analytics data from Jellyfish to measure engineering team output across a large dataset. This data included 300 million individual work events, such as commits and pull requests, from more than 700,000 employees. The dataset covered the period from 2021 through March 2026, allowing for a comprehensive look at trends in software development practices.

To determine the impact of AI tools, the team combined directly measured AI usage with analyses of GitHub activity. They identified when companies introduced AI coding assistants, which help auto-complete human-authored code, and AI coding agents, which write and submit code autonomously. The researchers then performed difference-in-differences regressions to compare key variables before and after the adoption of these tools at various organizations.

Impact on Development Workflows

The results showed a clear increase in raw code production when firms adopted AI coding agents. The study found a 30 percent increase in total lines of code generated, a 20 percent rise in total commits, and a 23 percent increase in pull requests on average. These metrics indicate that AI tools are effective at producing large volumes of code quickly and efficiently within the initial development phase.

However, this surge in code volume did not translate into improved software output at the firm level. The researchers noted that the code review process significantly increased in length, with pull requests more likely to require revisions. Reviewers also left more comments on the AI-generated code, creating a downstream constraint that absorbed the upstream efficiency gains from the AI tools.

Implications for Software Firms

The findings suggest that human code review remains a critical bottleneck for the efficiency of AI coding tools. Despite the ability of AI agents to generate functional code efficiently, the need for substantial human effort to review and verify that accuracy limits overall productivity. The study found little evidence that firms increased software output or reduced employment by integrating these AI technologies into their workflows.

This dynamic highlights the complexity of integrating AI into software development processes. While AI can accelerate the initial coding phase, the subsequent review and production stages impose constraints that prevent proportional gains in final output. Firms relying on AI coding agents may see increased activity metrics but not necessarily improved results or cost savings in terms of employment.

What should you do?

Security and technology professionals should monitor their development metrics to understand how AI tools impact their specific workflows. Teams should evaluate whether increased code volume is leading to longer review cycles or more revisions. It is important to assess the balance between AI-generated code and human review capacity to identify potential bottlenecks.

What to do and how to stay safe: AI coding agents

• Review internal metrics to see if AI adoption has increased pull request volume without increasing final software releases.
• Evaluate the time spent on code reviews to determine if AI-generated code requires more extensive verification than human-authored code.
• Assess whether your team's capacity for code review is becoming a bottleneck that negates upstream efficiency gains from AI tools.
• Monitor changes in development velocity and quality metrics to understand the real-world impact of AI coding agents on your specific workflow.

Step-by-step guide: Prompt Injection Warning Signs: How to Spot AI Manipulation

General security guidance from the Threat Times newsroom. It is not confirmed advice from the organisations named in this story.

Frequently asked questions

Does using AI coding agents increase the amount of software a firm produces?

No, the study found little evidence that firms increase software output by using AI coding agents, as efficiency gains are absorbed by downstream constraints.

How much more code do AI coding agents generate compared to baseline?

AI coding agents lead to a 30 percent increase in total lines of code, a 20 percent rise in commits, and a 23 percent increase in pull requests.

What was the source of the data used in this study?

The researchers used aggregated analytics data from Jellyfish, encompassing 300 million work events from over 700,000 employees at more than 700 firms.