AI coding agents generate more code, but not more software
Harvard study: AI coding agents raise code volume but not Jira-tracked output, as review time jumps 49%; commenters add review capacity and test bloat may erase the gains.
A Harvard study of Jellyfish analytics covering 300 million work events at more than 700 firms finds AI coding agents raise raw output—30% more lines of code, 20% more commits, 23% more pull requests—but not Jira-tracked issue or epic resolution.
The apparent productivity gain is absorbed by review: average PR review time rises 49%, changes requested nearly double, and comments per PR rise 35%, while AI handles only 23.3% of review comments and 10.8% of PRs. The authors report no significant employment changes and caution that the March 2026 cutoff predates newer agent capabilities.
Key takeaways
- The study uses aggregated Jellyfish data from more than 700,000 employees at over 700 software firms from 2021 through March 2026, with difference-in-differences regression around AI assistant and agent adoption.
- Agent introduction is associated with 30% more lines of code, 20% more commits, and 23% more pull requests, but no statistically significant change in Jira issue or epic resolution, or in issue size and complexity.
- Review process time from pull request submission to merge increases 49%; the share of PRs with changes requested nearly doubles, comments per PR rise 35%, and the share of workers doing reviews rises 14%.
- AI code review remains marginal: 80% of firms used some form by March 2026, but agents generated only 23.3% of review comments and 10.8% of pull requests.
- Researchers cannot attribute significant employment changes to AI; 95% of firms had implemented agents by the cutoff, and the authors note learning effects and rapid updates may alter results.
What people think
Analysis details
18 comments analyzed. Analyzed .
The sample selects active discussion branches and includes available parent comments. It may omit parts of the full thread. The analysis included 18 of 32 usable stored comments because of the input limit. Selected themes do not measure community opinion or how common a view is.
Context-dependent value and adoption of line-by-line code review
Comments debate whether line-by-line human review should be abandoned. One root argues code review is a growing bottleneck and that change may come first from startups adopting no-human-review, while replies qualify this: review need depends on stakes (e.g., UI color versus critical backend latency), some senior engineers dislike review, and others defend review for shared understanding, maintainability, skill transfer and tribal knowledge. A reply challenges the startup-adoption story by asking where the faster products are, saying the consumer/enterprise software landscape looks unchanged.
Paper data age, baselines and methodology
Replies scrutinize the study's timing and metrics. One notes academic papers describe the research period, here mostly 2025 models and initial deployment effects; another says the study was updated twice, last in August 2026. A detailed reply cites baseline 5,017 LoC/month, 5.34 PRs/month and 7.03-day review time as signs of old data, worries about unread code, and criticizes use of mean over median. A response challenges the claim that the authors are lying given stated collection dates from January 2021 to March 2026.
Limits of purely agentic code review versus human review
Comments argue agentic review has orthogonal weaknesses: it remains subpar on architecture and design, has false positives/negatives, and lacks the author's mental model. One reply counters that reconstructing a mental model independently is a key benefit of human review, especially for maintainability and shared understanding; another warns against treating code as a black box without human comprehension.
Agent-assisted IDE review and new review practices
A reply reports best results with agent-assisted code review in an IDE, using agents for a first correctness pass while the human builds a mental model. Another suggests starting from no review and inventing a new practice for agentic coding rather than modifying traditional review. A further reply proposes humans review requirements, specifications and test scenarios, though a response cautions that big upfront specifications previously failed and were replaced by continuous improvement and tighter release cycles.
Agent-generated test code bloat and redundancy
Comments describe agents producing repetitive test code, avoiding refactoring and duplicating helpers instead of reusing them, so integration tests can dwarf shipped code. One reply recounts auditing a 100k-line test suite, finding about 60k lines near useless due to overmocking, then deleting them and writing broader scenario tests that caught bugs.
AI velocity gains limited by non-coding bottlenecks
A reply disputes leadership expectations that AI will roughly double developer velocity: if coding is only about 10% of team time and 90% is design and analysis, doubling code-writing speed yields only about 5% overall improvement. Another reply invokes Amdahl's law and the concept of a bottleneck.