How much faster do developers code with AI assistants?
Randomized trials inside companies find gains of about 21 to 26 percent. Across all studies the results run from 19 percent slower, for experienced developers in their own mature codebases, to 56 percent faster on one simple lab task, so the evidence conflicts.
Across all 5 studies: from 19% slower to 56% faster.
- Medium confidence
- 5 primary sources
- 2023 to 2026
- Last checked
Every study on one scale
Chart values as text
- Scale: % faster or more tasks done (negative means slower). Shaded range: 21–26% in company trials; dashed, all studies: -19% to 56%.
- Peng et al., lab task (2023): 55.8% faster
- Cui et al., three companies (2025): 26% more pull requests
- Paradis et al., Google (2024): 21–26% faster
- METR, 2025 (2025): 19% slower
- METR, late-2025 rerun (2026): 4–18% faster (not significant)
Why medium confidence: Five randomized studies from four independent groups agree that the effect depends on the task and the developer. The two company estimates are close but have wide margins of error and measure different things, and the studies disagree on the sign for experienced developers in familiar codebases.
Sources
| Study | Creator | Year | Sample | Method | Figure | Link |
|---|---|---|---|---|---|---|
| The Impact of AI on Developer Productivity: Evidence from GitHub Copilot | Sida Peng (Microsoft Research), Eirini Kalliamvakou and Peter Cihon (GitHub), Mert Demirer (MIT Sloan) | 2023 | 95 freelance developers recruited on Upwork, mostly in India and Pakistan | Controlled experiment, 2022, on one task: writing a basic HTTP server in JavaScript | 55.8% less time (71 minutes against 161); 95% confidence interval 21% to 89% “the treated group completed the task 55.8% faster (95% confidence interval: 21-89%)” | arxiv.org ↗ |
| The Effects of Generative AI on High-Skilled Work: Evidence from Three Field Experiments with Software Developers | Kevin Zheyuan Cui, Mert Demirer, Sonia Jaffe, Leon Musolff, Sida Peng and Tobias Salz (Princeton, MIT, Microsoft, Wharton) | 2025 | 4,867 developers at Microsoft, Accenture and a Fortune 100 electronics maker | Three randomized experiments run by the companies over 2 to 8 months, pooled. Counts output; does not time tasks. | 26.08% more completed pull requests a week for developers who used GitHub Copilot (standard error 10.3%) “our analysis reveals a 26.08% increase (SE: 10.3%) in completed tasks among” | economics.mit.edu ↗ |
| How much does AI impact development speed? An enterprise-based randomized controlled trial | Elise Paradis, Satish Chandra and colleagues (Google) | 2024 | 96 full-time Google software engineers | Randomized trial on one enterprise task in Google's own tools, 2024. Google evaluated its own tools. | About 21% less time (21% to 26% across models); 96 minutes with AI against 114 without “this main effect was robust to covariates, making developers between 21% and 26% faster.” | arxiv.org ↗ |
| Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity | Joel Becker, Nate Rush, Beth Barnes and David Rein (METR) | 2025 | 16 experienced developers, 246 real tasks in mature open-source projects they had worked on for about 5 years | Randomized at the task level, February to June 2025, mostly Cursor with Claude 3.5 and 3.7 Sonnet | 19% more time per task with AI (95% confidence interval +2% to +39%) “we find that allowing AI actually increases completion time by 19%” | arxiv.org ↗ |
| We are Changing our Developer Productivity Experiment Design | METR | 2026 | 57 developers (10 from the first study, 47 new), more than 800 tasks | Same design from August 2025. Many developers declined tasks without AI, so METR calls the estimates unreliable. | 18% less time for returning developers and 4% less for new ones; both margins of error include zero “Among newly-recruited developers the estimated speedup is -4%” | metr.org ↗ |
Which developers gain the most from AI?
Newer and more junior ones, in the three-company trials: developers with short tenure raised output 27% to 39%, against 8% to 13% for long-tenured colleagues, and junior developers 21% to 40% against 7% to 16% for senior ones. These splits are not statistically significant on their own. F-12 compares the pattern across studies.
Do newer AI coding agents make developers faster?
There is no clean measurement yet. METR's late-2025 rerun, with 57 developers, estimated 4% to 18% less time with AI, but both estimates sit within the margin of error. Many developers declined to do tasks without AI, which METR says biases the data.
Numbers you may have seen
These figures circulate widely. Here is what the original source says.
- Developers code 55% faster with GitHub Copilot. This comes from one simple, self-contained lab task done by freelancers in 2022. METR's paper notes the task is unlike most software work.
- AI makes developers 19% slower. METR studied 16 experienced developers in large, mature codebases with early-2025 tools. METR says the result does not show that AI slows down most developers, and its 2026 rerun leans toward a speedup.
- Copilot makes developers 26% more productive. The 26% is the rise in pull requests completed per week for developers who used the tool, with a standard error of 10 points. It counts output, and measures neither time per task nor code quality.
More from What AI Saves
- F-083–5.4%How much time does AI save knowledge workers?
- F-0925–40%How much faster do people finish writing tasks with AI, and is the work better?
- F-1014–15%How much more do customer support agents get done with AI?
- F-1227–46%Does AI help less-experienced workers more than experienced ones?
- F-1314–25%When does AI make work slower or worse?
The whole chapter · Chapter poster · All of The AI at Work Fact Book