Video · 10 min

How to measure developer productivity with AI coding tools like Claude Code and Cursor

Oren Keinan, founder of Aidealy · Published November 10, 2026

By pressing "Play on YouTube", you agree to load this video from YouTube, a Google service. Google then receives data such as your IP address and may store data in your browser.
Details in our Cookie Policy and Privacy Policy.

Watch on YouTube (Opens in a new tab)More from Oren Keinan on YouTube (Opens in a new tab)Read Oren's take on LinkedIn (Opens in a new tab)Read the transcript

In short

When AI writes much of the code, measure developer productivity in pairs: each count next to a number that shows when the work is bad. Split each pair by the AI tool's own record, not by a guess.

AI now writes much of the code, and counts like commits can triple while the software a team releases grows far less.
I led R&D teams for 20 years, and I never had a good answer to one question:
what each engineer was really shipping. AI made that question harder.
The sections that follow give the 3 pairs I'd put on a report today, where each number comes from, and where to start.

Do teams measure developer productivity now that AI writes code?

Mostly not: in a 2025 survey of software professionals, most felt more productive with AI, but more than half used no objective number to measure their work.

Görkem Giray and colleagues, at five universities and research institutes, posted a study of generative AI adoption in software engineering in 2025. They surveyed about 200 software professionals from 38 countries. In that 2025 survey, about 95% said AI made them more productive. But 58% said they don't measure their work with any objective number at all.

So before you report that AI made your team faster, check how many of the merged pull requests needed a fix soon after.

Which developer productivity metrics still work when AI writes the code?

Metrics in pairs still work: one number shows speed or amount, and the other shows when the work is bad.

Economists at MIT, Wharton and Boston University followed more than half a million developers on GitHub for a 2026 working paper of the National Bureau of Economic Research (NBER), "Writing Code vs. Shipping Code". After the developers started to use AI coding agents, their commits roughly tripled, but their software releases rose by only about 30%. If you read only the commit count, AI looks like a big success.

These are the 3 pairs I'd put on a report today:

The number	Read it next to	Where both numbers come from
Merged pull requests	Their code quality and the fixes they needed after merge	Your Git host and your Git history
Review speed	Approved pull requests that were fixed later	Your Git host
AI spending	The merged changes it paid for	Your AI tools' bills and your Git host

The first number of each pair can look good while the work gets worse. The article on developer productivity metrics that get people fired shows how people cheat counts like commits and lines of code.

What should you put next to merged pull requests?

Put merged pull requests next to the share that needed a fix soon after the merge, and their code quality if you check it.

Both numbers come from your Git host, such as GitHub, and from your Git history. The merged pull requests are easy to list. The fixes are the hard part.

For every fix, you have to find the pull request that wrote the lines the fix changed. Git has a command for that, git blame, but it works on one file and one fix at a time. Then count the pull requests that needed a fix within a set window after the merge, for example 30 days. That share is the second number.

If you already run a code quality check on each pull request, add its result too. It shows whether each change increased code quality or lowered it.

To me, a merged pull request is finished work only when its code doesn't need fixing soon after. So never put the number of merged pull requests on a report by itself.

Two DORA metrics ask the same question of each deployment: did it need a fix? DORA (DevOps Research and Assessment) is Google's research program on how teams build and ship software. Use them if your team tracks deployments.

DORA's guide to its software delivery performance metrics was last updated in January 2026. It defines change fail rate as "The ratio of deployments that require immediate intervention following a deployment." That usually means a rollback or an urgent fix. Deployment rework rate is "The ratio of deployments that are unplanned but happen as a result of an incident in production." Both come from your deployment records and your incident records.

The same guide says that "speed and stability are not tradeoffs". A team can ship often and still need few fixes. The guide has 5 metrics in all, and the article on DORA metrics in the AI era covers each of them.

Review speed next to the approved pull requests fixed later

Read how long pull requests wait for review next to the approved pull requests that had to be fixed later.

Researchers at Carnegie Mellon University and Stanford University studied one company that pushed its engineers to double their output with AI. In their 2026 study of about 800 developers there, each reviewer had about twice as many pull requests to check as before. The share of pull requests that got any human review fell from 89% to 68%, as AI review tools overtook human review.

Researchers at Deakin University in Australia followed merged pull requests in open-source projects. In their 2026 study of follow-up fixes, merged pull requests from AI coding agents needed a follow-up fix about 1.6X as often as human ones.

Both halves of this pair come from your Git host. It keeps the time when someone asked for a review and the time of each review. GitHub's REST API reference for pull request reviews lists the reviews in time order. So the wait is a subtraction. But you have to do that subtraction for every pull request, in every repository. For the second number, reuse the fixes you traced for merged pull requests, and look at who approved each fixed change.

A quick approval of code that needs fixing a week later only looks fast. So I'd always read review speed next to what happened to the code in the weeks after the approval.

How do you measure AI spending against what it paid for?

Divide the month's AI bill by the pull requests merged that month, and read the result as a cost.

The research firm Gartner predicted in a June 2026 press release that by 2028, AI coding costs will be higher than the average developer's salary. That's a forecast. In the same release, Gartner analyst Nitish Tyagi said: "Most organizations still lack the maturity and frameworks to effectively measure cost versus business impact."

Evan Conaway of DORA wrote "Finding balance in the era of tokenmaxxing" in June 2026. He suggests a measure for AI spending, the "cost per accepted change". DORA's article doesn't define it. I count an accepted change as one that got merged. Conaway's article also asks what changes when you measure token use "by team, by application, or by use case".

The spending comes from your AI tools' bills and usage pages. The merged changes come from your Git host again.

Never rank anyone by AI spending.

How do you compare AI-assisted work without guessing?

Use the record your AI tools keep of the code they wrote.

A popular shortcut is an AI code detector, a tool that guesses from how code looks. It can scan any finished code, even old code. But a peer-reviewed study on detecting AI-generated source code, accepted at ICSE 2025, the International Conference on Software Engineering, tested the existing detectors. The authors found that "they all perform poorly and lack sufficient generalizability to be practically deployed." So don't guess.

Cursor keeps its own record of AI code. On its Team and Enterprise plans, Cursor's team analytics show what share of your committed code Cursor's AI wrote. On the Enterprise plan, Cursor also offers an early version of its AI Code Tracking API. For every commit, the API records how many lines Cursor's AI wrote and how many other lines the commit added.

Claude Code keeps its record differently. Claude Code's settings reference says that by default, every commit Claude Code makes carries a line that names it as a co-author. A team can disable that line, so check that it's still enabled.

Then mark each merged pull request as AI-assisted when the tool's record shows that AI wrote any part of its code. By hand, that means matching each tool's record to each pull request, one by one. Then show every pair twice: once for AI-assisted work and once for everything else.

What goes on the one page you show your CEO?

The 3 pairs, with one row for each service, and AI-assisted work shown next to everything else.

Each pair gets 2 columns: the number, and its partner. DORA's guide says its metrics "are best suited for measuring one application or service at a time". So are the pairs.

AI spending gets its own column at the end, marked as a cost. Put last quarter next to this quarter, so the first thing your CEO sees is the trend.

How to read one engineer's numbers

Use the same pairs to find where to look, and talk to the person before you decide anything.

Two of the pairs work for a single person. One shows the pull requests someone merged, next to the fixes those needed. The other shows the reviews they gave, next to the approved changes that had to be fixed later. If someone's pull requests often need fixes soon after the merge, start by looking at the work behind those fixes.

A number about a person can show you where to look. It can't tell you why. So before you decide anything based on it, talk to the person.

I founded Aidealy, which makes software that measures the work engineering teams ship. Its ranking of engineers, the Aidealy Productivity Rank, is built on pairs like these. Among other things, a merged pull request earns credit, a little more when it increases code quality, and loses points when its code has to be fixed soon after. Approving a change earns credit, and the approver shares the cost of a fix; keeping a teammate waiting for a review costs points. Aidealy tracks AI cost per pull request and never ranks anyone by it. For the code your engineers commit, Aidealy labels each line AI-written or human-written from AI changes it records itself, as the engineer works in Cursor or Claude Code. You don't configure dashboards. You don't write queries. You ask.

Where to start measuring developer productivity

Take the first number in your R&D report and put its pair right next to it.

If your report starts with deployments, put the share that needed a fix beside them. You don't need a year of data for that. Your Git host already holds last quarter's merged pull requests, their reviews and the fixes that came after. Your deployment records hold last quarter's deployments.

DORA's guide suggests a few ways to start small. One of them is "a source-available or commercial product that comes with pre-built integrations".

I'd never show the first number of a report without its pair. The next steps, in order:

Check that your AI tools keep their record: the co-author line for Claude Code, and a Team or Enterprise plan for Cursor.
Pick one window for fixes after the merge, and keep it every quarter.
Show the first pairs per service at your next leadership meeting.
Check them again next month, then repeat.

Sources

Görkem Giray, Onur Demirörs, Marcos Kalinowski and Daniel Mendez, "An Empirical Study of Generative AI Adoption in Software Engineering", arXiv, 2025: https://arxiv.org/abs/2512.23327
Mert Demirer, Leon Musolff and Liyuan Yang, "Writing Code vs. Shipping Code: Productivity Effects Across Generations of AI Coding Tools", NBER Working Paper 35275, 2026: https://www.nber.org/papers/w35275
DORA (DevOps Research and Assessment), "DORA's software delivery performance metrics", updated January 2026: https://dora.dev/guides/dora-metrics/
Hao He and colleagues, "AI Writes Faster Than Humans Can Review: A Longitudinal Study of an Enterprise 2x Mandate", arXiv, 2026: https://arxiv.org/abs/2607.01904
Wannita Takerngsaksiri, Nhat Duong and Scott Barnett, "Who Finishes the Job? A Study of Follow-Up Fixes and Commit Authorship on AI Coding Agent Pull Requests", arXiv, 2026: https://arxiv.org/abs/2609.26847
GitHub Docs, REST API reference for pull request reviews: https://docs.github.com/en/rest/pulls/reviews
Gartner, "Gartner Predicts AI Coding Costs Will Surpass Average Developer's Salary by 2028 as Token Consumption Surges", 2026: https://www.gartner.com/en/newsroom/press-releases/2026-06-24-gartner-predicts-ai-coding-costs-will-surpass-average-developer-salary-by-2028-as-token-consumption-surges
Evan Conaway, DORA, "Finding balance in the era of tokenmaxxing", 2026: https://dora.dev/insights/finding-balance-in-the-era-of-tokenmaxxing/
Hyunjae Suh and colleagues, "An Empirical Study on Automatically Detecting AI-Generated Source Code: How Far Are We?", ICSE 2025: https://arxiv.org/abs/2411.04299
Cursor Docs, team analytics: https://cursor.com/docs/account/teams/analytics
Cursor Docs, AI Code Tracking API: https://cursor.com/docs/account/teams/ai-code-tracking-api
Claude Code Docs, settings reference: https://code.claude.com/docs/en/settings-reference

Watch the video on YouTube

Transcript

Show the full transcript

0:00 Most feel more productive with AI, few measure it

In a 2025 university survey of 204 software professionals from 38 countries, about 95% said AI made them more productive. But 58% don't measure their work with any objective number at all. So how do you measure developer productivity when AI writes much of the code? With 3 pairs of numbers. By the end of this video, you'll know which 3 pairs to put on the report you already send.

You'll also know where each number comes from, and where to start. Some of these numbers are easy to find. Others need a lot of manual work, or software that collects them for you. I'll show you which ones, and how to get each one. I led R&D teams for 20 years, and I never had a good answer to one question: what each engineer was really shipping.

0:49 Which metrics still work when AI coding agents write the code

The metrics that still work when AI writes the code come in pairs. One number shows speed or amount. The other drops when the work is bad. Economists at MIT, Wharton and Boston University followed more than 500,000 developers on GitHub. After they started using AI coding agents, their commits roughly tripled. But their software releases rose by only about 30%.

Read only the first number, and AI looks like a big success. So here are the 3 pairs I'd put on a report today. For each pair, I'll show you where both numbers come from. And later in this video, I'll show you how to split each pair into the work AI helped write and everything else. With no guessing.

1:33 Pair 1: merged pull requests and the fixes after merge

The first pair is the number of pull requests your team merged, next to their code quality and the fixes they needed after the merge.

[On screen: illustrative example, not real data]

Both numbers come from your Git history and your Git host, like GitHub. The merged pull requests are easy to list. The fixes are the hard part. For every fix, you have to find which pull request wrote the lines it changed. Git has a command for that, called git blame. But it works one file and one fix at a time.

Then count the pull requests that needed a fix within a set time after the merge, for example 30 days. That share is the second number of the pair.

[On screen: illustrative example, not real data]

If you already run a code quality check on each pull request, add its result to this pair. It tells you whether the change increased code quality or lowered it. If your team also tracks its deployments, two of DORA's metrics ask the same thing about each deployment: whether it needed a fix. DORA is Google's research program on how teams build and ship software.

It publishes an official guide to its metrics, and the latest version is from January 2026. The first of these two DORA metrics is called change fail rate. It's the share of deployments that need immediate attention afterwards, usually a rollback or an urgent fix. A rollback means putting the previous version back. The second DORA metric is called deployment rework rate.

It's the share of deployments that weren't planned, and only happened because of a problem in production. The data for both DORA metrics comes from your deployment records and your incident records. For change fail rate, divide the deployments that needed an urgent fix or a rollback by all the deployments in the month. To me, a merged pull request is finished work only when its code doesn't need fixing soon after.

That's why the number of merged pull requests never goes on your report alone. DORA's guide also says that "speed and stability are not tradeoffs." That means you can ship often and still need few fixes. There's an earlier video on this channel about the DORA metrics in the AI era. It covers all 5 of them.

3:39 Pair 2: review speed and approved changes fixed later

The second pair is how long pull requests wait for review, next to the approved pull requests that had to be fixed later. Researchers at Carnegie Mellon and Stanford studied one company that pushed its engineers to double their output with AI. After the push, each reviewer had about twice as many pull requests to check. And the share that got any human review fell from 89% to 68%, as AI review tools took over.

Researchers at Deakin University in Australia studied pull requests in open-source projects. In those projects, merged pull requests from AI coding agents needed a follow-up fix about 1.6 times as often as human ones. Both halves of this pair come from your Git host. It keeps the time when someone asked for a review, and the time of each review.

On GitHub, both carry a time stamp. So the wait is a subtraction. But you have to do that subtraction for every pull request, in every repository. For the second number, use the fixes you found for the first pair. This time, look at who approved each fixed change. A quick approval of code that needs fixing a week later only looks fast.

So I'd always look at review speed next to what happened to the code in the weeks after the approval.

4:58 Pair 3: AI spending and what it paid for

The third pair is what you spend on AI, next to the merged changes that spending paid for. The research firm Gartner predicts that by 2028, AI coding costs will be higher than the average developer's salary. It also says most companies can't yet measure that cost against business results. DORA's article from June 2026 suggests a measure for this pair, called the cost per accepted change.

I count an accepted change as one that got merged. DORA also asks what changes when you measure token use by team, by application or by use case. The spending comes from your AI tools' bills and usage pages. The merged changes come from your Git host again. To get the cost per accepted change, divide the month's AI bill by the pull requests merged that month.

So read the spending as a cost, next to what it bought. And never rank anyone by it.

5:54 How to tell AI-written code apart: Claude Code and Cursor records

To compare AI-assisted work with everything else, use the record your AI tools keep of the code they wrote. A popular shortcut is an AI code detector, a tool that guesses from how the code looks. It can scan any finished code, even old code. But a peer-reviewed study from 2025 tested the existing detectors and found that "they all perform poorly."

So don't guess.

[On screen: illustrative example, not real data]

Cursor keeps its own record of AI code. On its Team and Enterprise plans, the team dashboard shows what share of your committed code Cursor's AI wrote. On the Enterprise plan, Cursor also offers an early version of a record for every commit. It shows how many lines its AI wrote, and how many other lines the commit added.

[On screen: illustrative example, not real data]

Claude Code keeps its record differently. By default, every commit it makes carries a line that names the AI as a co-author. If your team kept that default, your Git history already shows which commits it made. A team can switch that line off. So check that it's still on. Now mark each merged pull request as AI-assisted or not, from your AI tool's record.

By hand, that means matching each tool's record to each pull request. One by one. Call a pull request AI-assisted when its record shows that the AI wrote any part of the code. Then show every pair twice. That's how you split each pair into the work AI helped write and everything else, with no guessing.

7:21 What goes on the one page you show your CEO

[On screen: illustrative example, not real data]

The one page you show your CEO holds the 3 pairs, with one row for each service. AI-assisted work sits next to everything else. Each pair gets 2 columns: the number, and the number next to it.

[On screen: illustrative example, not real data]

DORA's guide says its metrics work best for one application or service at a time. The pairs work the same way.

[On screen: illustrative example, not real data]

AI spending gets its own column at the end, marked as a cost. And put last quarter next to this quarter, so the first thing your CEO sees is the trend.

7:51 How to read one engineer's numbers

[On screen: illustrative example, not real data]

To read 1 engineer's numbers, use the same pairs to find where to look. If someone's pull requests often need fixes soon after the merge, start by looking at the work behind those fixes. Pairs one and 2 work for a single person too. One shows the pull requests someone merged, next to the fixes those needed. The other shows the reviews they gave, next to the approved changes that had to be fixed later.

A number about a person can show you where to look. It can't tell you why. So before you decide anything based on it, talk to the person.

[On screen: illustrative example, not real data]

I'm the founder of Aidealy. We make software that measures the work engineering teams ship. And its ranking of engineers, the Aidealy Productivity Rank, is built on pairs like these. Among other things, a merged pull request earns credit. It earns a little more when it increases code quality. Aidealy counts every change merged into your main branch as reaching production.

Points come off when that code has to be fixed or reverted soon after, and more when it breaks sooner. Approving a colleague's change earns credit too, and the approver shares the cost when that change has to be fixed. Keeping a teammate waiting for a review costs points. We track what AI costs for each pull request. We just don't rank by it.

And for the code your engineers commit, Aidealy labels each line AI-written or human-written from the records Cursor and Claude Code keep. You don't configure dashboards. You don't write queries. You ask.

9:22 Where to start

Start here: take the first number in your R&D report and put its pair right next to it. If your report starts with deployments, put the share that needed a fix beside them. You don't need a year of data for that. Your Git host already holds last quarter's merged pull requests, their reviews and the fixes that came after.

And your deployment records hold last quarter's deployments. DORA's guide also says to start small. One of the ways it suggests is a product that already connects to your tools. Then check that your AI tools keep their record. For Claude Code, that's the co-author line. For Cursor, check which plan you're on. Pick one window for fixes after the merge, and keep it every quarter.

Then show the first pairs per service, at your next leadership meeting. Check them again next month. Then repeat. I'd never show the first number of a report without its pair. The sources I used are all linked below, so read them yourself before you take my word for any of this. If you want to keep going, pick one of the videos on the screen now.

Or subscribe for the next one.

See what your R&D is really doing.

Tell us what you're trying to figure out, and we'll get you set up to answer it on your own R&D: your engineers, your AI agents, and the code they ship together.