New research: Leading indicators of AI coding agent effectiveness
New research: Leading indicators of AI coding agent effectiveness
Introducing UPLIFT: Measuring AI Effectiveness in Engineering
Frameworks
Stephen Poletto
Stephen Poletto
ON THIS PAGE
For the past decade, DORA has been the shared language of software delivery performance. But DORA was built for a world in which humans wrote most of the code.
With agents now writing a growing share of software, leaders need new measures to assess the performance of their AI engineering practice.
Leaders need objective ways to assess questions such as:
Can I trust the output enough to stop inspecting every line?
What does it cost to produce a shipped outcome, in tokens and in people?
How much human attention and rework does each outcome burn?
Neither DORA nor total token spend ("tokenmaxxing") can answer these questions.
UPLIFT is a framework built for the AI era. It measures the effectiveness of your AI engineering practice across six dimensions: Uptake, Pace, Leverage, Investment, Focus and Trust.
For the past decade, DORA has been the shared language of software delivery performance. But DORA was built for a world in which humans wrote most of the code.
With agents now writing a growing share of software, leaders need new measures to assess the performance of their AI engineering practice.
Leaders need objective ways to assess questions such as:
Can I trust the output enough to stop inspecting every line?
What does it cost to produce a shipped outcome, in tokens and in people?
How much human attention and rework does each outcome burn?
Neither DORA nor total token spend ("tokenmaxxing") can answer these questions.
UPLIFT is a framework built for the AI era. It measures the effectiveness of your AI engineering practice across six dimensions: Uptake, Pace, Leverage, Investment, Focus and Trust.
Why UPLIFT?
Why UPLIFT?
Durable dimensions, evolving KPIs
While the six dimensions of UPLIFT are stable, the KPIs beneath them should change as you mature. Like SPACE before it, UPLIFT treats productivity as multidimensional and its metrics as context-dependent. What's different is that in the agentic era, the right metrics shift quickly as an organization's relationship with agents changes.
A team who has not yet invested in a robust harness and automated verification systems should not track auto-merge rate - it is not yet ready. A team running parallel agents in a cloud factory triggered off tickets has outgrown seat utilization. So each dimension comes with illustrative KPIs at three stages:
Assisted. AI is autocomplete and chat. Humans drive every change. The primary question is whether people are using it, and what it costs.
Agentic. Interactive agents are the norm. A human steers the session and reviews most diffs. The primary question is yield: how much agent work ships, and how much steering does it take (both pre-PR and in code review).
Factory. Agents becomes asynchronous. Work is kicked off by tickets, alerts and CI, many sessions run in parallel, and low-risk changes merge with sampled or no review. The primary question is autonomy: how much can the team ship without a human in the loop?
Most organizations have teams across stages, with a few teams running ahead of the rest. The KPIs should flex to earlier and more mature teams to shape behavior and goals in a stage-appropriate way.
Durable dimensions, evolving KPIs
While the six dimensions of UPLIFT are stable, the KPIs beneath them should change as you mature. Like SPACE before it, UPLIFT treats productivity as multidimensional and its metrics as context-dependent. What's different is that in the agentic era, the right metrics shift quickly as an organization's relationship with agents changes.
A team who has not yet invested in a robust harness and automated verification systems should not track auto-merge rate - it is not yet ready. A team running parallel agents in a cloud factory triggered off tickets has outgrown seat utilization. So each dimension comes with illustrative KPIs at three stages:
Assisted. AI is autocomplete and chat. Humans drive every change. The primary question is whether people are using it, and what it costs.
Agentic. Interactive agents are the norm. A human steers the session and reviews most diffs. The primary question is yield: how much agent work ships, and how much steering does it take (both pre-PR and in code review).
Factory. Agents becomes asynchronous. Work is kicked off by tickets, alerts and CI, many sessions run in parallel, and low-risk changes merge with sampled or no review. The primary question is autonomy: how much can the team ship without a human in the loop?
Most organizations have teams across stages, with a few teams running ahead of the rest. The KPIs should flex to earlier and more mature teams to shape behavior and goals in a stage-appropriate way.
The Six Dimensions
The Six Dimensions
The six dimensions of UPLIFT
U: Uptake
Uptake measures how much work flows through your organization's goal AI tooling, so its instrument moves as that frontier moves. Early on, you might track indicators that engineers are reflexively reaching for AI to assist their work. As agentic development becomes the norm, you may start tracking what share of PRs are produced by background/cloud agents.
The question: How much of our work flows through our goal AI tooling?
Stage | Illustrative KPIs |
|---|---|
Assisted | Agent WAU (% of engineers) |
Agentic | % PRs agent-authored; AI code ratio (% of merged LoC) |
Factory | % PRs from background/cloud agents |
P: Pace
Pace asks how much faster agents are making delivery end to end. It's valuable to track DORA-style PR cycle time and PR throughput. At the agentic stage, watch time spent waiting on review: it is where faster code generation usually stalls. In a factory, measure ticket-to-merge lead time and outcomes shipped per engineer-week.
The question: How fast does work move from intent to production?
Stage | Illustrative KPIs |
|---|---|
N/A | PR cycle time (p50); PR throughput (PRs/dev/wk); review wait time (p50); ticket-to-merge lead time (p50); epic/initiative cycle time (p50) |
L: Leverage
Leverage measures how much human attention is required for each shipped outcome. This includes the author steering the agent, and attention from reviewers downstream.
At the agentic stage, track turns to merge (prompts per 1,000 merged AI lines) and review cycles per PR. In a factory, measure how frequently you can ship work with zero human intervention.
Also track how many concurrent sessions each engineer runs. Engineers can only work in parallel when each session is autonomous enough to leave alone and switch to something else; if agents need babysitting, attention stays pinned to one session at a time. That makes parallelism both a signal of autonomy and a multiplier on leverage: more work shipped per unit of human attention.
The question: How much human attention does each outcome consume?
Stage | Illustrative KPIs |
|---|---|
Assisted | % engineers using agent mode |
Agentic | Turns per AI kLoC; review cycles per AI kLoC; concurrent sessions per engineer |
Factory | Zero-touch PR rate (%); auto-merge rate (%) |
I: Investment
Investment measures the cost of each outcome delivered.
Total and per-engineer spend are fine for budgeting in the assisted stage. Agentic teams should track cost per merged PR and session yield (e.g. the share of sessions and dollars that end in merged code rather than rework, abandonment or closed PRs).
Factories should measure fully loaded cost per outcome (tokens plus human hours) by work type and by model, which shows whether model selection is paying off.
The question: What does each shipped outcome cost?
Stage | Illustrative KPIs |
|---|---|
Assisted | Token spend ($ per engineer per month) |
Agentic | $ per merged PR; session yield (% of sessions merged) |
Factory | Fully loaded $ per outcome (tokens + human hours), by work type and model |
F: Focus
Focus asks where the newly augmented capacity of the team is being directed.
Start by seeing spend by team and repo. Then track the innovation ratio, the share of spend building new capability rather than sustaining existing systems.
Mature organizations add a strategic alignment ratio (spend that ladders up to declared priorities) and quantify the maintenance drag: the follow-up fixes and rework attached to each feature after it ships.
The question: Where is AI capacity being spent?
Stage | Illustrative KPIs |
|---|---|
Assisted | Token spend ($) by team and repo |
Agentic | Innovation ratio (% of spend on new capability) |
Factory | Strategic alignment ratio (% of spend on declared priorities); maintenance tail ($ per feature) |
T: Trust
Trust evaluates whether the quality of the work is good.
This means looking at DORA-style metrics like change failure rate and incident count, and also defect escape rates.
Trust is the gate on Leverage: autonomy should expand only as fast as Trust can prove it is safe.
The question: In our agentic system producing high-quality outputs?
Stage | Illustrative KPIs |
|---|---|
N/A | Change failure rate (%); Escaped defects @ 7d per AI kLoC |
The six dimensions of UPLIFT
U: Uptake
Uptake measures how much work flows through your organization's goal AI tooling, so its instrument moves as that frontier moves. Early on, you might track indicators that engineers are reflexively reaching for AI to assist their work. As agentic development becomes the norm, you may start tracking what share of PRs are produced by background/cloud agents.
The question: How much of our work flows through our goal AI tooling?
Stage | Illustrative KPIs |
|---|---|
Assisted | Agent WAU (% of engineers) |
Agentic | % PRs agent-authored; AI code ratio (% of merged LoC) |
Factory | % PRs from background/cloud agents |
P: Pace
Pace asks how much faster agents are making delivery end to end. It's valuable to track DORA-style PR cycle time and PR throughput. At the agentic stage, watch time spent waiting on review: it is where faster code generation usually stalls. In a factory, measure ticket-to-merge lead time and outcomes shipped per engineer-week.
The question: How fast does work move from intent to production?
Stage | Illustrative KPIs |
|---|---|
N/A | PR cycle time (p50); PR throughput (PRs/dev/wk); review wait time (p50); ticket-to-merge lead time (p50); epic/initiative cycle time (p50) |
L: Leverage
Leverage measures how much human attention is required for each shipped outcome. This includes the author steering the agent, and attention from reviewers downstream.
At the agentic stage, track turns to merge (prompts per 1,000 merged AI lines) and review cycles per PR. In a factory, measure how frequently you can ship work with zero human intervention.
Also track how many concurrent sessions each engineer runs. Engineers can only work in parallel when each session is autonomous enough to leave alone and switch to something else; if agents need babysitting, attention stays pinned to one session at a time. That makes parallelism both a signal of autonomy and a multiplier on leverage: more work shipped per unit of human attention.
The question: How much human attention does each outcome consume?
Stage | Illustrative KPIs |
|---|---|
Assisted | % engineers using agent mode |
Agentic | Turns per AI kLoC; review cycles per AI kLoC; concurrent sessions per engineer |
Factory | Zero-touch PR rate (%); auto-merge rate (%) |
I: Investment
Investment measures the cost of each outcome delivered.
Total and per-engineer spend are fine for budgeting in the assisted stage. Agentic teams should track cost per merged PR and session yield (e.g. the share of sessions and dollars that end in merged code rather than rework, abandonment or closed PRs).
Factories should measure fully loaded cost per outcome (tokens plus human hours) by work type and by model, which shows whether model selection is paying off.
The question: What does each shipped outcome cost?
Stage | Illustrative KPIs |
|---|---|
Assisted | Token spend ($ per engineer per month) |
Agentic | $ per merged PR; session yield (% of sessions merged) |
Factory | Fully loaded $ per outcome (tokens + human hours), by work type and model |
F: Focus
Focus asks where the newly augmented capacity of the team is being directed.
Start by seeing spend by team and repo. Then track the innovation ratio, the share of spend building new capability rather than sustaining existing systems.
Mature organizations add a strategic alignment ratio (spend that ladders up to declared priorities) and quantify the maintenance drag: the follow-up fixes and rework attached to each feature after it ships.
The question: Where is AI capacity being spent?
Stage | Illustrative KPIs |
|---|---|
Assisted | Token spend ($) by team and repo |
Agentic | Innovation ratio (% of spend on new capability) |
Factory | Strategic alignment ratio (% of spend on declared priorities); maintenance tail ($ per feature) |
T: Trust
Trust evaluates whether the quality of the work is good.
This means looking at DORA-style metrics like change failure rate and incident count, and also defect escape rates.
Trust is the gate on Leverage: autonomy should expand only as fast as Trust can prove it is safe.
The question: In our agentic system producing high-quality outputs?
Stage | Illustrative KPIs |
|---|---|
N/A | Change failure rate (%); Escaped defects @ 7d per AI kLoC |
Putting UPLIFT into Practice
Putting UPLIFT into Practice
Putting UPLIFT into practice
To put this framework in practice, decide at least one (or more) KPIs per dimension.
Place your team on the maturity curve.
Pick one KPI per dimension for your stage. Start from the illustrative tables and swap in what you can instrument reliably today. A simple metric you trust beats a sophisticated one you don't.
Give each KPI a goal and an owner. Review them on a regular cadence, and tie each change to the harness, environment or model defaults back to the KPI it should move.
Graduate when a KPI saturates. Once a metric stops being meaningful, such as agent usage near 100% or nearly every PR agent-authored, it has done its job. Keep it as a health check and adopt the next-stage KPI for that dimension, so the dashboard points at the next constraint.
Putting UPLIFT into practice
To put this framework in practice, decide at least one (or more) KPIs per dimension.
Place your team on the maturity curve.
Pick one KPI per dimension for your stage. Start from the illustrative tables and swap in what you can instrument reliably today. A simple metric you trust beats a sophisticated one you don't.
Give each KPI a goal and an owner. Review them on a regular cadence, and tie each change to the harness, environment or model defaults back to the KPI it should move.
Graduate when a KPI saturates. Once a metric stops being meaningful, such as agent usage near 100% or nearly every PR agent-authored, it has done its job. Keep it as a health check and adopt the next-stage KPI for that dimension, so the dashboard points at the next constraint.
More articles
Learn More
Frequently asked questions
Frequently asked questions
What counts as a contributor?
A contributor is anyone who commits code or is tracked across your connected tools (e.g., GitHub, GitLab). Non-engineering users like PMs or executives who only view dashboards are free.
Do you offer volume discounts?
Is there a minimum contract term?
Do you offer services and support?
Can I try Span before committing?
Let's talk about where your AI program is.
platform
Resources
© 2026 Attuned Inc.
Let's talk about where your AI program is.
platform
© 2026 Attuned Inc.
Let's talk about where your AI program is.
platform
© 2026 Attuned Inc.