Appendix B: Stage-by-Stage Metrics Guide
This appendix is reference material for Chapter 9. It maps what to measure, what to ignore, and what tools to use at each stage of the CTO role described in Chapter 1.
The governing principle is Lena Reinhard’s: "Metrics create incentives, that create behaviours, that create culture."[1] Every metric you introduce will change how the team works — sometimes in ways you intended, sometimes not. The second principle is Will Larson’s: "The number one measurement risk is measuring nothing because you’re trying to measure everything."[2] At every stage, the CTO should measure the minimum that produces actionable insight and resist the pressure to instrument more.
In 2025, DORA abandoned the four-tier performance model (elite, high, medium, low) that practitioners had been citing as benchmarks for six years, replacing it with seven behavioural archetypes.[3] The shift confirmed what Chapter 9 argues: DORA clusters were never targets. They were diagnostic patterns that changed every year. The benchmarks in this appendix are reference points for calibration, not goals to optimise toward.
Stage 1: Coder — Seed, 1–5 Engineers
What to measure. Three things, as Chapter 9 recommends: deployment frequency, recovery speed, and team morale. Deployment frequency is what Charity Majors calls "the heartbeat of your company. It should be regular. It should happen consistently. It should be boring."[4] At this stage, aim to deploy at least daily — LinearB’s benchmark data shows startups deploy 18% faster than enterprises.[5] Recovery speed means: when something breaks in production, how long does it take to fix? Track this informally. Team morale means: in your weekly one-to-ones, are people energised or frustrated? Reinhard’s guidance for this stage: "Everyone works more or less on everything and the team size is small enough so that everyone still has visibility into what’s going on. Use a rudimentary project management tool. At this stage, anything more would probably add too much overhead and slow teams down."[1]
What not to measure. Anything at the individual level. No lines of code, no pull requests per person, no story points, no velocity. Martin Fowler’s argument remains definitive: "Any true measure of software development productivity must be based on delivered business value. False measures only make things worse."[6] Larson is blunter about the tools that promise individual productivity measurement: "I’m personally convinced that these companies are selling products that harm the companies that use them rather than help them."[2]
Goal-setting. Informal. Dave Bailey’s "one metric that matters" principle applies: identify the single number that tells you whether the product is working, and orient the team around it.[7] If you introduce OKRs at this stage, you are spending process time that should be spent shipping.
Tools. GitHub or GitLab built-in analytics. No dedicated metrics platform. Cost: free.
Board reporting. At seed, you are unlikely to have a board that demands engineering metrics. If asked, report two numbers: how often you ship and whether production is stable. Frame engineering investment as a percentage of total spend — at seed, this is typically 40–95% of revenue, which is expected.[8]
Healthcare addendum. Track patching cadence (time from vulnerability disclosure to patch applied). Begin SOC 2 readiness documentation if you are pursuing enterprise healthcare customers. Audit log retention should be configured from day one — HIPAA requires six years minimum (Chapter 4).
Stage 2: Manager — Series A, 5–15 Engineers
What to measure. Everything from Stage 1, plus three additions. First, a developer experience signal — a quarterly or every-eight-weeks survey. Laura Tacho, CTO of DX, recommends this cadence: "Every 8 weeks is a good cadence. Running the survey twice a year or less doesn’t provide enough data."[9] At this team size, keep surveys short — five to seven questions on a Likert scale covering satisfaction, friction points, and confidence in the codebase. Tacho warns about anonymity at small scale: "Even basic info like job title can blow cover in small teams."[9] Second, track the ratio of new-capability work to maintenance work. If more than 40% of engineering capacity is going to maintenance, your technical debt load is affecting your ability to compete — the threshold Chelsea Troy identifies in Chapter 5. Third, begin tracking cycle time: the elapsed time from first commit to production deployment. LinearB’s benchmark for "good" cycle time is under 72 hours at the 75th percentile.[5]
What not to measure. Velocity as a cross-team comparison tool. Story points are a planning aid for individual teams, not a productivity metric. Do not rank engineers against each other on any quantitative dimension. Gergely Orosz warns: "Every new thing you start measuring will lead to engineers optimising to make that measure look better. A problem with measurements focused on effort and output is that they transform the engineering culture to one where 'slack time' is frowned upon."[10]
Goal-setting. Bailey’s lightweight OKR adaptation: four-to-six-week cycles, one or two key results per objective, scoped to customers and team.[7] Alternatively, Spotify’s DIBB framework (Data, Insight, Belief, Bet) if the company is product-led.[11] Marty Cagan’s diagnostic applies: "If a company is still using feature teams, then the OKR technique is going to be a cultural mismatch."[12]
Tools. Swarmia Lite (~$240/developer/year) or LinearB free tier for delivery metrics. A simple developer experience survey — DX’s platform, or a five-question homebrew form. Cost: $0–$3,600/year.
Board reporting. Report three categories, following Tomasz Tunguz’s recommendation: reliability (percentage of requests served successfully), availability (percentage served within latency targets), and a quality proxy such as support-ticket-to-active-user ratio.[13] Camille Fournier’s principle: never report Agile internals to the board. "The engineering department is not a standalone function, and it’s certainly not a factory cranking out code widgets."[14] Frame engineering capacity in business terms: "We are spending X% of engineering time on new capabilities, Y% on maintenance, and Z% on debt reduction."
Healthcare addendum. Add mean time to detect (MTTD) and mean time to contain (MTTC) for security incidents. Begin formal access-control reviews. If not already in progress, start the SOC 2 Type 2 audit cycle (Chapter 4). Consider compliance automation — Vanta or Drata — which maps SOC 2 controls to HIPAA requirements and reduces manual audit burden.
Stage 3: Director — Series B, 15–50 Engineers
What to measure. Everything from Stage 2, plus four additions. First, the full set of DORA metrics — deployment frequency, lead time for changes, change failure rate, and time to recover — now tracked from production systems rather than estimated in conversation. Use these as a diagnostic, not a scorecard: identify the most constrained metric and address that constraint (Chapter 9). Second, investment allocation: what percentage of engineering time goes to new capabilities, maintenance, and debt reduction? DX Core 4 benchmarks show that top-quartile tech companies spend roughly 6% more of their time on new capabilities than non-tech companies.[15] Swarmia recommends a Balance Framework target: 60% new capabilities and improvements, 15% developer productivity, 10% keeping-the-lights-on.[16] Third, cross-team delivery metrics: are handoffs between teams creating bottlenecks? Fourth, SLA compliance for customer-facing systems — 99.9% uptime (three nines, roughly 8.76 hours of downtime per year) is baseline for most SaaS; 99.99% (four nines, roughly 52 minutes per year) is the target for critical paths.
What not to measure. Do not treat the DORA tier labels as fixed targets — they shifted every year even before DORA abandoned them. Do not let board-level metrics cascade into team-level pressure. Kent Beck’s warning: "Trying to meet that need through metrics or even surveys inevitably poisons the data on which productivity measurement relies."[17] Hillel Wayne extends the point: "Even 100% honest pursuit of a metric, taken far enough, is harmful to your goals, and this is an inescapable consequence of the difference between metrics and values."[18]
Goal-setting. OKRs may begin to earn their keep at this stage, provided the engineering team is empowered to determine how to solve the problems they are assigned rather than being told what to build (Cagan’s distinction from Chapter 9). If the team is still operating as a feature factory, OKRs are a bureaucratic layer on a delivery function.
Tools. LinearB Business (~$49/month per contributor), Swarmia full platform, or DX for developer experience measurement. Compliance automation (Vanta, Drata, or Secureframe) becomes essential at this stage for healthcare companies. Cost: $9,000–$30,000/year.
Board reporting. Tunguz’s three metrics (reliability, availability, incident rate) remain the core.[13] Add investment allocation as a pie chart — the board needs to see where engineering time goes and whether the ratio is shifting in the right direction. Add a hiring and retention section: open roles, time to fill, regrettable attrition. Translate velocity into business language: not "we shipped 12 features" but "we reduced customer onboarding time from three weeks to four days." R&D as a percentage of revenue should be in the 25–35% range at this stage.[8]
Healthcare addendum. SOC 2 Type 2 audit on a 6–12-month cycle. Incident response time benchmarks: target detection within hours, containment within 24 hours. Full compliance dashboard tracking patching cadence, access-control reviews, encryption status, and tenant data isolation. Average healthcare data breach cost is $10.93 million (IBM Security) — the board needs to understand that compliance investment is risk reduction with a quantifiable upside.
Stage 4: Strategist — Series C+, 50+ Engineers
What to measure. Full DORA, SPACE, and DX Core 4 infrastructure. DX Core 4 combines one system metric (pull requests per engineer — top-quartile benchmark is 4.3 per week at small-to-medium tech companies), one perceptual metric (Developer Experience Index), one quality metric (change failure rate — benchmark range 2.8–4.6%), and one business-alignment metric (percentage of time on new capabilities versus maintenance).[15] Abi Noda’s benchmark: "For every one-point increase in DXI score, you save about 10 minutes per week per engineer."[15] At 50+ engineers, that is measurable in headcount-equivalent terms and becomes a compelling board narrative. Add financial metrics: R&D as a percentage of revenue (20–35% at this stage is typical; median at IPO is 23–26%), revenue per employee ($130,000+ for private SaaS, $283,000+ for public SaaS), and the Rule of 40 (growth rate + profit margin ≥ 40%).[8]
What not to measure. Individual productivity scores. McKinsey’s 2023 proposal to measure individual developer output provoked a unified backlash from Orosz, Beck, Noda, and Forsgren — the practitioners who created the frameworks McKinsey was citing.[10] The consensus was unambiguous: individual output metrics destroy the collaboration that makes teams productive. Tacho’s principle: "What I’m not measuring is also important. I’ve made a choice here not to directly measure things like PR count, commits, or even story points."[9]
Goal-setting. Full OKR infrastructure with business-outcome framing, or DIBB at scale. Board-level reporting that translates engineering metrics into the language of the Rule of 40 and capital efficiency.
Tools. Jellyfish, Faros AI, Cortex or CorralData for engineering intelligence and financial alignment. Full compliance platform. Custom executive dashboards. Cost: enterprise pricing.
Board reporting. The board at this stage is evaluating engineering as a financial function. Report: R&D efficiency (revenue per engineer, R&D as percentage of revenue), delivery health (DORA metrics in trend-line form, not single data points), team health (developer experience trend, attrition), and risk posture (compliance status, incident trends, security metrics). Fournier’s framing applies at every stage but is most critical here: the CTO who presents engineering as a cost centre will be managed as one. The CTO who presents engineering as a value-creation engine — with the metrics to prove it — will be treated as a strategic partner.
Healthcare addendum. Full compliance dashboard with board-level reporting. If pursuing FDA clearance for software as a medical device (SaMD), add regulatory submission metrics. Continuous monitoring across all SOC 2 trust service criteria. Breach notification readiness drills. Target mean time to restore critical systems under 24 hours — Censinet’s benchmark for healthcare organisations using automation is 4.2 hours.[19]
What to Avoid at Every Stage
Five measurement anti-patterns, each attributed to the practitioner who named it most precisely:
Individual productivity scoring. Larson: tools that measure individual developer output "harm the companies that use them."[2]
Lines of code, velocity points, or PRs as productivity proxies. Fowler: "Code that’s well designed and factored will be shorter because it eliminates the duplication. Copy and paste programming leads to high LOC counts and poor design."[6]
DORA clusters as fixed benchmarks. The 2025 report abandoned them. They were always statistical artefacts of annual survey data, not performance targets.[3]
Measurement disconnected from business outcomes. Larson: "Don’t ask your CEO to evaluate engineering through optimisation metrics."[2] If the metric cannot be translated into language the board understands, it is an internal diagnostic, not a reporting tool.
Gaming through honest pursuit. Wayne: "We have metrics in the first place because what we actually care about is nonquantifiable."[18] The team that optimises for the metric at the expense of the value the metric was supposed to represent has succeeded at measurement and failed at engineering.
This appendix provides the reference frame. Chapter 9 provides the argument for why these metrics matter and the communication language for presenting them. Chapter 6 provides the deployment infrastructure that makes the delivery metrics possible. Chapter 5 provides the debt framework that gives the maintenance-versus-new-capability ratio its meaning.