# A Delivery-Throughput Metric Cannot See Whether Knowledge Spread ## Claim A framework measuring how fast code moves through a pipeline is structurally unable to observe who else now knows how to do the work — which is exactly the dimension knowledge-sharing initiatives claim to change, and exactly why they can't prove their value using delivery metrics. <!-- atomicity: 1 idea ✓ --> ## Scope & conditions - Holds specifically for throughput/delivery-pipeline metrics (deployment frequency, lead time, change failure rate, time to restore) assessed against knowledge-distribution goals — not a general claim that these metrics are bad, only that they measure a different axis. - Doesn't establish that knowledge distribution is unmeasurable, only that the standard delivery framework wasn't built to measure it — the proposed alternative (bus factor, reuse rate, cross-team adoption) is a different instrument, not an absence of one. - Sharpens under AI-assisted development, where the unit that gets counted (PRs, then tokens) keeps changing while the underlying error — counting activity as a proxy for value — does not. ## Evidence "DORA... tells you how fast code moves, not how knowledge flows." The clip's central diagnostic: InnerSource claims to change who can do what across a team, and no deployment-frequency or lead-time number can register that change happening. The same activity-vs-outcome error recurs with a new unit each cycle — Amazon's internal "hero rank" leaderboard ranked engineers by AI tokens burned; within weeks engineers were "token maxing," running pointless tasks to climb the ranking, until compute costs spiked and the leaderboard was killed and replaced with a shipped-code measure. Lines of code, then pull requests, now tokens — the unit changes, the structural blindness to actual knowledge transfer does not. ## Links - [[DevRel MOC]] — the DevRel outcome no velocity framework can observe, which is why it goes unfunded - [[Open Source Governance MOC]] — why velocity frameworks are blind to the contributor capacity a project actually runs on - [[3. Resources/Permanent Notes/Prevented Work Is Invisible to Velocity Metrics]] — because extends; both name a specific kind of labor a throughput metric structurally can't see — that note's is defect-prevention, this note's is knowledge transfer, and both are invisible by construction rather than by miscalibration. - [[3. Resources/Permanent Notes/A Metric Is Vanity Only Against Your Own Outcome Data]] — because applies-in; DORA isn't vanity in general, it's vanity specifically against a knowledge-distribution outcome it was never built to predict. - [[3. Resources/Permanent Notes/Goodharts Law Becomes Automatic Under AI Tool Use]] — because evidence-for; the Amazon token-leaderboard story is a fresh, concrete, dated instance of that note's exact mechanism, from a different source entirely. ## Tests / falsifiers - Check whether DORA metrics show any measurable correlation with independent knowledge-distribution measures (bus factor, cross-team contribution rate) in a real dataset — a strong correlation would weaken this claim's force. - Try the rebuild-cost counterfactual ("if we didn't share this, how much time would we lose rebuilding it?") on one shared component and see whether it produces a number that survives finance scrutiny better than an activity count would. ## Maintenance - Last tended: 2026-08-16 - Next review: 2026-11-16 ## Source - [[3. Resources/Readwise/Articles/Beyond Pull Requests Quantifying the ROI of InnerSource|Beyond Pull Requests - Quantifying the ROI of InnerSource]] — InnerSource Commons community call, 2026-07-31; the DORA critique and the Amazon "hero rank" token-leaderboard account (unsourced in the talk, worth independent verification).