> [!warning] Names unreliable in this capture > Auto-transcript renders the speaker as "Ununice Hayes," company as "Aquamo," client as "Toshima." All three are likely mis-transcriptions and none could be verified. Do not cite any name from this note without checking the source video. > [!question] Lingering Questions > - Can the Amazon "hero rank" leaderboard story be confirmed independently, including the May withdrawal date? It carries the talk's central lesson and is entirely uncited. > - What was the measurement baseline for the textile case study, and over what period? None given. > [!tip] Article Ideas > - "Your AI metric is a Goodhart trap" — built on the Amazon token-leaderboard story, which is current, concrete, and involves a sophisticated organization. > [!link] Cross Reference > - [[3. Resources/Permanent Notes/Goodharts Law Becomes Automatic Under AI Tool Use]] — the Amazon leaderboard is a fresh, concrete instance of this exact mechanism. > - [[3. Resources/Permanent Notes/Prevented Work Is Invisible to Velocity Metrics]] — same shape as "DORA measures how fast code moves, not how knowledge flows": a throughput metric structurally blind to the labor/value that actually matters. > [!quote] Vocabulary > - **DORA** — DevOps Research and Assessment's four delivery metrics (deployment frequency, lead time, change failure rate, time to restore) > - **bus factor** — number of people who could leave before a component becomes unmaintainable; proposed here as a trackable knowledge-retention metric > - **token maxing** — running pointless tasks purely to climb a token-consumption leaderboard ## Metadata - Source: [[3. Resources/Readwise/Articles/Beyond Pull Requests Quantifying the ROI of InnerSource|Beyond Pull Requests - Quantifying the ROI of InnerSource]] - Category: Video (YouTube, InnerSource Commons community call) - Published: 2026-07-31 ## Author's thesis InnerSource ROI is hard to demonstrate because organizational measurement habits count throughput (deployments, PRs, now tokens) rather than knowledge distribution, which DORA cannot see by design; the fix is measuring outcomes (reuse, cross-team knowledge sharing, bus factor) against a rebuild-cost counterfactual, and the same legible-conventions practice that lets InnerSource work is what makes an organization ready for agents as contributors. ## Key arguments 1. **DORA measures pipeline speed, not knowledge distribution** — "it tells you how fast code moves, not how knowledge flows," which is precisely the dimension InnerSource claims to change and precisely why ROI proofs fail against delivery metrics. 2. **The metric unit keeps changing; the error does not** — lines of code, then PRs, now tokens burned. Amazon's internal "hero rank" leaderboard ranked engineers by AI tokens consumed; engineers ran pointless tasks to climb it, compute costs spiked, the leaderboard was killed in May and replaced with a shipped-code measure. 3. **Agents are contributors, so contributor-readiness is agent-readiness** — legible conventions, executable acceptance criteria, and tests committed before code serve an unfamiliar human team and an agent identically. ## Claim atoms ### DORA measures pipeline speed, not knowledge distribution > [!quote] ID:rw_01kywpb6dzgxd8bsqhk0fghb8b > it tells you how fast um code moves not how knowledge flows. *Where:* ~00:02:30. *Why:* precisely identifies what the industry-standard delivery framework structurally cannot observe — the exact dimension InnerSource claims to change, which is why its ROI proofs fail against DORA. ### The metric unit keeps changing; the error does not > [!quote] ID:rw_01kywpb6dzgxd8bsqhk0fghb8b > Amazon built a an internal a leaderboard called u hero rank ranking developers by how many AI tokens they burned. Within weeks engineers were token maxing running pointless tasks just to climb the rankings. *Where:* ~00:05:31. *Why:* a fresh, current Goodhart instance from a sophisticated organization, with the correct generalization attached — lines of code, PRs, tokens are all activity proxies for the same underlying error. ### Agents are contributors, so contributor-readiness is agent-readiness > [!quote] ID:rw_01kywpb6dzgxd8bsqhk0fghb8b > the key insight here is that agents are contributors. So same inner source patterns as humans um uh low on boarding costs, clear ownership, executable uh acceptance criteria. *Where:* ~00:10:32. *Why:* the most forward-looking claim in the talk — legible conventions and explicit acceptance criteria serve unfamiliar humans and agents identically. ## Permanent candidates - **A delivery-throughput metric cannot see whether knowledge spread** — the DORA critique, generalizable well past InnerSource; strongest candidate. ## Processing checklist - [ ] Verify the Amazon "hero rank" leaderboard story and its May withdrawal - [ ] Try the rebuild-cost counterfactual ("if we didn't share this, how much time would we lose rebuilding it?") on one shared component - [x] Connect into the relevant MOC — [[3. Resources/MOCs/DevRel MOC]] ## Links - [[3. Resources/MOCs/DevRel MOC]] — outcome-vs-activity metric design is core DevRel measurement territory. - [[3. Resources/MOCs/Open Source Governance MOC]] — InnerSource-as-agent-readiness reframes contributor-onboarding work as governance infrastructure. - [[3. Resources/Permanent Notes/A Delivery-Throughput Metric Cannot See Whether Knowledge Spread]] — the Permanent note this Literature note generated.