# AI ROI Requires a Pre-Existing Performance Baseline ## Claim An organization can't quantify what AI changed because it never measured its own performance before AI arrived, so there's nothing to compare the "after" against. <!-- atomicity: 1 idea ✓ --> ## Scope & conditions - Holds when an org has no documented pre-AI KPI, task-tracking, or "definition of done" for the role in question. - Breaks when the org already tracks a stable baseline metric — AI ROI becomes measurable there via uplift modeling (treatment vs. control period). - Doesn't establish which KPI is the right one to track, only that some tracked baseline must exist before an "AI ROI" claim is meaningful. ## Evidence Christopher Penn, answering a viewer's question about measuring AI's organizational ROI beyond individual time saved, reframes the question itself: without existing task, time, or output tracking, AI doesn't introduce a new measurement problem — it makes a pre-existing blind spot invisible instead of visible. His fix is Trust Insights' "Five P" framework (Purpose, People, Process, Platform, Performance), where Performance is defined as the quantitative "definition of done." > If you have no idea what your performance metrics are, you do not have an AI problem [...] you have a basic measurement problem. ## Links - [[AI & the Future of Knowledge Work MOC]] — why most organisations cannot answer what AI changed for them, and never could - [[3. Resources/Permanent Notes/A Metric Is Vanity Only Against Your Own Outcome Data]] — because extends; same author's broader claim that a metric only means something once tested against your own outcome data — this note names the specific outcome data an AI-ROI claim needs: the pre-AI baseline. - [[3. Resources/Permanent Notes/An Agent Evaluation Without an Incumbent Baseline Cannot Establish Value]] — because contrasts; that note requires a baseline one level down (agent vs. the incumbent tool it replaces), this one requires a baseline one level up (org performance vs. its own past). - [[3. Resources/MOCs/AI Cost & Tokenomics MOC]] — because applies-in; ROI claims are a cost/value question that this claim gates before any number is trustworthy. - [[3. Resources/Permanent Notes/Comparing Adopters With Non-Adopters Credits the Tool for the Adopters' Prior Trend]] — contrasts: adopters of a tool were already on a steeper trend than non-adopters, so an adopter comparison credits the tool for that trend. ## Tests / falsifiers - Find an organization with zero pre-AI baseline metrics that still produces a credible, externally validated AI ROI figure — this would falsify the claim. - Track whether orgs that build a baseline specifically to satisfy this argument actually produce more defensible ROI numbers than orgs using looser before/after anecdote. ## Maintenance - Last tended: 2026-08-16 - Next review: 2026-11-16 ## Source - [[3. Resources/Readwise/Podcasts/You Ask, I Answer Measuring AI ROI Beyond Time Saved]] — Christopher Penn, "You Ask, I Answer" YouTube series, 2026-08-11.