# Uplift Modeling Isolates a Treatment Effect From a Tracked Baseline ## Claim Once a KPI is being tracked, comparing a control period against a treatment period — uplift modeling — is what actually attributes a measured delta to the new input rather than to other confounders. <!-- atomicity: 1 idea ✓ --> ## Scope & conditions - Only applies once [[3. Resources/Permanent Notes/AI ROI Requires a Pre-Existing Performance Baseline]] is satisfied — a tracked KPI has to exist before there's anything to run a control/treatment comparison against. - The claim is that this is the right *shape* of method (control period vs. treatment period); it doesn't specify the estimator (propensity scores, two-model approach, class-transformation) or how to handle confounders — that's a verification gap, not a settled method. - Borrowed from biostatistics/marketing science; the source names it without walking through the mechanics, so treat the label as a pointer to verify, not a fully specified technique. ## Evidence Penn proposes this as the concrete follow-on once an organization has a "definition of done" per role: measure a control period, measure a treatment period after AI is introduced, and treat the delta between them as AI's attributable effect — rather than crediting AI with a change that a raw before/after comparison can't actually isolate from everything else that changed in the same window. > there's an entire branch of statistics called uplift modeling that we borrow from biostatistics [...] I know that in this period there was a control, in this period there was a treatment, and the treatment showed this delta compared to this control. ## Links - [[AI & the Future of Knowledge Work MOC]] — the method that turns a tracked number into an actual attribution - [[3. Resources/Permanent Notes/AI ROI Requires a Pre-Existing Performance Baseline]] — because extends; this note is the specific method that makes the baseline note's claim actionable once satisfied. - [[3. Resources/Permanent Notes/A Metric Is Vanity Only Against Your Own Outcome Data]] — because applies-in; a control/treatment delta is exactly the kind of test-against-your-own-data that distinguishes a real signal from a vanity read. - [[3. Resources/MOCs/AI Cost & Tokenomics MOC]] — because applies-in; this is the measurement mechanism a defensible AI cost/value claim would need to cite. - [[3. Resources/MOCs/Evidence & Claims Literacy MOC]] — the positive method for attributing a measured delta to its cause - [[3. Resources/Permanent Notes/Comparing Adopters With Non-Adopters Credits the Tool for the Adopters' Prior Trend]] — extended by: adopters of a tool were already on a steeper trend than non-adopters, so an adopter comparison credits the tool for that trend. ## Tests / falsifiers - Verify Penn's usage against a standard causal-inference uplift-modeling source (propensity/treatment-control framing) — if it diverges materially, this note's method description needs correction. - Find a published AI-adoption ROI claim that used a genuine control-vs-treatment comparison and check whether the reported delta survives scrutiny for confounders (seasonality, concurrent process changes, staffing shifts). ## Maintenance - Last tended: 2026-08-16 - Next review: 2026-11-16 ## Source - [[3. Resources/Readwise/Podcasts/You Ask, I Answer Measuring AI ROI Beyond Time Saved]] — Christopher Penn, "You Ask, I Answer" YouTube series, 2026-08-11.