# A Small Sample Cannot Prove a Sitewide Decay Claim ## Claim Scanning a small, non-random slice of a large content library and extrapolating the result to the whole library proves the sample exists, not that the whole population shares its properties. <!-- atomicity: 1 idea ✓ --> ## Scope & conditions - Applies whenever a diagnostic claim about a large population (a content library, a survey population, a codebase) rests on a convenience sample chosen by the person making the claim. - Doesn't mean the underlying finding is false — only that the sample doesn't establish its scope. ## Evidence The WPBeginner case: 30 of 3,000+ articles scanned, selection method undisclosed, extrapolated to "imagine what the other 3,000 look like." The same evidentiary shape recurs in the Discipleship Deficit corpus's survey-instrument claims — a different domain making the identical small-sample-to-population move. ## Links - [[DevRel MOC]] — the evidence standard for content-quality claims DevRel is often asked to make - [[3. Resources/Permanent Notes/Unmaintained Tutorial Content Is an SEO and Safety Liability]] — because contrasts; this note is the epistemic caution on that one's evidence base. - [[3. Resources/Literature Notes/The Discipleship Deficit (NotebookLM corpus distillation)]] — because evidence-for; same small-sample-to-population move in a different domain. ## Tests / falsifiers - Draw a stratified random sample from the same population and compare its distribution to the original convenience sample's — a close match would weaken this note's caution for that specific case. - Disclose and audit the selection method behind any "I scanned N of M" claim before treating the extrapolation as evidence. ## Maintenance - Last tended: 2026-08-16 - Next review: 2026-11-16 ## Source - [[3. Resources/Literature Notes/WPBeginner Collapse — Outdated Content, Not Just Promotion (EEAT)]] — the 30-of-3,000 sample and its extrapolation.