# Evidence & Claims Literacy MOC ## Synthesis additions (2026-10-17 — an older note registered: untracked failure layers) - [[3. Resources/Permanent Notes/A System Can Pass Every Tracked Metric and Still Fail Its User]] — a system can pass every metric its owners track and still fail the person using it, because the failure sits in a layer nobody instrumented. ## Synthesis additions (2026-10-17 — reading a vendor's study of its own adopters) - [[3. Resources/Permanent Notes/Comparing Adopters With Non-Adopters Credits the Tool for the Adopters' Prior Trend]] — a vendor's own sentence says adopters of its first fully autonomous agent were already on a steep upward trajectory, so an adopter comparison overstates the tool's effect. ## Synthesis additions (2026-10-15 — reading a vendor's speed claim) - [[3. Resources/Permanent Notes/A Ticket-to-Merge Clock Stops Before the Fix Reaches Users]] — ask where an agent-speed claim's clock starts and stops (Brad Vincent thread) ## Synthesis additions (2026-10-01 — historical and apologetic claims examined) - [[3. Resources/Readwise/Articles/Why Christianity Is True]] — Dan McClellan responds line by line to a clip arguing Christianity is historically true via apostle martyrdom, gospel reliability, and the spread of resurrection belief; his own dates are uncited on screen, captions garbled. - [[3. Resources/Readwise/Articles/This Proves King David Was Real!]] — Dan McClellan says the Tel Dan Stele's 'House of David' line is indirect evidence for a historical David, a narrower claim than the video title implies. ## Synthesis additions (2026-10-01 — AI in knowledge-work practice) - [[3. Resources/Readwise/Articles/This AI Does Your ENTIRE Systematic Literature Review in Minutes]] — Andy Stapleton demonstrates AnswerThis running a literature review and a PRISMA-style systematic review and calls it among the best such tools tested; a single-reviewer demo, not an evaluation. The recurring question underneath a lot of this vault: **when does a stated number, citation, or causal story actually license the conclusion drawn from it?** These notes were spread across nine domain MOCs — the theology ones filed under NAR, the vendor ones under Product, the security ones under Hosting — because the reading error arrived wearing whatever subject it came in on. The error is the same shape every time. ## What makes something evidence at all - [[3. Resources/Permanent Notes/Pre-Registration Is What Separates a Finding From a Story]] — declaring what you'll look for before looking is what makes a result able to fail - [[3. Resources/Permanent Notes/Disclosing Failures Does Not Make an Unfalsifiable Claim Falsifiable]] — reclassifying failures as mystery while counting successes as confirmation keeps a framework unfalsifiable - [[3. Resources/Permanent Notes/Testimony-as-Evidence Resists External Verification by Design]] — withholding the one checkable detail is what lets a proof narrative function rhetorically - [[3. Resources/Permanent Notes/A Ceiling Score Reported by the Scored Party Measures Test Fit]] — who administered the test changes what a ceiling score is evidence for; self-reported extremes measure test fit. - [[3. Resources/Permanent Notes/Pair One Named Case With a Population Statistic]] — the constructive counterpart — how to use a vivid case for salience while the statistic carries the warrant. - [[3. Resources/Permanent Notes/A Dated Precedent Beats an Abstract Structural Claim]] — a named dated case gives the reader something checkable; the abstraction gives them the author's confidence. - [[3. Resources/Permanent Notes/An Unspecified Denial Is an Assertion Not a Rebuttal]] — a denial that names no specific error is shaped so nothing could contradict it. - [[3. Resources/Permanent Notes/Sincerity and Correctness Need Separate Evidence]] — costly action evidences conviction and says nothing about whether the conviction is warranted. - [[3. Resources/Permanent Notes/A Checkable Admission Outranks an Unmethodical Number]] — the admission has a future observable attached; the personal probability does not. - [[3. Resources/Permanent Notes/An Authenticity Claim Without a Verification Mechanism Conveys Intent]] — where sincerity and imitation produce identical artifacts, the claim states intent and establishes nothing. ## Baselines and comparisons - [[3. Resources/Permanent Notes/A Verified Fraction Without a Comparison Baseline Cannot Be Scored]] — a verified percentage is uninterpretable until you know what the same process does to a comparable set - [[3. Resources/Permanent Notes/Event-Density Narratives Need a Control Decade to Count as Evidence]] — a 'remarkable convergence' is weak evidence unless tested against a control period - [[3. Resources/Permanent Notes/Uplift Modeling Isolates a Treatment Effect From a Tracked Baseline]] — comparing control against treatment is what attributes a delta to the input rather than a confounder - [[3. Resources/Permanent Notes/A Quality Label Without a Quantity Is Not a Claim You Can Check]] — 'human-reviewed' means nothing without the amount of review it names - [[3. Resources/Permanent Notes/An Alignment Claim Without a Reference Frame Is Unfalsifiable]] — "aligned" with no stated target is the same defect as "faster" with no baseline — nothing to be wrong against. - [[3. Resources/Permanent Notes/Genre-Wide Claims Need More Than One Catalog as Evidence]] — depth of evidence in one case gets mistaken for breadth across a population. - [[3. Resources/Permanent Notes/Self-Selected Testimonials Cannot Establish an Effect Size]] — the sample's bias does not shrink with its size, so no number of letters fixes it. - [[3. Resources/Permanent Notes/Radiocarbon Dating Sets a Floor Not a Ceiling for Writing on Wood]] — the measurement bounds one side of the range and reporting routinely presents it as both. ## Proxies mistaken for outcomes - [[3. Resources/Permanent Notes/Adoption and Satisfaction Are Not Evidence That a Teaching Method Works]] — spread, profit, and teacher satisfaction are selection pressures independent of whether learners learn - [[3. Resources/Permanent Notes/Adoption Frequency Does Not Measure Output Reliability]] — frequent use and reliable output are separate measurements - [[3. Resources/Permanent Notes/Measurement Scaffolding Is Not the Work It Measures]] — a reporting framework gets mistaken for the process, and teams execute the report - [[3. Resources/Permanent Notes/An AI Citation and an Organic Click Are Not Equivalent Units of Demand]] — counting AI visibility like organic traffic overstates reach because the units differ - [[3. Resources/Permanent Notes/Incremental Progress Is Invisible Because It Has No Headline Moment]] — attention tracks events, so a solved problem with no event registers as unsolved - [[3. Resources/Permanent Notes/Averaging Unweighted Sub-Scores Defeats the Threshold Built on Them]] — a high score on the part that did not matter carries the average past the trigger the threshold existed for. - [[3. Resources/Permanent Notes/Self-Assessment Without a Downward Mechanism Is a Ratchet]] — symmetric rules produce asymmetric outcomes when only one direction has an advocate in the room. ## How a claim smuggles in more than it states - [[3. Resources/Permanent Notes/A Disclaimed Comparison Is Still a Comparison]] — disclaiming a comparison and then making it lets it land without the author owning it - [[3. Resources/Permanent Notes/A Temporal Sequence Presented Without a Causal Claim Still Delivers One]] — sequence implies cause whether or not cause is asserted - [[3. Resources/Permanent Notes/A Chain of Individually Plausible Steps Is Not a Plausible Chain]] — each step being not-impossible does not make the conjunction likely - [[3. Resources/Permanent Notes/A Headline's Framing Is a Claim the Body Must Still Support]] — the title is where the smuggling usually happens, and the check needs only the piece itself. - [[3. Resources/Permanent Notes/Check a Shakeup Headline's Verb Against the Governance Action]] — ousted, forced, sidelined, and placed on leave described one board action across this cluster. - [[3. Resources/Permanent Notes/Report Proximity as a Fact Without Assigning Motive]] — publish the documented relationship and stop; the motive is the part the reporting cannot support. - [[3. Resources/Permanent Notes/An Unrelated Metric Beside an Unaddressed Allegation Is Diversion]] — the metric is usually accurate, which is what makes the substitution work. - [[3. Resources/Permanent Notes/Confidence in a Conclusion Cannot Exceed Confidence in Its Premise]] — the mechanical check — find the load-bearing premise and compare the hedging on each. - [[3. Resources/Permanent Notes/A Specific Number Borrows Authority From a Measurement It Never Had]] — precision is how measured results present themselves, so an untested default inherits their credibility. ## Sourcing discipline - [[3. Resources/Permanent Notes/Secondary Security Reporting Is a Triage Trigger Not a Citation]] — act on it, don't cite it for scope or version numbers - [[3. Resources/Permanent Notes/Vendor Usage Anecdotes Need Verification Before Adoption Decisions]] — self-reported vendor numbers are a signal to check, not evidence to act on - [[3. Resources/Permanent Notes/Original Reporting Identifies Itself by an Obtained Statement and a Named Non-Answer]] — two textual tells that separate original reporting from aggregation across a thirteen-outlet cluster. - [[3. Resources/Permanent Notes/Separate What an Outlet Saw From What It Attributes Secondhand]] — counting outlets measures reach, not confirmation; read for the attribution markers instead. - [[3. Resources/Permanent Notes/A Citation That Misses Its Anchor Is Worse Than No Citation]] — a present-but-wrong link discourages the check an absent one invites; found twice in this cluster. - [[3. Resources/Permanent Notes/A Single Internal Channel Is Still a Single Source]] — fifty quotes from one Slack thread is one vantage point with fifty lines. - [[3. Resources/Permanent Notes/Track Report Reaction and Denial as Three Separate Layers]] — three layers that can each be true or false independently, and usually arrive collapsed into one paragraph. - [[3. Resources/Permanent Notes/Named Brain Structures Without a Citation Are Rhetoric]] — a named region sounds like a citation while carrying none of a citation's obligations. - [[3. Resources/Permanent Notes/Scoped Disclosure Is More Credible Than Blanket Denial]] — a named boundary is testable; a categorical denial is unfalsifiable and usually untrue. - [[3. Resources/Permanent Notes/Citation-Checking Beats Citation-Trusting]] — footnote existence and footnote accuracy are separate properties; only the first is cheap to check. - [[3. Resources/Permanent Notes/Separate Alleged Harm From Allegations You Cannot Verify]] — per-claim rather than per-source: one thin allegation does not contaminate a piece's documented material. - [[3. Resources/Permanent Notes/Name the School Behind a Scholarly Claim]] — the gap between a serious minority position and the field consensus is where overclaiming happens. ## Gaps - No note yet on base rates specifically, which is the most common missing ingredient in these cases. - Nothing on how to write an honest uncertainty range in one's own published claims. - [[3. Resources/Permanent Notes/A Closed Validation Loop Cannot Detect Its Own Errors]] — the structural prerequisite to claim-testing: who is permitted to check, before what gets checked ## Synthesis additions (2026-10-05 — court filings and source reading) - [So Barry Abrahamson's Previously Sealed Testimony in the Automattic Trial...](3.%20Resources/Readwise/Tweets/So%20Barry%20Abrahamson's%20Previously%20Sealed%20Testimony%20in%20the%20Automattic%20Trial....md) — Duane Storey's thread (opinion) summarizes Dkt. 419 testimony and adds his own inferences, which the excerpt itself does not show. - [[3. Resources/Readwise/Articles/Exhibit Two to the Declaration of Kaitlin Keohane Filed at Dkt. No. 303-4, Refiled Pursuant to Court Order at Dkt. No. 381]] — Dkt. 419 excerpt records Abrahamson's sworn 30(b)(6) answers on a stats dashboard and update checks, ending on an unanswered question about what an update check sends. - [[3. Resources/Readwise/Articles/WPE v. Automattic, Dkt. 420 - Redacted Document Slip Sheet for Exhibit A (300-3)]] — One-page slip sheet marking a redacted Exhibit A as filed 'per court order', naming neither the order nor its grounds (filer's label). - [[3. Resources/Readwise/Articles/WPE v. Automattic, Dkt. 420-1 - Exhibit A, Mullenweg Deposition Excerpts (Redacted Re-filing)]] — A word-level diff against Dkt. 418-1 shows 964 words absent in this re-filing and none added; the text alone cannot show why. - [[3. Resources/Readwise/Articles/WPE v. Automattic, Dkt. 421 - Redacted Document Slip Sheet for Exhibit 1 (300-4)]] — One-page slip sheet marking a redacted Exhibit 1 as filed 'per court order', with no content, order, or reasons stated. - [[3. Resources/Readwise/Articles/WPE v. Automattic, Dkt. 421-1 - Exhibit 1, Mullenweg Deposition Transcript Excerpts (Redacted Re-filing)]] — A diff against Dkt. 418-2 found the same testimony with one missing nine-word run, so the redaction footprint here is small and its reason is unstated. - [Your partner isn’t a narcissist. He has autism. And so do you.](3.%20Resources/Readwise/Articles/Your%20partner%20isn’t%20a%20narcissist.%20He%20has%20autism.%20And%20so%20do%20you..md) — Penelope Trunk's 2022 opinion post argues narcissism is a subset of autism; the analysis found the linked research does not support the claim as worded (one partner-trait study reports r = 0.153), so treat it as a personal essay. - [[3. Resources/Permanent Notes/A Thread About an Excerpt Mixes Its Author's Inference Into What the Excerpt Says]] — sort each sentence of a thread about a filed excerpt into what the witness said, what the author inferred, and what came from elsewhere; only the first layer can be checked against the page. - [[3. Resources/Permanent Notes/Diffing a Redacted Re-filing Against Its Twin Shows Where Text Went, Not Why]] — a word-level diff of an original and a redacted re-filing shows where text is absent and how much, and says nothing about the reason or legal basis. ## Synthesis additions (2026-10-07 — popular-science evidence tiers) - [Your Feelings Aren't Reactions. They're Predictions.](3.%20Resources/Readwise/Articles/Your%20Feelings%20Aren't%20Reactions.%20They're%20Predictions..md) — Hidden Brain host Shankar Vedantam narrates psychologist Lisa Feldman Barrett's theory that emotions are the brain's predictions about body signals in context; the video states the prediction-machine picture as settled and the specific emotion claims as Barrett's, with no studies cited by name or sample sizes. ## Synthesis additions (2026-10-14 — self-description and traceability) - [[3. Resources/Permanent Notes/AI Output a Reader Must Act On Should Carry Its Sources by Default]] — a reader can't verify AI-assisted work without its sources and reasoning (from the Overcommitted survey episode). - [[3. Resources/Permanent Notes/An In-House Newsroom Earns Trust by Stating Its Independence Rule]] — a newsroom paid by the body it covers needs a stated rule for covering its own leaders and funders. - [[3. Resources/Readwise/Articles/Demand-Driven Context What 300 Incidents Reveal About Tribal Knowledge - Raj Navakoti]] — the speaker says the title's "300" was never a count (00:18:30), a specimen for [[3. Resources/Permanent Notes/A Headline's Framing Is a Claim the Body Must Still Support]]. ## Synthesis additions (2026-10-18 — registering previously unlisted notes) ### Permanent Notes - [[3. Resources/Permanent Notes/A Corporate Alliance Announcement Signals Coordination Not New Capital]] — A press release built from members' prior independent commitments signals coordination and PR, not new funding, until it discloses new dollar amounts. - [[3. Resources/Permanent Notes/A Credential Earns Trust Through Job-Relatedness Security and Path-Independence Together]] — A credential earns employer trust only when it proves job-relatedness, enforces exam security, and stays independent of any one educational path; missing one makes it a weaker signal. Closest-fit placement. - [[3. Resources/Permanent Notes/A Symbolic Decoding Method That Proves Its Opposite Proves Nothing]] — A symbolic-decoding method is worthless as evidence if the same rules can produce the opposite conclusion, as the podcast shows with a Hebrew letter-meaning trick. - [[3. Resources/Permanent Notes/Affiliate Best-Plugin Rankings Encode Reviewer Incentive Not Just Merit]] — A best-plugin ranking with affiliate links and a self-made rubric reflects reviewer incentive as much as merit, so treat it as a shortlist to re-verify. - [[3. Resources/Permanent Notes/An API Reversal Can Fix One Use Case While Leaving Adjacent Ones Broken]] — When a vendor reverses an API restriction, the fix covers only what the API contract does, so adjacent feed, timeline, and bulk uses can stay broken. - [[3. Resources/Permanent Notes/An Expertise Disclaimer Only Narrows Claims That Follow It]] — Saying I'm not a specialist narrows an argument only if every later claim stays inside the boundary just named; otherwise the disclaimer works as a credibility shield. - [[3. Resources/Permanent Notes/Commerce Inside a Crisis Narrative Raises a Trust Question Apart From the Crisis's Truth]] — A product promotion placed inside a crisis or threat narrative raises an incentive question, separate from whether the crisis itself is real. - [[3. Resources/Permanent Notes/Disclosing A Methods Limits Is What Makes Findings Usable]] — Publishing a measurement's accuracy and failure modes next to its headline number is what makes a finding usable, and selective disclosure is worse than none. - [[3. Resources/Permanent Notes/Editorial Rules of Thumb Are Probability Claims Not Mechanical Laws]] — Rules like prefer the harder reading state tendencies, not laws, so treating them as automatic verdicts gives wrong answers when other evidence disagrees. - [[3. Resources/Permanent Notes/Evidence Beats Assertion Once Assertion Becomes Free To Generate]] — When credentials are cheap to claim, the only credible signal left is a verifiable artifact someone else can inspect, though fakeable artifacts and ATS filters limit this. - [[3. Resources/Permanent Notes/Facts Don't Carry Meaning Until Someone Installs It]] — An event carries no meaning until a person installs one; the note proposes separating what a camera would record from inner-state claims and testing the assigned meaning. Closest-fit placement. - [[3. Resources/Permanent Notes/Greenwashing Disables Consumer Agency Not Just Brand Accuracy]] — Greenwashing's main harm is that vague eco claims stop consumers from making the better-informed choice, an epistemic harm beyond reputational damage. Closest-fit placement. - [[3. Resources/Permanent Notes/Identical Connectors Can Hide Categorically Different Capability]] — A standardized connector can carry different tiers of capability underneath, so appearance can't show what a cable or port supports, as with USB-C and WhatCable. Closest-fit placement. - [[3. Resources/Permanent Notes/Incident Root Causes Live Upstream of the Failed Component]] — The component that broke is rarely the root cause; a decision made earlier by someone outside their own domain of expertise usually is. Closest-fit placement. - [[3. Resources/Permanent Notes/Judge a Technology by Shipped Value Not the Funding Narrative]] — Judge a technology by the durable problems it provably solves, not by the funding narrative around it, and ask to see the working product. - [[3. Resources/Permanent Notes/Named Third-Party Research Partners Make Corporate Claims Checkable]] — Pairing a co-announced initiative with a named independent research partner turns a launch statement into something checkable later. - [[3. Resources/Permanent Notes/Shared-Source Citations Create an Illusion of Corroboration]] — Citing several outlets that all trace to one primary document reads as triangulation but gives no independent verification, because the outlets repeat the same source. - [[3. Resources/Permanent Notes/Ungrounded Content Gets Deleted Not Rewritten]] — When a draft contains a claim with no verifiable fact behind it, cut the line instead of rephrasing it, since better words on an empty claim stay empty. ### Literature Notes - [[3. Resources/Literature Notes/Best WordPress Booking Plugin 2026 (The Non Boring Version)]] — A video review scores five booking plugins on eight equal-weight tests and crowns BookingPress on a tiebreak, a ranking the note reads against its method. - [[3. Resources/Literature Notes/Mike Winger Goes NUCLEAR After These Threats]] — A 15-minute video reports threats against a Christian apologist's family, then makes polemical claims about Islam, including a disputed IQ argument, with a product promotion mid-video. Closest-fit placement. - [[3. Resources/Literature Notes/Sustainability Communication for Startups]] — Eleanor argues greenwashing comes down to language: replacing vague terms like eco-friendly with specific, evidenced claims, and being open about what you don't know. Closest-fit placement. - [[3. Resources/Literature Notes/You Ask, I Answer - Setting Test Timeline Expectations]] — A video on setting A/B test timelines covers stakeholders choosing risk tolerance and a sample-size calculator, and the note flags a misdefinition of confidence level. Closest-fit placement. - [[3. Resources/Literature Notes/Your Dashboards Are Production Systems. Start Monitoring Them Like One]] — Rupesh Ghosh argues dashboards need a fifth observability layer (Reliability, Performance, Capacity, Adoption, Governance) because current monitoring never asks if consuming the data was trustworthy. Closest-fit placement. - [[3. Resources/Literature Notes/watermarks-remover Supports OpenAI + Gemini + Claude - Literature Note]] — An MIT-licensed tool that stripped Claude text watermarks now covers OpenAI and Gemini, showing character-level in-band AI text watermarking is reversible by anyone motivated. Closest-fit placement.