# A Test Earns Its Place by Blocking a Ship A test belongs in my suite only if its failure would stop me shipping. That is the whole criterion. A test that goes red while the system's behavior is unchanged is not reporting a defect; it is reporting that I moved code around. Brian Okken states the rule at 00:15:49 of his Overcommitted interview: if a test can break because he refactored something while the functionality is still fine, "that's not a good indicator." Such a test is bound to structure, and structure is what refactoring changes on purpose. > [!quote] [00:05:01] > let's try to get the tests as high as possible so that we can refactor without breaking tests The mechanism is where change lands. Refactors move function boundaries; tests bound to function boundaries break; the system's public API changes only on a deliberate behavior change or a major version (00:04:19 to 00:05:01). So a test bound to the API fails only when behavior fails, and every red is a shipping decision. The practical consequence arrives at 00:22:25, relaying a Django developer who wrote small tests, then database tests, then Playwright tests through the UI: once the UI-level test passed, the earlier ones became disposable, because a small failure beside a passing large test never changes what ships. Coding agents sharpen the cost. "Move this over there" is now a one-line request (00:04:19), and a host reports at 00:20:58 that agent harnesses told to do TDD produced more test edits than code edits. Two things follow for me. The audit is mechanical: any test that failed last quarter without a shipped fix was a structure test by this definition. And the rule for agents is Okken's single standing instruction (00:22:25): put each test at the highest level that makes sense for the job. For a WordPress plugin that level is the public contract: hooks, REST routes, WP-CLI commands, rendered output. The claim breaks in three places. It prices only one side: a red system test takes longer to diagnose than a red unit test, and the episode never mentions that localization cost. It does not cover tests written to pin an algorithm's exact output during a rewrite, which are temporary by design. And the deletion rule is argued, not measured; nothing in the 41-minute episode is a number, so I check mutation coverage before I delete a lower test wholesale. <!-- atomicity: 1 idea ✓ --> ## Why This Matters Before you fix a failing test, ask whether you would have refused to ship over it. If the answer is no, the fix is to move the test up to the contract it should have been guarding, or to delete it. Give coding agents the same one-line rule and read the tests they add with the same question. ## Links - [[3. Resources/Permanent Notes/A Test That Cannot Return No Cannot Validate Yes]] — contrasts: that note filters tests by whether they can fail at all; this one filters by whether a failure would change what ships. Both measure a test by the information a red carries. - [[3. Resources/Permanent Notes/A Silent Check and a Broken Check Produce Identical Output]] — extends: a check you have never watched fail carries no information, and a check whose failure has no shipping consequence carries none either. - [[3. Resources/Permanent Notes/Real-Use Testing and Constructed Fixtures Catch Different Bug Classes]] — constrains: a lower test that covers an edge case, locale, or timing path no system test reaches adds coverage, not count, and survives this rule. - [[3. Resources/Permanent Notes/Cheap Generation Raises the Value of Deterministic Verification]] — applies-in: as agents make code and tests cheap to generate, the suite that stays valuable is the one anchored to behavior, since structure tests churn with every agent refactor. - [[3. Resources/MOCs/WordPress Platform Design MOC]] — applies-in: a plugin's public contract (hooks, REST routes, WP-CLI commands, rendered output) is the level where its tests belong. - [[3. Resources/Permanent Notes/A Green Pipeline Proves the Stages Ran Not That the Deployed Build Behaves]] — applied in: A green pipeline reports that each configured stage exited cleanly. ## Source - [[3. Resources/Readwise/Articles/Lean TDD With Brian Okken|Lean TDD With Brian Okken]] - https://www.youtube.com/watch?v=2Y7fOH3VleU