# The Maker Should Not Grade Its Own Homework The single highest-leverage structural decision in any automated workflow is to **separate the agent that produces work from the agent that checks it**. The producer is systematically too lenient on its own output — not through malice but through the same mechanism that makes a writer miss their own typos: the thing that generated the artifact has already convinced itself the artifact is right. Addy Osmani states it plainly: *"The most useful structural thing in a loop, by far, is splitting the one who writes from the one who checks. The model that wrote the code is way too nice grading its own homework. A second agent with different instructions and sometimes a different model catches the stuff the first one talked itself into"* (Source: [Loop Engineering.](3.%20Resources/Readwise/Tweets/Loop%20Engineering..md)). The checker's value comes precisely from *not sharing the maker's context* — different instructions, ideally a different model, and a mandate to refute rather than confirm. The deepest version of the claim applies the split to the **stop condition itself**. Osmani notes that Claude Code's `/goal` primitive works by having *"a fresh model decide if the loop is done instead of the one that did the work, the maker and checker split applied to the stop condition itself."* This matters because "done" is the most dangerous judgment in any loop: the maker is maximally biased exactly at the moment it declares completion. A separate verifier with a written, checkable success criterion converts "I think it's done" into "it provably meets the bar." Without that separation, an unattended loop is *"a loop making mistakes unattended"* — it ships confident wrong answers at machine speed. The claim is not "verify your work" (everyone says that). It's stronger and more specific: **verification by the same agent that produced the work is structurally unreliable, so the verifier must be a different agent.** Self-review degrades to rationalization; the producer defends its choices because it already made them. This is why a passing self-assessment and an independent check are not the same evidence, even when they reach the same conclusion most of the time — the cases where they diverge are exactly the cases that matter. ## Why This Matters For **AI agent safety**, this is the auditable control that makes unattended automation trustworthy. The maker/checker split is the agent-era version of separation of duties: it pairs naturally with [[3. Resources/Permanent Notes/Agent-First vs Gateway-First Are Two Architectures for Self-Hosted AI Agents]] (the checker is a second agent in the loop) and with [[3. Resources/Permanent Notes/Default-On AI Activation Inflates Adoption Metrics at the Cost of Consent]] (a vendor grading its own adoption metrics is the institutional form of the same bias — the party that benefits from "success" should not be the party that measures it). For **my own workflows**, this is the principle the vault-hygiene-checker embodies directly: a separate skeptical pass that audits a synthesis pass instead of trusting it. The synthesis-backlog's rule — *only mark `done` after the link/atom actually exists, verified by the checker* — is the same discipline. The reason it's needed is documented in my own history: an integration suite stayed red while unit tests were green and eleven PRs merged anyway, because a passing test was mistaken for a running test. That was a maker grading its own homework at the level of a whole codebase. For **DevRel and human work**, the principle generalizes past AI. It's why [[3. Resources/Permanent Notes/Friction Logging Is the DevRel Equivalent of Code Review]] works — a second set of eyes with a different mandate finds what the author talked themselves past — and why adversarial review beats self-review in any domain where the producer has a stake in the output. ## Counterpoints to Hold - **The checker can be wrong too.** A separate agent isn't an oracle; it can hallucinate problems or miss real ones. The split reduces correlated error, it doesn't eliminate error. (My own checker's first run flagged a link "dead" via a parsing quirk when it was actually *fragile* — the verdict was right, the reasoning wasn't. The fix was to verify the checker.) - **It costs more.** Osmani is explicit that sub-agents *"burn more tokens since each one does its own model and tool work, so spend them where a second opinion is worth paying for."* Not every action warrants a checker; trivial, reversible, or low-stakes work doesn't. - **Diminishing returns and infinite regress.** Who checks the checker? At some point a human (or a cheap deterministic test) has to be the ground truth. The split buys reliability up to the point where the verifier's own judgment becomes the binding constraint — which is why human review bandwidth, not the tooling, is often the real ceiling. - **Independence is the active ingredient, not redundancy.** Two identical agents reviewing the same work mostly share the same blind spots. The benefit comes from *different* instructions/model/lens, not from a second copy. <!-- atomicity: 1 idea ✓ --> ## Links - [[DevRel MOC]] — the review discipline behind friction logging and every second-pair-of-eyes practice here - [[Product & Startup Instincts MOC]] — separation of duties as a product decision, not only an agent-safety one - [[3. Resources/Permanent Notes/Instincts Are Reliable Ideas Are Not So Test Small Bets on Proven Foundations]] — applies-in: small bets and retention evidence keep founder hope from serving as the verdict on the founder's own idea. - [[An Agent's Default Persona Should Be Collaborator Not Assistant]] — complements: a pushback-capable persona reduces weak output before execution, while independent checking catches what still gets through afterward. - [[An Ops Agent Is Only as Capable as Its System Access]] — applies-in: powerful operational access makes independent verification more important because a mistaken action can change real systems. - [[Auditing the Build Not the Install Is What Makes Personal-Scale Coverage Possible]] — applies-in: deduplicated scanning expands coverage, but live reproduction remains the independent check that makes each finding trustworthy. - [[AI Buyers and AI Coworkers Are Opposite Design Postures]] — applies-in: the coworker interface's reviewable reasoning and easy reversal turn independent verification into a product behavior. - [[3. Resources/Permanent Notes/A Trusted Intermediary Makes a False Claim Durable]] — applies-in; independent source checking prevents an intermediary's authority from making an unsupported claim durable. - [[3. Resources/Permanent Notes/A Self-Referential Analysis Is Admissible for Structure Not Truth]] — direct instance: names the specific boundary that follows once the maker is grading its own homework — structural cataloguing survives, truth-verdicts don't - [[3. Resources/Permanent Notes/Intent Written on the Outside Compounds While Re-Derived Intent Is Debt]] — the other half of the loop: the checker needs an external, written success criterion to check *against*; intent-on-the-outside is what makes "done" verifiable - [[3. Resources/Permanent Notes/Agent-First vs Gateway-First Are Two Architectures for Self-Hosted AI Agents]] — the checker is a structural second agent; architecture determines whether that separation is even possible - [[3. Resources/Permanent Notes/Default-On AI Activation Inflates Adoption Metrics at the Cost of Consent]] — the institutional maker-grades-own-homework: the party that benefits measures the success - [[3. Resources/Permanent Notes/Friction Logging Is the DevRel Equivalent of Code Review]] — the human-domain instance of independent review beating self-review - [[3. Resources/Permanent Notes/AI-Assisted PKM Replaces Synthesis Work With the Appearance of Synthesis Work]] — a related self-deception: the appearance of having checked vs. having checked - [[3. Resources/Permanent Notes/A Nondeterministic Check Breaks the Loops Stop Condition]] — the necessary second property: independence makes a checker trustworthy, determinism makes its verdict *usable* — an independent but flaky checker still breaks an unattended loop (convergent restatement from the 2026-06-26 loop-engineering threads) - [[Stop Trusting AI; Start Verifying It with Ari Joury]] — evidence-for; Joury's deterministic core and critique-agent architecture applies maker/checker separation to regulated, client-facing AI output - [[3. Resources/Literature Notes/The Modern WooCommerce Developer Workflow]] — contrasts: WP-CLI and browser access strengthen an agent's self-check, but don't supply the independent judgment needed for a true checker - [[3. Resources/Permanent Notes/Bounded Action Spaces Not Better Models Make Agents Safe to Run]] — applies-in: independent checking is one boundary in the larger system that keeps fallible agents safe to run. - [[3. Resources/Permanent Notes/Code Review Is a Human Knowledge-Transfer Ritual Not Just a Quality Gate]] — applies-in: review needs an independent human conversation, not the maker's agent simulating both sides of the judgment loop - [[3. Resources/Permanent Notes/Default-Off Feature Flags Decouple Shipping Speed From Production Risk]] — extends: independent review bounds what merges, while default-off exposure bounds who encounters the result after merge. - [[3. Resources/Permanent Notes/Defense in Depth Concedes Perfect Safety Is Impossible and Optimizes for Narrow Costly Monitored Failure]] — applies-in: an independent checker is the detection layer that keeps a producer's inevitable errors observable. - [[3. Resources/Permanent Notes/Encoding What Good Looks Like Is What Makes Agent Verification Trustworthy]] — extends: checker independence only becomes actionable when an external specification defines the result the checker must verify. - [[3. Resources/Permanent Notes/Route AI Models by Temperament Not Just Cost Tier]] — applies-in: temperament-aware routing still requires an independent checker because the chosen model cannot certify its own failure behavior - [[3. Resources/Permanent Notes/Safe Agent Changes Come From Guardrails and Observability Not Trust]] — applies-in: independent checking is one guardrail that replaces trust in the agent with externally derived evidence - [[3. Resources/Permanent Notes/Sampling a Fraction of a Source Produces False Authority Over the Whole]] — applies-in: a checker must re-read the source rather than accept the sampling model's own fluent account of its coverage - [[3. Resources/Permanent Notes/Silent Model Intervention and Edited Safety Docs Break Trust Regardless of Measured Impact]] — applies-in: producer-controlled and editable evidence cannot independently verify the provider's claim about an undisclosed intervention - [[3. Resources/Permanent Notes/The Bottleneck Moved From Code Generation to Review and Judgment]] — applies-in: independent checking preserves review quality when generated output grows faster than human review capacity. - [[3. Resources/Permanent Notes/The Harness Now Outpaces the Model as the Source of Agent Capability Gains]] — applies-in: an independent checker is part of the harness that turns model output into verified work. - [[3. Resources/Permanent Notes/Zet · Self-verifying AI flags its own unsupported claims]] — contrasts: internal citation flags make uncertainty visible but remain evidence produced by the same system being evaluated. - [[3. Resources/Literature Notes/My AI Security Pipeline for WordPress Plugins]] — evidence-for: the security pipeline's run-it-to-prove-it rule shows why AI findings need an independently derived execution check. - [[3. Resources/Literature Notes/My AI-Assisted Content System That Keeps Me in the Loop with Kyle Balmer]] — applies-in: a human checkpoint supplies an external quality standard instead of accepting the producing agents' own verdict. - [[3. Resources/Readwise/Articles/The PHP Podcast 2026.07.30]] — evidence-for (added 2026-08-01): independent convergence on the mechanism from a production system — a model generating basketball summaries is fact-checked by a *second context* reading the same source, catching winner-inversions the generating context was committed to. Answers the obvious objection directly: a model that can detect an invalid bug report would not have declined to write one, because the generating context does not inherit the verifier's conclusion. - [Gonna Share a Few General Thoughts and Tips About How...](3.%20Resources/Readwise/Tweets/Gonna%20Share%20a%20Few%20General%20Thoughts%20and%20Tips%20About%20How....md) — extends (added 2026-08-01): Kudish moves the split *upstream of code* — review agents audit the generated plan for defects before any implementation exists — and adds the cheap in-session variant, asking the model to contradict you or form its own evidence-based opinion - [[3. Resources/Permanent Notes/Gate Contributions on Demonstrated Comprehension Not on Tooling]] — contrasts: where maintainer attention cannot fund an independent checker for every submission, the fallback is making the maker evidence its own comprehension — structurally weaker, and chosen only because the stronger option does not scale - [[3. Resources/Permanent Notes/Automation Removes the Social Checks a Process Silently Depended On]] — why the split became necessary: code review used to rest on an unwritten norm that the author had run the thing, and automation removed it silently - [[3. Resources/Readwise/Articles/Almost Timely News 🗞️ How to Expand and Improve Content With AI, Part 2|Almost Timely News — model council]] — evidence-for: Christopher Penn runs three independent referee models over the same draft, on the rule that the model which wrote the text is biased toward its own style and cannot be its sole judge - [[3. Resources/Permanent Notes/A Quality Gate Holds Only If the Agent It Checks Cannot Edit the Gate]] — extended by: A static-analysis gate checks an agent's work only when the agent has no write access to the gate's configuration. ## Open Questions - What's the right cheap proxy for a "different lens" checker when a second model isn't available — different prompt? adversarial framing? a deterministic test? - How many independent checkers before the marginal one stops catching anything new (the loop-until-dry question applied to verification)? - Where is self-review *adequate* — is there a class of low-stakes, easily-reversible work where the split's cost isn't justified, and can it be named precisely rather than by gut feel? ## Source - [Loop Engineering.](3.%20Resources/Readwise/Tweets/Loop%20Engineering..md) — "the model that wrote the code is way too nice grading its own homework"; `/goal` as the maker/checker split applied to the stop condition; verification stays on you because "done is a claim, not a proof" - [[3. Resources/Readwise/Articles/Effective Verification for Humans and Agents]] — a panel case for an external oracle, higher-level change summaries, and a named human signer when agent-produced code ships. - See also: [[AI Agent Safety MOC]] (verification/control layer) and the vault-hygiene-checker + synthesis-backlog skills as a working instance