# Stop Trusting AI; Start Verifying It with Ari Joury ## 🗂️ Triage — annotated highlights surfaced > [!note]- Cross Reference > - **Trust the AI to act, but verify at every step** — the working principle behind the maker/checker split; a second agent critiques the first before a human sees it. Connects to [[The Maker Should Not Grade Its Own Homework]]. #xref > - **LLMs optimize for plausibility, not accuracy** — the root-cause diagnosis that makes verification non-optional in regulated domains. #xref #aphorisms ## 🧭 Bib capsule Ari Joury (Wangari Global), "Stop Trusting AI; Start Verifying It," O'Reilly, 2026 — https://www.youtube.com/watch?v=AQCrFnNxnyM ## 🧠 Author's thesis (Adler Q1+Q2) Joury argues that treating AI coding tools as a "magic code completion" black box quietly produces architectural decay and, in regulated work, fluent nonsense — because LLMs are optimized for plausibility, not accuracy. The fix is to treat AI as an engineering component inside a verification architecture: hardcode anything calculable, generate the non-deterministic parts with one agentic layer, critique them with a second, trace every output back to source data, and only then let a human client see it. ## 🧩 Key arguments - Vague AI prompting causes silent architectural decay that no human reviewer would produce on purpose, so the debt is illegibility, not a single bug. - LLM output is optimized for plausibility, which is a fatal mismatch anywhere accuracy is regulated or client-facing (the Zurich Insurance near-miss). - "Trust but verify" is an architecture, not a slogan — deterministic cores, critique agents, and source-traceable numbers checked at every step. ## 🗒️ Notes — Literature atoms ### Vague AI prompting causes silent architectural decay > [!quote] > "Under the hood, what happens is that your code base just becomes spaghetti code in ways that a human would never even dare to do." - **Where**: ~[00:00:23](https://www.youtube.com/watch?v=AQCrFnNxnyM&t=23) - **Why**: Names the mechanism of AI-driven decay — accumulated illegibility from unreviewed output, not one visible failure. ### LLMs optimize for plausibility, not accuracy > [!quote] > "These LLMs were optimized for plausibility and not accuracy, despite my apparently say. And in financial services, that's a no-go." - **Where**: ~[00:01:37](https://www.youtube.com/watch?v=AQCrFnNxnyM&t=97) - **Why**: The root-cause diagnosis for the Zurich Insurance incident; explains why fluent output is the dangerous failure mode. ### Trust the AI to act, but verify at every step > [!quote] > "You trust the AI to do its job, but you're also verifying it at every single step." - **Where**: ~[00:04:08](https://www.youtube.com/watch?v=AQCrFnNxnyM&t=248) - **Why**: Restates the title as a working principle — continuous verification, not one gate at the end. ## 🔖 Permanent candidates - Deterministic-core-plus-agentic-critique architecture — When an AI pipeline feeds regulated or client-facing output, hardcode anything calculable and route only the generative parts through a second agent that critiques the first before a human ever sees it. (Likely already covered by the maker/checker Permanent below — verify before promoting.) ## Links - [[AI Agent Safety MOC]] — home MOC; this is an operational verification-architecture pattern for running agents in regulated work - [[The Maker Should Not Grade Its Own Homework]] — Joury's "critique agent checks the first agent" is a concrete instance of the maker/checker separation - [[Encoding What Good Looks Like Is What Makes Agent Verification Trustworthy]] — the verification only holds if the checker knows the standard; Joury's source-traceable numbers are that standard made explicit - [[CodeRabbit - How can you trust your AI agents]] — the same trust question from a code-review vendor's seat; both converge on independent verification over trust. - [[I Installed a "Safe" Package; My AI Tried to Steal My SSH Keys with Eran Sandler]] — the concrete supply-chain threat that makes "verify, don't trust" non-optional; Sandler supplies the attack, Joury the defense. ## ✅ Processing checklist - [x] Claim atoms carried from deep-read analysis - [x] Permanent candidates reviewed for promotion — covered by [[The Maker Should Not Grade Its Own Homework]] and [[Encoding What Good Looks Like Is What Makes Agent Verification Trustworthy]] - [ ] Version History Diff check