Your Agent Loop Is Not a Production System ![rw-book-cover](https://res.cloudinary.com/leaddev/image/upload/f_auto/q_auto/dpr_auto/c_limit,w_1920,h_1080/next/2025/07/rollercoaster-08.jpg) 🔗 **Source:** [Your Agent Loop Is Not a Production System](https://leaddev.com/ai/your-agent-loop-is-not-a-production-system) > [!note]+ Document Notes > tags: [ai-agents, agent-harness, human-in-the-loop, context-engineering, production-reliability] > author: Sriram Madapusi Vasudevan > source: LeadDev > reference: https://leaddev.com/ai/your-agent-loop-is-not-a-production-system > date created: 2026-09-28 08:20 > > > [!info] Generated analysis — not the author's words. Sections below are machine-written > > from the captured source. Only text inside `> [!quote]` blocks is verbatim from the author. > > Do not quote or attribute any other line to the named author. > > # 🏷️ Title: Your agent loop is not a production system > <!-- deep-read:sections start --> ## 🧭 Table of contents - [📝 Summary](#-summary) - [📖 Main content](#-main-content) - [🗒️ Notes](#️-notes) - [📚 References](#-references) - [🗣️ Paraphrase](#️-paraphrase) - [📰 Newsletter](#-newsletter) - [✅ Action items](#-action-items) - [🧪 Hypotheses](#-hypotheses) - [📊 Methodology](#-methodology) - [❓ Questions](#-questions) - [🔮 Future research](#-future-research) - [💡 Implications](#-implications) - [🧠 Learning styles assessment](#-learning-styles-assessment) - [📊 Accessibility assessment](#-accessibility-assessment) - [🌍 DEIB assessment](#-deib-assessment) - [✅ Logical validity](#-logical-validity) - [🛡️ DARVO analysis](#-darvo-analysis) - [🎯 Conclusion](#-conclusion) ### 🔗 References - [Your agent loop is not a production system (LeadDev, 2026-09-28)](https://leaddev.com/ai/your-agent-loop-is-not-a-production-system) — the article itself - [AWS DevOps Agent: working with directed actions](https://docs.aws.amazon.com/devopsagent/latest/userguide/working-with-devops-agent-working-with-directed-actions.html) — defines the "directed actions" boundary the whole piece builds on - [Keep the Terminal Relevant: Patterns for AI Agent Driven CLIs (InfoQ)](https://www.infoq.com/articles/ai-agent-cli/) — the author's own prior article, cited as the layer below this one - [Trustworthy Productivity: Securing AI Accelerated Development (InfoQ)](https://www.infoq.com/articles/secure-ai-development/) — the author's own prior article, cited as the layer below this one - [What is context engineering (LeadDev)](https://leaddev.com/ai/what-is-context-engineering) — background for the Context contract ## 📝 Summary - **What happened.** Sriram Madapusi Vasudevan, drawing on their work on AWS DevOps Agent, argues that an agent's tool-call loop (context in, tool call out, result in, next turn) is not the same thing as a production system. They propose five contracts a harness must enforce before any team gives an agent write access: context, state, tools, control, and completion. - **Key takeaways.** The article opens with a three-bullet takeaways box, stated up front: mutation, not model choice, is the production boundary, and once an agent can change state, approval, identity, attribution, and verification become first-class system concerns; approval, execution, and completion are three different proofs, since a tool can run successfully without the operational problem being solved; and the harness contracts should be standardized centrally while domain teams keep ownership of tool semantics, task policy, and the definition of done. - **The claim.** A demo loop can look durable because a confirmation box looks like authorization, a successful API response looks like proof, and a completed action looks like a resolved incident. In production those are three separate facts, and only a harness that treats them separately survives a restart, a cancellation, or a false claim of success. - **Why it matters.** As soon as an agent can mutate a resource (roll back a deployment, resize a fleet, close a ticket), the risk moves from "the model said something wrong" to "the model changed something wrong," and the blast radius becomes the design problem (¶3). - **Key stakeholders.** Engineering leaders deciding whether to give agents write access, platform teams who would own the durable-action record and execution gateway, security teams who own invariants and adversarial tests, and the operators who approve and can stop a run mid-flight. - **Risks and tradeoffs.** The article is built almost entirely from one production system, AWS DevOps Agent, and one incident-response example (a checkout-service rollback). It names Strands, LangGraph, the OpenAI Agents SDK, and Google ADK as convergent tooling but does not claim any of them implement the five contracts out of the box. - **Bottom line.** Frameworks give you interception points, not meaning. The organization still has to decide what counts as authorized, what counts as executed, and what counts as done, and encode that as durable state, not as a good prompt. The article closes with its own concrete rollout sequence for doing that: prove one read-only path first, add one operator-approved write, then deliberately break it before adding generated code, background workers, plugins, or child agents. ## 📖 Main content - **The core distinction.** A tool-call loop (¶9) is control flow: the model proposes, a tool executes, a result comes back. A harness is what has to remain true around that loop after a restart, a cancellation, a scope change, or a false claim of success (¶13). The article's throughline is that most agent engineering effort goes into the loop, and most production risk lives in the harness. - **Directed actions as the hinge.** AWS DevOps Agent's "directed actions" (¶4) are the concrete mechanism the whole argument hangs on: operations that mutate resources, off by default, requiring layered opt-in and per-action approval, executing with credentials scoped to that one approved operation, and attributable to the approving operator in CloudTrail. Everything else in the piece (the five contracts) is the author generalizing from what building that boundary required. - **Five contracts, not five features.** Context, state, tools, control, and completion (¶13) are framed explicitly as contracts, not framework components: "a checkpoint store or approval hook exists" is a component; "what must remain true after a restart" is a contract (¶13). The distinction matters because a team can install a checkpoint library and still have no approval semantics. - **Ownership model.** The article proposes a specific split (¶14): platform teams own the durable action record, context interfaces, execution gateway, identity propagation, interruption, observability, and evaluation substrate; domain teams own evidence sources, tool semantics, task policies, and completion criteria; security teams own hard invariants and adversarial tests; operators own approval thresholds, escalation, and the ability to inspect and stop work. "Centralize the contracts, not the intelligence" (¶15) is the compression of that split. - **The triad that collapses in a demo.** Approval proves authority, execution evidence proves what changed, and verification proves whether the goal was reached (¶20). The rollback example (¶16-19) shows these can diverge: the deployment API can confirm the rollback executed while the underlying incident (elevated 5xx rate) continues (¶46). A demo tends to conflate all three into "it worked." - **Terms defined in the piece.** A "directed action" is a mutating operation gated by approval and scoped credentials (¶4). "Runtime state" is approvals, budgets, identities, checkpoints, execution records, verification results, and the task graph, as distinct from "model context," which is the smaller view selected for one inference (¶28). A "failure test" is the article's own device: a specific adversarial scenario (kill the client after approval but before execution, cancel mid-delegation, falsely claim the incident is resolved) used to check whether a design decision actually holds (¶26, ¶31, ¶37, ¶41, ¶48). - **What's asserted versus demonstrated.** The five contracts, the ownership split, and the failure tests are the author's proposed design, presented with AWS DevOps Agent as the worked example, not as an independently validated framework. The piece names Strands, LangGraph, the OpenAI Agents SDK, and Google ADK as offering "useful primitives" (¶49) but is explicit that none of them supply the contract semantics on their own (¶50-51). - **Uncertain or contested.** (1) The article doesn't say how AWS DevOps Agent's own postcondition checks are implemented or tested, only that the boundary exists. (2) "Broadly comparable" framework primitives (interrupts, checkpoints, callbacks, events) are listed together in ¶49 without a comparison of how far each one actually goes toward durable authorization. (3) The five-contract model is presented as sufficient for the incident-response and coding-agent cases named; whether it holds for agents with no clear task-level postcondition (creative or advisory tasks) is not addressed. ## 🗒️ Notes #### Key takeaways (boxed list, before ¶0) - Mutation, not model choice, is the production boundary: once an agent can change state, approval, identity, attribution, and verification have to become first-class system concerns. - Approval, execution, and completion are three different proofs: a tool can run successfully without the operational problem being solved. - Standardize the harness contracts centrally, while domain teams retain ownership of tool semantics, task policy, and the definition of done. - These three bullets preview the piece's spine: the mutation boundary developed in ¶3-4, the approval/execution/completion split walked through in ¶16-22, and the centralize-contracts/domain-ownership split spelled out in ¶14-15. #### ¶0 Article dek - The dek states the thesis directly: agent frameworks make model-to-tool wiring easy; the harder work is the machinery around tools that can change production. - Names the five contracts (context, state, tools, control, completion) as the thing to standardize before scaling agent autonomy, and names AWS DevOps Agent as the source of the argument. #### ¶1–¶11 The loop is only control flow - Opens on a 2am incident-response scenario: an agent joining an on-call engineer mid-incident, not having a conversation (¶1). - While an agent stays read-only (inspecting alerts, correlating logs, testing hypotheses), bad recommendations are recoverable; the architecture changes the moment an operator says "apply it" (¶2-3). - AWS DevOps Agent's directed actions are disabled by default, require layered opt-in and per-action approval, run on scoped credentials, and are attributable in CloudTrail (¶4). - The author's own prior two pieces (agent-ready CLIs, then guardrails on the ReAct loop) set up this one as "the next layer": agent-ready tools ask whether an agent can use software reliably, guardrails ask whether it can act inside a defensible envelope, the harness asks whether those properties survive a real production run (¶7-8). - The tool-call loop itself (context in, tool call, result, next turn) is called out as "useful. It is not an operating model" (¶9-10); a production harness has to preserve the chain of evidence, proposal, approval, executing identity, what changed, and why the run continued or stopped (¶11). #### ¶12–¶15 Standardize five contracts before teams scale - Reframes the first engineering-leadership decision: not which toolkit (Strands, LangGraph, etc.) but what guarantees every team inherits before any agent gets write access (¶12). - Defines the five contracts (context, state, tools, control, completion) against framework components: a component tells you a mechanism exists, a contract tells you what must survive a restart, cancellation, scope change, or false success (¶13). - Proposes an explicit ownership split across platform, domain, security, and operator roles (¶14). - "Centralize the contracts, not the intelligence" — models and orchestration can change; the behavior that matters under stress should not (¶15). #### ¶16–¶22 A successful action is not a successful task - Walks a concrete incident: a 5xx spike after a deployment, correlated to a config change, proposed as a checkout-service rollback (¶16). - The approval surface should show the exact operation, target, material parameters, blast radius, and validity window, not the vague sentence "roll it back" (¶17). - After execution, the harness reads the deployment state back (did the intended release activate?) and then checks the incident-level postcondition (did the 5xx rate recover, did the alarm clear, does the root cause still fit) (¶18-19). - States the triad plainly: approval proves authority, execution evidence proves what changed, verification proves whether the goal was reached — three different facts that a demo collapses into one (¶20-21). #### ¶23–¶26 1. Context: pin the decision - Context assembled by concatenating instructions, messages, retrieved documents, and tool results works until information crosses trust boundaries or the world changes mid-run (¶23). - A policy, a runbook, and a log line are not interchangeable; a log line saying "ignore the previous instruction" is evidence, not an instruction — each item needs provenance, freshness, task scope, and a semantic role (¶24). - An approval has to stay pinned to the exact tool, operation, target, material parameters, evidence, risk, identity, approval window, and expected postconditions; if any approval-relevant field changes, the harness re-asks (¶25). - Failure test: change the target deployment after the approval request exists; the harness must use the pinned target or require fresh approval, never reinterpret the old decision against new context (¶26). #### ¶27–¶32 2. State: resume authority, not just chat - Conversation history, runtime state, and model context are related but not interchangeable; putting everything into one mutable message array makes the system look durable only while a single process stays alive (¶27-28). - The durable record doesn't need every model token — it needs what governs the next legal step: admitted evidence, the proposal, the approving policy, the approver, the finalized scope, whether that approval was already used, what executed, and what verification established (¶29). - On restart after approval but before execution, the model must not reconstruct authority from chat; the harness restores the exact scope and either resumes the permitted action or re-asks if the approval expired (¶30). - Failure test: stop the client after approval but before execution, resume on another host, and confirm the scope is restored, used at most once, and never widened by a fresh model turn (¶31). Closing line: a harness that restores conversation but not authorization "has memory, not durable state" (¶32). #### ¶33–¶37 3. Tools: give every mutation one path - A tool schema is a starting point; production teams still need to classify each tool as read-only, mutating, or destructive, define its identity/resource needs, dry-run capability, runtime limit, structured result shape, and the postcondition that proves the larger task succeeded (¶33). - Classification has to match real behavior everywhere: a write reached through generated code, MCP, a background worker, or a child agent is still a write (¶34). - Every mutation should pass through one gateway: validate, classify, authorize, request approval, finalize scope, execute, read back, verify, record. Two paths through that sequence means two security models (¶35). - Failure test: invoke the same write via a direct tool call, generated code, an MCP tool, and a child agent, and confirm every path gets the same classification, scope, approval requirement, and audit record (¶36-37). #### ¶38–¶42 4. Control: make human decisions durable - Human-in-the-loop as a yes/no dialog bolted onto a tool call is not enough; in production an approval behaves like a temporary capability naming the operation, resource, constraints, approver, expiry, and reuse rule (¶38). - The finalized scope may narrow, never widen, the original proposal; the decision must be serializable (to survive pause/resume), revocable until used, and attributable after execution (¶39). - Control also covers budgets (steps, time, cost, tool calls, delegation depth); cancellation must reach in-flight tools and child agents; delegation must reduce authority, never expand it (¶40). - Failure test: cancel while a child agent is mid-evaluation and confirm cancellation propagates with no new mutation starting (¶41). Closing line: human-in-the-loop "is a durable authorization transition," not a confirmation dialog (¶42). #### ¶43–¶48 5. Completion: verify the operational outcome - The model saying "done" is not a completion condition; a coding agent's done differs from an incident-response agent's done, and the definition belongs to the task, not the model (¶43-44). - Where practical, separate the verifier from the planner — the planner searches for a path to the goal, the verifier checks whether the goal was actually reached (¶45). - The rollback example returns: the directed action can succeed (previous version active) while the operational task (elevated 5xx rate) has not (¶46). - Reframes what leaders should measure: not tool-call count, token volume, or unattended run time (activity metrics), but verified completion rate, correction/escalation rate, recovery after interruption, operator effort, and cost per verified outcome (¶47). Failure test: force the model to report resolution while the alarm stays active, and confirm the harness rejects that terminal state (¶48). #### ¶49–¶52 Frameworks give you hooks, not an operating model - Names four converging primitives across the ecosystem: Strands' interrupts and interventions, LangGraph's checkpointed and resumable interrupts, the OpenAI Agents SDK's serializable paused approval state, and Google ADK's events/session state/callbacks (¶49). - Calls that progress, but explicit that "a checkpoint does not define your approval semantics," a callback doesn't prove alternate paths honor the same policy, and a trace doesn't prove the resource changed or the task succeeded (¶50). Closing line: "Frameworks provide interception points. Your organization still owns their meaning" (¶51). - The ecosystem's ease (plugins, generated code, background workers, multi-agent collaboration) multiplies the places state, authority, and completion can go wrong, even as it makes demos look more powerful (¶52). #### ¶53–¶56 Build one directed action before you scale autonomy - ¶53 states the build order as three named stages, given as an explicit checklist right after "I would build a new harness in three stages": 1. **Make one read-only path reconstructable.** Start with one model and one read-only tool. Add a durable event record, structured tool results, and a reliable stop mechanism. The first milestone is being able to explain the run, not making the agent look autonomous. 2. **Add one operator-directed write.** Require a concrete proposal, classification, task-scoped identity, per-action approval, resource read-back, and a verifiable postcondition — this is where the five contracts become real. 3. **Break it before you expand it.** Restart after approval, expire the decision, change the target, deny the action, force the tool to fail, cancel delegated work, and keep the postcondition false. Only after those paths are trustworthy should generated code, background work, plugins, or child agents get added. - This order is deliberately "boring" (¶54), because it forces the semantics to become clear before the number of execution paths explodes. - Models, prompts, and tool protocols will change faster than the systems around them (¶55). - The durable asset is the operational judgment encoded in the harness: what enters context, what persists, how intent is pinned, how authority narrows, where humans interrupt, how execution is attributed, and what counts as enough evidence to stop (¶56). #### ¶57–¶60 Trustworthy productivity has to survive the action - Connects back to the author's two earlier pieces explicitly: agent-ready interfaces make tools usable, guardrails keep the agent inside a defensible envelope, the harness carries those properties through a real production run (¶57). - Restates the layering as a one-line summary: agent-ready tools make action possible, guardrails make action bounded, the harness makes action inspectable, attributable, and verifiable (¶58). - Reframes the real question: not whether a model can call a tool, but whether the system can show what it was allowed to do, what it actually changed, and whether the result was good enough to stop (¶59). - Closing line: "The loop makes the agent useful. The harness is what makes it fit for production" (¶60). ## 📚 References - [AWS DevOps Agent — working with directed actions](https://docs.aws.amazon.com/devopsagent/latest/userguide/working-with-devops-agent-working-with-directed-actions.html) (primary; defines the directed-actions mechanism the article builds its argument from, ¶4) - [Keep the Terminal Relevant: Patterns for AI Agent Driven CLIs — InfoQ](https://www.infoq.com/articles/ai-agent-cli/) (the author's prior article on agent-ready CLIs, cited in ¶7) - [Trustworthy Productivity: Securing AI Accelerated Development — InfoQ](https://www.infoq.com/articles/secure-ai-development/) (the author's prior article on the ReAct-loop trust boundaries, cited in ¶7) - [How to decide on engineering guardrails — LeadDev](https://leaddev.com/software-quality/how-decide-engineering-guardrails) (supports the "Guardrails" layer named in ¶8) - [The Engineering Leadership Report 2026 — LeadDev](https://leaddev.com/the-engineering-leadership-report-2026/) (linked from "engineering leaders" in ¶12) - [What is context engineering — LeadDev](https://leaddev.com/ai/what-is-context-engineering) (linked from "context" in ¶13, background for the Context contract) - [Why everyone's suddenly talking about AI agents — LeadDev](https://leaddev.com/technical-direction/why-everyones-suddenly-talking-about-ai-agents) (linked from "An agent" in ¶9, general agent-loop framing) - [Incident response before the incident — LeadDev](https://leaddev.com/technical-direction/incident-response-before-the-incident) (linked from "incident response" in ¶1) - [Every decision creates policy — LeadDev](https://leaddev.com/software-quality/every-decision-creates-policy) (linked from "A policy" in ¶24) - [The best AI coding assistants — LeadDev](https://leaddev.com/ai/best-ai-coding-assistants) (linked from "tool" in ¶33, Tools contract) - [Rethinking your engineering budget during a downturn — LeadDev](https://leaddev.com/career-development/rethinking-your-engineering-budget-during-downturn) (linked from "budgets" in ¶40) - [An engineer's guide to Model Context Protocol (MCP) — LeadDev](https://leaddev.com/ai/an-engineers-guide-to-model-context-protocol-mcp) (linked from "Model Context Protocol (MCP)" in ¶34) - [Designing human-agent engineering teams — LeadDev](https://leaddev.com/ai/designing-human-agent-engineering-teams) (linked from "Human-in-the-loop" in ¶38, Control contract) - [Your AI coding tools buying checklist for 2026 — LeadDev](https://leaddev.com/ai/your-ai-coding-tools-buying-checklist-for-2026) (linked from "A coding agent" in ¶44, Completion contract) - [Strands — interrupts](https://strandsagents.com/docs/user-guide/concepts/interrupts/) (primary source for the Strands primitive named in ¶49) - [LangGraph — interrupts](https://docs.langchain.com/oss/python/langgraph/interrupts) (primary source for the LangGraph primitive named in ¶49) - [OpenAI Agents SDK — human in the loop](https://openai.github.io/openai-agents-python/human_in_the_loop/) (primary source for the OpenAI Agents SDK primitive named in ¶49) - [Google ADK — event loop](https://google.github.io/adk-docs/runtime/event-loop/) (primary source for the Google ADK primitive named in ¶49) - [Level frameworks for developing developers — LeadDev](https://leaddev.com/management/level-frameworks-developing-developers) (linked from "Frameworks" in ¶49, an internal cross-link rather than a source for the technical claim) ## 🗣️ Paraphrase - The tool-call loop everyone demos (model proposes, tool executes, result returns) is control flow. It says nothing about what survives a restart, a cancellation, a scope change, or a model that falsely claims success. That gap is what the author calls the harness. - AWS DevOps Agent's directed actions made the author draw the system boundary differently: an approval is not a one-turn chat answer, it's a durable, scoped, identity-attached decision that has to persist across pauses and resumptions. - Five contracts, not five framework features, are what a team should standardize before scaling agent autonomy: context (pin the decision), state (resume authority, not chat), tools (one gateway per mutation), control (durable, narrowing-only approval), completion (a task-defined postcondition, not "the model said done"). - A rollback can execute successfully while the incident it was meant to fix continues. Approval, execution, and verification are three separate facts, and collapsing them is exactly what a demo does and a production harness cannot. - Strands, LangGraph, the OpenAI Agents SDK, and Google ADK all now ship interrupt/checkpoint primitives. That's real progress, but a checkpoint is not an approval semantic and a trace is not proof the task succeeded — the organization still owns what those hooks mean. - The advice for building a new harness is to do it in three deliberately boring stages, because the goal is to make the semantics explicit before the number of execution paths multiplies past the point anyone can reason about them. ## 📰 Newsletter **Your agent's "it worked" might be lying to you.** [Sriram Madapusi Vasudevan](https://leaddev.com/ai/your-agent-loop-is-not-a-production-system) draws on building AWS DevOps Agent to argue that an agent's tool-call loop and its production harness are different problems. Their example: a rollback agent can get a clean "success" from the deployment API while the 5xx rate that caused the incident stays elevated. Approval proves authority, execution evidence proves what changed, verification proves whether the goal was reached — three facts a demo happily collapses into one. For anyone wiring agents into WordPress workflows (Abilities API calls, WP-CLI, MCP tools that can actually write), the useful takeaway is concrete: every mutating tool call should pass through one gateway (validate, authorize, approve, execute, read back, verify, record), whether it's invoked directly, through generated code, through MCP, or through a child agent. A `wp_update_post` call reached through three different paths still needs the same approval scope and the same audit trail. Read the [directed actions doc](https://docs.aws.amazon.com/devopsagent/latest/userguide/working-with-devops-agent-working-with-directed-actions.html) before you wire up your next write-capable agent step. ## ✅ Action items #### DevRel - Developer advocacy: build a short talk or post, "the five contracts your agent demo skips," using the rollback example (¶16-22) as the worked case. - Education: for a maker/checker launchd loop like Courtney's, write down which of the five contracts (context, state, tools, control, completion) each gate actually enforces today, and which ones it only assumes. The article's own three-step rollout (read-only path first, then one operator-approved write, then break it before expanding, ¶53) is a ready-made curriculum for that audit. - Developer experience: for any agent tool you document, state its classification (read-only, mutating, destructive) up front, matching the tool contract in ¶33. - Developer marketing: don't market "autonomous agent" features without naming what happens on cancellation, restart, or a false success claim — that's the gap this article is written to close. - Community management: when a vendor announces agent autonomy features, ask in public whether approval state survives a restart, not just whether a confirmation dialog exists. - Internal/external/cross-project: track whether Strands, LangGraph, the OpenAI Agents SDK, or Google ADK publish more than interrupt/checkpoint primitives (¶49-51) — i.e., whether any of them ship durable approval semantics, not just hooks for it. #### OSPO - Policy & Governance: evaluate any agent framework adoption against the five-contract checklist (¶13) before it reaches production, not after. - License Compliance: no direct licensing claim in this source; nearest relevance is auditing which agent SDK primitives (interrupts, checkpoints) a project depends on and under what license. - Security Management: adopt the article's "one gateway per mutation" rule (¶35) as a security review item: flag any tool reachable by more than one code path (direct call, generated code, MCP, child agent) that doesn't share one authorization gateway. - Software Inventory: record, per agent-facing tool, its mutation classification and whether it has a dry-run mode (¶33), as part of the tool inventory. - Developer Support: publish an internal reference for "what a durable approval record needs" (exact tool, operation, target, parameters, evidence, identity, window, per ¶25) so teams stop inventing their own shape. - Community Engagement: watch whether upstream frameworks add durable, serializable approval state rather than just richer interrupts. - Strategic Alignment: propose "verified completion rate" and "cost per verified outcome" (¶47) as KPIs alongside raw agent activity metrics. #### Community - Maintainers: document your own project's agent-facing mutation paths and whether they all pass through one gateway. - Contributors: reproduce one of the article's failure tests (e.g. ¶37's "same write through four paths") against a project's own agent tooling as a reproducible check. - Sponsors: fund a small write-up of what "durable approval state" looks like for a specific open-source agent stack, since the article names the gap but not a reference implementation. - Ecosystem: propose a shared vocabulary for the five contracts so teams comparing agent frameworks are comparing the same properties. #### WordPress - Core: no direct action from this source; nearest relevance is any future core support for agent-mediated content changes needing an approval/attribution model. - Core Performance: not applicable. The source is about authorization and verification, not performance. - Core AI: test whether the MCP Adapter and Abilities API give a mutating call a stable classification (read-only vs. mutating vs. destructive, per ¶33) regardless of whether it's invoked directly or through a child agent. Follow the article's own rollout order (¶53): prove a read-only Abilities call first, add one operator-approved write, then break it (restart mid-approval, deny it, force it to fail) before wiring in a child agent or generated code. - **Core Program Team:** add "does this agent-facing feature define a completion condition separate from the model's own claim" (¶43-45) as a cross-team review question for AI-facing proposals. - Mobile: not applicable. The source doesn't touch mobile clients. - Playground: prototype a Playground blueprint that demonstrates the "one gateway per mutation" pattern (¶35) for a WordPress write action, as a teaching example. - Community: WordCamp talk angle: "Your plugin's write action needs an approval contract, not a confirmation dialog" (¶38, ¶42). - Training: build a Learn lesson on the difference between a tool call succeeding and a task being verified done (¶43-46), using the rollback example. - Marketing: not applicable. No WordPress marketing angle in this source. - Support: document what a user should expect when an agent-initiated WordPress change is later found to not have resolved the underlying problem (¶46's pattern). - Design: not applicable. No UI design content in this source. - Documentation: add an "agent write actions" pattern to plugin developer docs: classify the action, define its postcondition, and say how it can be dry-run. - Photos: not applicable. - TV: not applicable. - Polyglots: not applicable. - Accessibility: not applicable to the source; no UI or media content to assess. - Meta: consider whether wordpress.org's own APIs need a durable-approval model for any future agent-initiated actions (plugin review automation, for instance). - CLI: audit WP-CLI commands that mutate state for whether they can be invoked with a dry-run flag and a structured result, matching the tool contract in ¶33. - Hosting: hosts offering agent-driven site management (deploys, scaling, rollbacks) should adopt the directed-actions pattern (scoped credentials, per-action approval, attribution) directly; this maps almost one-to-one onto hosting operations. - Tide: not applicable. - Openverse: not applicable. - Themes: not applicable. - Plugins: plugin authors exposing any mutating action to an agent (via Abilities or MCP) should classify it and gate it per the tool contract in ¶33-37. #### The WPOCC - Programs: propose a microgrant for a reference implementation of durable agent-approval state against a WordPress-facing agent tool, since the article names the requirement but not an open-source example. - Projects: a documentation sprint on "agent write actions for WordPress" covering classification, approval scope, and postcondition design. - Funding experiments: a milestone payout tied to a published, reproducible failure test (per ¶26, ¶31, ¶37, ¶41, ¶48) against a real WordPress agent integration. #### FAIR - Technical independence: the article's insistence that authority must be pinned to an exact scope and never silently widened (¶25, ¶39) is directly relevant to any FAIR discussion of agent-mediated changes to federated or self-hosted infrastructure. - Policies and covenants: the ownership split in ¶14 (platform/domain/security/operator) is a usable starting template for a covenant defining who is accountable for an agent's mutating action in a federated system. - Productized governance artifacts: add "does the approval record survive a restart" (¶30-32) as a scorecard item for any agent-facing governance tooling FAIR evaluates. ## 🧪 Hypotheses - H1: Teams that implement all five contracts (context, state, tools, control, completion) show a lower rate of "successful action, unresolved task" incidents (the ¶46 pattern) than teams that only implement a confirmation dialog and a checkpoint store. - H2: Approval decisions that are serialized as durable state (survivable across restart, per ¶30-32) reduce unintended re-execution or scope-widening compared with approvals tracked only in conversation history. - H3: Requiring one shared gateway for every mutation path (direct call, generated code, MCP, child agent, per ¶35-37) reduces the rate of unauthorized or misclassified writes compared with per-path authorization checks. - H4: Separating the verifier from the planner (¶45) reduces false "task complete" reports compared with a single model asserting both the plan and its own success. - H5: Framework-provided primitives (Strands interrupts, LangGraph checkpoints, OpenAI Agents SDK paused state, Google ADK callbacks, ¶49) reduce implementation time for a durable-approval harness but do not, by themselves, change the false-completion rate, because they provide hooks rather than approval semantics (¶50-51). ## 📊 Methodology - H1/H4: instrument a set of agent-driven incident-response runs with and without a task-level postcondition check; compare the rate of runs where a directed action executed successfully but the underlying alarm or metric did not recover. - H2: ablation test matching the article's own failure test in ¶31 — kill the process after approval, before execution, resume on a different host, and measure how often authority is (correctly) not reconstructed from chat history alone across implementations that do and don't serialize approval state. - H3: a controlled audit where the same mutating capability is invoked through four paths (direct tool, generated code, MCP, child agent) as in ¶37, scored on whether classification, scope, and audit record match across all four. - H5: a structured comparison across Strands, LangGraph, the OpenAI Agents SDK, and Google ADK, mapping each toolkit's built-in primitives against the five contracts to see how much of each contract still has to be built by the adopting team. ## ❓ Questions - What does AWS DevOps Agent actually do when a postcondition check fails after a directed action executes — retry, escalate, or halt? - How does the harness decide when an approval has "expired" versus is still valid, when the article says approval has a lifetime but doesn't give a default (¶5, ¶25)? - For agents with no clean task-level postcondition (research, drafting, advisory work), does the five-contract model still apply, or does "completion" collapse back to a human judgment call? - How much of the ownership split in ¶14 (platform/domain/security/operator) assumes a large organization with distinct platform and security teams, versus a small team wearing all four hats? - Which of Strands, LangGraph, the OpenAI Agents SDK, or Google ADK comes closest to supporting serializable, durable approval state today, versus only checkpointing conversation and tool calls? - Does "narrower scope, never wider" (¶39) have an exception case the article doesn't mention, such as an emergency break-glass action? - What's the actual cost (engineering time, latency) of routing every mutation through one shared gateway (¶35) compared with the per-tool authorization most teams likely have today? ## 🔮 Future research - A published reference implementation showing the five contracts enforced end to end on an open agent framework, not just AWS DevOps Agent. - Comparative study of framework primitives (interrupts, checkpoints, callbacks, paused state) against the specific durability properties the article requires (survives restart, used at most once, never silently widened). - Metrics research validating "verified completion rate" and "cost per verified outcome" (¶47) as leading indicators, versus the activity metrics the article says they should replace. - Applied work on completion criteria for agent tasks that don't have a clean external postcondition (creative, advisory, exploratory work). ## 💡 Implications - **Practice.** Treat "the agent's tool call returned success" and "the task is done" as two separate claims that need two separate proofs, and build the second proof (a postcondition check) before scaling any agent to write access. - **Policy.** An organization that lets an agent's directed action widen its own scope, or reconstructs authority from chat history after a restart, has effectively delegated a governance decision to whatever the model happens to say next. - **Security.** The "one gateway per mutation, regardless of path" rule (¶35-37) is a direct, testable security control: any mutating capability reachable outside that gateway is an unreviewed attack surface, whether reached by generated code, MCP, or a child agent. - **Education.** The five contracts and the five failure tests are teachable as a checklist independent of any specific framework, and could sit alongside prompt-engineering curricula as "harness engineering." - **Funding.** Nobody currently has a strong incentive to build and open-source a reference "durable approval state" implementation; this is a gap collectives or foundations could fund the way independent benchmarks fill a similar gap in agent cost claims. ## 🧠 Learning styles assessment - Reading level estimate: Flesch-Kincaid grade approximately 14-15 (roughly 2,244 words across 60 numbered paragraphs, many single-sentence paragraphs used for emphasis alongside long, clause-dense sentences and specialized terms like "directed action," "durable authorization transition," and "postcondition"). College-level technical/opinion writing. #### 👀 Visual learners - Present: a hero image and section subheadings (per the meta headings list), but no diagrams. - Improve: a sequence diagram of the propose → approve → execute → read-back → verify chain (¶17-20), and a table of the five contracts against their failure tests. #### 🔊 Aural learners - Present: none in the captured content. - Improve: an audio walkthrough of the rollback example (¶16-22), since it's the piece's clearest concrete scenario. #### 🗣️ Verbal learners - Present: the article defines its own terms carefully in prose ("directed action," "runtime state" vs. "model context," a "failure test") and repeats short, quotable closing lines per section (¶6, ¶32, ¶42, ¶51, ¶60). - Improve: a glossary collecting those definitions in one place, since they're currently scattered across the piece. #### 🧩 Kinesthetic learners - Present: five explicit "failure test" exercises (¶26, ¶31, ¶37, ¶41, ¶48) that read as runnable checks against a real system. - Improve: turn each failure test into an actual runnable script or checklist against a named framework (Strands, LangGraph) so a reader can execute it, not just read it. #### 🧠 Logical learners - Present: a numbered five-contract structure, explicit ownership splits (¶14), an opening takeaways box, and a named three-step build checklist with concrete milestones (¶53). - Improve: a decision tree mapping "which contract does this failure violate" for troubleshooting a real agent incident. #### 🧑‍🤝‍🧑 Social learners - Present: none; the article names one author and one production system throughout. - Improve: a linked discussion thread or a named second practitioner's account of implementing the same contracts on a different framework. #### 🧘‍♀️ Solitary learners - Present: the piece is self-contained, structured for a single sitting, with a clear progression from problem to five contracts to build order. - Improve: a self-check like "which of the five contracts does your current agent setup NOT enforce?" placed at the end. #### 🌿 Naturalistic learners - Present: none. - Improve: none obviously applicable; the source is about authorization and verification, not environmental or systems-ecology framing. ## 📊 Accessibility assessment #### Screen reader semantics - Present (per the captured markup's heading list): a clear, single-level sequence of section headings ("The loop is only control flow," "1. Context: pin the decision," etc.) that would read in order to a screen reader. - Improve: the hero image's alt text wasn't captured in the extracted markup, so it can't be verified; numbered subheadings ("1. Context," "2. State"...) should confirm they render as real heading elements, not styled paragraph text, so the numbering is announced consistently. #### Keyboard navigation - Not assessable from captured content. This needs a live-page audit of the leaddev.com article template. #### Contrast and readability - Not assessable for color contrast from captured content. On readability: several sentences run 30-40+ words with multiple embedded clauses (e.g. ¶25, ¶29), which raises cognitive load independent of any contrast issue. #### Text resizing and reflow - Not assessable from captured content. #### Media and captions - No video or audio in the article. Not applicable. #### Forms and interactions - No forms in the captured content. Not applicable. #### Standards reference - WCAG 2.2 AA mapping: 1.1.1 Non-text Content (hero image alt text, unverified from captured markup). 1.3.1 Info and Relationships is the relevant check for the numbered inline subheadings ("1. Context: pin the decision") actually being marked up as headings rather than bolded text, since that distinction only shows up in the rendered DOM, not in the Reader capture used here. ## 🌍 DEIB assessment #### Cultural breadth - Present: no geographic or regional framing either way; the argument is tool- and system-agnostic. - Improve: no non-US/English deployment or regulatory context (e.g., how attribution requirements differ across data-residency regimes) is discussed. #### Inclusive language - Present: gender-neutral throughout ("the operator," "the approver," "a team"); technical jargon is mostly defined inline. - Improve: terms like "task graph" (¶28) and "postcondition" (¶33) are used without a first-use definition for readers newer to agent engineering. #### Neurodiversity support - Present: numbered five-contract structure and five repeated "failure test" call-outs create a predictable, scannable pattern across the whole piece. - Improve: a summary table up top (contract → failure test → what it prevents) would reduce the working-memory load of tracking five parallel threads through prose. #### Representation - Present: a single named author and a single named production system (AWS DevOps Agent) throughout; no other practitioners are quoted. - Improve: a second practitioner's account from a different framework (LangGraph, Google ADK) would test whether the five contracts generalize beyond the author's own system. #### Psychological safety - Present: respectful, non-alarmist tone throughout; risk is framed as a design problem to solve, not a reason to avoid agents. - Improve: not applicable; there's no conflict or harm content in the source to soften further. ## ✅ Logical validity #### Strong reasoning - Causal, mechanism-based argument: the piece doesn't just assert "harnesses matter," it names the specific failure mode (approval reconstructed from chat after a restart, ¶30-32) and a specific test that would catch it (¶31). That makes the claim falsifiable rather than aspirational. - The rollback worked-example (¶16-22, ¶46) is used consistently across three different contracts (context, control, completion) rather than invented fresh for each section, which is a coherence strength — the same scenario shows why pinning context, scoping approval, and verifying completion are three distinct problems, not one. - Honest scoping: the article explicitly frames frameworks as giving "interception points," not meaning (¶51), which pre-empts the obvious objection ("just use LangGraph's checkpoints") without dismissing those tools. #### Fallacies - **Single-case generalization (hasty generalization).** The five-contract model is generalized from one production system (AWS DevOps Agent) and one worked incident-response scenario. The article doesn't test the model against a different domain (e.g., a coding agent, mentioned only briefly in ¶44) with the same rigor. Fix: a second worked example from a domain with a different completion criterion (a coding agent's test suite, say) would strengthen the generalization. - **Unstated scope condition.** The claim that delegation "should reduce authority, not expand it" (¶40) and that scope should be "narrower... never wider" (¶39) is stated as an absolute, with no discussion of an emergency-override or break-glass case a production on-call team might actually need. Fix: name the exception explicitly, or state that none exists and why. - No fallacy found in the core causal argument itself (harness properties failing under restart/cancellation/false-success) after checking for appeal to authority, circular reasoning, and false dichotomy — the piece consistently ties claims to a named mechanism (CloudTrail attribution, scoped credentials, serializable approval state) rather than to the author's title or to AWS's brand. #### Tone of voice - First-person, measured, and consistent throughout ("I have been working through this problem from the bottom up," ¶7). It's a practitioner essay, not a vendor pitch: AWS DevOps Agent is the example, not the product being sold, and the piece never states pricing, adoption numbers, or a call to buy anything. ## 🛡️ DARVO analysis - Deny: none. The article doesn't respond to any accusation; it's a proactive design argument, not a rebuttal. - Attack: none. No critics, competitors, or prior approaches are named as wrong; the closest thing is the mild critique that most agent demos "reduce to a small loop" that "is not an operating model" (¶9-10), which is a critique of a pattern, not a person or project. - Reverse victim/offender: none. - Not applicable to this source. It's a technical opinion piece with no conflict, accusation, or defensive posture to analyze. ## 🎯 Conclusion - Bottom line: the article's real claim is narrower than its headline problem statement — it's not "agents are risky," it's "an agent's tool-call loop and its production harness are different engineering problems, and the harness needs five specific, testable contracts (context, state, tools, control, completion) before write access scales." The evidence is one production system and one worked incident, generalized outward. The piece is not short on a practical answer, though: it opens with a three-bullet takeaways box naming the mutation boundary, the approval/execution/completion split, and the centralize-contracts split, and it closes with a concrete, three-step rollout sequence (make one read-only path reconstructable, add one operator-directed write, break it before you expand it, ¶53) rather than leaving the reader with only the five-contract theory. - Next steps: run the five failure tests (¶26, ¶31, ¶37, ¶41, ¶48) against Courtney's own launchd maker/checker loops to see which contracts they already enforce and which ones currently rely on "the checker script ran and exited 0" standing in for "the task is verified done." Follow the article's own build order to do it: confirm one read-only path is fully reconstructable first, add exactly one operator-approved write with the full approval record, then deliberately break it (restart mid-approval, deny it, force the tool to fail) before adding any generated code, background workers, or child agents. Read the AWS DevOps Agent directed-actions doc before that first write-capable step. ## 🔄 Refactor Appendix — Your agent loop is not a production system > Density: 2–5 Literature atoms (H3), 0–3 Permanent candidates (H4). > Defaults: 3 Literature, 1 Permanent. Resonance filter: if you can't > articulate in one line why a claim matters, drop it. ### 📚 Literature Atoms (H3) #### [H1] Approval must become durable, scoped state > [!quote] ¶5 > Approval could not be a transient answer attached to one conversation turn. **Context & use**: - From the passage where the author describes how building AWS DevOps Agent changed where they drew the system boundary. - Reframes "approval" from a chat-turn event into a piece of state with scope, identity, lifetime, and a reuse rule — directly applicable to any maker/checker gate that currently treats an approval as a one-shot log line. **Tags**: #xref **Highlight**: [view highlight](https://leaddev.com/ai/your-agent-loop-is-not-a-production-system) <!-- atomicity: 1 idea ✓ --> #### [H2] Approval, execution, and verification are separate facts > [!quote] ¶20 > Approval proves authority. Execution evidence proves what changed. Verification proves whether the goal was reached. **Context & use**: - Closes the rollback incident example, immediately after describing the deployment-state read-back and the incident-level postcondition check. - The clean three-way split is the single most reusable idea in the piece — it's the checklist for auditing whether "it worked" in a run log actually means the underlying problem is resolved. **Tags**: #xref #articleseed **Highlight**: [view highlight](https://leaddev.com/ai/your-agent-loop-is-not-a-production-system) <!-- atomicity: 1 idea ✓ --> #### [H3] Restoring chat isn't restoring authorization > [!quote] ¶32 > If a harness can restore the conversation but not the authorization, it has memory, not durable state. **Context & use**: - Closing line of the State contract section, after the failure test of killing the client after approval but before execution. - Sharp, quotable distinction between "the system remembers what was said" and "the system knows what it's still allowed to do" — a direct test to run against any resumable agent job. **Tags**: #xref #share **Highlight**: [view highlight](https://leaddev.com/ai/your-agent-loop-is-not-a-production-system) <!-- atomicity: 1 idea ✓ --> ### 🧠 Permanent Candidates (H4) #### [P1] A maker/checker gate needs a durable approval record, not just a passing exit code **Claim** (one sentence, Courtney's voice): A checker script exiting 0 tells me a tool call ran, not that the operation it approved is still the one that executed, still scoped the way I approved it, and still hasn't been reused past its intended window. **Evidence pointer**: [[#H1]] **Scope & conditions**: Holds for any loop where a run can pause, resume, or restart between approval and execution (which is exactly the launchd job shape). Weakens for single-shot, synchronous scripts with no pause window, where there's nothing for the approval to drift away from. **Links** (≥1 with annotation): - [[Maker/checker automation gates]] — because applies-in launchd job design - [[Context engineering]] — because extends - [[Vendor benchmark self-reporting]] — because contrasts on who verifies the claim of success **Optional**: `proof_state: working`. Tests/falsifiers: pause one of my own maker/checker loops between the checker's "approved" state and the maker's execution step, restart the process, and confirm the resumed run uses the original approved scope rather than re-deriving it from the last log line. If it silently re-approves against new state, the claim fails. <!-- atomicity: 1 idea ✓ --> ### ✏️ Fleeting Queue - [F1] Compare Strands interrupts, LangGraph checkpoints, OpenAI Agents SDK paused state, and Google ADK callbacks against the five-contract checklist to see which comes closest to durable approval semantics — https://leaddev.com/ai/your-agent-loop-is-not-a-production-system #fleeting #inbox - [F2] Read the AWS DevOps Agent directed-actions doc in full before wiring a first write-capable step into any personal agent loop — https://docs.aws.amazon.com/devopsagent/latest/userguide/working-with-devops-agent-working-with-directed-actions.html #fleeting #inbox ### → Things 3 (max 2 actions) - [ ] Run the ¶31 failure test (kill after approval, before execution, resume elsewhere) against one launchd maker/checker loop — production-loop audit #things - [ ] Write down which of the five contracts (context, state, tools, control, completion) my current agent gates actually enforce vs. only assume — harness gap-check #things <!-- deep-read:sections end --> 📄 **Full text:** [[Full Document Contents/Articles/Your Agent Loop Is Not a Production System.md|Your Agent Loop Is Not a Production System by Sriram Madapusi Vasudevan]] ## Highlights - Working on AWS DevOps Agent changed where I drew the system boundary. Approval could not be a transient answer attached to one conversation turn. Runs pause and resume. Targets can change. The same capability may be reachable through more than one execution path. The approval has to become a durable state with an exact scope, an identity, a lifetime, and an explicit reuse rule. Just as importantly, a successful tool response could prove that an action ran without proving that the incident was over. ([View Highlight](https://read.readwise.io/read/01m3kzpxax2989w5225pv0x08v)) - Note: [H1] Approval must become durable, scoped state - Tags: atom/literature/h1 - Approval proves authority. Execution evidence proves what changed. Verification proves whether the goal was reached. ([View Highlight](https://read.readwise.io/read/01m3kzpy4m9kt27a5g8ns5xvh6)) - Note: [H2] Approval, execution, and verification are separate facts - Tags: atom/literature/h2 - If a harness can restore the conversation but not the authorization, it has memory, not durable state. ([View Highlight](https://read.readwise.io/read/01m3kzpyxqj163s9a1qmkrffez)) - Note: [H3] Restoring chat isn't restoring authorization - Tags: atom/literature/h3