The control layers of AI 🔗 **Source:** [The control layers of AI](https://dri.es/the-control-layers-of-ai) > [!note]+ Document Notes > tags: [ai, workflows, enterprise-ai] > author: Dries Buytaert > source: dri.es > reference: [https://dri.es/the-control-layers-of-ai](https://dri.es/the-control-layers-of-ai) > date created: > > # 🏷️ Title: The control layers of AI > <!-- deep-read:sections start --> ## 🧭 Table of Contents - [📝 Summary](#-summary) - [📖 Main Content](#-main-content) - [🗒️ Notes](#️-notes) - [📚 References](#-references) - [🗣️ Paraphrase](#️-paraphrase) - [📰 Newsletter](#-newsletter) - [✅ Action Items](#-action-items) - [🧪 Hypotheses](#-hypotheses) - [📊 Methodology](#-methodology) - [❓ Questions](#-questions) - [🔮 Future Research](#-future-research) - [💡 Implications](#-implications) - [🧠 Learning Styles Assessment](#-learning-styles-assessment) - [📊 Accessibility Assessment](#-accessibility-assessment) - [🌍 DEIB Assessment](#-deib-assessment) - [✅ Logical Validity](#-logical-validity) - [🛡️ DARVO Analysis](#-darvo-analysis) - [🎯 Conclusion](#-conclusion) ### 🔗 References - Include links cited in or relevant to the article. --- ## 📝 Summary - What happened: A Salesforce/Vivint incident exposed AI agents skipping critical workflow steps, silently breaking customer surveys. - Why it matters: It shows that orchestration, guardrails, and deterministic workflows are as important as AI “intelligence.” - Key stakeholders: Enterprise software vendors, AI platform builders, workflow/orchestration and open‑source communities. - Risks and tradeoffs: Flexibility vs auditability, probabilistic LLMs vs deterministic reliability, vendor lock‑in vs open tooling. - Bottom line: The control/orchestration layer around AI will become essential infrastructure, and open platforms are well‑positioned to own it. --- ## 📖 Main Content - The article opens with the Vivint–Salesforce case: AI agents inside Salesforce’s Agentforce skipped steps so that customer satisfaction surveys stopped going out for weeks without anyone noticing. The core problem was not the model’s “intelligence” but the lack of structured, deterministic control around it. - Salesforce’s fix was *more workflow structure and guardrails*, not “smarter” models. This supports the author’s earlier thesis that future systems must mix deterministic workflows (for reliability and auditability) with AI decision-making (for flexibility and judgment). - Using tools like n8n or Activepieces, the Vivint flow could be: call ends → AI analyzes transcript and personalizes a message → workflow platform guarantees the email send. Here, “the AI does what it is good at and the automation platform ensures the send happens every time” ([workflow example](https://read.readwise.io/read/01kdtz2rpb4bbwhe287r861n8d?endLexposition=0000S_00002-007i&startLexposition=0000K&emphasizeText=the+customer+service+call+ends%2C+AI+analyzes+the+transcript+and+personalizes+the+survey+message)). - The author argues that LLMs are probabilistic by design, so anything that *can* be deterministic *should* be. That implies workflows, schedulers, and tool calls must handle reliability, retries, failure modes, and auditing. - Two architectures are contrasted: - **Outside‑in**: a deterministic workflow engine (e.g., n8n, Activepieces, Drupal’s ECA) is in charge and invokes AI for particular steps. This is more reliable and auditable but less flexible because you’re constrained to what the workflow designer anticipated. - **Inside‑out**: AI agents are in charge and call deterministic tools when needed—for example, Anthropic’s Claude Code invoking a real compiler instead of guessing whether code compiles ([inside‑out description](https://read.readwise.io/read/01kdtz2rpb4bbwhe287r861n8d?endLexposition=0000a_00000-004j&startLexposition=0000U&emphasizeText=The+second+approach+is+inside+out.+AI+agents+call+deterministic+tools+as+part+of+their+operation.)). This is more adaptive but harder to debug, explain to regulators, and can be more expensive at scale. - The author suggests these two approaches are *nested* into three conceptual layers: 1. **Outer layer – Enterprise workflow**: orders steps, handles integrations, and enforces business rules and audits. 2. **Middle layer – AI decision step**: makes judgment calls within a given step. 3. **Inner layer – Deterministic tools**: perform precise operations when invoked. - Not every use case needs all layers (e.g., pure code-generation workflows can lean on inside‑out AI plus tools), but complex enterprise flows with compliance and multi‑system integration *do* need an explicit outer orchestration layer. - Workflow automation platforms do not need to handle every AI use case; they only need to dominate the complex ones where visibility, control, and accountability are mandatory. That’s also where enterprises are willing to pay a premium. - Finally, when something becomes critical infrastructure, organizations prefer owning rather than renting it, echoing how Linux and PostgreSQL became dominant. If the orchestration/control layer becomes essential for enterprise AI, open source is the natural fit. Activepieces (open source) and n8n (source-available) are highlighted as examples the author would like to see succeed ([open-source orchestration argument](https://read.readwise.io/read/01kdtz2rpb4bbwhe287r861n8d?endLexposition=0000g_00000-000G&startLexposition=0000c&emphasizeText=If+the+orchestration+layer+becomes+essential+to+enterprise+AI%2C+open+source+will+be+the+natural+choice.)). --- ### 🗒️ Notes - AI capabilities are no longer the whole story; the *control layers* around AI determine whether systems are safe, reliable, and auditable in production. - The Vivint case shows that “AI skipping steps” can silently break important customer-facing processes, revealing a monitoring and orchestration gap rather than just a model quality issue. - Fixing such incidents usually means tightening workflows, adding guardrails and checks, and clarifying handoffs between AI and deterministic components—rather than simply retraining or swapping models. - LLMs’ probabilistic nature means they are inherently bad at guarantees that matter in enterprise contexts: making sure something happens exactly once, exactly on time, or exactly as specified. - Therefore, architects should default to deterministic mechanisms for: - Scheduling and triggering. - Idempotency and retries. - Audit logs and explainability. - Policy and compliance enforcement. - **Outside‑in pattern**: - A workflow engine owns the sequence of operations. - It calls AI where human-like judgment or personalization is needed. - It catches failures and unexpected outputs, and can route exceptions to humans or alternates. - **Inside‑out pattern**: - An AI agent receives a goal and decomposes it into steps. - It decides when to call specific tools (e.g., compilers, database queries, external APIs). - This makes the system feel more “autonomous” but reduces transparency and predictability. - These patterns are naturally layered: workflows (macro‑order), AI steps (meso‑decisions), tools (micro‑execution). - In simple or highly technical tasks (e.g., coding), inside‑out might be enough. In enterprise settings—with compliance, SLAs, cross‑system dependencies—explicit orchestration is non‑optional. - Commercial opportunity exists wherever enterprises pay to reduce risk and increase accountability. Orchestration platforms that integrate AI in a controlled way will likely capture significant value. - Historical analogies: Linux and PostgreSQL show that once an infrastructure layer becomes mission‑critical, open source often wins due to cost, control, extensibility, and ecosystem effects. - The author implicitly argues that open or source‑available workflow platforms can become for AI orchestration what Linux became for servers. --- ## 📚 References - Dries Buytaert – “The control layers of AI” – https://dri.es/the-control-layers-of-ai - Activepieces – https://www.activepieces.com/ - n8n – https://n8n.io/ - Anthropic Claude Code – https://www.anthropic.com/ --- ## 🗣️ Paraphrase - Paragraph A paraphrase: The Vivint–Salesforce incident exposed how AI agents, left without strong workflows, can silently skip crucial steps, prompting Salesforce to reinforce structure and guardrails instead of just chasing smarter models. - Paragraph B paraphrase: The author argues that because LLMs are inherently probabilistic, reliable systems must wrap them in deterministic workflows and tools, using either outside‑in or inside‑out architectures—or both layered together—especially in enterprise and compliance-heavy environments. --- ## 📰 Newsletter Today’s read from Dries Buytaert digs into *AI control layers*: how workflows, guardrails, and deterministic tools shape whether AI is actually safe to ship in production. For WordPress builders, the implications are big: think AI‑assisted editorial flows, moderation, and deployments that *must not* silently skip steps. Expect a growing role for open‑source orchestrators—possibly even inside WordPress itself—where Core, Meta, and plugin authors collaborate on shared patterns for reliable “AI in the loop” automation. --- ## ✅ Action Items #### DevRel - Map common developer workflows (CI/CD, content pipelines, moderation) into outside‑in diagrams showing where AI steps belong and where deterministic guarantees are required. - Design workshops or office hours on “AI + workflows” using tools like n8n/Activepieces plus CMS integrations; capture recordings and code samples. - Improve SDKs and quickstarts showing how to wrap LLM calls with retries, validation, and audit logging. - Publish case studies comparing naive “agent-only” prototypes vs orchestrated designs, with metrics on failures avoided. - Run a survey on developers’ biggest pain points with AI reliability and observability to inform docs and demos. #### OSPO - Draft governance for AI orchestration components: who owns workflows, how changes are approved, and how incidents are handled. - Ensure license clarity for workflow and AI libraries (particularly source-available tools), documenting acceptable use internally. - Integrate SBOM practices to include AI components and orchestration services, tracking critical paths in enterprise workflows. - Establish guidance for using open-source orchestrators in regulated environments, including data residency and audit trail practices. - Support upstream contributions to key orchestration projects used in-house. #### Community - Create example repositories that demonstrate three-layer designs (workflow → AI → tools) for common community use cases. - Host public clinics on debugging AI workflows and adding deterministic fallbacks. - Encourage contributors to add observability to AI features (structured logs, dashboards, tracing). - Work with sponsors to fund reliability and testing improvements for critical orchestration components. - Coordinate incident simulation drills focused on AI misbehavior inside workflows. #### WordPress - **Core**: Explore minimal primitives for describing workflows and events that can safely host AI steps (without baking in a specific vendor). - **Core Performance**: Measure the cost of inside‑out AI (agent at every step) versus outside‑in orchestration, and set performance budgets. - **Core AI**: Define responsible AI guidelines emphasizing deterministic wrappers, evaluation sets, and clear failure modes. - **Core Program Team**: Align AI feature timelines with infrastructure for logging, rollback, and monitoring across teams. - **Mobile**: Prototype mobile flows where AI suggests actions but deterministic controllers own publishing and destructive operations. - **Playground**: Offer sandboxes to experiment with AI‑driven workflows safely, including broken/edge-case scenarios. - **Community**: Add “AI + workflows” sessions to meetups/WordCamps, with patterns and anti-patterns. - **Training**: Produce lessons on integrating LLMs into content workflows with deterministic review/approval gates. - **Marketing**: Tell stories about reliable AI features—not just capabilities—highlighting guardrails and user trust. - **Support**: Track tickets related to AI misbehavior to identify workflow failures vs model issues. - **Design**: Clarify UX for AI suggestions vs guaranteed actions; design explicit confirmation and review steps. - **Documentation**: Document best practices for plugin authors adding AI (logging, retries, validation, human-in-the-loop). - **Photos / TV / Polyglots**: Consider AI-assisted tagging/captioning/translation only through controlled workflows with human review queues. - **Accessibility**: Ensure AI does not bypass accessibility checks; embed them as deterministic steps. - **Meta**: Improve .org tooling to visualize plugin/theme workflows that include AI. - **CLI**: Add commands enabling safe scripted AI operations guarded by flags, dry runs, and logs. - **Hosting / Tide / Openverse / Themes / Plugins**: Define how AI is allowed in pipelines (e.g., code review suggestions, content analysis) with deterministic enforcement and monitoring. #### The WPOCC - Fund experiments around AI‑orchestrated workflows that remain host-agnostic and open. - Support maintainer stipends for building and documenting reference architectures of AI + workflows in the WordPress ecosystem. - Pilot transparent funding for upstream orchestration projects that WordPress depends on. ### FAIR - Encourage architectures where critical AI orchestration metadata (logs, policies, signatures) live on infrastructure controlled by the community or operator, not just vendors. - Promote covenants against capture of orchestration layers (e.g., data portability, exit clauses, open APIs). - Package governance artifacts—charters, operator agreements, scoring rubrics—for evaluating AI workflow tools’ openness and reliability. --- ## 🧪 Hypotheses 1. Systems that combine deterministic workflows with AI steps will exhibit significantly fewer production incidents than AI-agent-only systems. 2. Enterprises will increasingly adopt outside‑in architectures for processes touching compliance, finance, or safety-critical operations. 3. Open-source or source-available workflow tools will gain disproportionate share of AI orchestration workloads over the next decade. 4. Inside‑out AI architectures will dominate in developer-centric use cases (e.g., coding) but not in multi‑stakeholder enterprise processes. 5. Adding explicit three-layer designs (workflow, AI decisions, deterministic tools) will improve auditability and regulator acceptance of AI systems. --- ### 📊 Methodology - **Comparative incident analysis**: Collect incident reports from organizations using AI agents with and without workflow orchestrators; compare frequency and severity of failures. - **Field experiments**: Implement the same business process using (a) outside‑in and (b) inside‑out architectures, measuring reliability, MTTR, and operational cost. - **Adoption trend study**: Track stars, downloads, and enterprise case studies for open vs closed orchestrators; correlate with cloud usage and AI adoption metrics. - **Regulatory readiness survey**: Survey compliance teams about which architectures they find more explainable and auditable. - **Usability tests**: Run tasks where developers debug failures in both architectures, measuring time to diagnosis and fix. --- ### ❓ Questions - Under what conditions does the flexibility of inside‑out agents outweigh their opacity and cost? - How should we standardize logs and traces for AI steps embedded inside workflows? - What minimal orchestration features must be present before an AI-enabled system can be considered “enterprise-ready”? - How do we prevent orchestration layers from becoming new points of lock‑in? - What role should open governance (beyond open source code) play in AI orchestration platforms? --- ### 🔮 Future Research - Formal models and best practices for composing AI agents with deterministic controllers. - Reference architectures for common domains (support, finance, healthcare) with clear patterns for AI insertion points. - Tooling for end‑to‑end simulation and testing of AI‑infused workflows, including adversarial scenarios and drift. - Governance frameworks for shared orchestration infrastructure across organizations. - Techniques for explaining AI‑driven workflow decisions to auditors and end users. --- ### 💡 Implications - **Practice**: Architects should treat workflows and orchestration as first-class design concerns, not just wires around the “real” AI. - **Policy**: Regulators may require explicit control layers, logs, and deterministic fallbacks for AI in sensitive processes. - **Security**: Attackers might target orchestrators, so they need strong auth, integrity, and tamper‑evident logs. - **Education**: Engineering curricula should teach AI systems design with control layers and failure modes, not just models and prompts. - **Funding**: Investing in open orchestration infrastructure may yield outsized leverage across many AI use cases. --- ### 🧠 Learning Styles Assessment - Reading level estimate: Approx. Flesch–Kincaid Grade 10–11 (conceptually advanced but written in clear, short sentences). #### 👀 Visual learners - Present: Mental model of three nested layers; concrete, stepwise examples. - Improve: Add diagrams showing data and control flow between workflow, AI, and tools; side‑by‑side outside‑in vs inside‑out charts. #### 🔊 Aural learners - Present: None explicitly. - Improve: Provide a short audio summary or podcast episode walking through the Vivint incident and the two architectures. #### 🗣️ Verbal learners - Present: Clear definitions (outside‑in, inside‑out, orchestration), examples, and analogies (Linux/PostgreSQL). - Improve: More story-like case studies showing before/after states for real teams. #### 🧩 Kinesthetic learners - Present: Conceptual workflows that could be implemented in tools like n8n. - Improve: Hands-on labs where readers build a small workflow with AI and deterministic steps, plus checklists. #### 🧠 Logical learners - Present: Structured comparison of two architectures and a three-layer model. - Improve: Decision trees for choosing architectures; explicit cost–benefit tables. #### 🧑‍🤝‍🧑 Social learners - Present: Implied collaboration between ops, compliance, and engineering. - Improve: Discussion prompts and group exercises, e.g., designing an AI-safe workflow for a critical business process. #### 🧘‍♀️ Solitary learners - Present: A self-contained argument that can be read and reflected on privately. - Improve: Reflection questions about readers’ own systems and where AI should/shouldn’t be inserted. #### 🌿 Naturalistic learners - Present: Indirect analogies to ecosystems of infrastructure (Linux, PostgreSQL). - Improve: Use ecological metaphors (e.g., keystone species/infrastructure) and consider environmental impact of inefficient architectures. --- ### 📊 Accessibility Assessment #### Screen reader semantics - Present: Standard article structure with headings and paragraphs (assuming typical blog markup). - Improve: Ensure proper heading levels, descriptive alt text for the abstract image, and language attributes. #### Keyboard navigation - Present: Likely standard keyboard navigation if theme is accessible. - Improve: Confirm skip links, clear focus states, and that interactive components (if any) are reachable by keyboard. #### Contrast and readability - Present: Depends on site theme; text appears simple and uncluttered. - Improve: Verify contrast ratios, line height, and font size; avoid low-contrast accents for important information. #### Text resizing and reflow - Present: Modern blogs typically responsive. - Improve: Test at 200% zoom; ensure no clipping; respect prefers-reduced-motion for any animations. #### Media and captions - Present: Static images only. - Improve: Provide alt text and image descriptions; consider optional audio version. #### Forms and interactions - Present: Minimal (likely comments or newsletter). - Improve: Ensure labels, error messages, and touch targets meet WCAG guidance. #### Standards reference - Map gaps and checks to WCAG 2.2 AA criteria for perceivable, operable, understandable, and robust content. --- ### 🌍 DEIB Assessment #### Cultural breadth - Present: Enterprise software and open source examples with global relevance. - Improve: Highlight non‑US/EU case studies and diverse organizations adopting open orchestration. #### Inclusive language - Present: Neutral, technical language with minimal jargon. - Improve: Add a short glossary for less familiar terms (orchestration, probabilistic, inside‑out). #### Neurodiversity support - Present: Short paragraphs, clear argument flow. - Improve: More subheadings, bullet lists, and consistent use of patterns for comparing architectures. #### Representation - Present: Mostly technology and companies, not people. - Improve: Include stories from diverse teams or regions using these patterns. #### Psychological safety - Present: Non‑blaming tone about failures (“learned it the hard way”). - Improve: Add explicit encouragement for transparent incident reporting and learning. --- ### ✅ Logical Validity #### Strong reasoning - **Inductive & causal**: From a concrete failure (Vivint surveys) to the general claim that AI needs deterministic orchestration; strength comes from clear causal chain and alignment with known LLM properties. - **Analogical**: Comparison to Linux/PostgreSQL as past examples of open infrastructure winning once layers become critical; plausible, though contingent on similar dynamics. #### Fallacies - Possible **hasty generalization**: One case study (Salesforce/Vivint) plus a few tools may not fully justify broad predictions; could be strengthened with more empirical cases. - Potential **appeal to tradition/precedent**: Assuming open source will win orchestration because it won servers/databases; may need explicit discussion of countervailing forces (cloud platforms, closed SaaS). #### Tone of voice - Tone is pragmatic and forward‑looking, with phrases like “Salesforce learned it the hard way” and “I would love to see them succeed” conveying calm critique and cautious optimism. --- ### 🛡️ DARVO Analysis - **Deny**: No denial or minimization of AI risks; the article is frank about reliability problems. - **Attack**: No attacks on individuals or organizations; Salesforce is used constructively as a learning example. - **Reverse victim/offender**: No attempt to recast perpetrators as victims; the focus is on systemic design issues, not blaming or victimhood. - Overall, there is no DARVO pattern; the rhetoric is analytical rather than defensive. --- ## 🎯 Conclusion The article argues that AI’s real power in enterprises depends on the *control layers* around it. LLMs are inherently probabilistic, so reliable systems must wrap them in deterministic workflows and tools. Two architectures—outside‑in and inside‑out—will coexist, often as nested layers. As these orchestration layers become essential infrastructure, open-source platforms are well-positioned to play the same role Linux and PostgreSQL did in prior waves. Practical next steps: audit current or planned AI features for missing control layers, choose where deterministic workflows must own the process, and design AI as a “judgment component” inside robust, observable, and auditable systems. --- ## 🧱 Refactor Appendix (append after 🎯 Conclusion) ### 📚 Literature Split Pad - Purpose: Stage Konik CEE units that become per-source Literature notes. - Split level: **H3**. - Include verbatim quotes under 25 words and a [view highlight] link. ### [H1] 5–10-word present-tense claim > [!quote] [{Title or entity}]({ARTICLE_URL}) by [[{Author or [MISSING]}]] via [[{Site or [MISSING]}]]. Published {DATE_OR_[MISSING]}. Accessed . > “Short quote, verbatim, ≤25 words.” Two lines: context and use. Cite data if relevant. [view highlight]({ARTICLE_URL}#H1) #xref #articleseed ID:rw_[MISSING] ### [H2] 5–10-word present-tense claim > [!quote] [{Title or entity}]({ARTICLE_URL}) > “Short quote, verbatim, ≤25 words.” Two lines: context and use. [view highlight]({ARTICLE_URL}#H2) #xref ID:rw_[MISSING] --- ### 🔖 Atomic Split Pad (Zettel Candidates) - Purpose: Promote reusable claims into Atomic notes. - Split level: **H4**. - Each item: one idea, present tense, specific, testable. #### [Z1] 5–10-word present-tense claim - Scope/condition in one line. - Pointer to page/time or dataset. - Links: [[Parent-Concept]] [[Related-Idea]] #### [Z2] 5–10-word present-tense claim - Scope/condition in one line. - Pointer to page/time or dataset. - Links: [[Parent-Concept]] [[Related-Idea]] --- ### ✏️ Fleeting Queue (Extract by Selection) - Purpose: Capture quick actions or sparks for Fleeting notes. - Extraction: **Select one line** → “Refactor: Extract selection to new note (content only).” - Keep each line short and self-contained. - [F1] Short action — context/{ARTICLE_URL} #fleeting #inbox - [F2] Short action — context/{ARTICLE_URL} #fleeting #inbox - [F3] Short action — context/{ARTICLE_URL} #fleeting #inbox --- <!-- deep-read:sections end -->