Your Error Budgets Don’t Know AI Exists  🔗 **Source:** [Your Error Budgets Don’t Know AI Exists](https://leaddev.com/ai/your-error-budgets-dont-know-ai-exists) > [!note]+ Document Notes > tags: [error-budgets, slos, ai-assisted-development, engineering-leadership, software-reliability] > author: LeadDev > source: LeadDev > reference: https://leaddev.com/ai/your-error-budgets-dont-know-ai-exists > date created: 2026-09-30 10:13 > > > [!info] Generated analysis — not the author's words. Sections below are machine-written > > from the captured source. Only text inside `> [!quote]` blocks is verbatim from the author. > > Do not quote or attribute any other line to the named author. > > # 🏷️ Title: Your error budgets don't know AI exists > <!-- deep-read:sections start --> ## 🧭 Table of contents - [📝 Summary](#-summary) - [📖 Main content](#-main-content) - [🗒️ Notes](#️-notes) - [📚 References](#-references) - [🗣️ Paraphrase](#️-paraphrase) - [📰 Newsletter](#-newsletter) - [✅ Action items](#-action-items) - [🧪 Hypotheses](#-hypotheses) - [📊 Methodology](#-methodology) - [❓ Questions](#-questions) - [🔮 Future research](#-future-research) - [💡 Implications](#-implications) - [🧠 Learning styles assessment](#-learning-styles-assessment) - [📊 Accessibility assessment](#-accessibility-assessment) - [🌍 DEIB assessment](#-deib-assessment) - [✅ Logical validity](#-logical-validity) - [🛡️ DARVO analysis](#️-darvo-analysis) - [🎯 Conclusion](#-conclusion) - [🔄 Refactor appendix](#-refactor-appendix--your-error-budgets-dont-know-ai-exists) ## 📝 Summary - **What happened.** LeadDev published a practitioner opinion piece on 2026-09-30 in its AI section. The captured text has no byline, so this analysis credits the site. The narrator is an unnamed senior engineering leader who writes from a leadership seat ("As senior leaders, we set the tone," ¶34). This is commentary and advocacy built on one hypothetical scenario and the author's own review experience. It isn't a study. - **The claim.** Error budgets and SLOs measure how much failure a service has consumed. They don't measure whether the people who own the service understand why a change behaves the way it does. AI cuts the cost of producing a change but not the cost of understanding it (¶13), so a team can ship fast, roll back cleanly and still be unable to explain a failure (¶6). - **The scenario.** A change fails, rollback is quick, the 99.9% SLO holds, and the after-action review produces "what changed and when" but no explanation of why (¶5-¶7). - **The prescription.** Talk to the owning team of your primary critical service without the go-to engineer and ask them to explain the changes, alternatives and risks (¶22-¶23). Read recurring conflict (reopened MRs, rollbacks that become debates, decisions escalated upward) as telemetry (¶24-¶25). Don't build a comprehension score (¶29). Treat a change that depends on concentrated knowledge, single points of failure or only AI models as riskier (¶30). Give the team authority to push a release back while the service is still healthy (¶33). - **Why it matters.** The piece argues dashboards can stay green while organizational understanding thins, so the first visible signal is an outage, not a metric (¶20). - **Key stakeholders.** Senior engineering leaders who approve changes, owning teams and their go-to engineers, SRE and platform teams who run SLOs, and AI tool vendors. - **Risks and tradeoffs.** Delay has a cost: the author would postpone the next change "by an hour or a day a week" (¶21). The piece offers no data on incidence, and no test for what counts as enough comprehension. The claim that fewer people hold a mental model (¶20) rests on the author's experience. - **Commercial interest.** LeadDev sells events. ¶35-¶36 in the capture is a promotion for LeadDev Berlin on November 9 and 10, 2026. The link list also points to a coding-assistant roundup and a buying checklist (see References). The captured text has no disclosure statement. - **Capture gaps.** The "Key takeaways:" line (¶0) has no bullets in the capture. ¶1 to ¶3 read like the missing bullets, as they sit right after the empty heading and summarize the piece, but the capture doesn't label them as such. Treat that as an inference, not a fact. ¶35-¶36 is an event ad that landed mid-article. - **Bottom line.** The gap between shipping a change and understanding it is a real management concern and the remedies are cheap to try. The specific claims that AI widens the gap, and that conflict patterns predict outages, are the author's observations, not results. ## 📖 Main content **The core argument.** SLOs measure service reliability. With a 99.9% target, the error budget is 0.1% downtime, "roughly 43 minutes per month" (¶9). Leaders use that number to decide whether to keep changing a service at the current pace (¶10). The author says budgets and SLOs "were never intended" to show how well a team understands a service (¶11). That understanding used to come as a side effect of writing code, questioning a review comment and chasing a failed test (¶11). With AI a team can produce a working change while doing less of that work (¶12). **The "unnamed accounting."** ¶13-¶14 say output scales and understanding doesn't. The error budget still prices failure. What changed is "the collateral behind the change": you can no longer infer the owners can explain a change, challenge its assumptions or modify it safely when the expected path breaks (¶14). **The old versus new workflow.** Before frontier models, shipping more software meant building a mental model, hitting walls, and learning discernment from a mentor or Stack Exchange (¶16). Now developers use models to write changes and tests, code review is increasingly agentic, and documentation is generated before anyone explains a decision in their own words (¶17). Leaders see productivity, and the effort that produced understanding has disappeared (¶18). **Evidence of process is not explanation.** In reviews the author has sat through, everyone could point to the ticket, approver, test result and rollback, and the conversation stalled when asked why the system behaved that way (¶18-¶19). The author says risk rises even as delivery numbers improve, because fewer people can explain a consequential change "without going back to the prompt or the tool" (¶20). **Detection and decision.** The author gives a probe (a conversation with the owning team minus the go-to engineer, ¶22-¶23), a set of leading indicators (recurring conflict, ¶24-¶25) and a decision rule (¶30). The rule: a material change that depends on concentrated knowledge, single points of failure or solely on AI models is riskier than a similar change the team can explain and operate without them. The author calls this a leadership decision (¶21, ¶39). **Terms.** - *SLO* is a target for service reliability, here 99.9% uptime. - *Error budget* is the allowed unreliability implied by the SLO, here 0.1% downtime. - *MR* is a merge request, GitLab's term for a pull request. - *Burn* means consuming error budget (¶39). **Contested or unverified.** (1) That comprehension per change is falling because of AI: asserted in ¶13 and ¶20 with no measurement. (2) That recurring conflict is an early signal of missing understanding (¶24-¶26): plausible, and no data accompanies it. (3) That postponing a change costs less than learning about the gap in production (¶21, ¶39): a judgment, not a computed tradeoff. (4) That a comprehension score is a "bad idea" (¶29): stated with one supporting sentence about dashboards treating numbers as judgment. (5) The 43-minute figure in ¶9 matches a 30-day month at 99.9% (43.2 minutes); verify against your own window. ## 🗒️ Notes #### 📄 ¶0-¶7 The green dashboard and the failed change - ¶0 is "Key takeaways:" with no bullets in the capture. ¶1-¶3 carry the three takeaways in effect: dashboards can look fine while nobody can explain a failure, watch recurring conflict, and skip the comprehension score. - ¶2 lists the leading indicators: reopened MRs, rollbacks turning into debates, decisions escalated that teams should settle themselves. - ¶4-¶5 set the scene: healthy metrics, faster delivery, rework in margins, agentic code review and QA, then a consequential failure with a quick rollback and minimal customer impact. - ¶6-¶7: teams can show what changed and the approval chain and can't explain why it failed. The author says the dashboards "did not lie" and were asked something they weren't built to answer. #### 📄 ¶8-¶14 The unnamed error budget accounting - ¶9: a 99.9% SLO gives a 0.1% downtime allowance, roughly 43 minutes per month. - ¶10: for leadership, the budget shows how much unreliability a service can absorb before pace or risk gets reconsidered. - ¶11-¶12: understanding used to be a by-product of writing code, defending it in review and debugging tests. AI lets a team skip much of that work. - ¶13: AI cuts the cost of producing a change without cutting the cost of understanding it, so an organization can create more change with less comprehension per change. - ¶14: the budget still prices failure, and what changed is the collateral behind the change. #### 📄 ¶15-¶21 The work got cheaper - ¶16: before frontier models, shipping more meant time in the problem, hitting walls and learning discernment from a mentor or Stack Exchange. - ¶17: models write changes and tests, agentic reviews come back annotated, and documentation is generated before anyone explains the decision. Leaders read this as what they paid for. - ¶18-¶19: reviews can show ticket, approver, test result and rollback and still fail on "why did the system behave that way?" - ¶20: risk goes up while delivery numbers improve, since fewer people hold a mental model. Nothing on the dashboard has to turn red. - ¶21: if a team must rebuild a material change from prompt history to explain it, the author wants to know before approving the next one and prefers a delay of an hour or a day a week. #### 📄 ¶22-¶27 Testing for understanding and reading conflict as telemetry - ¶22-¶23: talk to the team that owns your primary critical service without the go-to engineer. Remaining members should explain what changed, what alternatives were considered and what risks are inherent. - ¶23: if they can't do so without AI or the key engineer, the risk is already in the operating model. - ¶24-¶25: a lack of understanding shows first as recurring conflict: a trivial MR reopened repeatedly, a routine rollback that becomes a debate, escalations to leadership. - ¶26: individually minor, together they suggest a team carrying responsibility without understanding. - ¶27: the org chart says who's accountable, and the real knowledge is somewhere else (a third party, a prompt or a model). #### 📄 ¶28-¶34 From comprehension to change risk - ¶29: the first impulse is a comprehension score, which the author rejects because dashboards already treat a number as judgment. - ¶30: the decision rule: changes that depend on concentrated knowledge, single points of failure or solely on AI models are riskier than changes the team can explain and operate without them. - ¶31: labeling AI-produced changes as higher risk isn't a verdict on AI work. Existing assurances are incomplete, and the author adds how much operating knowledge exists inside the owning team. - ¶32: if losing one engineer forces the team to rebuild comprehension from old prompts, that's a single point of failure. The bar is that an absence or attrition doesn't make the next change "archeology." - ¶33: give the team authority to push a release back while the service is healthy. Insisting on an SLO breach first means telling them to come back after something has failed. - ¶34: leaders set the tone. Slowing down when understanding has thinned is the cost of keeping control, not resistance to AI adoption. #### 📄 ¶35-¶39 Event promotion and close - ¶35-¶36: an insert for LeadDev Berlin, November 9 and 10, 2026, with a tagline about the gap between leadership expectations and what's possible. It sits mid-article in the capture. - ¶37: back to the scenario: service restored, customers unaffected, error budget intact, no explanation of the failure. The author won't let the green dashboard erase that. - ¶38: change got cheaper and faster, understanding didn't. Leaders should decide what level of understanding is required before the next change moves. The author says both speed and understanding are achievable with AI. - ¶39: waiting for the SLO to burn makes production the place where the organization finds what it stopped understanding. ## 📚 References The captured link list has 15 entries, and all 15 point to leaddev.com. None points to a study, dataset, SRE text or vendor documentation. Anchors and URLs below come from that list. What each page says isn't checked here because nothing was fetched. **Links tied to a phrase in the text** - [measure you traditionally trust](https://leaddev.com/ai/the-8-software-engineering-metrics-ai-broke) sits on the ¶4 sentence about services looking healthy by every trusted measure. The URL title suggests an article on engineering metrics AI broke, and it's the nearest link to the piece's central claim. The capture doesn't say which metrics it covers. - [Writing the code](https://leaddev.com/velocity/writing-code-was-never-the-bottleneck) anchors the ¶11 list of what builds understanding. - [Code reviews](https://leaddev.com/software-quality/level-your-code-reviews) anchors the agentic code review sentence in ¶17. - [Risk](https://leaddev.com/technical-direction/managing-technical-risk) anchors the ¶20 "Risk has increased" sentence. - [mental model](https://leaddev.com/leadership/4-mental-models-more-effective-engineering-leadership) anchors the phrase in ¶20. It points to a general leadership mental-models page, so it isn't a source for the claim that fewer people hold a system mental model. - [AI](https://leaddev.com/ai/as-ai-helps-us-write-more-code-whos-catching-the-bugs) is the second "AI" anchor (¶20-¶21 region; the list doesn't give the exact paragraph). It reads as related reading on AI-written code and bug detection. - [lack of understanding](https://leaddev.com/leadership/ai-makes-critical-thinking-harder-to-build-in-junior-engineers) anchors ¶24 and points to LeadDev's piece on critical thinking in junior engineers, an opinion piece of the same kind (see the finished analysis of that article in this vault's deep-read set). - [AI models](https://leaddev.com/ai/frontier-ai-models-haemorrhage-sensitive-data) anchors the phrase in ¶27 or ¶30 about reliance on a model. The URL title concerns sensitive data leakage, which is a different topic than comprehension. - [losing one engineer](https://leaddev.com/hiring/3-ways-to-bounce-back-from-sudden-team-cuts) anchors ¶32. It points to a page about sudden team cuts, adjacent to the bus-factor idea and not a study of it. - [AI has made change cheaper and faster,](https://leaddev.com/ai/ai-generated-code-sparks-production-confidence-crisis) anchors the ¶38 sentence and points to a LeadDev page whose URL title mentions a production confidence crisis. It could be the closest thing to supporting evidence, and its contents aren't visible here. - [engineering teams](https://leaddev.com/ai/designing-human-agent-engineering-teams) anchors the early-text phrase "engineering teams" (¶6 region). **Navigation, promotional or self-referential links** - [AI](https://leaddev.com/ai/best-ai-coding-assistants) and [With AI](https://leaddev.com/ai/your-ai-coding-tools-buying-checklist-for-2026) point to a coding-assistant roundup and a 2026 buying checklist. Both are commercially adjacent to the article's topic, and the second one is anchored on "With AI" in ¶12, which reads as a sponsored-content style insertion or an editorial cross-link. The capture can't tell which. - The first entry (empty anchor) and the second entry (anchor text "elephant mouse disagree with your boss manager") both point to the article's own URL. The second anchor text is unrelated to the piece and reads like a stray tag or related-story label picked up from page markup. **Claims with no matching link in the captured markup** - The 99.9% SLO and 43-minute figure (¶9). It's a worked example, and no SRE source is linked. - That understanding per change falls under AI use (¶13, ¶20). - That recurring conflict predicts outages (¶24). - The decision rule in ¶30, which is the author's own. **Also relevant (not in the link list, to verify)** - Google's SRE Book chapter on service level objectives and the SRE Workbook chapter on implementing SLOs define error budgets and burn-rate alerting. Verify the current text before citing. - [Article URL](https://leaddev.com/ai/your-error-budgets-dont-know-ai-exists), published 2026-09-30. - No byline appears in the captured text, so no author profile is available. ## 🗣️ Paraphrase The author describes a service that looks healthy on every measure leaders trust. A consequential change fails, the rollback is quick and the SLO holds, yet in the review no one on the team can say why the change broke things. They can show what changed, when, who approved it and that it passed QA. The author's point is that the error budget was never built to tell you whether people understand the system they're changing. AI, the author says, makes producing code, tests, reviews and documentation cheaper, and doesn't do the same for understanding. Teams used to learn a system by writing the code and arguing about it in review. Now they can ship without that, and the paperwork of a change (ticket, approval, test result, rollback) can look complete while the explanation is missing. The author suggests a simple probe: ask the owning team of a critical service to explain recent changes without their go-to engineer and without asking an AI. Watch for repeated fights over small decisions as a warning sign. The author doesn't want a comprehension score. The proposed rule is that a change relying on one person's knowledge, or only on AI, is riskier than one the team can explain and run. The team should be allowed to hold a release while the service is still healthy, so nobody has to burn error budget to justify caution. The author says slowing down for understanding isn't resistance to AI. ## 📰 Newsletter **Your dashboard is green. Can your team explain the last change?** LeadDev's new opinion piece argues that SLOs and error budgets track how much failure a service absorbed and say nothing about whether the owning team understands why a change works or breaks. It's one author's view built on a hypothetical incident, and it comes with a cheap probe: ask the team that owns your critical service to explain recent changes without their go-to engineer or an AI, and treat repeated MR reopenings and rollback debates as signals. For WordPress teams, the parallel is plugin and theme review. A patch can arrive with tests, a green CI run and an AI-written description while the author can't say what it assumes. Hosting and Core Performance teams that run reliability targets could add a "who can explain this change?" check before big releases. Link: [the article](https://leaddev.com/ai/your-error-budgets-dont-know-ai-exists). The piece also carries a LeadDev Berlin event promo, so read it as a publisher's editorial with a commercial neighbor. ## ✅ Action items #### DevRel - Developer advocacy: draft a talk outline on "SLOs measure failure, not understanding," with the ¶6 scenario as the opening and the ¶22-¶23 probe as the takeaway. - Education: add an exercise where learners explain a passing, AI-generated change's assumptions before an instructor reveals the bug. - Developer experience: in reliability and CI quickstarts, add a short "who can explain this change without the prompt?" checklist. - Developer marketing: cite this piece as a practitioner opinion. Don't present its comprehension claims as measured. - Community management: no direct action from this source. Nearest relevance: the ¶24-¶25 conflict pattern (reopened MRs, escalations) maps to moderation signals in contributor channels. - Internal/external/cross-project: agree with SRE and support on one shared definition of "understands the change" before using it in reviews. #### OSPO - Policy and governance: add an expectation that contributors of AI-assisted changes can explain alternatives considered and risks, echoing ¶23. - License compliance: no direct action from this source. - Security management: treat "the team can't explain why it works" as a review-depth trigger for security-sensitive patches. The source doesn't cover security directly. - Software inventory: per ¶30 and ¶32, record for each critical component whether more than one maintainer can explain it (a single-point-of-failure column). - Developer support: offer office hours on explaining and safely modifying AI-generated code. - Community engagement: no direct action from this source. - Strategic alignment: add a bus-factor style indicator per critical service to executive reporting, and don't invent a comprehension score, per ¶29. #### Community - Maintainers: run the ¶22 probe on one critical component. Ask co-maintainers to explain the last three material changes without the lead maintainer. - Contributors: in "first issue" guides, ask for a short note on what the patch assumes and what alternatives were rejected. - Sponsors: when funding critical projects, ask for a second maintainer who can explain the change history, per ¶32. - Ecosystem: no direct action from this source. Nearest relevance: a cross-project session on AI-assisted release decisions. #### WordPress - Core: no direct action from this source. Nearest relevance: release leads could ask component maintainers for a "who can explain this?" check on high-risk changes. - Core Performance: performance budgets and regressions are the WordPress analogue of the error budget in ¶9-¶10. Consider adding an owner-can-explain check next to any budget-based release gate. - Core AI: for guidance on AI-assisted contribution, include the ¶23 standard (explain the change, alternatives and risks without reaching for AI). - **Core Program Team:** no direct action from this source. Nearest relevance: retrospectives could look for the ¶24-¶25 conflict pattern. - Mobile: no direct action from this source. - Playground: build a lab where an AI-suggested change passes the tests and fails in a realistic scenario, so learners must explain why. - Community: no direct action from this source. Nearest relevance: a WordCamp talk on reading team conflict as telemetry. - Training: add a "explain this change without the prompt" exercise to contributor learning paths. - Marketing: no direct action from this source. - Support: no direct action from this source. - Design: no direct action from this source. - Documentation: add a handbook note on documenting the "why" of a change in the contributor's own words, since ¶17 says generated documentation can precede the explanation. - Photos: no direct action from this source. - TV: no direct action from this source. - Polyglots: no direct action from this source. - Accessibility: no direct action from this source. - Meta: no direct action from this source. Nearest relevance: .org services with SLOs could apply the ¶33 rule that a team may hold a release while healthy. - CLI: no direct action from this source. - Hosting: the article's SLO and error budget framing applies directly. Add a pre-release "owning team can explain this change" gate alongside SLO burn alerts, and give on-call teams explicit authority to hold a release while the service is healthy (¶33). - Tide: no direct action from this source. - Openverse: no direct action from this source. Nearest relevance: ingestion pipelines with reliability targets could use the same ownership probe. - Themes: no direct action from this source. Nearest relevance: reviewers can ask AI-assisted authors to state assumptions. - Plugins: same as Themes, with extra depth on security-adjacent patches. #### The WPOCC - Programs: if The WP Open Community Collective funds maintainers, count "second person who can explain this component" as an outcome. - Projects: no direct action from this source. - Funding experiments: no direct action from this source. Nearest relevance: a small grant to write a template for the ¶22 probe. ### FAIR - Technical independence: no direct action from this source. Nearest relevance: single points of failure in metadata or signing knowledge, per ¶32. - Policies and covenants: no direct action from this source. - Productized governance artifacts: add a "critical knowledge holders" row to any operator scorecard. ## 🧪 Hypotheses 1. Teams with AI-assisted changes can restate a material change's assumptions and alternatives at a lower rate than teams that wrote the changes by hand, measured in a blind after-action exercise. 2. Recurring conflict signals from ¶25 (reopened MRs, rollback debates, escalations) in a quarter predict a higher incident rate in the following quarter for the same service. 3. Services where at least two engineers can explain each of the last five material changes have fewer "unable to explain cause" post-incident reviews than services where one engineer can. 4. Giving a team explicit authority to hold a release while the SLO is healthy reduces incidents caused by changes the team flagged as poorly understood. 5. Reviews that require the author to explain the change without the prompt history catch more wrong assumptions than reviews based on approval evidence alone. 6. A single-point-of-failure flag per change (¶30) has better predictive value for post-release incidents than an AI-authorship flag. ## 📊 Methodology - **H1.** Recruit 30 to 40 engineers. Give each a set of matched changes, half produced with AI and half by hand. Blind graders score written explanations of assumptions, alternatives and risks against a rubric. Report effect size and inter-rater agreement. - **H2.** Pull 12 months of MR, rollback and escalation data across 20 services. Define the conflict indicators up front (an MR reopened 3 or more times, a rollback with a follow-up thread over a set length). Model next-quarter incidents while controlling for team size, deploy frequency and service age. - **H3.** Survey teams on who can explain recent changes, then join it to post-incident reviews tagged by cause knowledge. Use a comparison of teams with one versus multiple explainers, matched by service tier. - **H4.** Roll out hold authority to half of a set of comparable teams for 2 quarters. Track releases held, incidents tied to flagged changes, and lead time. Randomize by team. - **H5.** Insert a planted flawed assumption in a sample of PRs. Compare catch rates under approval-evidence review and explain-without-the-prompt review. - **H6.** Retrospectively label 100 post-release incidents for authorship (AI, human, mixed), knowledge concentration and severity. Compare the predictive value of each flag with a simple model. - **Controls throughout.** Stratify by tenure and stack, record the AI tools and versions used, and pre-register outcomes. ## ❓ Questions - What data supports the claim that comprehension per change is falling (¶13), and does any linked LeadDev page contain it? - How large is the delay cost in ¶21 ("an hour or a day a week") compared with the incident cost it aims to avoid? - How would a team tell "cannot explain the change" from "can explain it but doesn't want to reopen the debate"? - If not a score, what evidence shows that comprehension is thin, beyond a conversation and conflict signals? - Do the conflict signals in ¶25 also appear in teams with no AI use, and what does that imply for the causal story? - Who gets the authority to hold a release (¶33), and what stops it from becoming a veto that slows every change? - How does the ¶30 rule treat a change written by a person who has since left, with no AI involved? - Do SLO tools need a new signal, or is this a governance problem that no dashboard can carry? ## 🔮 Future research - **Measurement of comprehension.** The source rejects a score and gives no alternative instrument. A cheap, structured explain-the-change protocol could be validated against incident outcomes. - **Conflict as a leading indicator.** No data links reopened MRs or rollback debates to later outages. A retrospective study across teams could test it. - **Cost of holding releases.** Compare the delay cost of the author's proposal with incident cost in real deployment histories. - **Knowledge concentration.** Study how AI-assisted workflows change bus factor per service over time. - **Vendor design.** Test whether AI assistants that record rationale, alternatives and risks at change time improve later explanation quality. ## 💡 Implications - **Practice.** Add an explain-the-change check to approval for consequential changes, next to the existing evidence chain of ticket, approval, tests and rollback (¶18). - **Policy.** Give owning teams explicit authority to hold a release while the service is healthy (¶33), and write down who can override it. - **Security.** Changes nobody can explain are harder to patch safely under pressure. Treat unexplained changes in sensitive areas as a review-depth trigger. The source doesn't address security itself. - **Education.** Training should include explaining and defending changes without the tool that produced them. - **Funding.** Funders of critical open source can ask for more than one knowledgeable maintainer per critical component (¶32). - **Observability.** SLO and error budget tooling stays a failure-cost meter. Understanding needs a different, human signal. ## 🧠 Learning styles assessment - **Reading level estimate.** Flesch-Kincaid Grade Level 8.6 and Flesch Reading Ease 60.7 on the captured text (1,643 words, 108 sentences, computed with a simple syllable heuristic, so treat it as an estimate). That's plain English at roughly a 9th-grade level. #### 👀 Visual learners - Present: no diagram or chart appears in the captured text. - Improve: a timeline of the scenario (change, failure, rollback, review, gap) and a diagram showing what the error budget measures against what it doesn't. #### 🔊 Aural learners - Present: no audio version appears in the captured text. - Improve: a 3-minute audio summary and a discussion prompt built on the ¶22 probe. #### 🗣️ Verbal learners - Present: the "unnamed accounting" and "collateral behind the change" metaphors (¶14) and the scenario framing give words for the idea. - Improve: a short glossary defining SLO, error budget, MR and burn for readers outside SRE. #### 🧩 Kinesthetic learners - Present: the ¶22-¶23 conversation probe is a concrete exercise a leader can run this week. - Improve: a script with sample questions and a one-page scoring guide for the explain-without-AI session. #### 🧠 Logical learners - Present: ¶9 gives arithmetic (0.1% of a month is roughly 43 minutes), and ¶30 states a decision rule. - Improve: a decision table for when a change should be held, with the three conditions from ¶30 as columns. #### 🧑‍🤝‍🧑 Social learners - Present: the piece centers team conversation, escalation patterns and leadership tone. - Improve: retro prompts such as "which decisions did we reopen last month, and why?" #### 🧘‍♀️ Solitary learners - Present: the reflective scenario invites a leader to test their own service. - Improve: a private worksheet: pick one critical service, list its last three material changes, and rate how many owners can explain each. #### 🌿 Naturalistic learners - Present: the "collateral" and "archeology" imagery gives some concrete anchors. - Improve: an ecosystem analogy for knowledge spread across a team versus concentrated in one person. ## 📊 Accessibility assessment Judged from the captured text and link list only. The rendered page, alt text and styles weren't inspected. #### Screen reader semantics - Present: the text arrives as short paragraphs. Several lines read as headings ("The error budget accounting we never had to name," "The work got cheaper," "Comprehension to change risk"), but the capture doesn't show heading levels. - Improve: the capture holds 15 links. Anchors include single generic words ("AI" twice, "Risk," "With AI"), one empty anchor and one unrelated phrase, "elephant mouse disagree with your boss manager." That risks WCAG 2.2 AA 2.4.4 (Link Purpose, In Context). An empty link would fail 4.1.2 if it's a real link. #### Keyboard navigation - Present: unknowable from captured text. - Improve: verify focus order, skip link and visible focus (2.4.1, 2.4.3, 2.4.7) on the live page, including the mid-article event promo. #### Contrast and readability - Present: contrast can't be seen from text. Paragraphs are short and the reading level is around grade 9. - Improve: check body text at 4.5:1 (1.4.3) and check the promo block's text colors. #### Text resizing and reflow - Present: unknowable from captured text. - Improve: test at 200% zoom and 320 CSS px (1.4.4, 1.4.10). #### Media and captions - Present: no media referenced in the captured text. The text mentions no audio, video, captions or transcript. - Improve: the "Key takeaways:" block (¶0) has no bullets in the capture. If it's a graphic or a collapsed element, confirm it's exposed as text (1.3.1). The ¶35-¶36 promo may be an image or embed and needs a text equivalent (1.1.1). #### Forms and interactions - Present: no forms in the captured text. - Improve: the event promo probably has a call to action. Check its label, purpose and target size (2.4.6, 2.5.8). #### Standards reference - Gaps visible from the capture: 2.4.4 (generic and unrelated link text), 4.1.2 (empty anchor), 1.3.1 (¶0 bullets missing, ¶35-¶36 structure unclear). Everything else needs the rendered page. ## 🌍 DEIB assessment #### Cultural breadth - Present: the framing is workplace-generic and names no country. The event promo names Berlin. - Improve: the argument rests on one leadership seat and one hypothetical. Add cases from different company sizes, regions and regulated industries. #### Inclusive language - Present: "engineers," "teams" and "leaders" are used without gendered terms. The text defines SLO and error budget in context. - Improve: idioms and jargon ("archeology," "collateral," "go-to engineer," "burn") add load for translators. "MR" isn't expanded. #### Neurodiversity support - Present: short paragraphs and a scenario-then-remedy structure. The rule in ¶30 is stated in one sentence. - Improve: the "Key takeaways" list is empty in the capture, and the remedies aren't summarized in one place. #### Representation - Present: one unnamed narrator and an implied team of engineers. - Improve: no engineers, SREs or team members are quoted, so the perspective of the people being tested by the ¶22 probe is inferred, not reported. #### Psychological safety - Present: ¶31 says AI-produced work "is not lesser by any means," and ¶34 frames slowing down as accountability, not resistance. - Improve: the ¶22 probe could feel like an exam. Add guidance for running it so engineers aren't shamed if they can't answer. ## ✅ Logical validity This is an opinion and advocacy piece from a practitioner, so the assessment weights internal coherence and how far claims exceed the evidence offered. LeadDev's commercial neighbors (event promo, coding-assistant roundup, buying checklist) are noted under References. #### Strong reasoning - **Scope of the metric, ¶9-¶11.** An SLO at 99.9% is about availability, and the arithmetic in ¶9 matches a 30-day month (43.2 minutes). The claim that it says nothing about team understanding follows from what the metric counts, without needing outside data. - **Cost asymmetry, ¶13.** Output scales with AI and understanding doesn't necessarily, and the hedge "does not necessarily" is honest. A structural claim like this is testable. - **Process versus explanation, ¶18-¶19.** Having a ticket, approver, test result and rollback proves process happened. It doesn't prove anyone can explain the behavior. This is a valid distinction and it doesn't depend on AI. - **Decision rule, ¶30 and ¶33.** Conditioning risk on concentrated knowledge, single points of failure and sole AI dependence is coherent, and it uses standard bus-factor logic. Letting a healthy service hold a release avoids the perverse incentive of needing a breach first. - **Concession, ¶31 and ¶38.** The author says AI work isn't lesser and that speed and understanding are both achievable. That limits overreach. #### Fallacies - **Anecdotal evidence, ¶18.** "I have sat in enough reviews where everyone could point to the typical evidence." One leader's experience carries the empirical weight, and the count is unstated. Fix: cite incident data or state the limit. - **Hypothetical as evidence, ¶4-¶7 and ¶37.** The opening scenario is constructed ("In our scenario"). It illustrates and doesn't prove that such gaps are common. Fix: label it illustrative and add a real, anonymized case. - **Correlation implied as prediction, ¶24-¶26.** Recurring conflict is offered as a leading indicator, and the same behavior has other causes (unclear ownership, personality clashes, poor requirements). Fix: state alternative explanations and propose a test. - **Possible false dilemma, ¶21 and ¶34.** The framing offers speed at all cost or delay for understanding. Middle paths (lightweight explain-the-change checks, pairing) get little space. Fix: compare graded options. - **Unsupported empirical claim, ¶13 and ¶20.** That understanding per change is falling and fewer people hold mental models has no number in the captured text, and none of the linked pages is tied to it by the anchor text. Fix: cite the source or say "in my experience." - **Nostalgia framing, ¶16.** Pre-AI teams are described as building mental models by living in the problem, which ignores teams that shipped without understanding before AI. Fix: acknowledge uneven outcomes in the old system. - **Conflict of interest, not stated.** The publisher sells events and links to a coding-tools buying checklist. The capture has no disclosure. Fix: a one-line note on relationships, if any. #### Tone of voice - Sober, first-person and advisory. The tone is that of a senior leader speaking to peers ("As senior leaders, we set the tone," ¶34; "That is a leadership decision," ¶21). It uses controlled imagery ("archeology," "collateral") and short declarative closers ("That is too late, and that decision belongs to us," ¶39). It reads as a risk memo and offers no data. ## 🛡️ DARVO analysis Not applicable to this source. DARVO describes a defensive response to an accusation, and this is a leadership opinion piece with no accuser or accusation. For completeness: **Deny.** Nothing is denied, and ¶31 and ¶38 concede AI's value. **Attack.** No critics are attacked, and the pressure lands on leaders' own metrics and incentives (¶34). **Reverse victim/offender.** No such move appears. The nearest edge is ¶27, which places responsibility on organizations that name an owner while the real knowledge sits elsewhere. That's a structural critique, not a recasting of blame. ## 🎯 Conclusion The article's durable point is that an error budget prices failure and doesn't price understanding, so a green dashboard can hide an organization that can't explain its own changes (¶14, ¶20). The remedies are cheap to try: run the ¶22 probe on one critical service, watch for recurring conflict, judge risk by whether the team can explain and operate a change without one person or a model (¶30), and let the team hold a release while the service is healthy (¶33). The evidence is thin. The piece is opinion built on a hypothetical incident and the author's review experience, and none of the 15 links points to data on the claim. Two capture issues matter: the "Key takeaways" bullets are missing at ¶0, and the LeadDev Berlin promo (¶35-¶36) sits inside the text. The SRE background (error budget definitions, burn rates) isn't linked, so verify it in Google's SRE Book and Workbook. **Next steps for readers:** run the ¶22 conversation for one critical service and record what the team couldn't explain. Add "who can explain this change?" to your release gate for one quarter and compare incidents. Before citing the article's claims, read the linked LeadDev pages on metrics AI broke and production confidence, since they may hold the data. **To verify (no network in this pass):** Google SRE Book and Workbook definitions of SLO, error budget and burn-rate alerts. That 99.9% over 30 days equals 43.2 minutes (arithmetic check). Contents of the two linked LeadDev pages on AI-broken metrics and the production confidence crisis. Whether the ¶12 "With AI" and ¶4 links are editorial or promotional placements. The original rendering of ¶0 to recover the key takeaways. ## 🔄 Refactor Appendix — Your error budgets don't know AI exists > Density: 2–5 Literature atoms (H3 pad), 0–3 Permanent candidates (H4 pad). > Emitted: 4 Literature, 2 Permanent. Resonance filter applied: the event promo (¶35-¶36) and the > arithmetic in ¶9 were dropped as low-value for the vault. > > Konik claim-header rules: present tense, declarative, ≤10 words, searchable, unique. > > Atomicity: one declarative sentence per atom. "and"/"also" → split. > > Tag pool: #bmf · #addendum · #xref · #articleseed · #storystem · #share · #aphorisms ### 📚 Literature Split Pad #### [H1] AI cuts change cost but not understanding cost > [!quote] [¶13] > AI can reduce the cost of producing a change without reducing the cost of understanding it. **Context & use**: - ¶13, the center of the article's argument. - A structural claim the source states without data. Keep it paired with a note that it needs testing. **Tags**: #xref #articleseed <!-- atomicity: 1 idea ✓ --> #### [H2] Error budgets price failure, not comprehension > [!quote] [¶14] > The budget is still pricing failure. What changed is the collateral behind the change. **Context & use**: - ¶14, the "unnamed accounting" idea in one line. - Usable as a short framing for reliability reviews and postmortem templates. **Tags**: #xref #aphorisms <!-- atomicity: 1 idea ✓ --> #### [H3] Recurring conflict is early telemetry > [!quote] [¶24] > This lack of understanding is visible as recurring conflict before it becomes an outage. **Context & use**: - ¶24, introducing the reopened MR, rollback debate and escalation pattern in ¶25. - The claim is a plausible leading indicator with no data in the source. Treat it as a hypothesis. **Tags**: #xref #share <!-- atomicity: 1 idea ✓ --> #### [H4] Teams need authority to hold releases early > [!quote] [¶33] > If we insist on an SLO breach, or burning error budget first, we are telling them to come back after something has failed. **Context & use**: - ¶33, in the passage arguing a team should be able to push a release back while the service is healthy. - Directly usable as a governance argument for a release hold policy. **Tags**: #xref #addendum <!-- atomicity: 1 idea ✓ --> ### 🧠 Atomic Split Pad #### [P1] Reliability metrics don't measure understanding **Claim** (one sentence, Courtney's voice): A service level objective counts failure that already happened, so a green dashboard says nothing about whether the owners can explain the next change. **Evidence pointer**: [[#H1]] · [[#H2]] **Scope & conditions**: Holds for availability-style SLOs and error budgets. It weakens where teams also track change comprehension by other means, and the source gives no measure of comprehension. **Links**: - [[Error budgets and SLOs]] — because extends - [[Bus factor]] — because contrasts on measuring people risk versus service risk - [[Release gates for WordPress hosting]] — because applies-in pre-release checks at a host <!-- atomicity: 1 idea ✓ --> #### [P2] Hold authority must precede the breach **Claim** (one sentence, Courtney's voice): A team that can pause a release only after an SLO breach will learn about its knowledge gaps in production. **Evidence pointer**: [[#H4]] · [[#H3]] **Scope & conditions**: Applies where owning teams face delivery pressure from above. It doesn't cover cases where the hold power is abused as a veto, which needs a defined override path. **Links**: - [[Psychological safety]] — because extends - [[Incident review practices]] — because evidence-for the value of early signals - [[AI-assisted contribution policy]] — because applies-in maintainers asking for an explanation before merge <!-- atomicity: 1 idea ✓ --> ### ✏️ Fleeting Queue - [F1] Read the linked LeadDev pages on metrics AI broke and the production confidence crisis to see whether they support the ¶13-¶20 claims, https://leaddev.com/ai/the-8-software-engineering-metrics-ai-broke #fleeting #inbox - [F2] Recover the "Key takeaways" bullets (¶0) and confirm what ¶1-¶3 are on the live page, since the capture lost the bullets and the ¶35-¶36 promo landed mid-text #fleeting #inbox - [F3] Check Google's SRE Book and Workbook on error budgets and burn rates against the article's framing (no network in this pass) #fleeting #inbox - [F4] Try the ¶22 probe on one critical service or plugin: ask the owners to explain the last three changes without the lead maintainer or an AI #fleeting #inbox - [F5] Article seed: "Your dashboard doesn't know your team stopped understanding the code." How WordPress release gates could add an explain-it check #fleeting #inbox #articleseed ### → Things 3 (max 2 actions) - [ ] Read the two linked LeadDev pages on AI-broken metrics and production confidence and note what data they hold, reading, 30 min #things - [ ] Draft a one-page "explain this change" probe for a critical service using the ¶22-¶23 questions, hosting reliability practice #things <!-- deep-read:sections end --> 📄 **Full text:** [[Full Document Contents/Articles/Your Error Budgets Don’t Know AI Exists.md|Your Error Budgets Don’t Know AI Exists by LeadDev]] ## Highlights - AI can reduce the cost of producing a change without reducing the cost of understanding it. Code, tests, documentation, reviews, and even parts of QA can now be produced with far less human contact than before. Output scales, but understanding does not necessarily scale with it. The organization can create more change while carrying less comprehension per change. ([View Highlight](https://read.readwise.io/read/01m3sazn2vzkwypfj9t8z73c3y)) - Note: [H1] AI cuts change cost but not understanding cost - Tags: atom/literature/h1 - This is where the unnamed accounting lives. The error budget still tells you how much failure the service has consumed. Now you can no longer infer that the people responsible for that service can explain why a change behaves the way it does, or if they can challenge its assumptions or safely modify it when the expected path breaks. The budget is still pricing failure. What changed is the collateral behind the change. ([View Highlight](https://read.readwise.io/read/01m3sazqahfb41smtnhajg3k37)) - Note: [H2] Error budgets price failure, not comprehension - Tags: atom/literature/h2 - This lack of understanding is visible as recurring conflict before it becomes an outage. You have to be open to reading it as telemetry. ([View Highlight](https://read.readwise.io/read/01m3sazstgxzy7t8sxhq8dtwcj)) - Note: [H3] Recurring conflict is early telemetry - Tags: atom/literature/h3 - We have a team rushing to make changes to a critical service and system they do not clearly understand. If you have gone through the exercise above, it should be clear to them what’s missing in relation to comprehension. This team needs the authority to push a release back, and so should the teams supporting its delivery, while the service is healthy. If we insist on an SLO breach, or burning error budget first, we are telling them to come back after something has failed. ([View Highlight](https://read.readwise.io/read/01m3sazw8fersh1jrsynpc8w3x)) - Note: [H4] Teams need authority to hold releases early - Tags: atom/literature/h3