Is AI Killing Your Software Quality Standards?  🔗 **Source:** [Is AI Killing Your Software Quality Standards?](https://www.youtube.com/watch?v=yTL4IKTjcuI) > [!note]+ Document Notes > tags: [ai-code-quality, engineering-standards, code-review-automation, software-development-ai, technical-governance, ai-assisted-development] > author: [MISSING] > source: Deep Dev > reference: https://www.youtube.com/watch?v=yTL4IKTjcuI > date created: {{date:YYYY-MM-DD}} {{time:HH:mm}} > > # 🏷️ Title: Is AI killing your software quality standards? > <!-- deep-read:sections start --> ## 🧭 Table of Contents - [📝 Summary](#-summary) - [📖 Main Content](#-main-content) - [🗒️ Notes](#️-notes) - [📚 References](#-references) - [🗣️ Paraphrase](#️-paraphrase) - [📰 Newsletter](#-newsletter) - [✅ Action Items](#-action-items) - [🧪 Hypotheses](#-hypotheses) - [📊 Methodology](#-methodology) - [❓ Questions](#-questions) - [🔮 Future Research](#-future-research) - [💡 Implications](#-implications) - [🧠 Learning Styles Assessment](#-learning-styles-assessment) - [📊 Accessibility Assessment](#-accessibility-assessment) - [🌍 DEIB Assessment](#-deib-assessment) - [✅ Logical Validity](#-logical-validity) - [🛡️ DARVO Analysis](#-darvo-analysis) - [🎯 Conclusion](#-conclusion) ## 📝 Summary - **What happened**: A panel discussion explored how AI code generation is transforming software quality standards and code review processes across engineering organizations. - **Why it matters**: Traditional human-only code review cannot scale to match AI's code generation volume, requiring fundamental shifts in testing, standards enforcement, and engineering culture. - **Key stakeholders**: Engineering teams, CTOs, product directors, and organizations using AI code generation tools like Copilot, with particular focus on safety-critical systems. - **Risks and tradeoffs**: Risk of losing code quality control versus gaining development velocity; maintaining human accountability while leveraging AI automation; potential skill atrophy as humans review less code. - **Bottom line**: Standards still matter but must evolve - focus shifts from documenting "what" to "why," from human-scale review to AI-assisted governance, and from static rules to dynamic, enforceable guardrails. ## 📖 Main Content The fundamental challenge facing engineering organizations is that AI can now generate code faster than humans can meaningfully review it. This creates a scalability crisis where traditional quality gates become bottlenecks rather than safeguards. **Standards Evolution**: Engineering standards must shift from human-centric to AI-partnership models. Clean, readable, maintainable code remains crucial because: 1. Humans still debug production issues at critical moments 2. Standards provide the control interface between humans and AI agents 3. AI agents perform better on well-structured, consistent codebases **Documentation Philosophy Change**: The focus must shift from documenting "what the code does" (which AI can infer) to documenting "why decisions were made" - business context, architectural tradeoffs, and intent that AI cannot derive independently. **Governance Through Skills and Rules**: Organizations are implementing "skills" - packaged bundles of rules with context about when and how to apply them, rather than flat rule lists. This enables different standards for different code areas (payment processing vs. internal tooling). **Risk-Based Review Strategy**: Teams are adopting confidence-based review processes where AI-generated code receives different scrutiny levels based on risk assessment and AI confidence scores. High-confidence changes in low-risk areas may ship with minimal human review. ## 🗒️ Notes #### ⏱️ [00:06:42](https://www.youtube.com/watch?v=yTL4IKTjcuI&t=402) Vernon on scaling challenges - Traditional testing didn't scale even without AI - now "rocket fuel" added to code output - Identity shift required: testers must become enablers providing tools rather than gatekeepers - Mixed approach needed: deterministic code uses traditional methods, AI-generated code needs new approaches #### ⏱️ [00:12:15](https://www.youtube.com/watch?v=yTL4IKTjcuI&t=735) Almog on enduring value of standards - Standards more important than before because humans still debug critical production issues - Clean code serves as interface between humans and AI - lowering standards means losing control - AI agents work better on clean, consistent codebases - "garbage in, garbage out" #### ⏱️ [00:18:30](https://www.youtube.com/watch?v=yTL4IKTjcuI&t=1110) Dominic on architectural documentation - Moving toward documenting architectural decisions explicitly for AI context - Using code review agents that understand company-specific API standards - Risk-based approach: different review rigor for different code areas #### ⏱️ [00:22:45](https://www.youtube.com/watch?v=yTL4IKTjcuI&t=1365) AI review vs human review discussion - AI reviews handle volume and context that humans cannot - Need different tools for generation vs review to avoid bias - Human-in-the-loop remains essential for accountability and safety #### ⏱️ [00:28:20](https://www.youtube.com/watch?v=yTL4IKTjcuI&t=1700) Dominic on governance through AI - Using AI to enforce communication standards in project updates - Skills that prompt engineers to think through problems rather than just report status - Codifying coaching practices that previously required manual intervention #### ⏱️ [00:34:10](https://www.youtube.com/watch?v=yTL4IKTjcuI&t=2050) Almog on governance implementation patterns - Converting tribal knowledge into enforceable rules - Skills as "next generation rules" with context and enforcement methods - Connecting same standards across generation and review phases - Analytics on rule performance to create feedback loops #### ⏱️ [00:45:15](https://www.youtube.com/watch?v=yTL4IKTjcuI&t=2715) Vernon on Bainbridge's paradox - Automation leaves only difficult edge cases for humans - Humans get worse at spotting rare problems due to infrequency - AI's "jagged frontier" - excellent at complex tasks, fails at simple ones #### ⏱️ [00:48:30](https://www.youtube.com/watch?v=yTL4IKTjcuI&t=2910) Dominic on shifting quality left - AI automates data gathering for debugging, reducing manual investigation time - Engineers focus on judgment rather than log collection and correlation - Confidence scoring allows automated triage and assignment ## 📚 References - [Kodo AI Code Review Platform](https://kodo.ai) - [Inspiring Senior Living Platform](https://inspiring.com) - [Phrase Localization Platform](https://phrase.com) - Lisanne Bainbridge's 1983 paper on automation paradox - Ethan Mollick's "jagged frontier" concept ## 🗣️ Paraphrase Engineering organizations face a fundamental scaling challenge as AI generates code faster than humans can review it. While traditional code review processes become bottlenecks, the solution isn't to abandon standards but to evolve them for AI partnership. Standards remain crucial because humans still handle critical debugging, clean code provides the control interface with AI, and AI performs better on consistent codebases. However, the focus must shift from documenting what code does to why decisions were made, capturing business context and architectural tradeoffs that AI cannot infer. Organizations are implementing "skills" - contextual rule bundles that specify when and how standards apply to different code areas. Risk-based review processes use AI confidence scores to determine review rigor, with high-confidence changes in low-risk areas potentially shipping with minimal human oversight. The human role evolves from comprehensive reviewer to governance designer and edge-case handler, though Bainbridge's automation paradox warns that humans may become less skilled at spotting the rare critical issues that remain their responsibility. ## 📰 Newsletter 🤖 **Is AI Killing Code Quality? Not If You Do It Right** The latest Deep Dev panel tackled every engineering leader's burning question: how do we maintain quality when AI generates code faster than we can blink? The answer isn't what you'd expect. **The Plot Twist**: Standards matter *more* now, not less. But they need to evolve from human-scale to AI-partnership scale. Think "skills" instead of rules - contextual bundles that know when and where to apply standards. **What's Working**: Teams are using risk-based review (high confidence + low risk = minimal review), documenting *why* instead of *what*, and treating AI as a partner that needs clean code to perform well. **The Reality Check**: We're all figuring this out together. Even the experts admit they don't have all the answers yet. 🔗 **Worth Your Time**: - Watch the full discussion for real-world implementation strategies - Check out Kodo's AI code review platform for automated standards enforcement - Consider how your team's "tribal knowledge" could become enforceable rules The future isn't about choosing between AI or humans - it's about designing systems where both work together without compromising on quality. ## ✅ Action Items #### DevRel - Document AI-assisted development best practices for developer education programs - Create workshops on evolving code review practices in AI-augmented workflows - Develop guidelines for maintaining code quality standards with AI pair programming - Build example repositories showing effective human-AI collaboration patterns #### OSPO - Review contribution policies for AI-generated code acceptance criteria - Establish guidelines for AI tool usage in open source contributions - Create standards for documenting AI assistance in commit messages and PR descriptions - Develop security scanning policies that account for AI-generated code patterns #### Community - Facilitate discussions on evolving code review practices in maintainer forums - Create templates for documenting architectural decisions that AI can understand - Develop mentorship programs for transitioning to AI-assisted development workflows - Establish best practices for risk-based code review in open source projects #### WordPress - **Core**: Evaluate how AI-generated patches should be reviewed and attributed - **Core Performance**: Establish performance testing standards for AI-generated optimizations - **Documentation**: Create guidelines for documenting "why" decisions in architecture - **Plugins**: Develop review criteria for AI-assisted plugin submissions - **Themes**: Create standards for AI-generated theme code quality and maintainability - **Security**: Implement additional scrutiny processes for AI-generated security-related code #### The WPCC - Fund research into effective governance models for AI-assisted open source development - Support development of tools that help maintainers enforce standards at scale - Create grants for projects developing AI-assisted code review capabilities ## 🧪 Hypotheses 1. **Standards Evolution Hypothesis**: Engineering standards will shift from prescriptive "what" documentation to contextual "why" documentation as AI handles implementation details 2. **Risk-Based Review Hypothesis**: Organizations will adopt confidence-scoring systems for AI-generated code that automate low-risk, high-confidence changes 3. **Skill Atrophy Hypothesis**: Developers will lose code review skills as AI handles routine reviews, making them less effective at catching critical edge cases 4. **Quality Interface Hypothesis**: Code quality standards will become the primary interface for human-AI collaboration in software development 5. **Governance Automation Hypothesis**: Tribal knowledge and informal best practices will be systematically converted into enforceable, AI-readable rules 6. **Tooling Specialization Hypothesis**: Different AI tools will specialize in code generation vs. code review to avoid bias and improve quality ### 📊 Methodology - **Comparative Analysis**: Study code quality metrics before and after AI adoption across different review strategies - **Longitudinal Studies**: Track developer skill retention and code quality outcomes over 6-12 month AI adoption periods - **A/B Testing**: Compare traditional review vs. confidence-scored review approaches on similar codebases - **Ethnographic Research**: Observe how teams naturally evolve their practices when introducing AI code generation - **Survey Research**: Collect data on developer confidence, stress levels, and perceived code quality across different governance models ### ❓ Questions - How do we measure code quality when the definition of "good code" changes for AI partnership? - What governance structures prevent teams from over-relying on AI confidence scores? - How do we maintain developer skills for handling edge cases they rarely encounter? - What role should regulatory compliance play in defining minimum human review requirements? - How do different AI tools' biases compound when used together in generation and review? ### 🔮 Future Research - Development of standardized metrics for AI-human collaborative code quality - Investigation of optimal confidence scoring algorithms for different code domains - Studies on long-term effects of reduced human code review on developer expertise - Research into effective training methods for human reviewers in AI-augmented environments - Exploration of regulatory frameworks for accountability in AI-assisted critical systems ### 💡 Implications **Practice**: Engineering teams must redesign their entire quality assurance workflows, moving from human-centric to AI-partnership models while maintaining accountability. **Policy**: Organizations need new governance frameworks that define appropriate uses of AI in different risk contexts and maintain clear human responsibility chains. **Security**: Traditional security review processes may miss AI-generated vulnerabilities that follow new patterns, requiring updated threat models and detection methods. **Education**: Computer science and engineering curricula must evolve to teach AI collaboration skills alongside traditional programming and review techniques. **Funding**: Investment in AI-assisted development tools should prioritize explainability and human oversight capabilities rather than pure automation. ### 🧠 Learning Styles Assessment **Reading level estimate**: Grade 14-16 (undergraduate/professional level). Uses technical vocabulary and complex concepts but maintains clear structure. #### 👀 Visual learners - **Present**: Video format with multiple speakers, screen sharing capabilities - **Improve**: Add architectural diagrams showing human-AI workflow integration, code review process flowcharts, before/after comparison visuals #### 🔊 Aural learners - **Present**: Panel discussion format, multiple perspectives and voices - **Improve**: Provide chapter markers, key quote highlights, audio summary of main points #### 🗣️ Verbal learners - **Present**: Clear definitions of "skills," "confidence scoring," and technical terms - **Improve**: Add glossary of AI development terms, more analogies for complex concepts #### 🧩 Kinesthetic learners - **Present**: Real-world examples from practicing CTOs and engineering leaders - **Improve**: Add hands-on exercises, implementation checklists, step-by-step migration guides #### 🧠 Logical learners - **Present**: Clear problem-solution structure, cause-effect relationships in AI adoption - **Improve**: Add decision trees for review strategies, quantitative metrics for success measurement #### 🧑‍🤝‍🧑 Social learners - **Present**: Panel format with multiple perspectives, Q&A interaction - **Improve**: Discussion prompts for teams, collaborative worksheets for standards definition #### 🧘‍♀️ Solitary learners - **Present**: Individual reflection opportunities through Q&A questions - **Improve**: Self-assessment tools for current AI readiness, personal implementation planning templates #### 🌿 Naturalistic learners - **Present**: Evolution metaphors for changing standards and practices - **Improve**: Ecosystem thinking about AI tools, sustainable development practices with AI ### 📊 Accessibility Assessment #### Screen reader semantics - **Present**: Video with spoken content, clear speaker identification - **Improve**: Provide detailed transcript with speaker tags, screen reader compatible slides #### Keyboard navigation - **Present**: Standard video controls, Q&A interface - **Improve**: Keyboard shortcuts for video navigation, accessible timestamp links #### Contrast and readability - **Present**: Standard video interface with adequate contrast - **Improve**: High contrast mode option, customizable subtitle appearance #### Text resizing and reflow - **Present**: Video format with scalable playback - **Improve**: Responsive transcript layout, adjustable text size for supplementary materials #### Media and captions - **Present**: Transcript provided post-discussion - **Improve**: Real-time captions during live discussion, downloadable transcript with timestamps #### Forms and interactions - **Present**: Q&A submission interface - **Improve**: Clear error messages, accessible form validation, multiple submission methods ### 🌍 DEIB Assessment #### Cultural breadth - **Present**: Speakers from multiple geographic regions (Canada, UK, Israel) - **Improve**: Include perspectives from developing markets, different regulatory environments #### Inclusive language - **Present**: Professional, accessible technical language - **Improve**: Avoid assumptions about team sizes, tool access, or organizational maturity levels #### Neurodiversity support - **Present**: Clear structure with defined segments and topics - **Improve**: Provide content outline, summary bullets, predictable transition patterns #### Representation - **Present**: Mix of technical and product leadership perspectives - **Improve**: Include diverse organizational sizes, industry verticals, role types #### Psychological safety - **Present**: Speakers openly admit uncertainty and ongoing learning - **Improve**: Acknowledge different risk tolerances, varying AI adoption stages, financial constraints ### ✅ Logical Validity #### Strong reasoning - **Almog's standards argument**: Strong deductive reasoning - if humans debug critical issues AND AI works better on clean code AND standards provide control, then standards remain essential - **Vernon's scaling analysis**: Sound inductive reasoning from existing automation patterns to current AI challenges - **Dominic's risk-based approach**: Logical application of risk management principles to code review processes #### Fallacies - **Appeal to novelty**: Assuming AI-generated code requires completely new approaches without considering proven software engineering principles - **False dichotomy**: Framing choices as either "AI or human" rather than collaborative approaches - **Hasty generalization**: Drawing broad conclusions about AI capabilities from limited organizational experiences #### Tone of voice - **Collaborative and humble**: Speakers consistently acknowledge uncertainty and learning ("we don't know what the hell we're doing") - **Practically focused**: Emphasis on real-world implementation over theoretical ideals - **Problem-oriented**: Honest about challenges while remaining solution-focused ### 🛡️ DARVO Analysis No DARVO patterns detected. The discussion maintains accountability focus, with speakers taking responsibility for quality outcomes rather than deflecting to AI tools. When challenges are acknowledged, they're framed as shared industry problems requiring collective solutions, not attempts to reverse responsibility. ## 🎯 Conclusion AI is not killing software quality standards - it's forcing them to evolve rapidly. The organizations that thrive will be those that view AI as a partner requiring new interfaces and governance models, not a replacement for human judgment. Standards become more important, not less, but must shift from documenting implementation details to capturing intent and context that only humans can provide. The practical path forward involves risk-based review processes, AI-readable rule systems, and maintaining human accountability while leveraging AI's scale advantages. Success requires honest acknowledgment that we're all figuring this out together, combined with systematic experimentation and measurement of what actually works. ## 🔄 Refactor Appendix — Is AI killing your software quality standards? ### 📚 Literature Atoms #### AI agents require enforced standards not just documentation > [!quote] ID:rw_1 > Don't just write your standards down. You need to make them enforceable. If it existed only in a document or in someone else's head they will never be a reality only a wish. **Context & use**: - Essential for governance frameworks when AI generates majority of codebase **Tags**: #ai-governance #engineering-standards #xref #### Clean code serves as human-AI communication interface > [!quote] ID:rw_2 > Clean code is clear names, small functions, consistent pattern. It's not just about looking nice. This is the interface between humans and AI. **Context & use**: - Architectural decisions for codebases with mixed human-AI authorship **Tags**: #code-quality #human-ai-interface #xref #### AI agents perform better on consistently structured codebases > [!quote] ID:rw_3 > Agents actually work better on clean code. AI models give better reviews, better suggestions, and make fewer mistakes when the codebase follow consistent patterns. **Context & use**: - Technical argument for maintaining code standards in AI-heavy environments **Tags**: #ai-performance #code-standards #xref #### Documentation must shift from describing what to explaining why > [!quote] ID:rw_4 > We need to shift towards documenting the intent and the context and not just the description of what the code does document the why not the what. **Context & use**: - Updating documentation practices when AI can infer functionality but not reasoning **Tags**: #documentation #ai-era-practices #xref #### Traditional testing approaches become inadequate with AI code volume > [!quote] ID:rw_5 > Having all of your testing go through the human tester didn't scale anyway. And now we've added rocket fuel to the process of outputting code. **Context & use**: - Strategic planning for quality assurance in high-velocity AI development environments **Tags**: #testing-strategy #scalability #xref #### Skills package rules with context and enforcement mechanisms > [!quote] ID:rw_6 > Skills is a package that bundles rules together with the context about when you should apply them and also how you should enforce them. **Context & use**: - Implementation strategy for contextual governance systems beyond static rule lists **Tags**: #skills-framework #rule-enforcement #xref <!-- deep-read:sections end --> 📄 **Full text:** ![[Full Document Contents/Articles/Is AI Killing Your Software Quality Standards.md|Is AI Killing Your Software Quality Standards? by YouTube]] ## Highlights - keep your standards alive when agents are writing most of the code today? It's one of the most important questions in engineering right now. So thanks for having me. Thank you, Alog. And next we'll move on to Dominique. >> Hi everyone. My name is Dominique. I'm the CTO at Inspiring. Um we build a full stack connected care platform for senior living. So we help residents that live in senior living communities with hardware, AI models on the edge, software. We have a huge data platform. So very very complex ecosystem with a ([View Highlight](https://read.readwise.io/read/01kv3sqpg6dsgt7sfttxcny14q)) - Tags: 👻 ai highlighted - standards and tests with integrity when AI agents begin producing code at a massive scale? >> I have no bloody idea. Like this is this is this is why I was super duper excited to be on the panel because we are we are literally trying to figure this out. And so, um, a slightly more serious answer is one important idea that that I have as we figure this out is actually this isn't ([View Highlight](https://read.readwise.io/read/01kv3sqpgsff76es66ez74zr8e)) - Tags: 👻 ai highlighted - the answer isn't new because having all of your testing go through the human call, the tester, forget AI, that didn't scale anyway. And now we've added rocket fuel to the process of outputting code. It's like good luck if you don't have any automated tests. Good luck if you don't have any pipelines or you know all that cool stuff that we've been trying to convince organizations in the industry to do for like the past 20 years. That's now table stakes. If you're not doing any of that, good luck. So that's one ([View Highlight](https://read.readwise.io/read/01kv3wqt0128m493ts77qn4dd3)) - Note: Human-gated testing never scaled; AI added "rocket fuel" to code output. Automated tests and pipelines — what we've pushed for 20 years — are now table stakes, not nice-to-haves. - Tags: testing-strategy, scalability, atom/literature/h1 - thing. Um but the second reason is about control standards is how you keep track of what agents are doing. If you let agents write messy, inconsistent code, you lose the ability to check what they did. Uh clean code is yeah it's clear names, small functions, consistent pattern. It's not just about looking nice. This is the interface between humans and AI. And when you lower those standards, you're giving up control and ([View Highlight](https://read.readwise.io/read/01kv3sqpjdw5ntx3nen5mkdjfv)) - Tags: 👻 ai highlighted - thing. Um but the second reason is about control standards is how you keep track of what agents are doing. If you let agents write messy, inconsistent code, you lose the ability to check what they did. Uh clean code is yeah it's clear names, small functions, consistent pattern. It's not just about looking nice. This is the interface between humans and AI. And when you lower those standards, you're giving up control and ([View Highlight](https://read.readwise.io/read/01kv3wpnk3rp0146b5abnext1g)) - Note: Clean code is the human–AI interface, not cosmetics: clear names, small functions, consistent patterns are how you keep the ability to check what agents did. Lowering standards = giving up control. - Tags: atom/literature/h1, code-quality, human-ai-interface - it's it's very dangerous and even irresponsible when I'm I'm the one that should be owning the ship to ship this product. Um, and third, and this is something that we see very clearly at Kodo, agents actually work better on clean code. A models give better reviews, better suggestions, and make fewer mistakes when when the codebase follow the consistent patterns. And it makes sense, right? Garbage garbage in, garbage out. We all know that. If your codebase is a ([View Highlight](https://read.readwise.io/read/01kv3wq1kk26yqt2majwmpjr6w)) - Note: Agents work better on clean code: models give better reviews/suggestions and make fewer mistakes when the codebase follows consistent patterns. Garbage in, garbage out. - Tags: ai-performance, atom/literature/h1, code-standards - mess, the agents working on, they produce messier outputs. it makes the codebase even worse and it makes everything messier and that's a really really bad cycle. So standards break that that cycle. Uh so yes standards matter but um some of our standards were built in a world where humans were the main bottleneck and that world is changing now. um core idea uh I uh clarity, maintainability, ([View Highlight](https://read.readwise.io/read/01kv3sqpkg6fwz8q4nmn6vxwqc)) - Tags: 👻 ai highlighted - trade-offs were made what's the business context I think we need to shift towards documenting the intent and the context and not just the description of what the code does document the why not the what so to wrap it up the question isn't whether the standards should exist yeah they should exist when AI agents write most of the code standard becomes the main way that we keep quality and control but we also need to be honest about which standards will still still serve us. Yeah. Right. After the whole ([View Highlight](https://read.readwise.io/read/01kv3wqcne5vgfd8ycvnec974j)) - Note: "Document the why, not the what" — shift docs toward intent, context, and trade-offs. When agents write most of the code, standards become the main way to keep quality and control. - Tags: documentation, atom/literature/h1, ai-era-practices - rules together with the context about when you should apply them and also how you should enforce them. Uh think of it this way. You might have a set of rules that only relevant for when you're working on your kernel code or your payment layer, authentication layer. These are different rules. So you want the skills to say uh these are the rules. This is where you need to apply them and here's how to check them. Maybe ([View Highlight](https://read.readwise.io/read/01kv3wrfapgqqrwnvzctx11jt3)) - Note: "Skills" package rules + context + enforcement: which rules, where they apply (kernel vs payment vs auth layer), and how to check them — so scoped standards travel with the work. - Tags: skills-framework, rule-enforcement, atom/literature/h2 - >> Fantastic. Thank you. And uh wrapping up will be along >> I say um don't just write your standards down. You need to make them enforceable. It's really really important. uh if it existed only in a document or in someone else's head they will never be a reality only a wish. So make them enforceable. >> Brilliant. Thank you. Thank you so much ([View Highlight](https://read.readwise.io/read/01kv3wpbv2d9ak131qrgwqf3yh)) - Note: "Don't just write your standards down. You need to make them enforceable." Standards living only in a doc or someone's head are a wish, not a reality — critical when agents write the code. - Tags: ai-governance, atom/literature/h1, engineering-standards