How We Introduced Quality Engineering and Made Everything Better!  🔗 **Source:** [How We Introduced Quality Engineering and Made Everything Better!](https://www.youtube.com/watch?v=8FISCXU2BNQ) > [!note]+ Document Notes > tags: [quality-engineering, continuous-deployment, dora-metrics, test-automation, shift-left, developer-experience] > author: Stuart Thomas > source: LeadDev (YouTube) > reference: https://www.youtube.com/watch?v=8FISCXU2BNQ > date created: 2026-10-15 17:43 > > > [!info] Generated analysis — not the author's words. Sections below are machine-written > > from the captured source. Only text inside `> [!quote]` blocks is verbatim from the author. > > Do not quote or attribute any other line to the named author. > > # 🏷️ Title: How we introduced quality engineering and made everything better! > <!-- deep-read:sections start --> ## 🧭 Table of contents - [📝 Summary](#-summary) - [📖 Main content](#-main-content) - [🗒️ Notes](#️-notes) - [📚 References](#-references) - [🗣️ Paraphrase](#️-paraphrase) - [📰 Newsletter](#-newsletter) - [✅ Action items](#-action-items) - [🧪 Hypotheses](#-hypotheses) - [📊 Methodology](#-methodology) - [❓ Questions](#-questions) - [🔮 Future research](#-future-research) - [💡 Implications](#-implications) - [🧠 Learning styles assessment](#-learning-styles-assessment) - [📊 Accessibility assessment](#-accessibility-assessment) - [🌍 DEIB assessment](#-deib-assessment) - [✅ Logical validity](#-logical-validity) - [🛡️ DARVO analysis](#️-darvo-analysis) - [🎯 Conclusion](#-conclusion) - [🔄 Refactor appendix](#-refactor-appendix--how-we-introduced-quality-engineering-and-made-everything-better) ## 📝 Summary - **What happened.** LeadDev published a 17:32 conference talk, uploaded 2026-10-14, in which Stuart Thomas (self-introduced at 00:00:07; the name comes from the transcript's own introduction, and the rest of that sentence is garbled) tells two stories. The first covers a quality and engineering transformation at a company the captions spell "Zuper," "Zupla" and "Zupller" (spelling unstable, so the real name is unconfirmed). The second covers carrying those lessons to a company the captions spell "Humara" (also unconfirmed). - **The claim.** A quality team that started with no coding experience moved a product from fortnightly manual releases to on-demand continuous deployment, and the effect went beyond the original lead-time goal. - **The numbers given.** Before: releases every 2 weeks, 3 weeks from earliest commit to production, recovery measured in days, change failure rate "close to 100%," regression testing that took two people multiple days. After: multiple deployments a day, lead time in hours, recovery in minutes, change failure rate under 5%. - **Why it matters.** The talk ties a DORA metric improvement (deployment frequency, lead time, mean time to recover, change failure rate) to specific practices: automated UI smoke tests, a deployment tool, tests-first work, short-lived branches, feature toggles, ephemeral environments, and alerting. - **Key stakeholders.** The quality team, the head of engineering, product and data colleagues who test under the shift-left model, business stakeholders who needed risk communication, and on-call engineers. - **Risks and tradeoffs.** Every figure is a single speaker's retrospective with no data shown in the transcript. Two organizations are anonymized by caption garble, the second story's results are given as "almost zero" without a number, and the causal story credits quality engineering for changes the speaker also says were shared with engineering. - **Bottom line.** The practices are standard and well aligned with DORA research. The transferable lesson is the sequencing: build trust with small, announced, reversible steps before automating the last manual hop. ## 📖 Main content **The starting point.** The first product shipped once a fortnight with all-manual testing. Regression took two people multiple days. A release meant taking the system offline for hours. There were multiple production instances, deployed days apart on purpose, because the team expected a change failure and wanted to fix it before touching the second environment. That choice is why deployment frequency was 2 weeks but lead time was 3. **The first three moves.** - **Automated testing, started by people who had never coded.** The quality team picked the same language the engineers used (the caption says "C," likely C# but unconfirmed), automated the release-time smoke tests through the UI, and ran them beside the manual runs. The automated runs caught more than tired humans did, so the team kept going and prioritized tests that were slow, error-prone, or disliked. - **Unblocked deployments.** Releases had been staff remote-accessing cloud servers and running copied scripts. The team introduced Octopus Deploy so people could see, review, change, and improve the scripts. The speaker argues the brand didn't matter. Having a visible, editable tool did, because it empowered people. - **Shift left, defined in two parts.** Part one is collaboration: anyone in delivery can test, and defining "good" up front means writing tests before code. That can be testable acceptance criteria in a manual environment, or red-green with predetermined UI selectors and API tests. Part two is process: an always-deployable main branch, short-lived feature branches, and feature toggles that let incomplete work ship dark and turn on for a test user. **Ephemeral test environments.** Single-use environments are built, deployed to, tested, and destroyed. The speaker calls them essential to an always-deployable main branch. They isolate changes and remove the queue for static test environments. CI builds, deploys, and runs automated tests without a human. **Trust as the main lever.** Each change was announced to stakeholders early, with its risks, mitigations, and a recovery plan. The speaker names process trust as the single most important part of the changes, ahead of how new or shiny the tooling was. **The deployment ramp.** Fortnightly became weekly, then twice a week, then daily, then twice daily. Early rungs were manual button-presses timed when engineers were available to support. The last step automated everything: a GitHub tag creates a release, the tag deploys to staging, regression runs, green results trigger production, and alerts go to Slack and, out of hours, to the on-call team through "Ops Genie" (likely Opsgenie). **Outcome and lessons.** The team reached continuous deployment on demand, with the system staying online during deploys. Not everything worked first time. One unpredicted issue: a daytime deployment emptied a cache, and the system ran badly with an empty cache, so the team fixed the underlying performance problem to allow daytime deploys. The speaker's stated lesson is that unknown unknowns happen and a recovery plan makes them survivable. **The second company.** The existing guided-selling product had a three-weekly cadence, five people spending a week on manual regression, and a low-quality "testim" suite (the caption spelling; likely the Testim tool) that nobody trusted. The team modernized test automation and deployment tooling and reached on-demand deploys with a low change failure rate. A new virtual-sales-agent product got CI/CD built for multiple daily deploys from the start, using ephemeral environments again. **What a quality engineer is.** The speaker's closing definition: someone who identifies problems and systemic friction and fixes them, so the software experience improves for everyone. The speaker says every change was led or heavily influenced by quality. **Contested or unverifiable in this source.** No slide data is in the transcript. The figures "close to 100%" and "less than 5%" are unsourced. The 2-weeks-to-3-weeks relationship is internally consistent. The claim that DevX and trust drive better code is asserted, not shown. ## 🗒️ Notes #### ⏱️ [00:00:07](https://www.youtube.com/watch?v=8FISCXU2BNQ&t=7) Introduction and the two stories - Speaker introduces self as Stuart Thomas, publishing on a site the captions render as "the qualityduck.co.uk" (verify the domain), on quality testing, developer experience, and leadership. The caption "bolding middle-aged white guy far to Ellis" is garbled and not interpreted here. - Story one: a quality and engineering transformation. Story two: carrying the learnings to a second company. - The internal headline at the first company was reduced lead time to deployment, the stated business goal, but the speaker says the impact was bigger. #### ⏱️ [00:01:34](https://www.youtube.com/watch?v=8FISCXU2BNQ&t=94) Baseline: fortnightly, manual, near-100% failure - Deployment frequency 2 weeks, earliest-commit-to-production 3 weeks, because multiple production instances were deployed days apart. - Recovery measured in days. First step was automated testing, started by a quality team with no coding history. > [!quote] [00:01:34] > our change failure rate was close to 100%. #### ⏱️ [00:03:02](https://www.youtube.com/watch?v=8FISCXU2BNQ&t=182) Why the automated tests won - Automated tests ran beside the manual ones, then replaced the slow, error-prone, and disliked scenarios. > [!quote] [00:03:02] > not only were the automated tests quicker, they were better at detecting issues than the tired humans were. - Outcome much later: a full regression pack and higher release confidence. #### ⏱️ [00:04:31](https://www.youtube.com/watch?v=8FISCXU2BNQ&t=271) Octopus Deploy and the meaning of shift left - Octopus Deploy replaced remote-access script runs. The speaker says the tool mattered because people could see, review, and improve the process. - Shift left part one: collaboration. Everyone in delivery can test, which reduces bottlenecks and over-the-fence handoffs. - Shift left means defining good up front: acceptance criteria in a testable format, or red-green tests. #### ⏱️ [00:06:43](https://www.youtube.com/watch?v=8FISCXU2BNQ&t=403) Branching strategy and feature toggles - Earlier test automation frees time for exploratory testing, which finds the edge cases that bite in production. - Branching practices named: always-deployable main, short-lived feature branches (fewer merge conflicts), feature toggles (ship incomplete features dark, enable for a test user). - Smaller, more frequent releases give feedback before an engineer has moved on to the next thing. #### ⏱️ [00:08:12](https://www.youtube.com/watch?v=8FISCXU2BNQ&t=492) Ephemeral environments - Single-use environments are built, deployed, tested, and destroyed. They isolate work and end the queue for static test environments. - CI builds the change, deploys it, and runs automated tests with no human interaction. Fast feedback lowers cognitive load. #### ⏱️ [00:09:40](https://www.youtube.com/watch?v=8FISCXU2BNQ&t=580) Building trust with stakeholders - Every change was communicated early, with risks, mitigations, and a recovery plan. The stated effect: stakeholders knew the team wasn't "cowboys." - Rule drawn from this: avoid big steps, build confidence in small chunks. #### ⏱️ [00:10:23](https://www.youtube.com/watch?v=8FISCXU2BNQ&t=623) The deployment ramp - Deployment frequency moved fortnightly, weekly, twice weekly, daily, twice daily. Early rungs were manual triggers timed for engineer support. - Final automation: GitHub tag creates the release, deploys to staging, runs automated regression, then deploys to production on green. #### ⏱️ [00:11:53](https://www.youtube.com/watch?v=8FISCXU2BNQ&t=713) Alerting and the before and after - Alerts go to Slack, and to the on-call team out of hours via "Ops Genie." Without alerting, an end-of-day deployment could fail while everyone logged off. - Start state: 100% manual testing, two-weekly releases, three-weekly lead time, recovery in days, change failure near 100%. - End state: continuous deployment on demand, regression automated, no customer impact during deploys. > [!quote] [00:12:37] > We've moved from low confidence and high risk to high confidence and low risk. - DORA metrics after: multiple deploys a day, lead time in hours, recovery in minutes, change failure rate under 5%. #### ⏱️ [00:13:24](https://www.youtube.com/watch?v=8FISCXU2BNQ&t=804) Lessons and the cache incident - Not everything works first time; a recovery plan makes that acceptable. Unknown unknowns exist. - Concrete case: a daytime deployment emptied a cache, the system performed poorly with it empty, and the team fixed the underlying performance issues so daytime deploys were safe. #### ⏱️ [00:14:07](https://www.youtube.com/watch?v=8FISCXU2BNQ&t=847) Second company: the guided selling engine - Speaker's claim: better developer experience yields better code, because developers get information when they need it. - Starting state at the second company: three-weekly deploys, five people for a week of manual regression, a low-quality Testim suite nobody trusted, high change failure rate. - A greenfield product had little process but room to get it right early. #### ⏱️ [00:15:36](https://www.youtube.com/watch?v=8FISCXU2BNQ&t=936) Results at the second company - Guided selling engine: on-demand deploys, low change failure rate, fast recovery after test automation and deployment tooling modernization. - Virtual sales agent: CI/CD built for multiple daily deploys from day one, with ephemeral environments, and an "almost zero" change failure rate (no number given). #### ⏱️ [00:16:17](https://www.youtube.com/watch?v=8FISCXU2BNQ&t=977) What a quality engineer is, and the close - Two constants across both journeys: quality engineers improve things for everybody, and teams should move fast and learn rather than break things. - The speaker apologizes to organizers for running fast, invites questions on social media or at office hours. The transcript ends at 00:17:02; the final ~30 seconds of the 17:32 runtime carry no captured speech. ## 📚 References The transcript itself contains no links; the following come from the video description or are primary sources for named entities. The video description is empty, so nothing else is pointed to by the speaker. - [How we introduced quality engineering and made everything better! — LeadDev](https://www.youtube.com/watch?v=8FISCXU2BNQ) — the source talk, 17:32, uploaded 2026-10-14. - [LeadDev](https://leaddev.com) — publisher of the conference talk. - [DORA metrics](https://dora.dev) — the four delivery metrics (deployment frequency, lead time, time to restore service, change failure rate) the speaker measures against; the caption spells it "Dora." - [Octopus Deploy](https://octopus.com) — the deployment tool the speaker names. - [Opsgenie](https://www.atlassian.com/software/opsgenie) — likely the "Ops Genie" paging tool named at 00:11:53. - [Testim](https://www.testim.io) — likely the "testim" automation suite named at 00:14:51. - The speaker's own site, rendered "qualityduck.co.uk" in captions: no URL is confirmed, so check the spelling before citing it. ## 🗣️ Paraphrase The talk walks through two cases of a quality team changing how software ships. At the first company, releases happened once a fortnight, testing was manual, and nearly every deploy failed in some way. The quality team started by automating the smoke tests it already ran by hand, compared automated and manual results side by side, and found the machines caught more than tired people did. It then replaced hand-run deployment scripts with a visible deployment tool, adopted tests-first work, short-lived branches, toggles, and disposable test environments, and raised deploy frequency one step at a time, announcing each risk and recovery plan to stakeholders as it went. By the end, deploys were on demand with the system staying up, and the four delivery metrics had improved by orders of magnitude on the speaker's account. One surprise, a cache emptied by daytime deploys, forced a performance fix. At the second company, the same approach turned a three-weekly manual process into on-demand deploys for the old product and daily-plus deploys for a new one. The speaker closes by defining a quality engineer as someone who finds systemic friction and fixes it for everybody. ## 📰 Newsletter **Ship smaller, announce louder.** LeadDev's 17-minute talk from Stuart Thomas packs a full delivery-pipeline rebuild into one story: fortnightly manual releases, near-100% change failure, then continuous deployment with under 5% failure. The part worth stealing is the order of operations. Automate the smoke tests you already run by hand, put the deploy scripts in a tool everyone can read, then raise deploy frequency one rung at a time (fortnightly, weekly, twice weekly, daily, twice daily), telling stakeholders the risk and the recovery plan before each step. For WordPress teams, the same ladder applies to plugin releases, theme updates, and the release-party checklist. Short-lived branches, feature toggles, and ephemeral environments map neatly onto Playground blueprints and wp-env. The speaker's cache-flush surprise on a daytime deploy is a reminder to test with an empty object cache. Source: [the talk on YouTube](https://www.youtube.com/watch?v=8FISCXU2BNQ), and the [DORA research](https://dora.dev) behind the metrics. ## ✅ Action items #### DevRel - Developer advocacy: give a talk or office-hours session on the deployment-frequency ramp, with the five rungs named and one measured metric per rung. - Education: build a workshop where learners write the failing UI selector test first, then the change, using the red-green pattern the talk describes at 00:05:59. - Developer experience: add a "tests before code" quickstart with a Playground-based ephemeral environment so a docs example can be run and discarded. - Developer marketing: if citing this talk, cite the mechanism (small steps, announced risk, recovery plan) and flag that the percentage figures are one speaker's account. - Community management: add a retrospective prompt to community events asking which process step lacks a way to tell people something went wrong. - Internal/external/cross-project: agree on a shared DORA baseline definition so deploy-frequency numbers compare across projects. #### OSPO - Policy and governance: write down the rule that main stays deployable and feature work ships behind toggles, with a named decision owner. - License compliance: no direct action from this source; nearest relevance: deployment tooling choices (such as Octopus Deploy) add third-party license and terms review. - Security management: treat the tag-triggered automatic production deploy as a privileged path; require signed tags and a protected-branch rule. - Software inventory: record the deployment tool and the paging tool as inventory entries with owners and availability targets. - Developer support: publish guidance on rollback and recovery plans before any release-frequency increase. - Community engagement: no direct action from this source; nearest relevance: share the "announce risks, mitigations, recovery" template with upstream maintainers. - Strategic alignment: set a four-metric DORA dashboard as a KPI set, reviewed quarterly. #### Community - Maintainers: pilot weekly instead of fortnightly releases in one repo, with the manual-trigger-first step the speaker describes at 00:10:23. - Contributors: document how contributors can run the full automated check locally or in a throwaway environment. - Sponsors: no direct action from this source; nearest relevance: report change failure rate and time to recover as sponsor-facing impact numbers. - Ecosystem: run a shared incident drill that rehearses a bad deploy and the out-of-hours alert path. #### WordPress - Core: no direct action from this source; nearest relevance: the always-deployable-trunk idea matches release hygiene discussions. - Core Performance: add an empty-cache load test to performance runs, based on the cache incident at 00:13:24. - Core AI: no direct action from this source; nearest relevance: test automation that gates AI-assisted changes before deploy. - **Core Program Team:** share the ramp-up pattern (manual trigger first, then schedule, then full automation) for any cross-team release cadence change. - Mobile: no direct action from this source; nearest relevance: automated smoke tests for the release checklist. - Playground: publish an ephemeral-environment recipe (build, deploy, test, destroy) as a Blueprint-driven pattern. - Community: no direct action from this source; nearest relevance: a retrospective format for meetup and WordCamp release talks. - Training: write a lesson on shift left covering collaboration and process separately, as the talk splits them. - Marketing: no direct action from this source; nearest relevance: do not repeat unsourced "under 5%" figures in campaigns. - Support: ask whether release-day regressions show up in tickets, and track them as a change failure proxy. - Design: no direct action from this source. - Documentation: document the release checklist as automated steps with the manual ones labeled and owned. - Photos: no direct action from this source. - TV: no direct action from this source. - Polyglots: no direct action from this source; nearest relevance: automated string-freeze checks before release. - Accessibility: add automated axe checks to the pipeline so accessibility is part of "defining what good looks like." - Meta: apply tags-trigger-deploy plus staged regression to .org tooling releases, measured against current lead time. - CLI: no direct action from this source; nearest relevance: smoke tests for command coverage on every tag. - Hosting: document how hosts handle cache flush during daytime deploys, since the talk found it as a hidden risk. - Tide: no direct action from this source; nearest relevance: compare Tide's pre-release checks with an automated regression gate. - Openverse: no direct action from this source; nearest relevance: ephemeral environments for ingestion pipeline changes. - Themes: add an automated smoke suite that runs on theme review submissions. - Plugins: add the same smoke suite for plugin review and publish a recovery plan template for authors. #### The WPOCC - Programs: no direct action from this source; nearest relevance: fund a fellowship track on test automation for maintainers. - Projects: sponsor a documentation sprint on release pipelines for volunteer-run plugins. - Funding experiments: require recipients to report DORA-style before and after numbers for any tooling grant. ### FAIR - Technical independence: ensure deploy and signature checks can run without a single hosted vendor tool. - Policies and covenants: require a published recovery plan for any change to a release path. - Productized governance artifacts: add a release-cadence and change-failure row to operator scorecards. ## 🧪 Hypotheses 1. Teams that automate their existing manual smoke tests first, instead of designing a new suite, reach a trusted regression pack faster than teams that begin from a clean design. 2. Raising deployment frequency in five or fewer steps, with manual triggers first, produces fewer change failures than a single jump to daily automated deploys. 3. Announcing risks, mitigations, and a recovery plan before each change measurably lowers stakeholder escalations per release. 4. Ephemeral environments cut median wait-for-test-environment time to under 5 minutes versus a queue on shared static environments. 5. Daytime deploys on a cache-dependent system raise p95 latency for the first 10 minutes unless cache warming is in place. 6. A change failure rate near 100% before the transformation is partly a measurement artifact: it falls sharply when the definition of "failure" is fixed across periods. ## 📊 Methodology - **H1.** Compare two matched teams over 12 weeks: one automates its existing release smoke tests, one builds a new suite. Measure time to a regression pack covering 80% of scenarios and defect escape rate. - **H2.** Run a stepped-wedge rollout across 8 repositories: some move through weekly, twice weekly, and daily steps, others jump directly. Record change failure rate and time to recover per step. - **H3.** Count stakeholder escalations per release before and after adopting a written risk, mitigation, and recovery note. Use a fixed template and a 6-month window. - **H4.** Time the interval from "ready to test" to "first test running" across static and ephemeral environments, sampling at least 200 pull requests. - **H5.** Log latency at 1-minute resolution around daytime deploys, with and without cache warming, over at least 30 deploys. - **H6.** Re-derive the failure rate from incident and rollback records using one definition, and compare with the team's reported figure. - **Controls.** Fix the DORA definitions before the study, and record team size and product type because both change deploy cadence. ## ❓ Questions - What exact definition of "change failure" produced "close to 100%" and "less than 5%," and was it the same in both periods? - How long did each step of the deployment ramp last before the team moved to the next one? - Which language did the quality team start with (the caption says "C")? Was it C#? - What proportion of the speedup came from tests, what from the deployment tool, and what from branching and toggles? - How did the team handle database migrations in continuous deployment with no downtime? - What was the recovery plan stakeholders were told about, and was it ever used for real? - At the second company, what number stands behind "almost zero" change failure rate? - What did the ephemeral environments cost per run, and how was data seeded? ## 🔮 Future research - **Sequencing of quality practices.** Whether test automation before deployment tooling, or the reverse, matters to outcomes is untested here, since both happened together. - **Trust as a measurable input.** The speaker calls trust the most important factor, but no measure of it is given. A stakeholder-confidence survey tied to deploy frequency would test it. - **Quality engineer role definition.** The talk's definition (finds systemic friction, fixes it) overlaps with platform and SRE roles. A role-boundary study would clarify. - **Cache and warm-state risk.** The empty-cache incident suggests a class of daytime deploy hazards that deserves a catalog. - **Generalization.** Two companies, one speaker. A multi-team replication would test whether the pattern holds in regulated or hardware-adjacent settings. ## 💡 Implications - **Practice.** Start with automation of the manual checks already trusted, run them beside humans, and only then retire the manual run. The talk ties trust in the tests to that overlap at 00:03:02. - **Policy.** Require a written risk, mitigation, and recovery note for every release-path change, as the talk models at 00:09:40. - **Security.** An automated tag-to-production path concentrates privilege. Protect tags, require green regression before the production step, and keep the paging route working out of hours. - **Education.** Teach shift left as two ideas, collaboration and process, rather than "move tests earlier." - **Funding.** Fund tooling and test-automation upskilling for existing QA staff, since the first company's quality team started with no coding experience and the speaker describes that as workable. ## 🧠 Learning styles assessment - **Reading level estimate.** Flesch-Kincaid Grade Level on the transcript: roughly grade 9 to 10. Flesch Reading Ease lands near 65, plain conversational English with domain terms (DORA, red-green, feature toggles, ephemeral environments) that suit practitioners. #### 👀 Visual learners - Present: the talk is likely slide-supported, though the transcript shows no on-screen content, so this is unverified. - Improve: a single before and after DORA table and a diagram of the tag-to-staging-to-production flow would carry the numbers the speaker only says aloud. #### 🔊 Aural learners - Present: continuous narration with a clear two-story structure announced at 00:00:07. - Improve: the speaker runs fast (apologizes at 00:17:02), so a slower closing summary of the five deploy-frequency rungs would help listeners. #### 🗣️ Verbal learners - Present: shift left is defined in two parts, and a quality engineer is defined at 00:16:17. - Improve: DORA, red-green, and ephemeral environment are used with brief or no definition for non-specialists. #### 🧩 Kinesthetic learners - Present: the ramp from manual trigger to full automation is a step-by-step path a team can copy. - Improve: no repository, pipeline example, or exercise is linked in the transcript or description. #### 🧠 Logical learners - Present: before and after metrics against four named measures at 00:11:53 and 00:12:37. - Improve: show the derivation of each metric and the measurement window, and name the failure definition. #### 🧑‍🤝‍🧑 Social learners - Present: invitation to connect on social media or at office hours at 00:17:02. - Improve: add a discussion prompt such as "which rung of the ramp is your team on?" #### 🧘‍♀️ Solitary learners - Present: short, self-contained, and replayable. - Improve: a self-check list ("do you trust your regression pack?") would convert the talk into a reflection exercise. #### 🌿 Naturalistic learners - Present: no explicit natural-world framing. - Improve: the throwaway-environment idea has a resource-use angle (compute destroyed after each run) that could be quantified. ## 📊 Accessibility assessment #### Screen reader semantics - Present: YouTube provides the landmarks and player controls; a machine transcript exists. - Improve: on-screen slides (if any carried the metric table) aren't voiced in full. WCAG 2.2 AA 1.1.1 (Non-text Content) and 1.2.5 (Audio Description, Prerecorded) apply to any chart or table shown only visually. #### Keyboard navigation - Present: the YouTube player is keyboard operable with visible focus. - Improve: the video adds no barrier of its own. 2.1.1 and 2.4.7 pass through the platform. #### Contrast and readability - Present: unverifiable from a transcript. - Improve: conference slides often use small type; confirm 1.4.3 (Contrast Minimum) on any published slide deck. #### Text resizing and reflow - Present: the player scales and captions resize separately. - Improve: text burned into slides doesn't reflow, so 1.4.4 (Resize Text) and 1.4.10 (Reflow) fail for baked-in text. #### Media and captions - Present: auto-captions exist, which is how this analysis was made. - Improve: captions garble the speaker's surname context, both company names, "Ops Genie," "Dora," "testim," and the speaker's site. That is a 1.2.2 (Captions, Prerecorded) failure in substance. The video has no chapter list, which weakens 2.4.5 (Multiple Ways) navigation, and the description is empty. #### Forms and interactions - Present: none in the video. - Improve: contact routes (socials, office hours) are spoken only. Put them in the description as selectable text. #### Standards reference - Primary gaps against WCAG 2.2 AA: **1.2.2** (inaccurate captions on proper nouns), **1.2.5** (no audio description for visual-only content, unverified), **1.1.1** (any chart carried visually), **1.4.4 / 1.4.10** (baked-in text), and **2.4.5** (no chapters, empty description). Fixing captions and adding chapters is the cheapest high-impact change. ## 🌍 DEIB assessment #### Cultural breadth - Present: the examples are two software companies; the speaker's site is UK-based by domain suffix. - Improve: add examples from regulated, non-English-language, or non-SaaS settings, where release constraints differ. #### Inclusive language - Present: plain terms, no exclusionary language. The speaker self-describes at 00:00:07 in a way the captions garble. - Improve: "cowboys" at 00:09:40 and "wild west" at 00:14:51 are idioms that don't translate cleanly; "unplanned ad hoc releases" says the same with a fact. #### Neurodiversity support - Present: a clear two-story structure and a consistent before, steps, after pattern. - Improve: the speaker runs fast and says so at 00:17:02. Chapter markers and a closing recap would lower the processing load. #### Representation - Present: one speaker; the talk credits "people," "engineers," "stakeholders," and a quality team that began without coders. - Improve: no named colleagues or customer voice appears, so credit for the transformation reads as team-level but unattributed. #### Psychological safety - Present: the talk models a safe-to-fail stance at 00:13:24 (failures are acceptable when a recovery plan exists) and early risk communication. - Improve: the quality team's initial skepticism from the head of engineering at 00:02:18 is mentioned lightly; naming how it was resolved would help teams in the same position. ## ✅ Logical validity #### Strong reasoning - **Sequencing by risk, 00:10:23.** Building deploy frequency in steps, with manual triggers first, tests the pipeline under supervision before automation removes the humans. The argument holds because each step has an observable pass condition (it ran fine) before the next. - **Side-by-side validation, 00:03:02.** Running automated tests beside manual ones until the automated runs proved better at detecting issues is a controlled comparison and the strongest empirical move in the talk. - **Causal mechanism for ephemeral environments, 00:08:12.** Isolating work, removing the environment queue, and enabling an always-deployable main follow directly from single-use environments. - **Risk communication, 00:09:40.** Telling stakeholders the risk, the mitigation, and the recovery plan before acting is a sound way to earn trust because the plan is stated before the outcome is known. - **Honest failure disclosure, 00:13:24.** Reporting the cache-emptying incident raises credibility. To clarify further, state how the incident was detected and how long it lasted. #### Fallacies - **Unsourced quantification, 00:01:34.** "our change failure rate was close to 100%" and "less than 5%" at 00:12:37 carry no data source or definition. Fix: show the measurement method and window. - **Post hoc attribution, 00:16:17.** "Quality engineers make everything better for everybody" is drawn from two cases in which quality led or influenced the changes. Other causes (engineering investment, tooling) aren't separated. Fix: name which metric moved after which specific change. - **Hasty generalization, 00:16:17.** Two companies are offered as proof of a general rule. Fix: scope the claim to teams with similar starting points. - **Single-factor emphasis, 00:08:57.** "Trusting process is probably that single most important part" is a ranking with no comparison against the other changes. Fix: say what happened to each metric when trust was weaker. - **Vague quantifier, 00:15:36.** "almost zero change failure rate" gives no number. Fix: give the rate and the denominator. #### Tone of voice - Warm, brisk, conversational, with self-deprecating asides ("Great way to enter learning to code" at 00:02:18). Second-person and first-person plural throughout ("we did it," "you need to define what good looks like"). Confidence is highest on outcomes and lowest on mechanisms. The speaker closes by apologizing for speed, which signals a time-constrained conference slot rather than evasion. ## 🛡️ DARVO analysis Not applicable to this source. DARVO (Deny, Attack, Reverse Victim and Offender) describes a response to accusation or criticism, and this is a conference experience report with no accuser and no defensive posture. - **Deny:** nothing is denied. The speaker volunteers failures: a near-100% starting failure rate, a cache incident, and the claim that not everything works first time. - **Attack:** no critics or competitors are attacked. The nearest challenge is the initial skepticism from the head of engineering at 00:02:18, which the speaker treats as understandable. - **Reverse victim/offender:** no such move appears. ## 🎯 Conclusion The talk's durable idea is an order of operations. Automate the checks people already trust, run them beside the manual process until they win, replace invisible deployment steps with a tool anyone can read, and raise deploy frequency one rung at a time while telling stakeholders the risk and the recovery plan first. The speaker reports the result as on-demand deploys, hours of lead time, minutes of recovery, and under 5% change failure at the first company, with a similar pattern at the second. The figures are unsourced in this transcript, the company names are garbled by the captions, and the credit given to quality engineering is not separated from shared engineering work. **Next steps for readers:** pick one release path, write the risk, mitigation, and recovery note for the next frequency increase, and measure the four DORA metrics with one fixed definition before and after. Confirm the company, tool, and site names against the speaker's own materials before citing them. ## 🔄 Refactor Appendix — How we introduced quality engineering and made everything better! > Density: 2–5 Literature atoms (H3 pad), 0–3 Permanent candidates (H4 pad). > Emitted: 4 Literature, 2 Permanent. Resonance filter applied: the tool-name and > company-name details were dropped as unconfirmed caption garble. > > Konik claim-header rules: present tense, declarative, ≤10 words, searchable, unique. > > Atomicity: one declarative sentence per atom. "and"/"also" → split. > > Tag pool: #bmf · #addendum · #xref · #articleseed · #storystem · #share · #aphorisms ### 📚 Literature Split Pad #### [H1] Defining good means writing tests first > [!quote] [00:05:16] > You need to define what good looks like. And defining what good looks like means writing tests before code. **Context & use**: - 00:05:16, the speaker's definition of the second half of shift left. - Pairs with red-green UI and API tests at 00:05:59, where the selectors are known before the change exists. **Tags**: #xref #share <!-- atomicity: 1 idea ✓ --> #### [H2] Process trust outranks tool novelty > [!quote] [00:08:57] > Trusting process is probably that single most important part of the whole set of changes. **Context & use**: - 00:08:57, just before the stakeholder risk-communication passage at 00:09:40. - A ranking claim with no comparison data, so keep it as the speaker's judgment, not a finding. **Tags**: #xref #aphorisms <!-- atomicity: 1 idea ✓ --> #### [H3] Daytime deploys exposed an empty-cache hazard > [!quote] [00:13:24] > a daytime deployment was emptying a cache and the system was performing poorly with an empty cache which impacted end users. **Context & use**: - 00:13:24, the unknown-unknowns example from the first company. - Concrete, testable risk for any team moving from scheduled to on-demand deploys. **Tags**: #xref #articleseed <!-- atomicity: 1 idea ✓ --> #### [H4] Quality engineers fix systemic friction > [!quote] [00:16:17] > a quality engineer is somebody that identifies problems, systemic frictions and issues and makes them better. **Context & use**: - 00:16:17, the closing role definition after both stories. - Broad enough to overlap platform and SRE roles, so scope it before reuse. **Tags**: #xref #share <!-- atomicity: 1 idea ✓ --> ### 🧠 Atomic Split Pad #### [P1] Raise deploy frequency in announced, reversible steps **Claim** (one sentence, Courtney's voice): I get more trust and fewer failures by raising release frequency one announced step at a time, with a manual trigger before automation. **Evidence pointer**: [[#H2]] · [[#H3]] **Scope & conditions**: Holds when each step has an observable pass condition and a recovery plan. Breaks when stakeholders can't be told in advance, when the system has hidden warm-state dependencies (such as an emptied cache), or when the change failure definition shifts between steps. **Links**: - [[Release cadence and DORA metrics]] — because extends - [[Big-bang release plans]] — because contrasts on step size and reversibility - [[WordPress plugin release process]] — because applies-in staged cadence changes <!-- atomicity: 1 idea ✓ --> #### [P2] Write the check before the change **Claim** (one sentence, Courtney's voice): A defined, failing test before the change makes "done" checkable and frees time for exploratory testing. **Evidence pointer**: [[#H1]] **Scope & conditions**: Applies to UI changes with predictable selectors, API behavior, and acceptance criteria. Weaker where the right behavior is still unknown, such as early prototypes. **Links**: - [[Test-driven development]] — because extends - [[Exploratory testing]] — because evidence-for the time it frees - [[Acceptance criteria in testable form]] — because applies-in manual environments <!-- atomicity: 1 idea ✓ --> ### ✏️ Fleeting Queue - [F1] Confirm the speaker's company names and site spelling (captions give "Zuper," "Zupla," "Zupller," "Humara," "qualityduck.co.uk") before citing any of them — https://www.youtube.com/watch?v=8FISCXU2BNQ #fleeting #inbox - [F2] Check what definition of change failure gives "close to 100%" and "less than 5%" at 00:01:34 and 00:12:37 #fleeting #inbox - [F3] Open question: how much of the DORA improvement belongs to tests, deploy tooling, branching, or environments? The talk bundles all four #fleeting #inbox - [F4] Article seed: "The empty-cache test" — daytime deploys fail in places nobody predicted, so test with cold state #fleeting #inbox #articleseed - [F5] Check whether our own release checklists have a manual-trigger rung before full automation #fleeting #inbox ### → Things 3 (max 2 actions) - [ ] Verify company, tool, and site names from the speaker's own site before citing this talk — research, 10 min #things - [ ] Add an empty-cache run to the next release-readiness checklist — release process #things <!-- deep-read:sections end --> 📄 **Full text:** [[Full Document Contents/Articles/How We Introduced Quality Engineering and Made Everything Better!.md|How We Introduced Quality Engineering and Made Everything Better! by Stuart Thomas]] ## Highlights - [00:04:31] then a tool that people could see the scripts that were being run, see how the process was working and people then could review it, change it, update it, improve it. Octopus deploy, it didn't matter that it was Octopus, not the Octopus guys or like me saying that bit. What mattered was having a tool. That tool empowered people and empowerment was such a huge part of the journey. Shift left feels like a bit of a loaded term these days. We hear it said a lot and and I think a lot of the time we don't then get a a description of what it really means. And for me it falls [00:05:16] into two separate parts. The first is all about collaboration. Everybody working in software delivery, whether they are technical or not, can test and probably should test in certain circumstances. By having everybody involved, quality, software, data, product, you can help to reduce some of that bottleneck and over the fence culture. But to do this, you need to do some work up front. And that's where the shift left comes in. You need to define what good looks like. And defining what good looks like means writing tests before code. ([View Highlight](https://read.readwise.io/read/01m50s3yad0cpry7xtyf5a7eqh)) - Note: [H1] Defining good means writing tests first - Tags: atom/literature/h1 - [00:08:57] Our CI pipelines would build the application and the changes, deploy it and run automated tests with no human interaction. And further because of the fast feedback, it reduces cognitive load. Something that throughout all of this process became really apparent early on and we we doubled down on was building trust. Trusting process is probably that single most important part of the whole set of changes. It didn't matter how good or shiny or cutting edge the new thing was that we were introducing. What mattered was that people trust us and what we [00:09:40] were doing. So all the way through every change, every improvement, every experiment, we communicated it early with stakeholders ahead of time and told them what the risks were of what we were doing. And then we told them how we were going to mitigate those risks and if it did go wrong, what our plan was to recover. This meant then that our stakeholders had confidence that we weren't being cowboys. We were we were doing things correctly and that actually if something did go wrong, they could trust us to make it right and make it right quickly. Things we learned though was to to do this, we had to avoid taking big steps. We had to build confidence that it was ([View Highlight](https://read.readwise.io/read/01m50s40ywd757yaeyzywknqvv)) - Note: [H2] Process trust outranks tool novelty - Tags: atom/literature/h2 - [00:13:24] What we learned though was not everything works first time and that that was okay as long as you had a plan on how to handle it and to recover from it not working out. And we learned that we will encounter issues that no one predicted. there are unknown unknowns and that they will have an impact on what you're trying to do. At Zupller, we ran into an issue with the product where a daytime deployment was emptying a cache and the system was performing poorly with an empty cache which impacted end users. We had to fix underlying performance issues in order to do those daytime deployments so that the system could run with an empty cache. The main thing though was to to remember [00:14:07] that you will adapt and overcome to any issues that you find there is a solution. But the DevX and TR trust are essential to making all of this work. The more you improve the developer experience, the better the code will be, the better the system will be because the developers are getting the information they need when they need it. So I want to talk now a little bit about the company I moved on to uh Humara. Humara is a dual product company. It's got its existing inmarket product called a guided selling engine and a bit like the story at Zupla. It had a uh three-w weekly deployment cadence in this case ([View Highlight](https://read.readwise.io/read/01m50s43nrbjg04h32x5405psz)) - Note: [H3] Daytime deploys exposed an empty-cache hazard - Tags: atom/literature/h3 - [00:16:17] consistent across both journeys. Quality engineers make everything better for everybody. And teams need to be able to move fast and learn things, not break them. But you might now be thinking, well, what what is a quality engineer? We've talked about DevOps. We've talked about improving deployments. We've not really talked much about testing apart from a bit of test automation. All of these journeys had one key thing in common. every change was that they were either led by quality or hugely influenced by quality. And a quality engineer is somebody that identifies problems, systemic frictions and issues and makes them better. They find a way [00:17:02] to solve those problems and make the experience of the software better for everybody. I've run really fast, so I'm really sorry to the uh the organizers on that. But um that is it. If you do have questions, connect with me on the socials or come see me at the office hours. Thank you. It super ([View Highlight](https://read.readwise.io/read/01m50s46w4fyyt7jd2w4m65k4x)) - Note: [H4] Quality engineers fix systemic friction - Tags: atom/literature/h3