AI Productivity at Enterprise Scale  🔗 **Source:** [AI Productivity at Enterprise Scale](https://www.youtube.com/watch?v=ZgxsSEUPzgI) > [!note]+ Document Notes > tags: [ai-skills-assessment, mental-models, developer-productivity, pr-throughput, ai-adoption-vs-value, engineering-leadership, learning-intervention, dx-metrics, leaddev, skiller-whale, vendor-talk, uk-enterprise, correlation-not-causation, ai-training-roi] > author: LeadDev > source: youtube.com > reference: https://www.youtube.com/watch?v=ZgxsSEUPzgI > date created: 2026-09-03 07:20 > > > [!info] Generated analysis — not the author's words. Sections below are machine-written > > from the captured source. Only text inside `> [!quote]` blocks is verbatim from the speaker. > > Do not quote or attribute any other line to the named author. > > > [!warning] Four things to know before trusting this note > > **1. Vendor talk in a sponsored slot.** The speaker co-founded the training company whose results he presents, says his co-founder "owns the budget for conference sponsorship," and closes with "come talk to us at the stand." Every number is the vendor's own assessment data or a customer's self-reported estimate. He states the correlation/causation caveat himself, twice. > > **2. Speaker and company names are ASR guesses.** Auto-captions render the speaker as "Howell" and the company as "Skillow" / "skill well." The likeliest reading is Hywel Carver of Skiller Whale (UK tech-team training), a co-founder named "Haley" in the captions. Not verified against the LeadDev programme; treat the identification as inferred. > > **3. Thin source.** 10 min 40 s, 1,779 words, 22 caption paragraphs, zero outbound links. Slides are referenced ("yellow lines here are before, blue lines here are after") but not captured; the chart claims below are as spoken. > > **4. The talk withholds one answer on purpose.** The XML-plus-max-tokens thought experiment ends with "Come see me on the stand if you want to know." The note does not supply the answer as if the speaker had. > > Published to the LeadDev channel 2026-09-02; recorded at LDX3 (LeadDev's London event). Channel name used as author per the video convention. > > # 🏷️ Title: AI productivity at enterprise scale > <!-- deep-read:sections start --> ## 🧭 Table of Contents - [📝 Summary](#-summary) - [📖 Main Content](#-main-content) - [🗒️ Notes](#️-notes) - [📚 References](#-references) - [🗣️ Paraphrase](#️-paraphrase) - [📰 Newsletter](#-newsletter) - [✅ Action Items](#-action-items) - [🧪 Hypotheses](#-hypotheses) - [📊 Methodology](#-methodology) - [❓ Questions](#-questions) - [🔮 Future Research](#-future-research) - [💡 Implications](#-implications) - [🧠 Learning Styles Assessment](#-learning-styles-assessment) - [📊 Accessibility Assessment](#-accessibility-assessment) - [🌍 DEIB Assessment](#-deib-assessment) - [✅ Logical Validity](#-logical-validity) - [🛡️ DARVO Analysis](#-darvo-analysis) - [🎯 Conclusion](#-conclusion) ### 🔗 References - Source: https://www.youtube.com/watch?v=ZgxsSEUPzgI (LeadDev channel, LDX3 talk) - Named in the talk, not linked: DX (developer-experience analytics vendor; supplied the PR-throughput comparison and the time-savings survey), a DX report on AI adoption vs. throughput, "two of the big four high street banks in the UK," "one of the top three telecoms in the UK," "two of the top 10 retailers in the UK." - Zero `<a href>` links in the captured transcript. Nothing in this talk is cited in a way a reader can follow. ## 📝 Summary **What this is.** A ten-minute vendor talk from a UK tech-training company presenting, for the first time in eight years, trends from its own AI skills assessments of software engineers across UK and European companies. The pitch is a single sentence the speaker returns to three times: mental models, not tips and tricks, are what turn AI adoption into productivity. **The evidence chain, in the order given.** - *How skills are measured.* Confidence questions ("how confident are you using agentic LLM systems to write code") plus demonstration questions (explain how tokenization works). The speaker argues self-assessment alone is "kind of flawed" because "people don't normally know what they don't know." - *Baseline distribution.* Perceived LLM understanding is bimodal: skeptics who understand little and expect little, and a second peak who understand deeply and expect large gains. The two correlate. Measured ability shows "a pretty broad range." - *Where untrained developers are already strong.* AI tool usage and security. His explanation: tool usage "is really like AI doing function calls," and security was always close to developers' work. - *Where they're weak.* Customizing AI systems: building RAG, knowing when to fine-tune, agentic workflows. "This is what we really want people to be doing most." - *Perceived gaps differ from measured gaps.* People feel weak on understanding LLMs and coding effectiveness; measured weakness is in building systems. - *Intervention.* Four to six learning sessions, then reassessment. Peak score moves from about 40% to about 70%. Confidence and expected productivity impact rise too. - *Productivity.* With DX, two groups in one organization measured on PR throughput, both previously given a vendor's tool training. The studied group "roughly doubled" throughput while the rest of the org "slowly trailed downwards." - *ROI.* At one of the UK's largest retailers, a DX survey after training: 44% estimate saving 1–3 hours a week, 31% estimate 3–6, 17% estimate 10 or more. DX extrapolates to £51.6 million; the vendor says its fee was under 0.1% of that. - *The counter-case.* A DX report found a 65% increase in AI adoption produced an 8% increase in PR throughput. "Adoption is not the same thing as gaining value. Like buying and using a saw does not turn you into a carpenter." **Bottom line.** The 65%-to-8% figure and the saw line are the transferable parts; they're the vendor's argument against its competitors' "tool-based training" and they hold regardless of who's selling. The £51.6M and the doubled throughput are self-reported or single-org comparisons the speaker himself declines to call causal. For Courtney the talk is a compact statement of the training-design position she already holds from the instructional-design side: teach the model of the system, not the incantation. ## 📖 Main Content **The structure of the argument.** Three moves. First, establish that measured skill and perceived skill diverge, which justifies assessment over self-report and, conveniently, justifies the vendor's product. Second, show that a short intervention moves measured skill (40% → 70%) and then moves an outcome metric (PR throughput, self-estimated hours). Third, contrast with the industry pattern (65% adoption → 8% throughput) to argue that adoption without mental models wastes money. **The mental-model example is the best teaching moment in the talk.** The markdown tip ("ask Claude for markdown because it's token-efficient") is good advice that stops being advice the moment you run a local model, pay by seat rather than by token, or need a format that isn't markdown. The XML-plus-max-tokens question is the test: someone with a mental model of decoding and stop conditions can predict whether they'll get terse XML or truncated XML; someone with a tip list can't. He then withholds the answer to drive stand traffic, which is fair for a sponsored slot and a small loss for the recording. **What the data can and can't carry.** - The bimodal perception chart and the correlation between understanding and expected impact are descriptive and plausible; no sample size is given. - The strengths/gaps pattern (tool usage and security strong, system-building weak) is the most interesting finding and the least argued. His post-hoc explanation for tool usage is reasonable; the security explanation ("something that should always have been close to the work of developers") is a hope, not a mechanism, given how often security is the gap in other surveys. - The 40% → 70% jump is pre/post on the vendor's own assessment after the vendor's own course. Teaching to the test is the obvious confound and isn't addressed. - The PR-throughput comparison is two groups in one company, both previously trained by an "AI vendor" on tools. Doubling versus slight decline is a striking split, but "roughly doubled" from what base, over what period, with what team sizes, is unspecified. The speaker says "this is not the whole story." - The £51.6M is a DX extrapolation from a self-reported time-savings survey. Self-reported hours saved is the weakest productivity measure there is, and extrapolating it across an organization multiplies the error. The under-0.1%-of-that fee line is a sales figure. - The 65% → 8% DX report figure is the one external number. It's also secondhand; the report isn't named. **What's missing.** No control group that received no training. No time horizon on any effect. No mention of what the four to six sessions cover beyond "mental models." No costs to the client beyond the fee (engineer hours in sessions). No failure cases. No definition of "AI adoption" in the DX report figure. **The rhetorical frame.** The speaker pre-empts objections well: "science is quite hard," "your mileage may vary," "we can never be confident of the difference between correlation and causation," "I have to include this slide I'm afraid." That candor is disarming and it's also the vendor's strongest credibility move; the note gives it credit and also notes it doesn't change what the numbers can bear. ## 🗒️ Notes - **The saw line is the reusable atom.** "Adoption is not the same thing as gaining value. Like buying and using a saw does not turn you into a carpenter." It compresses the whole talk and it's vendor-neutral. - **65% adoption → 8% throughput** is the number to carry forward, with the caveat that it's a secondhand citation of an unnamed DX report. Worth finding the primary. - **Strengths where nobody trained them: tool usage and security.** If this generalizes, the training dollar should go to system-building (RAG, fine-tune decisions, agent workflows), not to "how to prompt." That matches where the WordPress AI team's Abilities API work sits: developers can call tools; fewer can design the system around them. - **Perceived vs. measured gaps diverge.** Engineers feel weak at "understanding LLMs" and "coding with them" but test weak at building systems. A curriculum built from a needs survey would teach the wrong thing. This is the instructional-design finding in the talk. - **Teaching to the test is unaddressed.** The 40% → 70% pre/post is on the vendor's assessment after the vendor's course. It shows the course teaches what the assessment measures, which is necessary and not sufficient. - **Both PR-throughput groups had prior vendor tool training.** So the comparison isn't "training vs. none," it's "mental-model training on top of tool training vs. tool training alone." That's a narrower and more interesting claim than "training doubles throughput." - **The XML question is a good interview question** for anyone who claims LLM fluency, whatever the answer turns out to be. - **ASR artifacts:** "Howell" (likely Hywel), "Skillow" and "skill well" (likely Skiller Whale), "very tur" (terse), "Haley" (co-founder). Quotes left as captured. - **No links, no slides in the capture.** Every chart claim is as spoken. ## 📚 References - LeadDev, *AI productivity at enterprise scale*, YouTube, published 2026-09-02, recorded at LDX3 (London). Speaker identified by caption as "Howell" from "Skillow." - DX (developer-experience analytics) — named as the source of the PR-throughput comparison, the retailer time-savings survey and its £51.6M extrapolation, and the report showing 65% adoption increase → 8% throughput increase. None linked; the report is unnamed. - Cross-reference (Reader, 2026-09-02): *Designing a WordPress site in under 10 minutes with AI* (Peatling: "the models are non-deterministic," make each feature reliable before promising it) — the same mental-model-over-tips stance from a product builder's side. - Cross-reference (Reader, 2026-09-02): *Why everyone is moving to Stylex?* and *AI Struggles: A Developer's Story* — practitioner accounts of AI tooling where the gap was system understanding, not tool access. - Cross-reference (vault): Courtney's instructional-design track and the learning-path skill — the perceived-vs-measured-gap finding is a needs-analysis caution for both. ## 🗣️ Paraphrase We've assessed engineers' AI skills for years and here's what the data says. Self-assessment is unreliable because people don't know what they don't know, so we test whether they can explain how the thing works. Untrained developers are already decent at using AI tools and at security. They're weak at building their own LLM systems, which is where the value is. They think their weakness is understanding LLMs and coding with them; it isn't. After four to six sessions on mental models, scores go from about 40% to about 70%. In one company, a team we trained roughly doubled PR throughput while everyone else drifted down. At a big retailer, most engineers said they save one to six hours a week; a vendor extrapolated that to £51.6 million. Meanwhile an industry report found a 65% jump in AI adoption bought only an 8% jump in throughput. Buying a saw doesn't make you a carpenter. Mental models are the missing piece, and yes, we sell them. ## 📰 Newsletter **A training vendor's numbers say AI adoption without understanding buys almost nothing.** In a ten-minute LDX3 talk, the co-founder of a UK tech-training firm presented eight years of skills-assessment data with an unusually candid frame: "science is quite hard," "your mileage may vary," and a closing admission that the slide exists because his co-founder owns the sponsorship budget. The findings worth keeping: developers who've had no formal AI training already test well on tool usage and security, and badly on building their own systems (RAG, fine-tuning decisions, agent workflows). They also misjudge their own gaps, feeling weakest at "understanding LLMs" when the measured weakness is elsewhere. After four to six sessions, assessment scores rose from roughly 40% to 70%; one trained team roughly doubled PR throughput while the rest of its company drifted down; a retailer's engineers self-reported time savings that a DX extrapolation turned into £51.6 million. Against that, he set a DX report finding that a 65% increase in AI adoption produced an 8% increase in throughput. "Buying and using a saw does not turn you into a carpenter." The pre/post scores are on the vendor's own test, the throughput comparison is one company, and the pounds are self-reported hours multiplied out. The 65-to-8 gap is the number that survives the discount. ## ✅ Action Items **DevRel** - The perceived-vs-measured-gap finding is a case for assessment-first curriculum design in any developer education program: survey what people want, test what they can do, teach the second. `#action/h3` - The XML-plus-max-tokens thought experiment is a ready-made workshop opener for LLM fluency. `#action/h4` **OSPO** - Not applicable to this source. **Community** - Not applicable to this source. **WordPress teams** (Core, Core Performance, Core AI, Core Program Team, Mobile, Playground, Community, Training, Marketing, Support, Design, Documentation, Photos, TV, Polyglots, Accessibility, Meta, CLI, Hosting, Tide, Openverse, Themes, Plugins) - **Training:** if the strengths/gaps pattern holds, learn.wordpress.org AI content should weight system design (when to retrieve, when to call an ability, how to bound an agent) over prompt tips. `#action/h3` - **Core AI:** the "developers are already good at tool usage" finding supports the Abilities API bet: the scarce skill is designing the system around the tool, which is where documentation and examples should concentrate. `#action/h3` - All other teams: not applicable to this source. **The WPOCC** - Not applicable to this source. **FAIR** - Not applicable to this source. **Personal (Courtney)** - **Find the DX report** behind "65% adoption increase → 8% throughput increase." If it's real and recent, it's the one citable number in the talk and belongs in any argument about AI training design. `#action/h2` - **Steal the assessment-first pattern for your learning-path work.** The talk's cleanest finding is that self-reported gaps and measured gaps diverge; any curriculum you scope from a needs survey alone inherits that error. `#action/h2` - **Keep the saw line** for the next time someone equates seat licenses with productivity. `#action/h3` - **Don't cite the £51.6M or the doubled throughput without the caveats.** Self-reported hours extrapolated by a vendor, and a one-company two-group comparison the speaker himself won't call causal. `#action/h3` ## 🧪 Hypotheses - **H-A:** Developers without formal LLM training score higher on tool-use and security items than on system-design items across assessment vendors, not just this one. *Testable against any published skills-assessment breakdown.* - **H-B:** Self-reported AI skill gaps and measured gaps diverge in the same direction (people over-report weakness in "understanding" and under-report weakness in "building") in WordPress developer surveys. *Testable with a paired survey plus practical task at a WordCamp workshop.* - **H-C:** The 40% → 70% pre/post gain would shrink on an assessment not written by the training vendor. *Testable only with an independent instrument.* - **H-D:** The 65% → 8% adoption-to-throughput ratio comes from a DX report that measures adoption as license activation, not as sustained use. *Testable by reading the report.* ### 📊 Methodology Vendor-collected assessment data, described but not shown: no sample sizes, no time window beyond "over the last couple of years," no instrument published. Outcome data comes from a third party (DX) in two forms: one within-company two-group comparison on PR throughput, and one self-reported time-savings survey extrapolated to organization scale. The speaker names the correlation/causation limit twice and offers no control condition. Pre/post scores are on the vendor's own assessment after the vendor's own course. The one external figure (65% → 8%) is secondhand and unnamed. The appropriate reading is: plausible descriptive patterns, unquantified effect sizes, one strong rhetorical contrast. ### ❓ Questions - Which DX report is the 65%-to-8% figure from, and how does it define "adoption"? - What was the base PR throughput of the "roughly doubled" team, over what period, and how large was it relative to "the rest of the org"? - What do the four to six sessions cover? Is "mental models" a curriculum or a brand? - Has the assessment been validated against any external measure of LLM competence? - What is the answer to the XML-plus-max-tokens question the speaker withheld, and does the assessment score it? - Why would untrained developers score well on security in this dataset when security is a common gap elsewhere? Is it the item design? ### 🔮 Future Research - Locate and read the DX report; if the 65/8 figure holds up, write it into the learning-path material with the primary citation. - Run a small paired study at a WordPress developer event: self-rated confidence on five AI skill areas, then a practical task in each. Compare the gap directions with this talk's claim. - Compare this vendor's "mental models" framing with the WordPress Training team's existing AI lesson plans to see whether they teach mechanism or recipe. - Track whether LeadDev publishes the slides; the strengths/gaps chart is the finding most worth seeing as data. ### 💡 Implications **For training design.** Needs surveys measure what people feel; assessments measure what they can do. This talk's data says the two point at different gaps. Any AI curriculum scoped from the first will teach prompting to people who need system design. **For engineering leaders buying AI.** The seat-license-equals-productivity assumption has a number against it now, even if secondhand: 65% more adoption, 8% more throughput. The purchase decision and the training decision are separate line items. **For the WordPress AI effort.** If developers already handle tool calls well and struggle with system design, the Abilities API's documentation burden is the architecture around the ability, not the ability itself. **For vendor claims.** A speaker who names his own caveats is easier to trust and no more correct. The candor here is genuine and the numbers still can't bear causal weight. ## 🧠 Learning Styles Assessment **Auditory** — the native mode; conversational, ten minutes, one speaker, clear signposting ("first," "on the other hand," "anyway"). **Visual** — served in the room by charts (bimodal distribution, yellow/blue pre-post lines, strengths/gaps), not served in the capture; the recording without slides loses the evidence. **Read/write** — moderately served by the transcript; the key numbers are spoken clearly enough to note. **Kinesthetic** — one thought experiment (XML plus max tokens) invites the listener to predict; the answer is withheld, which frustrates rather than teaches. **Sequential** — well served; the argument runs measurement → baseline → intervention → outcome → contrast. **Global** — served by the repeated thesis (mental models) and the saw analogy. **Overall:** a well-structured conference talk for an auditory-sequential audience of engineering leaders, dependent on slides the video doesn't show. ## 📊 Accessibility Assessment Assessment of the YouTube capture and transcript. Not a WCAG audit of the video player or the LeadDev site. #### Captions and transcript Auto-generated captions only, with proper-noun errors ("Howell," "Skillow," "very tur"). No human-reviewed captions or transcript detected. WCAG 1.2.2 (captions, prerecorded) is met only nominally. #### Audio description None. Every chart is described by pointer ("yellow lines here," "in blue here"), so a listener without the slides gets no data. WCAG 1.2.3/1.2.5 not met for the visual content. #### Alternative text and non-text content Slides are not in the capture; no alt text exists for them. #### Keyboard navigation, contrast, reflow Not assessable from the capture; standard YouTube player behavior applies. #### Cognitive load Short, single-thread, repeated thesis, plain register. Numbers arrive quickly (three percentages in one sentence at the retailer survey); a listener would need to replay. #### Standards reference WCAG 2.2 AA (1.2.2 Captions, 1.2.3 Audio Description or Media Alternative, 1.2.5 Audio Description). Captions present but unedited; audio description absent. ## 🌍 DEIB Assessment #### Cultural breadth UK and European enterprise data, UK reference points ("high street banks," pounds). The speaker says so; the framing is honest about scope. #### Inclusive language Neutral. "People," "folks," "developers," "teams." No gendered examples. #### Neurodiversity support Plain speech, one idea at a time, repeated thesis. Good for listeners who need structure. The withheld answer and fast number clusters cut the other way. #### Representation One speaker, one vendor, one analytics partner. Customers are anonymized ("one of the UK's largest retailers"). No engineer's voice from the trained cohorts. The co-founder is named as the person who writes the talks and holds the budget, which credits her work in a sponsored slot. #### Psychological safety The talk treats skill gaps as normal and measurable rather than as individual failings, and names "skeptics" without derision. That's a constructive frame for leaders taking the message back to teams. ## ✅ Logical Validity **Claims and their support.** - *Self-assessment alone is flawed* — supported by a mechanism (people don't know what they don't know) and by the perceived-vs-measured gap data. Reasonable. - *Untrained developers are strong on tool use and security, weak on system building* — descriptive claim from vendor data; no numbers given. Plausible, unquantified. - *Training raises scores 40% → 70%* — pre/post on the vendor's own instrument. Confounded by teaching to the test; not addressed. - *Trained team roughly doubled PR throughput while the org declined* — one company, two groups, no base rates, both groups had prior vendor tool training. Speaker disclaims causation. Weak as evidence, honest as presented. - *£51.6M saved* — self-reported hours extrapolated by a third party. Weakest number in the talk; presented with a sales comparison to fees. - *65% adoption increase → 8% throughput increase* — secondhand from an unnamed DX report. The strongest rhetorical point and unverifiable from the talk. - *Therefore mental models are the missing key* — the conclusion follows from the framing more than from the data; the data are consistent with it and don't establish it. **Fallacies.** No formal fallacies. Two structural weaknesses: post-hoc explanation for the strengths pattern, and conclusion-by-elimination (tool training didn't work, ours did, therefore mental models). The saw analogy is apt and does no argumentative work beyond restating the claim. **Internal consistency.** Consistent. The caveats at the start match the hedges at the end. **Verdict.** Valid as a well-hedged vendor argument; invalid as evidence for effect sizes. The one transferable finding, perceived vs. measured gaps, is also the one that doesn't depend on the vendor's outcome numbers. ## 🛡️ DARVO Analysis Not applicable to this source. There is no accusation, victim, or reversal. The nearest structure is the vendor positioning "tool-based training from an AI vendor" as the thing that failed before its own intervention succeeded; that's competitive framing, not blame reversal, and the speaker names his own conflict of interest. ## 🎯 Conclusion A candid ten-minute sales talk with one finding worth keeping on its own terms and one number worth chasing to its source. The finding: engineers' self-reported AI skill gaps (understanding LLMs, coding with them) don't match their measured gaps (building systems around LLMs), and untrained engineers already test well on tool use and security. The number: a DX report showing 65% more AI adoption bought 8% more PR throughput. Everything else, the 40% → 70% scores, the doubled throughput, the £51.6M, is the vendor's own instrument or a customer's self-estimate, and the speaker says as much. For Courtney the talk is a compact argument for assessment-first curriculum design and for concentrating WordPress AI education on system design rather than prompt tips, both positions she can hold without the vendor's numbers. The saw line does the rest. ## 🔄 Refactor Appendix — AI productivity at enterprise scale ### 📚 Literature Atoms (H3) #### [H1] Adopting an AI tool is not the same as gaining value from it > [!quote] ID:rw_01m1kmwxq6s6c71p96h9fs5mkg > "What we do know is that adoption is not the same thing as gaining value. Like buying and using a saw does not turn you into a carpenter. And AI is very expensive. So if you're using it without skill, it's potentially just a waste of money without significant benefit." The talk's thesis in one analogy; vendor-neutral and reusable. Tags: `#atom/literature/h1` #### [H2] A 65% rise in AI adoption produced an 8% rise in PR throughput > [!quote] ID:rw_01m1kmwxq6s6c71p96h9fs5mkg > "DX put out this report where they found that companies that were adopting AI most rapidly saw a very minor change in PR throughput where 65% adoption increase in AI um led to 8% increase in throughput." Secondhand, unnamed report; the one external number in the talk and the one to verify. Tags: `#atom/literature/h2` #### [H3] Four to six sessions moved the assessment peak from about 40% to about 70% > [!quote] ID:rw_01m1kmwxq6s6c71p96h9fs5mkg > "We also measure a big increase in technical capabilities. So before the peak is around 40% score, afterwards it's around 70%." Pre/post on the vendor's own instrument after the vendor's own course; shows the course teaches the test. Tags: `#atom/literature/h3` #### [H4] Untrained developers' measured gap is building LLM systems, not using LLM tools > [!quote] ID:rw_01m1kmwxq6s6c71p96h9fs5mkg > "If you look at common gaps, the most common areas of weakness for people who haven't been formally trained, uh we found that they were in customizing AI systems. So, building your own sort of LLM based bits of product, things like rag systems or knowing when to fine-tune LLMs and agentic workflows." Paired with the claim that tool usage and security are already strong, this redirects the training dollar. Tags: `#atom/literature/h4` #### [H5] People's felt gaps and measured gaps point at different skills > [!quote] ID:rw_01m1kmwxq6s6c71p96h9fs5mkg > "if you look at where people think they're weak and strong, um, you see very different gaps to what you measure in their abilities, the gaps that people feel are in their understanding of LLMs and their effectiveness at coding with them. People know that they don't know this stuff." The instructional-design finding: a needs survey alone would scope the wrong curriculum. Tags: `#atom/literature/h5` ### 🧠 Permanent Candidates (H4) #### [P1] Self-reported skill gaps are a poor guide to curriculum because they diverge from measured gaps Engineers in this dataset felt weakest at understanding LLMs and coding with them; they tested weakest at building systems around LLMs. Any AI curriculum scoped from a needs survey inherits that misdirection. Assess first, then teach the measured gap. *Candidate; one vendor's data, but it matches the general finding that self-assessment tracks confidence, not competence, and it's directly actionable in Courtney's learning-path work.* Links: [[Assessment-First Curriculum Design]] · [[Create Learning Path]] · [[Instructional Design MOC]] · [[AI Tooling MOC]] Tags: `#atom/zettel/z1` #### [P2] AI seat adoption and AI productivity are separate purchases A 65% increase in adoption bought an 8% increase in throughput in the cited DX data. Licensing the tool and building the skill to use it are different line items with different owners, and the second is where the gap is. *Candidate; depends on a secondhand figure that needs its primary source before this hardens.* Links: [[AI Adoption vs Value]] · [[Rewarding AI Usage Manufactures Usage Not Value]] · [[Developer Productivity Metrics]] · [[AI Tooling MOC]] Tags: `#atom/zettel/z2` ### ✏️ Fleeting Queue - The XML-plus-max-tokens question as an interview or workshop prompt: predict terse vs. truncated and explain why. `#atom/fleeting/f1` - "Tool usage is really like AI doing function calls" is why developers pick it up unaided; the Abilities API docs can lean on that intuition. `#atom/fleeting/f1` - Both throughput groups had prior vendor tool training; the real comparison is mental-model training layered on tool training, not training vs. none. `#atom/fleeting/f1` - A speaker naming his own conflict of interest and caveats raises trust without raising evidence quality; keep the two separate when reading vendor talks. `#atom/fleeting/f1` ### → Things 3 (max 2 actions) 1. **Find the DX report** behind "65% adoption increase → 8% throughput increase" and record how it defines adoption. `#action/h2` 2. **Add an assessment-first step to the learning-path skill**: pair any self-rated needs survey with a short practical check before scoping content. `#action/h2` <!-- deep-read:sections end --> 📄 **Full text:** [[Full Document Contents/Articles/AI Productivity at Enterprise Scale.md|AI Productivity at Enterprise Scale by LeadDev]] ## Highlights - found that they were in customizing AI systems. So, building your own sort of LLM based bits of product, things like rag systems or knowing when to fine-tune ([View Highlight](https://read.readwise.io/read/01m1kn7mjneb7wevdq7ec47gmb)) - Note: H4 — Untrained developers' measured gap is building LLM systems (RAG, fine-tune decisions, agent workflows), while tool usage and security are already strong. Redirects the training dollar. #atom/literature/h4 - Tags: atom/literature/h4 - technical capabilities. So before the peak is around 40% score, afterwards ([View Highlight](https://read.readwise.io/read/01m1kn7hmvz5ynzaf4e0a5q6e8)) - Note: H3 — Pre/post assessment peak moves from ~40% to ~70% after 4–6 sessions. Vendor's own instrument after the vendor's own course; teaching-to-the-test unaddressed. #atom/literature/h3 - Tags: atom/literature/h3 - What we do know is that adoption is not the same thing as gaining value. Like buying and using a saw does not turn you into a carpenter. And AI is very ([View Highlight](https://read.readwise.io/read/01m1kn79sdsmx5dyf0jdxmwxmm)) - Note: H1 — Adopting an AI tool is not the same as gaining value from it; the saw/carpenter line is the talk's thesis and it's vendor-neutral. #atom/literature/h1 - Tags: atom/literature/h1 - PR throughput where 65% adoption increase in AI um led to 8% increase in throughput. And to me this all makes ([View Highlight](https://read.readwise.io/read/01m1kn7eqznd2eydgfn7bcnmqg)) - Note: H2 — 65% adoption increase → 8% PR-throughput increase, citing an unnamed DX report. The one external number in the talk; find the primary. #atom/literature/h2 - Tags: atom/literature/h2