Why AI Is Going Vertical (Again) | Dianne Penn ![rw-book-cover](https://d34adp677peecb.cloudfront.net/static/images/article4.6bc1851654a0.png) 🔗 **Source:** [Why AI Is Going Vertical (Again) | Dianne Penn](https://www.youtube.com/watch?v=tivaWTTVRhY#transcript) > [!note]+ Document Notes > tags: [ai-product-management, model-evaluation, anthropic, product-strategy, user-research, vertical-ai] > author: Lenny Rachitsky and Dianne Penn > source: Lenny's Podcast > reference: https://www.youtube.com/watch?v=tivaWTTVRhY > date created: 2026-08-04 10:08 EDT > > > [!info] Generated analysis — not the author's words. Sections below are machine-written > > from the captured source. Only text inside `> [!quote]` blocks is verbatim from the author. > > Do not quote or attribute any other line to the named author. > > # 🏷️ Title: Why AI is going vertical (again) | Dianne Penn (Anthropic) > <!-- deep-read:sections start --> ## 🧭 Table of Contents - [📝 Summary](#-summary) - [📖 Main Content](#-main-content) - [🗒️ Notes](#️-notes) - [📚 References](#-references) - [🗣️ Paraphrase](#️-paraphrase) - [📰 Newsletter](#-newsletter) - [✅ Action Items](#-action-items) - [🧪 Hypotheses](#-hypotheses) - [📊 Methodology](#-methodology) - [❓ Questions](#-questions) - [🔮 Future Research](#-future-research) - [💡 Implications](#-implications) - [🧠 Learning Styles Assessment](#-learning-styles-assessment) - [📊 Accessibility Assessment](#-accessibility-assessment) - [🌍 DEIB Assessment](#-deib-assessment) - [✅ Logical Validity](#-logical-validity) - [🛡️ DARVO Analysis](#-darvo-analysis) - [🎯 Conclusion](#-conclusion) - [🔄 Refactor Appendix](#-refactor-appendix--why-ai-is-going-vertical-again-dianne-penn-anthropic) ### 🔗 References - [Source video](https://www.youtube.com/watch?v=tivaWTTVRhY) - References named in the transcript are contextual leads; verify them independently before reuse. ## 📝 Summary - **What happened:** Dianne Penn describes product management inside Anthropic as a bridge between fast-moving model research and specific user outcomes. Model capability alone doesn't create adoption; products, harnesses, context, feedback translation, and evaluations turn general models into useful vertical experiences. - **Why it matters:** The episode offers an inside account of research-product work and the changing PM role. It is valuable testimony from a leader at one company, not an independent measure of Anthropic's performance or safety. - **Key stakeholders:** AI product managers, researchers, engineers, founders, users in specialized domains, evaluators, safety teams, and hiring candidates. - **Risks and tradeoffs:** Vendor-centered evidence, product names mistranscribed in captions, unclear claims about inclusion, rapid-plan churn, concentrating advanced capabilities inside labs, and treating eval scores as complete representations of user value. - **Bottom line:** Build the model and product surface together, translate feedback into reproducible failures, and keep first-principles user judgment central as production gets faster. ## 📖 Main Content Dianne Penn describes product management inside Anthropic as a bridge between fast-moving model research and specific user outcomes. Model capability alone doesn't create adoption; products, harnesses, context, feedback translation, and evaluations turn general models into useful vertical experiences. The episode offers an inside account of research-product work and the changing PM role. It is valuable testimony from a leader at one company, not an independent measure of Anthropic's performance or safety. The source is an interview or educational presentation, not a controlled study. Claims about products, markets, organizations, and outcomes should be treated as speaker testimony unless a cited source independently confirms them. ### 🗒️ Notes #### ⏱️ [00:30](https://www.youtube.com/watch?v=tivaWTTVRhY&t=30) Penn links the impact of Opus 4 - Penn links the impact of Opus 4.5 to Claude Code as the product vehicle through which users experienced the model. #### ⏱️ [15:57](https://www.youtube.com/watch?v=tivaWTTVRhY&t=957) Rapid capability changes require teams to update plans when new evidence appears - Rapid capability changes require teams to update plans when new evidence appears. #### ⏱️ [23:57](https://www.youtube.com/watch?v=tivaWTTVRhY&t=1437) Labs is framed as a place for discontinuous bets outside the core roadmap - Labs is framed as a place for discontinuous bets outside the core roadmap. #### ⏱️ [40:52](https://www.youtube.com/watch?v=tivaWTTVRhY&t=2452) Research product managers are expected to reason from user value rather than repeat old SaaS playbooks - Research product managers are expected to reason from user value rather than repeat old SaaS playbooks. #### ⏱️ [47:00](https://www.youtube.com/watch?v=tivaWTTVRhY&t=2820) The team translates vague user feedback into reproducible failure categories and evals - The team translates vague user feedback into reproducible failure categories and evals. #### ⏱️ [83:31](https://www.youtube.com/watch?v=tivaWTTVRhY&t=5011) Penn argues that easier building increases the need for detailed, user-centered product judgment - Penn argues that easier building increases the need for detailed, user-centered product judgment. ## 📚 References - [Original video](https://www.youtube.com/watch?v=tivaWTTVRhY) - Not applicable beyond the source video; the transcript doesn't supply a complete, verified bibliography. ## 🗣️ Paraphrase - Anthropic's product identity formed through repeated experiments that exposed model behavior to users. - A frontier model and a fitting product surface can amplify each other's adoption. - Fast capability jumps make static roadmaps unreliable. - Labs explores large bets that may not fit the core roadmap. - Research product managers embed with researchers where changes directly affect users. - Vague feedback becomes useful only after a team reproduces and classifies the failure. - Evals can act like executable requirements while PRDs still hold context and alignment. - As implementation gets easier, deciding what is worth building becomes harder and more important. ## 📰 Newsletter The striking line from Anthropic's Dianne Penn is that evals are becoming executable requirements for product teams. A model may be capable, but users meet it through a product, a harness, and a context. WordPress's parallel is familiar: shipped behavior beats a proposal, and a test case beats a vague complaint. ## ✅ Action Items #### DevRel - Turn vague developer feedback into reproducible cases with clear scope and expected behavior. #### OSPO - Fund independent evaluation and document decision rights across research, product, and community stakeholders. #### Community - Invite domain experts and affected users into evaluation design before a pilot scales. #### WordPress #### Core - Not applicable to this source; it doesn't identify a direct action for this team. #### Core Performance - Not applicable to this source; it doesn't identify a direct action for this team. #### Core AI - Use task-specific evals with field replay before promoting model-assisted features. #### Core Program Team - Track cross-team assumptions that change after model or API updates. #### Mobile - Not applicable to this source; it doesn't identify a direct action for this team. #### Playground - Provide reproducible browser fixtures for model and harness experiments. #### Community - Not applicable to this source; it doesn't identify a direct action for this team. #### Training - Teach eval design, user interviews, and failure reproduction together. #### Marketing - Not applicable to this source; it doesn't identify a direct action for this team. #### Support - Not applicable to this source; it doesn't identify a direct action for this team. #### Design - Test whether model behavior and product surface support the same user goal. #### Documentation - Publish eval scope, known gaps, and replay steps near feature guidance. #### Photos - Not applicable to this source; it doesn't identify a direct action for this team. #### TV - Not applicable to this source; it doesn't identify a direct action for this team. #### Polyglots - Not applicable to this source; it doesn't identify a direct action for this team. #### Accessibility - Include disabled users and assistive-technology workflows in eval coverage. #### Meta - Measure real contributor outcomes, not only model or interface activity. #### CLI - Not applicable to this source; it doesn't identify a direct action for this team. #### Hosting - Not applicable to this source; it doesn't identify a direct action for this team. #### Tide - Not applicable to this source; it doesn't identify a direct action for this team. #### Openverse - Not applicable to this source; it doesn't identify a direct action for this team. #### Themes - Not applicable to this source; it doesn't identify a direct action for this team. #### Plugins - Not applicable to this source; it doesn't identify a direct action for this team. #### The WPOCC - Support public, provider-neutral eval methods and domain-expert participation. ### FAIR - Keep evaluation artifacts portable, inspectable, and governed outside a single vendor's private scoring system. ## 🧪 Hypotheses - Model launches paired with a purpose-built product surface will produce more sustained adoption than model-only launches. - Structured eval cases will shorten time from user complaint to research action. - Teams that revisit roadmaps after capability jumps will ship more useful features than teams that preserve prior plans. - First-principles user research will predict successful vertical applications better than copying established SaaS patterns. - PMs who prototype directly with models will translate research capabilities into actionable requirements faster. ### 📊 Methodology - Compare adoption cohorts across model-only and model-plus-product launches while controlling for marketing exposure. - Track feedback items from receipt through reproduction, classification, eval creation, model change, and user outcome. - Run prospective roadmap studies that record which assumptions changed after each model release. - Audit eval suites for coverage, construct validity, demographic breadth, and mismatch with field behavior. - Interview researchers, PMs, and users separately to test whether internal definitions of value agree. ### ❓ Questions - When does a specialized harness create durable vertical value rather than a temporary wrapper? - How are eval suites protected from Goodhart's law? - Who decides that a vague complaint represents a large enough user problem? - What user groups remain absent from internal feedback loops? - How should product teams expose uncertainty when model behavior changes between releases? ### 🔮 Future Research - Independent replication of claims about model-product adoption effects. - Cross-company comparisons of research product management structures. - Long-term study of vertical AI products after general models improve. - Accessibility and internationalization audits of eval datasets and product surfaces. - Governance research on capability access concentrated inside frontier labs. ### 💡 Implications - Practice: turn vague feedback into reproducible cases before asking research teams for changes. - Policy: record eval ownership, coverage limits, and decisions made from each score. - Security: include misuse, prompt injection, tool boundaries, and unsafe fallback behavior in release gates. - Education: teach PMs to prototype, inspect model behavior, and reason from user needs. - Funding: support independent evaluation and domain experts who can challenge lab-centered evidence. ### 🧠 Learning Styles Assessment - Reading level estimate: Flesch–Kincaid grade 10–12 based on conversational syntax and specialized vocabulary; an exact score requires the cleaned transcript. #### 👀 Visual learners - Present: a spoken framework and examples, with limited dependence on visuals in the captured transcript. - Improve: add a labeled model, process diagram, and before/after example with text alternatives. #### 🔊 Aural learners - Present: the interview or narration format supports listening and speaker emphasis. - Improve: add a short chaptered audio summary and identify speakers consistently. #### 🗣️ Verbal learners - Present: definitions, analogies, repeated terms, and concrete stories. - Improve: add a glossary and a plain-language recap after each major idea. #### 🧩 Kinesthetic learners - Present: practical prompts or reflection exercises can be attempted during viewing. - Improve: add a bounded exercise, expected result, and verification checklist. #### 🧠 Logical learners - Present: frameworks and causal claims organize the discussion. - Improve: distinguish assumptions, evidence, mechanism, and measurable outcome in a table. #### 🧑‍🤝‍🧑 Social learners - Present: dialogue exposes agreement, challenge, and clarification. - Improve: provide moderated discussion prompts and a peer-review rubric. #### 🧘‍♀️ Solitary learners - Present: the transcript supports self-paced review and reflection. - Improve: add a worksheet, pause points, and answer key for self-checks. #### 🌿 Naturalistic learners - Present: Not applicable to this source; environmental systems aren't a central teaching device. - Improve: connect resource use and lifecycle effects only where evidence supports the link. ### 📊 Accessibility Assessment #### Screen reader semantics - Present: the saved transcript provides text access, but source-video heading and landmark quality wasn't verified. - Improve: use a logical heading hierarchy, identify speakers, and supply descriptive link text. WCAG 2.2: 1.3.1, 2.4.6. #### Keyboard navigation - Present: Not verifiable from transcript content alone. - Improve: test every player and transcript control by keyboard with visible focus. WCAG 2.2: 2.1.1, 2.4.7. #### Contrast and readability - Present: Not verifiable from the transcript. - Improve: test 4.5:1 normal-text contrast, 3:1 large text, line spacing, and readable line length. WCAG 2.2: 1.4.3, 1.4.12. #### Text resizing and reflow - Present: transcript text can support reflow, but the source page wasn't audited. - Improve: test 200% zoom and 320 CSS-pixel reflow without loss. WCAG 2.2: 1.4.4, 1.4.10. #### Media and captions - Present: a machine-captured transcript exists. - Improve: publish corrected captions, speaker labels, and timestamps; add audio description when visuals carry meaning. WCAG 2.2: 1.2.2, 1.2.5. #### Forms and interactions - Present: Not applicable to the spoken content; player controls still require testing. - Improve: ensure accessible names, target size, status messages, and error recovery. WCAG 2.2: 2.5.8, 4.1.2, 4.1.3. #### Standards reference - This is a content-based review, not a conformance claim. A WCAG 2.2 AA audit needs the rendered page, player, captions, keyboard flow, and assistive-technology testing. ### 🌍 DEIB Assessment #### Cultural breadth - Present: examples come mainly from a US/English-speaking professional context. - Improve: include workers and users across regions, labor systems, languages, and income levels. #### Inclusive language - Present: the conversational language is generally approachable but includes field-specific shorthand. - Improve: define jargon and avoid idioms where they obscure meaning. #### Neurodiversity support - Present: chaptered notes and a transcript support review at a chosen pace. - Improve: add predictable summaries, visual structure, and low-distraction text. #### Representation - Present: the source represents named professional experiences, not a representative sample. - Improve: add voices from affected groups with different roles, seniority, disability, and access to resources. #### Psychological safety - Present: the discussion is candid about uncertainty and tradeoffs. - Improve: separate critique from personal judgment and point people toward practical support when the topic affects livelihood or identity. ### ✅ Logical Validity #### Strong reasoning The model-plus-vehicle example offers a coherent causal account of adoption, and the feedback-to-eval workflow explains how product work can become actionable for researchers. #### Fallacies The episode risks success-story selection by emphasizing Anthropic launches that worked. Claims about inclusive access and product impact need independent outcome measures. “Evals are the new PRDs” is a memorable compression, but it becomes a false dichotomy if read literally; Penn later clarifies that PRDs still serve a different purpose. #### Tone of voice Reflective, optimistic, technical, and candid about uncertainty and rapid change. ### 🛡️ DARVO Analysis Not applicable to this source. The interview discusses product practice and organizational learning, not an allegation-response pattern. ## 🎯 Conclusion Build the model and product surface together, translate feedback into reproducible failures, and keep first-principles user judgment central as production gets faster. Next steps: test the strongest claims against independent evidence, run a bounded real-world case, and record conditions that change the result. ## 🔄 Refactor Appendix — Why AI is going vertical (again) | Dianne Penn (Anthropic) > Density: 5 Literature atoms and 1 Permanent candidate. > Literature quotes match the associated master Readwise highlights. ### 📚 Literature Atoms (H3) ### [H1] Models need products to reach users > [!quote] ID:rw_1040672517 > What was magical about Opus 4.5 is we also now not just had a model but a vehicle, a great product experience like Claude Code. **Context & use**: - 00:30 in the source. - A capability becomes valuable through a product experience that makes it usable. **Tags**: #xref **Highlight**: [view highlight](readwise://open?highlight=1040672517) <!-- atomicity: 1 idea ✓ --> ### [H2] New evidence should change product plans > [!quote] ID:rw_1040672518 > the adaptability of when you're faced with new information: how do you then make better decisions versus keeping the same plan? **Context & use**: - 15:57 in the source. - Fast model progress rewards roadmap revision rather than plan defense. **Tags**: #xref **Highlight**: [view highlight](readwise://open?highlight=1040672518) <!-- atomicity: 1 idea ✓ --> ### [H3] First principles reveal actual user value > [!quote] ID:rw_1040672519 > rather than pattern matching, what you used to do in consumer product or B2B SaaS, but actually figuring out what is the user value? **Context & use**: - 40:52 in the source. - Old category patterns can obscure new user behavior. **Tags**: #xref **Highlight**: [view highlight](readwise://open?highlight=1040672519) <!-- atomicity: 1 idea ✓ --> ### [H4] Evals make feedback executable > [!quote] ID:rw_1040672520 > It's basically test-driven development for PMs; you write the test first. **Context & use**: - 47:00 in the source. - Reproducible cases shorten the distance from complaint to research action. **Tags**: #xref **Highlight**: [view highlight](readwise://open?highlight=1040672520) <!-- atomicity: 1 idea ✓ --> ### [H5] User-centered work grows more important > [!quote] ID:rw_1040672521 > the role of people who are user-centric, who go into the details of understanding what users are trying to accomplish **Context & use**: - 83:31 in the source. - Cheap implementation raises the premium on judgment about what deserves to exist. **Tags**: #xref **Highlight**: [view highlight](readwise://open?highlight=1040672521) <!-- atomicity: 1 idea ✓ --> ### 🧠 Permanent Candidates (H4) #### [P1] Product surfaces turn general capability into vertical value **Claim** (one sentence, Courtney's voice): A general model becomes a vertical product when context, tools, feedback, and evaluation organize it around a specific user outcome. **Evidence pointer**: [[#H1]] **Scope & conditions**: This holds when domain workflows and data create measurable value; thin branding around the same generic interaction doesn't establish vertical differentiation. **Links**: - [[Model-product fit]] — because extends - [[Evaluation-driven development]] — because applies-in AI product management - [[User research]] — because evidence-for vertical value <!-- atomicity: 1 idea ✓ --> ### ✏️ Fleeting Queue - [F1] Compare eval-driven product work with WordPress release-test gates. https://www.youtube.com/watch?v=tivaWTTVRhY #fleeting #inbox - [F2] Map which parts of a vertical product remain valuable as base models improve. https://www.youtube.com/watch?v=tivaWTTVRhY #fleeting #inbox ### → Things 3 (max 2 actions) - [ ] Draft one eval from a vague user complaint and state its coverage limit — source follow-up #things - [ ] List the domain context and tools required for one vertical workflow — source follow-up #things <!-- deep-read:sections end --> 📄 **Full text:** [[Full Document Contents/Articles/Why AI Is Going Vertical (Again) Dianne Penn.md|Why AI Is Going Vertical (Again) | Dianne Penn by YouTube]] ## Highlights - Something Gary Tan's been talking about.If you are willing to spend $100,000 ayear right now in tokens, you are livingthe way somebody in 2028 is going tolive. ([View Highlight](https://read.readwise.io/read/01kz6j628fv4c9gw7s2kwhzy8b)) - Note: Large token budgets offer a preview of more AI-intensive product work. - Tags: atom/literature/h1, vertical-ai - Prototypes, products like cloud code, tag, and others,but it's very hard to predict the exactmoment or the exact model, and so theadaptability of when you're faced withnew information: how do you then makebetter decisions versus keeping the sameplan? And so, like, that agility is reallyimportant. I think another piece is, withthat, how do you actually be thinkingvery first principles and reason throughwhat's next? What's the so what? How dowe invest in new products? How do weinvest in explaining the differences tousers? So, a lot of the, a lot of theexperiences, I think, of being in thatexponential is that pace, understandinghow you operate and make betterdecisions, and then applying that firstprinciples thinking to then do somethingthat maybe we pull up a plan that, uh, wewould be expecting a few months fromnow, but now the model can actually do, uh,and work on, and actually bring that touser. So, this is things like co-work. ([View Highlight](https://read.readwise.io/read/01kz6j7fmf7tx79z8j2hb42gzx)) - Note: Fast capability change rewards revising the plan when new evidence arrives. - Tags: product-strategy, atom/literature/h2 - So what we actually look for, and thetraits and how we evaluate, uhgeneralists like PMs, generalists likeresearch product managers, have actuallybeen the same. Um. So I think some ofthose traits, number one, is firstprinciples thinking.And this is really, uh, rather thanpattern matching, what you used to do inlet's say, consumer product or B2B SaaS,um, but actually figuring out, in thismoment, for this user group with thistechnology, what is the user value? ([View Highlight](https://read.readwise.io/read/01kz6j7h7phwfg5xyh1j32ecrn)) - Note: First-principles product work starts with user value, not inherited category patterns. - Tags: user-research, atom/literature/h3 - like 99.9%. And so, it's no longer a painpoint. Uh, but in the early days, it wastaking the user feedback, figuring outactually what they mean, can wereproduce it, is it consistent, is it abig issue, and then figuring out how touh, standardize it in a way that can beconsumable for researchers. It'sBasically, test-driven development forPMs is the world we're living now. Uh,where you write the test first. So, isthis just a core part of the productmanagement job now at Enthropic? Writingbells? ([View Highlight](https://read.readwise.io/read/01kz6j7gdx8rx3m361cqvbqzg9)) - Note: Reproducible evals turn vague feedback into executable product requirements. - Tags: model-evaluation, atom/literature/h4 - Skill sets in many ways of things likethinking from a first principles mannerabout what you solve. I think,Fundamentally, you didn't ask me this, butthere is this question in the communityof do we still need PMs when the modelsare so capable? When engineers areleaning in, um, I think the role of peoplewho are user-centric, who go into thedetails of understanding what users aretrying to accomplish, bubbling that up inan actionable manner, and doing therelentless work to do that — that, tome, is a core of a product person, and Iactually think we need more of that. ([View Highlight](https://read.readwise.io/read/01kz6j7jepdhemnvcjd41wwtsy)) - Note: Cheaper implementation raises the value of detailed user-centered judgment. - Tags: ai-product-management, atom/literature/h5 ## Links - [[3. Resources/Permanent Notes/AI's Leverage Comes From the Data Foundation Not the Model]] — supports: vertical systems gain from domain data and workflow context. - [[3. Resources/Permanent Notes/Context Not Constraints Is What Unhobbles an Agent]] — applies-in: domain-specific context makes general capability operational.