Why AI is going vertical (again) | Dianne Penn (Anthropic) ![rw-book-cover](https://d34adp677peecb.cloudfront.net/static/images/article1.be68295a7e40.png) 🔗 **Source:** [Why AI is going vertical (again) | Dianne Penn (Anthropic)](https://www.youtube.com/watch?v=tivaWTTVRhY) > [!note]+ Document Notes > tags: [ai-product-management, model-evaluation, anthropic, product-strategy, user-research, vertical-ai] > author: Lenny Rachitsky and Dianne Penn > source: Lenny's Podcast > reference: https://www.youtube.com/watch?v=tivaWTTVRhY > date created: 2026-08-04 10:08 EDT > > > [!info] Generated analysis — not the author's words. Sections below are machine-written > > from the captured source. Only text inside `> [!quote]` blocks is verbatim from the author. > > Do not quote or attribute any other line to the named author. > > # 🏷️ Title: Why AI is going vertical (again) | Dianne Penn (Anthropic) > <!-- deep-read:sections start --> ## 🧭 Table of Contents - [📝 Summary](#-summary) - [📖 Main Content](#-main-content) - [🗒️ Notes](#️-notes) - [📚 References](#-references) - [🗣️ Paraphrase](#️-paraphrase) - [📰 Newsletter](#-newsletter) - [✅ Action Items](#-action-items) - [🧪 Hypotheses](#-hypotheses) - [📊 Methodology](#-methodology) - [❓ Questions](#-questions) - [🔮 Future Research](#-future-research) - [💡 Implications](#-implications) - [🧠 Learning Styles Assessment](#-learning-styles-assessment) - [📊 Accessibility Assessment](#-accessibility-assessment) - [🌍 DEIB Assessment](#-deib-assessment) - [✅ Logical Validity](#-logical-validity) - [🛡️ DARVO Analysis](#-darvo-analysis) - [🎯 Conclusion](#-conclusion) - [🔄 Refactor Appendix](#-refactor-appendix--why-ai-is-going-vertical-again-dianne-penn-anthropic) ### 🔗 References - [Source video](https://www.youtube.com/watch?v=tivaWTTVRhY) - References named in the transcript are contextual leads; verify them independently before reuse. ## 📝 Summary - **What happened:** Dianne Penn describes product management inside Anthropic as a bridge between fast-moving model research and specific user outcomes. Model capability alone doesn't create adoption; products, harnesses, context, feedback translation, and evaluations turn general models into useful vertical experiences. - **Why it matters:** The episode offers an inside account of research-product work and the changing PM role. It is valuable testimony from a leader at one company, not an independent measure of Anthropic's performance or safety. - **Key stakeholders:** AI product managers, researchers, engineers, founders, users in specialized domains, evaluators, safety teams, and hiring candidates. - **Risks and tradeoffs:** Vendor-centered evidence, product names mistranscribed in captions, unclear claims about inclusion, rapid-plan churn, concentrating advanced capabilities inside labs, and treating eval scores as complete representations of user value. - **Bottom line:** Build the model and product surface together, translate feedback into reproducible failures, and keep first-principles user judgment central as production gets faster. ## 📖 Main Content Dianne Penn describes product management inside Anthropic as a bridge between fast-moving model research and specific user outcomes. Model capability alone doesn't create adoption; products, harnesses, context, feedback translation, and evaluations turn general models into useful vertical experiences. The episode offers an inside account of research-product work and the changing PM role. It is valuable testimony from a leader at one company, not an independent measure of Anthropic's performance or safety. The source is an interview or educational presentation, not a controlled study. Claims about products, markets, organizations, and outcomes should be treated as speaker testimony unless a cited source independently confirms them. ### 🗒️ Notes #### ⏱️ [00:30](https://www.youtube.com/watch?v=tivaWTTVRhY&t=30) Penn links the impact of Opus 4 - Penn links the impact of Opus 4.5 to Claude Code as the product vehicle through which users experienced the model. #### ⏱️ [15:57](https://www.youtube.com/watch?v=tivaWTTVRhY&t=957) Rapid capability changes require teams to update plans when new evidence appears - Rapid capability changes require teams to update plans when new evidence appears. #### ⏱️ [23:57](https://www.youtube.com/watch?v=tivaWTTVRhY&t=1437) Labs is framed as a place for discontinuous bets outside the core roadmap - Labs is framed as a place for discontinuous bets outside the core roadmap. #### ⏱️ [40:52](https://www.youtube.com/watch?v=tivaWTTVRhY&t=2452) Research product managers are expected to reason from user value rather than repeat old SaaS playbooks - Research product managers are expected to reason from user value rather than repeat old SaaS playbooks. #### ⏱️ [47:00](https://www.youtube.com/watch?v=tivaWTTVRhY&t=2820) The team translates vague user feedback into reproducible failure categories and evals - The team translates vague user feedback into reproducible failure categories and evals. #### ⏱️ [83:31](https://www.youtube.com/watch?v=tivaWTTVRhY&t=5011) Penn argues that easier building increases the need for detailed, user-centered product judgment - Penn argues that easier building increases the need for detailed, user-centered product judgment. ## 📚 References - [Original video](https://www.youtube.com/watch?v=tivaWTTVRhY) - Not applicable beyond the source video; the transcript doesn't supply a complete, verified bibliography. ## 🗣️ Paraphrase - Anthropic's product identity formed through repeated experiments that exposed model behavior to users. - A frontier model and a fitting product surface can amplify each other's adoption. - Fast capability jumps make static roadmaps unreliable. - Labs explores large bets that may not fit the core roadmap. - Research product managers embed with researchers where changes directly affect users. - Vague feedback becomes useful only after a team reproduces and classifies the failure. - Evals can act like executable requirements while PRDs still hold context and alignment. - As implementation gets easier, deciding what is worth building becomes harder and more important. ## 📰 Newsletter The striking line from Anthropic's Dianne Penn is that evals are becoming executable requirements for product teams. A model may be capable, but users meet it through a product, a harness, and a context. WordPress's parallel is familiar: shipped behavior beats a proposal, and a test case beats a vague complaint. ## ✅ Action Items #### DevRel - Turn vague developer feedback into reproducible cases with clear scope and expected behavior. #### OSPO - Fund independent evaluation and document decision rights across research, product, and community stakeholders. #### Community - Invite domain experts and affected users into evaluation design before a pilot scales. #### WordPress #### Core - Not applicable to this source; it doesn't identify a direct action for this team. #### Core Performance - Not applicable to this source; it doesn't identify a direct action for this team. #### Core AI - Use task-specific evals with field replay before promoting model-assisted features. #### Core Program Team - Track cross-team assumptions that change after model or API updates. #### Mobile - Not applicable to this source; it doesn't identify a direct action for this team. #### Playground - Provide reproducible browser fixtures for model and harness experiments. #### Community - Not applicable to this source; it doesn't identify a direct action for this team. #### Training - Teach eval design, user interviews, and failure reproduction together. #### Marketing - Not applicable to this source; it doesn't identify a direct action for this team. #### Support - Not applicable to this source; it doesn't identify a direct action for this team. #### Design - Test whether model behavior and product surface support the same user goal. #### Documentation - Publish eval scope, known gaps, and replay steps near feature guidance. #### Photos - Not applicable to this source; it doesn't identify a direct action for this team. #### TV - Not applicable to this source; it doesn't identify a direct action for this team. #### Polyglots - Not applicable to this source; it doesn't identify a direct action for this team. #### Accessibility - Include disabled users and assistive-technology workflows in eval coverage. #### Meta - Measure real contributor outcomes, not only model or interface activity. #### CLI - Not applicable to this source; it doesn't identify a direct action for this team. #### Hosting - Not applicable to this source; it doesn't identify a direct action for this team. #### Tide - Not applicable to this source; it doesn't identify a direct action for this team. #### Openverse - Not applicable to this source; it doesn't identify a direct action for this team. #### Themes - Not applicable to this source; it doesn't identify a direct action for this team. #### Plugins - Not applicable to this source; it doesn't identify a direct action for this team. #### The WPOCC - Support public, provider-neutral eval methods and domain-expert participation. ### FAIR - Keep evaluation artifacts portable, inspectable, and governed outside a single vendor's private scoring system. ## 🧪 Hypotheses - Model launches paired with a purpose-built product surface will produce more sustained adoption than model-only launches. - Structured eval cases will shorten time from user complaint to research action. - Teams that revisit roadmaps after capability jumps will ship more useful features than teams that preserve prior plans. - First-principles user research will predict successful vertical applications better than copying established SaaS patterns. - PMs who prototype directly with models will translate research capabilities into actionable requirements faster. ### 📊 Methodology - Compare adoption cohorts across model-only and model-plus-product launches while controlling for marketing exposure. - Track feedback items from receipt through reproduction, classification, eval creation, model change, and user outcome. - Run prospective roadmap studies that record which assumptions changed after each model release. - Audit eval suites for coverage, construct validity, demographic breadth, and mismatch with field behavior. - Interview researchers, PMs, and users separately to test whether internal definitions of value agree. ### ❓ Questions - When does a specialized harness create durable vertical value rather than a temporary wrapper? - How are eval suites protected from Goodhart's law? - Who decides that a vague complaint represents a large enough user problem? - What user groups remain absent from internal feedback loops? - How should product teams expose uncertainty when model behavior changes between releases? ### 🔮 Future Research - Independent replication of claims about model-product adoption effects. - Cross-company comparisons of research product management structures. - Long-term study of vertical AI products after general models improve. - Accessibility and internationalization audits of eval datasets and product surfaces. - Governance research on capability access concentrated inside frontier labs. ### 💡 Implications - Practice: turn vague feedback into reproducible cases before asking research teams for changes. - Policy: record eval ownership, coverage limits, and decisions made from each score. - Security: include misuse, prompt injection, tool boundaries, and unsafe fallback behavior in release gates. - Education: teach PMs to prototype, inspect model behavior, and reason from user needs. - Funding: support independent evaluation and domain experts who can challenge lab-centered evidence. ### 🧠 Learning Styles Assessment - Reading level estimate: Flesch–Kincaid grade 10–12 based on conversational syntax and specialized vocabulary; an exact score requires the cleaned transcript. #### 👀 Visual learners - Present: a spoken framework and examples, with limited dependence on visuals in the captured transcript. - Improve: add a labeled model, process diagram, and before/after example with text alternatives. #### 🔊 Aural learners - Present: the interview or narration format supports listening and speaker emphasis. - Improve: add a short chaptered audio summary and identify speakers consistently. #### 🗣️ Verbal learners - Present: definitions, analogies, repeated terms, and concrete stories. - Improve: add a glossary and a plain-language recap after each major idea. #### 🧩 Kinesthetic learners - Present: practical prompts or reflection exercises can be attempted during viewing. - Improve: add a bounded exercise, expected result, and verification checklist. #### 🧠 Logical learners - Present: frameworks and causal claims organize the discussion. - Improve: distinguish assumptions, evidence, mechanism, and measurable outcome in a table. #### 🧑‍🤝‍🧑 Social learners - Present: dialogue exposes agreement, challenge, and clarification. - Improve: provide moderated discussion prompts and a peer-review rubric. #### 🧘‍♀️ Solitary learners - Present: the transcript supports self-paced review and reflection. - Improve: add a worksheet, pause points, and answer key for self-checks. #### 🌿 Naturalistic learners - Present: Not applicable to this source; environmental systems aren't a central teaching device. - Improve: connect resource use and lifecycle effects only where evidence supports the link. ### 📊 Accessibility Assessment #### Screen reader semantics - Present: the saved transcript provides text access, but source-video heading and landmark quality wasn't verified. - Improve: use a logical heading hierarchy, identify speakers, and supply descriptive link text. WCAG 2.2: 1.3.1, 2.4.6. #### Keyboard navigation - Present: Not verifiable from transcript content alone. - Improve: test every player and transcript control by keyboard with visible focus. WCAG 2.2: 2.1.1, 2.4.7. #### Contrast and readability - Present: Not verifiable from the transcript. - Improve: test 4.5:1 normal-text contrast, 3:1 large text, line spacing, and readable line length. WCAG 2.2: 1.4.3, 1.4.12. #### Text resizing and reflow - Present: transcript text can support reflow, but the source page wasn't audited. - Improve: test 200% zoom and 320 CSS-pixel reflow without loss. WCAG 2.2: 1.4.4, 1.4.10. #### Media and captions - Present: a machine-captured transcript exists. - Improve: publish corrected captions, speaker labels, and timestamps; add audio description when visuals carry meaning. WCAG 2.2: 1.2.2, 1.2.5. #### Forms and interactions - Present: Not applicable to the spoken content; player controls still require testing. - Improve: ensure accessible names, target size, status messages, and error recovery. WCAG 2.2: 2.5.8, 4.1.2, 4.1.3. #### Standards reference - This is a content-based review, not a conformance claim. A WCAG 2.2 AA audit needs the rendered page, player, captions, keyboard flow, and assistive-technology testing. ### 🌍 DEIB Assessment #### Cultural breadth - Present: examples come mainly from a US/English-speaking professional context. - Improve: include workers and users across regions, labor systems, languages, and income levels. #### Inclusive language - Present: the conversational language is generally approachable but includes field-specific shorthand. - Improve: define jargon and avoid idioms where they obscure meaning. #### Neurodiversity support - Present: chaptered notes and a transcript support review at a chosen pace. - Improve: add predictable summaries, visual structure, and low-distraction text. #### Representation - Present: the source represents named professional experiences, not a representative sample. - Improve: add voices from affected groups with different roles, seniority, disability, and access to resources. #### Psychological safety - Present: the discussion is candid about uncertainty and tradeoffs. - Improve: separate critique from personal judgment and point people toward practical support when the topic affects livelihood or identity. ### ✅ Logical Validity #### Strong reasoning The model-plus-vehicle example offers a coherent causal account of adoption, and the feedback-to-eval workflow explains how product work can become actionable for researchers. #### Fallacies The episode risks success-story selection by emphasizing Anthropic launches that worked. Claims about inclusive access and product impact need independent outcome measures. “Evals are the new PRDs” is a memorable compression, but it becomes a false dichotomy if read literally; Penn later clarifies that PRDs still serve a different purpose. #### Tone of voice Reflective, optimistic, technical, and candid about uncertainty and rapid change. ### 🛡️ DARVO Analysis Not applicable to this source. The interview discusses product practice and organizational learning, not an allegation-response pattern. ## 🎯 Conclusion Build the model and product surface together, translate feedback into reproducible failures, and keep first-principles user judgment central as production gets faster. Next steps: test the strongest claims against independent evidence, run a bounded real-world case, and record conditions that change the result. ## 🔄 Refactor Appendix — Why AI is going vertical (again) | Dianne Penn (Anthropic) > Density: 5 Literature atoms and 1 Permanent candidate. > Literature quotes match the associated master Readwise highlights. ### 📚 Literature Atoms (H3) ### [H1] Models need products to reach users > [!quote] ID:rw_1040672517 > What was magical about Opus 4.5 is we also now not just had a model but a vehicle, a great product experience like Claude Code. **Context & use**: - 00:30 in the source. - A capability becomes valuable through a product experience that makes it usable. **Tags**: #xref **Highlight**: [view highlight](readwise://open?highlight=1040672517) <!-- atomicity: 1 idea ✓ --> ### [H2] New evidence should change product plans > [!quote] ID:rw_1040672518 > the adaptability of when you're faced with new information: how do you then make better decisions versus keeping the same plan? **Context & use**: - 15:57 in the source. - Fast model progress rewards roadmap revision rather than plan defense. **Tags**: #xref **Highlight**: [view highlight](readwise://open?highlight=1040672518) <!-- atomicity: 1 idea ✓ --> ### [H3] First principles reveal actual user value > [!quote] ID:rw_1040672519 > rather than pattern matching, what you used to do in consumer product or B2B SaaS, but actually figuring out what is the user value? **Context & use**: - 40:52 in the source. - Old category patterns can obscure new user behavior. **Tags**: #xref **Highlight**: [view highlight](readwise://open?highlight=1040672519) <!-- atomicity: 1 idea ✓ --> ### [H4] Evals make feedback executable > [!quote] ID:rw_1040672520 > It's basically test-driven development for PMs; you write the test first. **Context & use**: - 47:00 in the source. - Reproducible cases shorten the distance from complaint to research action. **Tags**: #xref **Highlight**: [view highlight](readwise://open?highlight=1040672520) <!-- atomicity: 1 idea ✓ --> ### [H5] User-centered work grows more important > [!quote] ID:rw_1040672521 > the role of people who are user-centric, who go into the details of understanding what users are trying to accomplish **Context & use**: - 83:31 in the source. - Cheap implementation raises the premium on judgment about what deserves to exist. **Tags**: #xref **Highlight**: [view highlight](readwise://open?highlight=1040672521) <!-- atomicity: 1 idea ✓ --> ### 🧠 Permanent Candidates (H4) #### [P1] Product surfaces turn general capability into vertical value **Claim** (one sentence, Courtney's voice): A general model becomes a vertical product when context, tools, feedback, and evaluation organize it around a specific user outcome. **Evidence pointer**: [[#H1]] **Scope & conditions**: This holds when domain workflows and data create measurable value; thin branding around the same generic interaction doesn't establish vertical differentiation. **Links**: - [[Model-product fit]] — because extends - [[Evaluation-driven development]] — because applies-in AI product management - [[User research]] — because evidence-for vertical value <!-- atomicity: 1 idea ✓ --> ### ✏️ Fleeting Queue - [F1] Compare eval-driven product work with WordPress release-test gates. https://www.youtube.com/watch?v=tivaWTTVRhY #fleeting #inbox - [F2] Map which parts of a vertical product remain valuable as base models improve. https://www.youtube.com/watch?v=tivaWTTVRhY #fleeting #inbox ### → Things 3 (max 2 actions) - [ ] Draft one eval from a vague user complaint and state its coverage limit — source follow-up #things - [ ] List the domain context and tools required for one vertical workflow — source follow-up #things <!-- deep-read:sections end --> ## Highlights - What was magical about Opus 4.5 is we also now not just had a model but a vehicle, a great product experience like Claude Code. - Note: Model capability reaches users through a product surface designed for the work. - the adaptability of when you're faced with new information: how do you then make better decisions versus keeping the same plan? - Note: Fast model progress makes adaptation a core product-management competency. - rather than pattern matching, what you used to do in consumer product or B2B SaaS, but actually figuring out what is the user value? - Note: First-principles product thinking matters when precedent ages faster than the technology. - It's basically test-driven development for PMs; you write the test first. - Note: Evals turn vague user complaints into reproducible, actionable product requirements. - the role of people who are user-centric, who go into the details of understanding what users are trying to accomplish - Note: Capable models increase the value of detailed user understanding rather than eliminating product work.