# AI Product Manager's Handbook — Work the Book > [!warning] Source boundary > This analysis covers the local **2024 Packt Early Access EPUB** supplied for this session. The archive passes an integrity test, but it is not the finished second edition. Its opening roadmap lists 20 substantive chapters; the EPUB contains Chapters 1–12 and ends after an unfinished case study. Packt's finished November 2024 edition is listed as 488 pages and includes the missing sections. No conclusions about Chapters 13–20 are inferred from titles alone. > [!info] Attribution key > **Author says** = explicit in the available EPUB. **Author implies** = a reasonable reconstruction. **Third party** = a source cited by the author. **Illustrative case** = the fictional Waterbear/Akeira example. **We infer / We conclude** = analysis produced in this session. **Still uncertain** = source or evidence is inadequate. ## Prompt 0 — Retrieval and vault setup ### Ingestion record | Field | Result | |---|---| | Selected source | `/Users/courtneyrobertson/Downloads/AI Product Managers Handbook, Second Edition (Early Access) (Irene Bratsis) (z-library.sk, 1lib.sk, z-lib.sk).epub` | | Selection assumption | 🔶 No EPUB was bundled with the pasted prompt. The newest newly downloaded EPUB, timestamped 10:37 on 2026-08-24, was selected because it most closely matched the request. | | Embedded identity | *AI Product Manager's Handbook*, Irene Bratsis, Packt, ISBN 978-1-83588-284-9, Early Access | | Integrity | ZIP/EPUB validation passed; no compressed-data errors | | Extracted coverage | 4,591 lines; about 112,700 words; title material plus Chapters 1–12 | | Missing material | Planned Chapters 13–20, preface/index/back matter, and the completed ending of Chapter 12 | | Readwise | Exact title/author/ISBN search returned no document; no reader-created highlights were available | | Layer 1 treatment | Local EPUB remains the source of truth. This note paraphrases it and uses short excerpts only. | | Passage IDs | `EP-####` marks system-selected EPUB evidence; it does **not** mean a Readwise highlight. | ### Source-completeness diagnosis The source contains several production-state signals: - The welcome page calls the chapters drafts that will be updated. - The opening roadmap names 20 chapters, but only 12 are present. - Chapter 12 has no summary or references and contains the editorial note “Maybe we can create a double sided table.” - Its final paragraph promises the next section on adding AI to traditional products, but that section is absent. - The finished publisher listing confirms later content exists. See [Packt's finished-edition record](https://www.packtpub.com/en-us/product/ai-product-managers-handbook-9781835882849). ### Highlight index These passages were selected by the system to support the analysis. | ID | Chapter / location | Source lane | Passage significance | |---|---|---|---| | EP-0001 | Welcome, lines 26–53 | Author says | Early Access status and 20-chapter roadmap | | EP-0002 | Ch. 1, 93–111 | Author says | AI/ML boundary, ANI, and harms from weak oversight | | EP-0003 | Ch. 1, 251–281 | Author says | Data infrastructure plus CI/CD/CT/CM operating loop | | EP-0004 | Ch. 1, 356–389 | Author says | Shadow, A/B, and canary deployment patterns | | EP-0005 | Ch. 2, 626–640 | Author says | Real-world, varied training data and readiness | | EP-0006 | Ch. 2, 648–681 | Author says | Ether-price model comparison and high-score caution | | EP-0007 | Ch. 2, 718–738 | Author says | Monitoring, drift, documentation, and user interpretation | | EP-0008 | Ch. 2, 744–786 | Author says / third party | Ethics, 2023 US executive order, EU principles, downstream harm | | EP-0009 | Ch. 3, 869–923 | Author says | Historical roots, eugenics, exclusion, and demographic gaps | | EP-0010 | Ch. 3, 1081–1140 | Author says | Generative-model taxonomy and transformer discussion | | EP-0011 | Ch. 3, 1165–1201 | Author says | Explainability, accuracy, data, and compute tradeoffs | | EP-0012 | Ch. 4, 1282–1429 | Author says | Commercial examples and four-part strategy model | | EP-0013 | Ch. 5, 1503–1657 | Author says / third party | Finance, government, health, and AI-for-good applications | | EP-0014 | Ch. 6, 1774–1849 | Author says | Four-stage AI-product lifecycle | | EP-0015 | Ch. 6, 1857–1998 | Author says | Cross-functional team and stack decisions | | EP-0016 | Ch. 6, 2004–2054 | Author says | Productization, customization, and selling value | | EP-0017 | Ch. 6, 2060–2068 | Author says | Waterbear/Akeira explicitly introduced as a sample case | | EP-0018 | Ch. 7, 2281–2305 | Author says | Scope → package → delivery → feedback loop | | EP-0019 | Ch. 7, 2309–2507 | Author says | AI/software differences, uncertainty, MLOps, retraining | | EP-0020 | Ch. 8, 2627–2706 | Author says | Domain, vertical, strategy, and “Why AI?” gate | | EP-0021 | Ch. 8, 2868–2918 | Author says | Business and technical metrics; metric-definition problem | | EP-0022 | Ch. 9, 3067–3238 | Author says | Macro/micro AI taxonomy | | EP-0023 | Ch. 9, 3306–3361 | Author says / third party | Ethics, performance, and safety failure cases | | EP-0024 | Ch. 10, 3453–3583 | Author says | User-centered design, iteration, documentation, trust | | EP-0025 | Ch. 11, 3810–3930 | Author says | North-star metrics, KPIs, and OKRs | | EP-0026 | Ch. 11, 3942–4098 | Author says / third party | Growth stack, data collection, cost, and stale tools | | EP-0027 | Ch. 12, 4267–4538 | Author says | Alignment, psychological safety, boundaries, feedback, testing | | EP-0028 | Ch. 12, 4544–4591 | Illustrative case | Unfinished Akeira case and abrupt source ending | ### Cluster map | Cluster | Chapters | Main topics | Rationale | Representative passages | |---|---:|---|---|---| | Foundations and market framing | 1–5 | Definitions, infrastructure, modeling, deployment, commercialization, social impact | Explains the technical and market terrain before product design | EP-0002, EP-0003, EP-0006, EP-0011, EP-0013 | | Building an AI-native product | 6–12 | Lifecycle, team, productization, verticalization, design, metrics, management | The complete available operating sequence | EP-0014, EP-0018, EP-0020, EP-0024, EP-0027 | | Missing integration and career arc | 13–20 | Existing-product integration and AI-PM career | Named by the opening roadmap but absent from this snapshot | EP-0001, EP-0028 | ## Prompt 1 — Setup, contract, baselines, biases, and priors ### Reading contract - **Why I am reading:** 🔶 Assumed purpose: decide which ideas can guide AI-related product and technical-communication work, and which require correction or current verification. - **What I hope to understand:** How the author connects data, models, deployment, product discovery, commercialization, design, metrics, ethics, and team leadership. - **Practical use:** Build a risk-aware AI product checklist, teaching material, vocabulary, and small experiments. - **Success criteria:** Recover the whole available argument; separate operating advice from promotional examples; test major technical claims; expose missing accessibility, governance, and evidence requirements. - **Evidence that would change my mind:** Reproducible model results, independent outcome studies, clear case-study provenance, complete source methods, and current primary documentation. - **What the source cannot establish:** Outcomes from its fictional case; causal returns from named companies; current vendor status; the content of missing chapters; universal AI-PM practices. - **Constraints:** The source is a partial Early Access draft; no exact Readwise record exists; the reader supplied no personal highlights, purpose statement, or prior-belief inventory. ### Prior beliefs and incentives | Lane | Baseline | |---|---| | Reader-provided information | No beliefs, motives, or incentives were supplied for this book. | | System inference | 🔶 Product advice is most useful when it joins user value, operational reliability, governance, and measurement. | | System incentive risk | The analysis may favor documented controls and testability over inspirational framing. | | Professional relevance | 🔶 The reader's technical-communication and product interests may make terminology, teaching, and evidence quality especially useful. This is contextual, not a claim about belief. | | Resistance watch | Avoid rejecting accessible explanations merely because they simplify; reject only when simplification changes the decision. | ### Open questions inventory **Definitions and mechanisms** 1. What is the minimum evidence needed to call a product “AI-powered”? 2. When does an AI feature require a dedicated AI-product lifecycle instead of an ordinary software lifecycle? 3. How should model, data, product, and policy changes be versioned together? 4. What is the practical boundary between AIOps and MLOps? 5. Which model metrics represent user outcomes rather than laboratory performance? **Evidence and credibility** 6. Which named company outcomes have independent, causal support? 7. Does the Ether-price example avoid leakage, temporal contamination, and test-set reuse? 8. How were commercial “successes” selected, and where are failed comparison cases? 9. Which statistics come from primary studies rather than media or vendor summaries? 10. Can the fictional Akeira case be taught without being mistaken for evidence? **Application and failure modes** 11. What decision rule should reject AI when rules, search, or ordinary software will work? 12. How should teams set thresholds when false positives and false negatives harm different groups? 13. What rollback, fallback, and appeal paths must exist before launch? 14. How should product teams test model drift, interface drift, and policy drift separately? 15. What makes a north-star metric unsafe for a mental-health or high-stakes product? 16. How should total cost include human review, incident response, retraining, energy, and compliance? **Context and counterarguments** 17. Does “AI-native” still describe products assembled from third-party foundation models and APIs? 18. When do bespoke AI services remain profitable without becoming standardized products? 19. What does the framework miss about procurement, copyright, security, and supply-chain risk? 20. How does an existing-product integration path differ from the native-product path? **DEIB, multicultural use, and accessibility** 21. Which affected communities participate in problem definition, testing, launch approval, and incident review? 22. How are subgroup error rates, intersectional harms, and proxy variables measured? 23. What disability-access requirements apply to data collection, interfaces, explanations, and appeals? 24. How do language, region, law, labor, connectivity, and device access change the advice? 25. When does personalization become surveillance or essentialist segmentation? **Unanswered by this snapshot** 26. How does the finished edition repair the draft's technical errors? 27. What do Chapters 13–20 add about integration and career development? 28. Is Waterbear/Akeira fully labeled as fictional in the finished layout? ### Priors seed table Confidence ranges are structured judgments, not measured probabilities. | Placeholder | Initial belief | Prior | Reason | Evidence needed | |---|---|---:|---|---| | P-01 | Data and operations matter as much as model choice | 80% | Common production pattern | Incident and lifecycle evidence | | P-02 | Problem-first selection beats adding AI for its own sake | 85% | Avoids solutionism | Portfolio comparison | | P-03 | Continuous monitoring is necessary | 90% | Models and contexts change | Monitoring-to-remediation records | | P-04 | A single score can't establish readiness | 90% | Metrics omit context and costs | External/subgroup validation | | P-05 | Productization can reduce delivery variance | 70% | Standardization often helps | Unit economics and variance data | | P-06 | AI projects have more uncertainty than deterministic software | 75% | Data and model behavior add variables | Matched project data | | P-07 | More data is not automatically better | 85% | Quality, consent, and fit matter | Learning curves and harm analysis | | P-08 | Explainability needs depend on use and affected people | 85% | Stakes differ by decision | User and regulator evidence | | P-09 | Funding and user counts do not prove product quality | 95% | They measure adoption/capital | Retention, outcomes, harms | | P-10 | Psychological safety improves risk discovery | 70% | Reporting depends on power and trust | Climate plus incident data | | P-11 | Accessibility must be part of the product lifecycle | 90% | Access barriers arise at every stage | Disabled-user testing | | P-12 | Regulations and vendor/tool advice will age quickly | 95% | Known high-drift domain | Scheduled primary-source review | ### Drift assumptions High-drift areas include model families, product names, knowledge dates, prices, compute requirements, vendor availability, benchmarks, salary figures, privacy rules, AI regulation, standards, and public policy. The book's own warning that executive orders can be revoked proved accurate: US Executive Order 14110 was revoked in 2025 by [Executive Order 14179](https://www.federalregister.gov/documents/full_text/html/2025/01/31/2025-02172.html). Treat all named tools and legal summaries as dated until checked. ## Prompt 2 — Inspectional reading ### Book map | Ch. | Actual title in source | Apparent purpose | Role in argument | Evidence / confidence | |---:|---|---|---|---| | 1 | Understanding the Infrastructure and Tools for Building AI Products | Define AI/ML and operational foundations | Establish technical terrain | Direct text; high | | 2 | Model Development and Maintenance for AI Products | Join product stages to training, deployment, drift, and ethics | Turn concepts into lifecycle | Direct text; high | | 3 | Machine Learning and Deep Learning Deep Dive | Broaden model vocabulary and tradeoffs | Add technical fluency | Direct text; high | | 4 | Commercializing AI Products | Classify markets and competitive strategy | Connect models to market choices | Direct text; high | | 5 | AI Transformation and Its Impact on Product Management | Survey sector use and social effects | Expand opportunity and responsibility | Direct text; high | | 6 | Understanding the AI-Native Product | Define stages, team, stack, and selling | Begin native-product operating model | Direct text; high | | 7 | Productizing the ML Service | Standardize scope, delivery, and feedback | Explain repeatability and MLOps | Direct text; high | | 8 | Customization for Verticals, Customers, and Peer Groups | Specialize by domain and define value metrics | Add market specificity | Direct text; high | | 9 | Macro and Micro AI for Your Product | Map technologies to features and failures | Recap options and cautions | Direct text; high | | 10 | Product Design for the AI Native Product | Center users, iteration, trust, and story | Add design discipline | Direct text; high | | 11 | Benchmarking Performance, Growth Hacking, and Cost | Select metrics and operating tools | Define success signals and economics | Direct text; high | | 12 | Managing the AI Native Product | Manage alignment, people, data, and testing | Close the native-product section | Direct but unfinished; high | | 13–20 | Named in roadmap; absent | Integration and career arc | Would complete the book's promise | Titles only; content unavailable | ### Central project - **Main problem:** Product managers need enough technical, commercial, ethical, and organizational fluency to guide AI products without becoming model engineers. - **Proposed answer:** Treat AI as a continuing product-and-service system: start from data and user value, build with a cross-functional team, deploy in controlled stages, monitor and retrain, specialize by market, measure value, and maintain alignment and trust. - **Method:** Explanatory synthesis, personal experience, named company examples, secondary references, and a fictional continuing case. - **Audience:** Aspiring and practicing PMs with some technical exposure. - **Reader assumptions:** AI adoption will keep expanding; organizations want AI; product managers bridge business and technical teams. - **Rhetorical modes:** Teach, explain, persuade, advocate, warn, and sell confidence in the AI-PM role. ### Provisional summary *Provisional because eight planned chapters are missing.* Bratsis presents AI product management as a cross-functional discipline that joins data infrastructure, model development, market strategy, product design, ethics, measurement, and organizational leadership. The strongest through-line is operational: an AI product does not end at model training. Teams must keep data available and representative, stage deployments, watch drift, test outputs, listen to users, manage costs, and maintain shared product direction. The book also asks PMs to understand enough machine learning to question metrics and tradeoffs while avoiding the idea that technical skill alone defines the role. Its advice is practical when it turns values into recurring work—monitoring, documentation, feedback, and stakeholder communication. Its evidence is weaker when funding, growth, press, or fictional case outcomes stand in for product efficacy. The Early Access state matters: technical simplifications, production notes, stale tools, and an abrupt ending reduce authority. Use the book as an operating-question generator, not as a current technical or legal reference. ### Reading hotspots | Area | Why it matters | |---|---| | Ch. 1 continuous maintenance | Closest thing to the book's technical operating spine | | Ch. 2 model comparison and ethics | Contains both a useful caution and a flawed inference about overfitting | | Ch. 3 explainability and cost | Key product tradeoffs plus several technical overstatements | | Ch. 6 lifecycle and team | Core AI-native product model | | Ch. 7 productization loop | Most reusable business-to-operations bridge | | Ch. 8 “Why AI?” and metrics | Best defense against solutionism; metric taxonomy needs correction | | Ch. 9 failure cases | Counterweight to promotional examples | | Ch. 10 design and documentation | Best user-centered section; accessibility remains underspecified | | Ch. 11 metrics and cost | Highly actionable but tool and statistic drift is high | | Ch. 12 people and governance | Useful team practices; incomplete case weakens closure | ### Guiding questions for deeper reading 1. What must be true before AI is the right solution? 2. Which book claims describe AI generally, and which describe only supervised ML? 3. Does each metric match the decision and cost of error? 4. Are test data truly held out and time-aware? 5. What user harm can occur even when aggregate accuracy improves? 6. Who can appeal an automated result, and to whom? 7. Which case studies demonstrate outcomes rather than popularity? 8. How does the product work when the model or vendor is unavailable? 9. What data is necessary, and what data should not be collected? 10. How are disabled and multilingual users included in research and testing? 11. What evidence would cause a team to stop an AI feature? 12. How do monitoring and incident response connect to roadmap decisions? 13. What human labor stays hidden behind “automation”? 14. How should high-stakes products change discovery, testing, and launch gates? 15. What does the finished edition correct or add? ### Reading strategy **Recommendation: selective deep read with current-source verification.** Read Chapters 1–2 and 6–12 closely for operating practices. Skim broad taxonomies and promotional company lists. Verify technical definitions, model evaluations, laws, standards, vendors, prices, and statistics before reuse. Obtain the finished edition before relying on integration or career guidance. ## Prompt 3 — Classification and terminology engine ### Classification | Dimension | Classification | |---|---| | Domain | AI product management | | Subdomains | ML lifecycle, MLOps, commercialization, product design, metrics, governance | | Genre | Technical-business handbook | | Argument | Explanatory and prescriptive, supported by examples | | Primary method | Practitioner synthesis rather than systematic research | | Intended use | Career orientation and operating reference | | Comparison traditions | Product discovery, service productization, responsible AI, MLOps, growth/product analytics | ### Controlled vocabulary `Evidence` points to system-selected EPUB passages. | Term | Author's working definition | Operational definition for use | Equivocation risk | Example / counterexample | Evidence | |---|---|---|---|---|---| | AI | Broad computer-science umbrella including learning and rule-based systems | A system performing tasks associated with inference, prediction, generation, or decision support | Marketing label can hide method | Expert system / ordinary form validation | EP-0002 | | ML | System improves from past examples | A model fitted from data to generalize to new inputs | “Learning” can imply online autonomy | Spam classifier / fixed rules | EP-0002 | | DL | Neural-network methods with layered representation learning | Multi-layer neural model trained for a defined task | Not every neural net is equally opaque | CNN / linear regression | EP-0010 | | ANI | AI applied to bounded tasks | Task-scoped capability without general competence | “Weak” can understate harm | Recommender / human-like general agent | EP-0002 | | Generative AI | Models generating text, image, code, or other content | Probabilistic system producing new outputs from learned distributions | Book narrows it too much to GANs and LLMs | Diffusion image model / logistic regression | EP-0010 | | LLM | Large language model trained on extensive text or multimodal data | Foundation model that predicts tokens and supports language tasks | Product name and model are often conflated | GPT-family model / scripted chatbot | EP-0010 | | Data pipeline | Flow that makes data available to models | Versioned ingestion, validation, transformation, storage, and delivery | “Fresh” is not always “fit” | Batch feature pipeline / copied spreadsheet | EP-0003 | | MLOps | Practices for deploying, observing, updating, and governing ML systems | Joint model-data-code lifecycle with monitoring and rollback | Book sometimes treats it as AIOps | Model registry and drift alert / generic server monitoring | EP-0019 | | AIOps | AI applied to IT operations | Detection or automation for operational telemetry | Not interchangeable with MLOps | Incident clustering / model retraining | EP-0019 | | Drift | Change that degrades model fitness or validity | Measured change in input, concept, performance, or policy context | Retraining isn't always the remedy | Label relationship shifts / seasonal change already modeled | EP-0007 | | Hyperparameter | Configuration chosen outside model fitting | Tunable setting fixed before or around training | Book first says every model has one, later says not all | Tree depth / learned coefficient | EP-0005 | | Overfitting | Model fits training patterns that fail to generalize | Gap shown through honest validation, not one high score | High test score alone isn't proof | Train 99%, external 60% / high external score | EP-0006 | | Explainability | Ability to account for model output or process | Explanation sufficient for the affected decision, user, and appeal | Global interpretability and local explanation differ | Reason codes / vague feature-importance chart | EP-0011 | | Productization | Make a service repeatable and marketable | Bound scope, package, delivery, quality, support, and feedback | Bespoke services can remain viable | Standard API tier / one-off consulting | EP-0018 | | AI-native | Product built around AI capability | Product whose core value or operation depends on model behavior | API-built products blur the boundary | Generative editor / calculator with AI marketing | EP-0014 | | Verticalization | Tailor a product to an industry or cohort | Adapt workflow, data, controls, language, and measures to context | Can become stereotyping | Clinical workflow adaptation / gender-essentialist packaging | EP-0020 | | North-star metric | Single measure intended to align product value | Leading value measure paired with guardrails | Can reward harmful engagement | Successful task completion / time-on-app for distress | EP-0025 | | KPI | Measure tied to a business objective | Decision-linked indicator with owner and threshold | Every metric can be mislabeled KPI | Incident resolution rate / page-view count alone | EP-0025 | | Accuracy | Proportion correct across all predictions | `(TP + TN) / all cases` for classification | Book groups it with distinct metrics | Balanced classifier / imbalanced 99% trivial model | EP-0021 | | Precision | Share of positive predictions that are correct | `TP / (TP + FP)` | Not specificity | Flagged fraud cases that are fraud / recall | EP-0021 | | Recall | Share of actual positives found | `TP / (TP + FN)` | Also called sensitivity, not precision | Detected disease cases / specificity | EP-0021 | | Psychological safety | Conditions that permit questions and risk reporting | Observable ability to raise concerns without retaliation | Warm culture claims may mask power | Logged dissent and response / praise-only channel | EP-0027 | ### Contested language - **“AI,” “intelligence,” and “learning”** invite anthropomorphism. - **“Accuracy”** is used as a family label for precision, specificity, recall, and F1 even though these answer different questions. Google's current ML reference defines them separately: [classification metrics](https://developers.google.com/machine-learning/crash-course/classification/accuracy-precision-recall). - **“Success,” “best,” and “done right”** often refer to funding, attention, growth, or author preference rather than user benefit or safety. - **“Representative data”** lacks a defined population, sampling plan, subgroup measure, and consent standard. - **“Personalization”** can mean helpful adaptation, profiling, surveillance, or demographic essentialism. - **“Ethical” and “safe”** remain values unless converted into requirements, tests, owners, and response paths. - **“AI-native”** and **“traditional software”** suggest a cleaner binary than modern API-based products support. ### Category-error watchlist | Risk | Where it appears | Correction | |---|---|---| | Adoption or funding → product efficacy | Commercial examples | Measure outcomes, retention, total cost, and harms | | High test score → overfitting | Ether model comparison | Audit split, leakage, external validity, and time horizon | | Accuracy → precision/recall/specificity | Ch. 8 metrics | Select distinct metric by cost of error | | AIOps → MLOps | Ch. 7 | Separate AI for IT operations from ML lifecycle operations | | Illustration → empirical case study | Waterbear/Akeira | Label fictional and use only for discussion | | More data → better product | Several chapters | Test marginal value, quality, minimization, consent, and bias | | User engagement → user welfare | Design/growth chapters | Pair engagement with well-being and harm guardrails | | Technical capability → practical outcome | Broad sector examples | Require workflow, human, legal, and access evidence | ## Prompt 4 — Claims inventory, dependency graph, and evidence expectations ### Claims inventory Source labels: **EA** explicit author, **IA** implied author, **3P** cited third party, **IC** illustrative case, **RI** reader/system inference. | ID | Claim | Type / scope | Source | Expected evidence | Serious falsifier | Evidence | |---|---|---|---|---|---|---| | C-001 | AI-product performance depends on data and operations, not model choice alone. | causal / broad | EA | Production incidents and lifecycle studies | Model-only choices explain outcomes across matched systems | EP-0003 | | C-002 | A product should start with a real problem and a defensible reason to use AI. | prescriptive / broad | EA | Portfolio outcomes for problem-first selection | AI-first portfolios outperform without added harm/cost | EP-0020 | | C-003 | Continuous integration, delivery, training, and monitoring form the operating loop. | methodological | EA | Operating records and standards | Stable systems succeed without comparable controls | EP-0003 | | C-004 | Representative real-world data improves performance and ethical fitness. | causal | EA | Sampling and subgroup outcome studies | Broader sampling fails to improve affected outcomes | EP-0005 | | C-005 | A high held-out score alone can indicate overfitting. | methodological | IA | Reproducible validation and leakage audit | External/temporal performance remains strong | EP-0006 | | C-006 | Continuous monitoring is both reliability and ethics work. | evaluative | EA | Detection, impact, and remediation records | Monitoring never detects or reduces consequential failures | EP-0007 | | C-007 | DL opacity raises product risk when decisions need explanation or contest. | causal / conditional | EA | Task-specific explanation and user studies | Explanations add no value in affected high-stakes decisions | EP-0011 | | C-008 | AI-products need controlled deployment and rollback patterns. | prescriptive | EA | Incident comparisons by launch pattern | Immediate full launch performs equally with no added risk | EP-0004 | | C-009 | Funding, growth, and user counts indicate AI-product success. | evaluative | IA | Retention, outcomes, unit economics, harm | High-growth cases show weak value or serious harm | EP-0012 | | C-010 | AI PMs need technical fluency but relational and strategic skills matter more than technical depth. | evaluative | EA | Role-performance and team studies | Technical depth predicts success after controlling for context | EP-0027 | | C-011 | AI-native product development follows ideation, data management, R&D, and deployment. | methodological | EA | Cross-company process comparison | Common successful paths require materially different stages | EP-0014 | | C-012 | PMs should help set model-performance thresholds. | prescriptive | EA | Governance roles and decision records | Threshold ownership elsewhere yields better accountability | EP-0014 | | C-013 | Productization makes ML delivery more consistent across customers. | causal | EA | Delivery variance and unit economics | Bespoke delivery matches consistency and economics | EP-0018 | | C-014 | AI products add uncertainty, data sensitivity, scaling cost, and retraining needs beyond ordinary software. | descriptive | EA | Matched project and operating data | Comparable non-AI systems show equal burdens | EP-0019 | | C-015 | AIOps and MLOps can be treated interchangeably. | definitional | EA | Vendor-neutral standards | Standards distinguish their objects and tasks | EP-0019 | | C-016 | Domain and market fluency are required to adapt AI products responsibly. | prescriptive | EA | Cross-domain failures and outcomes | Generic design performs equally across regulated contexts | EP-0020 | | C-017 | Precision, specificity, recall, and F1 are forms of classification accuracy. | definitional | EA | Accepted metric definitions | Primary technical references define distinct measures | EP-0021 | | C-018 | AI failure prevention requires ethics, performance, and safety testing. | prescriptive | EA | Postmortems and test effectiveness | Failures persist despite appropriate testing and response | EP-0023 | | C-019 | AI-product design requires user research, validation, iteration, documentation, and trust work. | methodological | EA | Product outcomes and research records | Model-centered design outperforms without these practices | EP-0024 | | C-020 | More user data usually improves personalization and product stickiness. | causal | IA | Controlled privacy-minimal comparison | Data-minimized or on-device systems perform as well | EP-0024 | | C-021 | A north-star metric can align teams around product value. | causal / conditional | EA | Decision and outcome histories | Metric optimization causes harmful divergence | EP-0025 | | C-022 | Price rarely blocks a purchase when a product performs best. | predictive | EA | Price elasticity and win/loss data | Qualified buyers reject chiefly on total cost | EP-0026 | | C-023 | Psychological safety improves feedback and risk discovery. | causal | RI from EA | Climate, reporting, and incident data | Safety measures don't change detection or response | EP-0027 | | C-024 | Akeira's metrics and adoption demonstrate the framework's success. | evaluative | IC | Real product records and independent outcomes | Case is fictional or numbers are invented | EP-0017, EP-0028 | | C-025 | The available source is sufficient to represent the finished second edition. | methodological | RI | Chapter/page comparison | Publisher record shows substantial missing content | EP-0001, EP-0028 | ### Dependency graph ```mermaid flowchart TD C2["C-002: Start with a real problem"] --> C11["C-011: Choose lifecycle"] C1["C-001: Data + operations matter"] --> C3["C-003: CI/CD/CT/CM"] C4["C-004: Fit-for-purpose data"] --> C7["C-007: Explain and contest"] C11 --> C12["C-012: Set thresholds"] C3 --> C8["C-008: Controlled deployment"] C3 --> C6["C-006: Monitor and remediate"] C13["C-013: Productize delivery"] --> C16["C-016: Adapt by domain"] C16 --> C19["C-019: User-centered design"] C12 --> C21["C-021: Value metric + guardrails"] C6 --> C21 C10["C-010: Relational + strategic PM skill"] --> C23["C-023: Psychological safety"] C23 --> C6 X1["If C-002 fails"]:::risk --> X2["Solutionism, cost, weak value"]:::risk X3["If C-003/C-006 fail"]:::risk --> X4["Unsafe or stale production behavior"]:::risk classDef risk fill:#fee,stroke:#a33,color:#600; ``` Foundational claims: C-001, C-002, C-004. Bridge claims: C-003, C-011, C-013, C-016. Practical conclusions: C-008, C-012, C-019, C-021, C-023. Independent weak claims: C-009, C-015, C-017, C-022, C-024. Failure of those weak claims does not overturn the operating spine. ### Claims-to-questions map | Claims | Open questions | |---|---| | C-001–C-004 | Q1–Q5, Q11, Q14 | | C-005–C-008 | Q7, Q12–Q14 | | C-009–C-010 | Q6, Q8, Q19 | | C-011–C-016 | Q2, Q4, Q11, Q17–Q20 | | C-017–C-021 | Q5, Q12, Q15, Q21–Q25 | | C-022–C-025 | Q6, Q10, Q26–Q28 | ## Prompt 5 — Quantitative rigor setup ### Bayesian update policy 1. Start from the explicit priors in Prompt 1. 2. Treat a company blog, vendor case, press story, and repeated retelling of one event as one dependent evidence family. 3. Increase confidence most for reproducible results, independent replications, primary legal or standards text, production postmortems, and matched comparisons. 4. Decrease confidence for missing denominators, obsolete tools, unconfirmed numbers, fictional examples, unclear populations, leakage risk, and causal language supported only by association. 5. Use ranges to show judgment. Do not turn subjective updates into measured probabilities. | Label | Range | |---|---:| | Very low | 0–20% | | Low | 21–40% | | Uncertain | 41–60% | | Moderate | 61–75% | | High | 76–90% | | Very high | 91–100% | ### Priors and posteriors | Claim | Prior | Evidence considered | Dependence concern | Posterior | Update | |---|---:|---|---|---:|---| | C-001 | 80% | Repeated operating examples and current NIST lifecycle framing | Mostly conceptual | 87% | Moderate increase | | C-002 | 85% | “Why AI?” gate plus cost/risk logic | No portfolio study | 88% | Small increase | | C-003 | 80% | Detailed loop and later MLOps discussion | Same author framework | 85% | Small increase | | C-004 | 80% | Ethics and fit arguments | No subgroup results | 82% | Small increase | | C-005 | 55% | High score shown, but no reproducible split; image uses test data in cross-validation | Evidence quality weak | 35% | Moderate decrease | | C-006 | 90% | Drift and downstream harm logic; current risk frameworks | Frameworks not outcome trials | 93% | Small increase | | C-009 | 45% | Funding and adoption examples | Promotional and selected | 20% | Strong decrease | | C-010 | 65% | Detailed role account | Practitioner opinion | 70% | Small increase | | C-013 | 70% | Clear repeatability loop | No unit-economics comparison | 74% | Small increase | | C-017 | 50% | Book wording | Current technical definitions contradict grouping | 10% | Strong decrease | | C-021 | 70% | Coherent metric practice | Goodhart risk and no causal study | 68% | No meaningful update | | C-023 | 70% | Concrete reporting practices | Outcomes not measured | 74% | Small increase | | C-024 | 40% | Case presents success | EPUB labels case as sample; no real evidence | 5% | Strong decrease | | C-025 | 50% | 12 chapters are substantial | Official final edition contains missing sections | 5% | Strong decrease | ### Calibration warnings - The Ether example compares scores without the dataset, code, time split, feature set, leakage audit, or external validation. - Figure 2.2 runs cross-validation on `X_test, Y_test`, and the displayed list used for a manual mean does not match the displayed cross-validation output. - Figure 2.3 duplicates an import and gives no preprocessing or temporal-validation details. - A single R² does not establish usefulness, stability, or absence of leakage. - Company, market, salary, and conversion statistics often lack denominators or independent sources. - Akeira numbers are illustrative, not observations. - Precision, recall, specificity, accuracy, F1, RMSE, and MAE are not interchangeable. - Watch publication, survivor, selection, authority, and vendor bias. ### Non-quantitative fallback Use: **strongly decreases confidence**, **moderately decreases confidence**, **no meaningful update**, **moderately increases confidence**, or **strongly increases confidence**. Always state why. ## Prompt 6 — Analytical reading scaffold ### Chapter analysis slots | Ch. | Purpose and thesis | Claims / terms | Evidence and examples | Counterexample, assumption, or question | Role / reader response | |---:|---|---|---|---|---| | 1 | Define the terrain; data operations determine performance | ANI, ML/DL, pipeline, CI/CD/CT/CM | Infrastructure descriptions; deployment patterns | PCA loses information; not every model has tuned hyperparameters | Strong operating foundation; verify definitions | | 2 | Join NPD to model readiness and ethics | loss, overfit, drift, threshold | Ether model; EU ethics; monitoring | High score isn't itself overfit; test data appear reused | Central but technically unreliable example | | 3 | Give PMs model vocabulary and tradeoffs | CNN, RNN, GAN, VAE, diffusion, transformer | Primary-paper links plus secondary summaries | Stable Diffusion isn't simply a transformer; GPT-4 size unconfirmed | Useful map, not a technical authority | | 4 | Frame commercialization strategy | B2B/B2C, blue/red ocean, four strategies | Named companies and Beekin experience | Funding/adoption don't prove efficacy; labor/copyright gaps | Inspiration outweighs evidence | | 5 | Survey sector effects and responsibility | MVP, reverse thinking, AI for Good | Government, health, finance, disability examples | Sweeping forecasts; promotional cases; privacy caveats weak | Broadens moral and market scope | | 6 | Define native-product lifecycle, team, stack, and sales | ideation, feature engineering, productization | Role breakdown and Akeira sample | “Six problem types” incomplete; raw-data-to-LLM advice unsafe | Core product operating model | | 7 | Make ML service delivery repeatable | productization, MLOps, feedback | Scope-package-deliver-feedback loop | AIOps ≠ MLOps; B2B/B2C binary is too tidy | Most portable practical model | | 8 | Specialize by domain and measure value | vertical, peer group, KPI, “Why AI?” | Sector use cases and metric lists | Metric conflation; profiling and consent gaps | Good decision gate, weak measurement detail | | 9 | Connect AI categories to product features and failures | macro/micro AI, expert system, fuzzy match | Tay, Pixellot, Watson, Google DR | Taxonomy oversimplified; disability access absent | Failure cases improve balance | | 10 | Center users, iteration, documentation, and trust | validation, user obsession, clarity | Design sequence; Duolingo; Akeira sample | More data not always better; no concrete access testing | Strong user focus, incomplete safeguards | | 11 | Align metrics, growth signals, and cost | NSM, KPI, OKR, PLG, CDP | Metric questions and tool stack | Google Optimize already closed; “cost doesn't matter” unsupported | Actionable but high drift | | 12 | Manage alignment, people, data, and testing | vision, safety, empowerment, feedback | Operational questions and team practices | Opinion-based outcomes; unfinished fictional case | Humane management close without source closure | ### Argument spine 1. AI is a broad product capability, while ML is data-trained behavior. 2. AI product outcomes depend on data flow, model behavior, infrastructure, and continuous operations. 3. PMs need enough technical fluency to frame problems, question tradeoffs, and coordinate specialists. 4. Product work moves from ideation through data management, research/development, deployment, and ongoing management. 5. Market and domain context determine useful customization, risk, and value measures. 6. Design, documentation, feedback, metrics, and team alignment turn model output into a sustainable product. 7. Ethics must live across the lifecycle because users and non-users can be harmed. 8. **Gap:** The source does not complete the promised integration and career arc, and several measurement claims undermine its technical authority. ### Evidence ledger | ID | Claims | Type | Source | Independence | Quality | Limitation | |---|---|---|---|---|---|---| | E-001 | C-001, C-003 | Practitioner synthesis | Ch. 1 | Low | B | No comparative outcome data | | E-002 | C-008 | Deployment-pattern explanation | Ch. 1 | Medium | B | No incident comparison | | E-003 | C-004 | Normative + technical argument | Ch. 2 | Low | B | “Representative” undefined | | E-004 | C-005 | Personal model screenshots | Ch. 2 | Low | D | Not reproducible; test reuse concern | | E-005 | C-006 | Drift discussion + cited survey | Ch. 2 | Mixed | C | Survey comes through a podcast | | E-006 | C-007 | Conceptual explanation | Ch. 3 | Low | C | Opacity overstated | | E-007 | C-009 | Company examples | Ch. 4 | Low | D | Selection and promotional bias | | E-008 | C-011, C-012 | Lifecycle framework | Ch. 6 | Low | B | Roles vary by organization | | E-009 | C-013 | Productization loop | Ch. 7 | Low | B | No unit-economics test | | E-010 | C-014 | Comparative reasoning | Ch. 7 | Low | B | Differences not measured | | E-011 | C-016 | Domain cases | Ch. 8 | Mixed | C | Mostly broad examples | | E-012 | C-017 | Metric list | Ch. 8 | Low | F | Contradicted by primary definitions | | E-013 | C-018 | Failure cases | Ch. 9 | Medium | B | Some retellings may distort originals | | E-014 | C-019, C-020 | Design advice | Ch. 10 | Low | B/C | No controlled outcomes; data minimization absent | | E-015 | C-021, C-022 | Metrics and pricing advice | Ch. 11 | Low | C/D | Time-sensitive and weakly supported | | E-016 | C-023 | Management practices | Ch. 12 | Low | B | Outcomes asserted, not measured | | E-017 | C-024 | Fictional case | Chs. 6–12 | None | F for empirical use | Useful only as a scenario | | E-018 | C-025 | EPUB vs publisher record | Local source + Packt | High | A | Finished content not analyzed | ### Reading plan and completion criteria - Deep read: Chapters 1–2 and 6–12. - Selective read: Chapters 3–5, focusing on definitions, evidence, and counterexamples. - External checks: publisher completeness, law and standards, metrics, tools, model claims, and high-stakes cases. - Attention level: high for author/source distinctions; high for any number used in a decision; medium for broad examples. - Stop and synthesize after lifecycle, productization, measurement, and management sections. Prompt 7 may begin because every available chapter has a slot, claims have stable IDs, the argument spine exists, major evidence is graded, and missing-source boundaries are explicit. ## Prompt 7 — X-ray analysis and fair criticism ### Strongest reasonable reconstruction **Author says:** AI product management is not model selection with a roadmap attached. AI products join uncertain model behavior, changing data, infrastructure, user needs, market positioning, operating cost, and social effects. The PM's job is to maintain alignment across these moving parts. That requires technical vocabulary, disciplined discovery, representative data, controlled deployment, continuous observation, user feedback, meaningful metrics, clear communication, and shared ethical responsibility. **We infer:** The strongest version of the argument is conditional: when AI is the right tool, a PM should turn uncertain model capability into a bounded service with explicit value, risk, ownership, measures, fallback, and learning loops. Technical specialists own much of model implementation; the PM owns the coherence of the product decision and helps ensure no discipline's local optimization becomes the user's systemic harm. **Still uncertain:** The book does not show that this division of labor or its four-stage lifecycle outperforms alternatives. The source is a practitioner handbook, not a controlled study. ### Major-claim evaluation | Claim | Support | Grade | Strongest objection | Current conclusion | |---|---|---:|---|---| | C-001 | Repeated lifecycle and operating discussion | B | Conceptual, not comparative | Accept as strong operating principle | | C-002 | “Why AI?” gate and cost/risk logic | B | No portfolio evidence | Adopt as decision rule | | C-003 | Detailed CI/CD/CT/CM sequence | B | Terms and implementations vary | Adopt with organization-specific design | | C-004 | Representative-data argument | B- | Population, consent, and subgroup tests undefined | Repair to “fit-for-purpose, lawful, tested data” | | C-005 | High model score plus warning | D | High held-out performance is not proof of overfit; method may leak | Reject as stated; require honest validation | | C-006 | Drift and harm analysis | B | Monitoring without action is theater | Accept only with thresholds, owners, and response | | C-007 | Opacity and contestability discussion | C+ | Not all DL is equally opaque; explanation quality is contextual | Narrow by decision and audience | | C-008 | Shadow/A-B/canary descriptions | B | Some high-stakes tests cannot expose users to unknown harm | Adopt with risk-tiered approval | | C-009 | Named-company popularity and growth | D | Capital and adoption don't prove efficacy or justice | Reject as a success measure | | C-010 | Role and management discussion | C+ | Role needs differ by company and product | Keep as a hiring hypothesis, not universal law | | C-011 | Four-stage lifecycle | B- | Stages hide governance, procurement, retirement, and incident response | Expand the lifecycle | | C-012 | PM threshold involvement | B- | Final accountability may sit with clinical, risk, legal, or model owners | Keep PM participation; define RACI | | C-013 | Productization loop | B | Standardization may not fit bespoke work | Adopt when repeated demand and economics support it | | C-014 | Data, cost, uncertainty, retraining differences | B | Some non-AI software shares these traits; not every AI system retrains | Treat as risk questions, not defining laws | | C-015 | AIOps and MLOps interchangeability | F | They manage different objects and outcomes | Reject | | C-016 | Domain fluency | B | External experts and participatory research may matter more than PM fluency | Expand to affected-community expertise | | C-017 | Metric grouping | F | Primary references define different formulas and error tradeoffs | Reject and replace with task-specific metrics | | C-018 | Ethics/performance/safety tests | B | Tests can miss deployment shifts and social harms | Adopt with monitoring and redress | | C-019 | User-centered design sequence | B | “User obsession” may exclude non-users who bear harm | Expand stakeholder scope | | C-020 | Data volume → personalization/stickiness | D | Minimization, on-device methods, and contextual design can match value | Reject the volume assumption | | C-021 | North-star alignment | B- | One measure invites proxy gaming | Use one value signal plus guardrails | | C-022 | Price rarely blocks the best product | F | Budgets, switching costs, and procurement routinely constrain choice | Reject | | C-023 | Psychological safety → risk discovery | C+ | Needs behavioral and incident evidence | Adopt as a testable management hypothesis | | C-024 | Akeira proves success | F | The case is fictional | Use only as a teaching scenario | | C-025 | Snapshot represents finished edition | F | Eight planned chapters are absent | Reject | Grades: **A** direct and independently supported; **B** substantial but limited; **C** suggestive or incomplete; **D** weak or poorly matched; **F** unsupported or contradicted; **N/A** non-empirical. ### Prior updates The main updates appear in Prompt 5. The largest decreases apply to C-005, C-009, C-017, C-024, and C-025. The strongest retained conclusions are C-001, C-002, C-003, C-006, C-008, C-013, C-018, and the repaired C-019. Evidence remains dependent because most comes from one handbook and its selected examples. ### Bias, rhetoric, and counter-observations | Issue | Evidence | Effect | Counter-observation | Repair | |---|---|---|---|---| | Selection and survivor bias | Successful named products dominate Chs. 4–5 | High for commercialization claims | Ch. 9 includes failures | Add matched failed, ordinary, and non-AI alternatives | | Authority bias | Large firms, funding, and media coverage imply quality | High | Scale can reveal some operating demand | Measure user outcomes, retention, harms, and economics | | Correlation-to-causation | AI use is credited for growth or conversion | High | Some cases may have credible internal attribution | State association unless a counterfactual exists | | Equivocation | Accuracy, precision, recall, and specificity are blurred | High for model decisions | The book correctly names several metrics | Define each formula and error cost separately | | Hasty generalization | Personal model and practitioner cases support broad advice | Moderate | Practitioner examples can generate hypotheses | Label experience evidence and require validation | | Appeal to novelty | AI adoption often appears necessary for competitiveness | Moderate | The “Why AI?” question pushes back | Make non-AI baseline mandatory | | Anthropomorphism | Models “learn,” “understand,” and “decide” | Moderate | Plain language aids new readers | Pair shorthand with operational definitions | | Solutionism | Social, health, and public problems become AI opportunities | High in high-stakes contexts | The text repeatedly raises ethics | Require system, policy, labor, and nontechnical options | | Fiction-evidence blur | Akeira gains adoption and doubled metrics | High | It is first introduced as a sample | Repeat “fictional scenario” at every use | | False binary | AI-native versus traditional; B2B versus B2C | Moderate | Categories can orient beginners | Use continua and hybrid cases | These are not all formal fallacies. Several are evidence or rhetoric weaknesses. The book's strongest repair is already inside it: ask “Why AI?”, treat monitoring as ongoing, and share ethical responsibility. The analysis extends that repair through non-AI baselines, risk tiers, affected-community participation, and redress. ### Attribution audit - **Author says:** the lifecycle, productization loop, sector examples, metric advice, and management practices. - **Author implies:** that growth and visibility often demonstrate success; that more data often improves personalization. - **Third parties say:** many market statistics, company outcomes, research findings, laws, and ethical principles. - **Illustrative case:** every Waterbear/Akeira claim and number. - **Evidence shows:** the local snapshot is incomplete; its metric taxonomy contains a material error; at least one named tool was already closed; its executive-order discussion aged as predicted. - **We conclude:** retain the operating questions, reject the proof-by-popularity pattern, and add governance, access, security, retirement, and redress gates. ## Prompt 8 — Systems, context, and intellectual lineage ### Context map | Context | Available to the author / in source | Later or current context | Effect on reading | |---|---|---|---| | Technical | Classical ML, early generative-AI boom, ChatGPT-era LLM framing | Model families, tools, system cards, evaluation practice, and product capabilities changed quickly | Treat architecture and product claims as dated examples | | Regulatory | 2023 US EO 14110; EU AI Act approval process | EO 14110 was revoked in 2025; the EU Act's prohibited-practice and AI-literacy rules began in 2025, GPAI governance began in 2025, and broader application began in 2026 with later dates for some high-risk systems | Replace static legal summaries with jurisdiction-specific review | | Standards | Ethics principles and scattered references | [NIST AI RMF](https://www.nist.gov/itl/ai-risk-management-framework), its [Generative AI Profile](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf), and [ISO/IEC 42001](https://www.iso.org/standard/42001) give more explicit governance structures | Add Govern/Map/Measure/Manage and management-system ownership | | Market | AI excitement, funding, platform examples, high growth expectations | Vendor closures, acquisitions, pricing shifts, and API dependence expose platform risk | Add exit plans and total-cost review | | Labor | Automation framed partly as relief from repetitive or traumatic work | Hidden review, labeling, moderation, creative, and support labor remain material | Track job quality and human escalation, not only headcount savings | | Culture | US-heavy product and startup frame; broad claims about consumers | Global laws, languages, connectivity, cultural norms, and collective harms differ | Limit transfer without local research | | Publishing | Early Access draft | Finished second edition published November 2024 | Obtain finished source before teaching missing sections | The current EU timeline is summarized on the [European Commission's AI Act page](https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai). The US policy change is recorded in [Executive Order 14179](https://www.federalregister.gov/documents/full_text/html/2025/01/31/2025-02172.html). These later facts are current-context additions, not Bratsis claims. ### Institutional and economic incentives - Packt benefits from an accessible, broad, tool-rich handbook and from releasing chapters early. - Vendors and startups benefit when adoption, funding, and product claims read as proof of value. - PMs benefit from a role definition that makes them the bridge among leadership, market, data, engineering, design, and ethics. - Organizations may reward launch speed, growth, and cost reduction more visibly than avoided harm, accessibility, or long-term maintenance. - Regulators, affected communities, workers, customers, and non-users may assign different costs to the same AI feature. ### Intellectual lineage ```mermaid flowchart LR A["Classical product management"] --> P["AI product management"] B["Data science + statistical learning"] --> P C["DevOps / continuous delivery"] --> M["MLOps"] --> P D["Service productization"] --> P E["Lean / agile discovery"] --> P F["Human-centered design"] --> P G["Responsible AI + tech ethics"] --> P H["Growth analytics / PLG"] --> P P --> I["Problem-value fit"] P --> J["Model-data operations"] P --> K["Governance + redress"] P --> L["Market + organizational alignment"] ``` ### Missing perspectives The source needs more from disabled users and accessibility practitioners; affected non-users; labor and creative workers; Indigenous data-governance traditions; Global South product teams; public-interest technologists; security and privacy engineers; procurement and legal teams; clinicians and regulators for high-stakes examples; small organizations without mature data infrastructure; and people who reject an AI system or cannot access it. ## Prompt 9 — DEIB, multicultural, accessibility, and reading-level audit ### Inclusion strengths - The book repeatedly treats bias, privacy, fairness, downstream non-user harm, and human autonomy as product concerns. - It mentions racial and gender exclusion in AI history and calls for varied data and stakeholder voices. - Sector examples include assistive technology and public-interest projects. - Chapter 12 values psychological safety, work boundaries, open feedback, and shared ethics responsibility. - The prose introduces technical ideas without assuming an engineering degree. ### Exclusion and omission risks - Disability access rarely becomes a product requirement, test plan, procurement rule, or KPI. - “Accessible” often means understandable or searchable, not perceivable, operable, compatible with assistive technology, or co-designed with disabled people. - Demographic personalization can slide into profiling and essentialist product packaging. - The fictional mental-health assistant lacks clinical validation, crisis handling, informed consent, data-retention limits, age safeguards, disability access, and clear human escalation. - US startup and enterprise assumptions dominate. Legal, linguistic, cultural, infrastructure, and labor differences receive little method. - Replacing content moderators with algorithms hides the need for human review, labor protection, and contextual judgment. - High-stakes examples often understate appeals, accountability, and the people harmed by false positives or false negatives. - A screenshot of a biased ChatGPT output shows a social-media post rather than a reproducible test. ### Multicultural transferability | Element | Transferability | Main risk | Adaptation | |---|---|---|---| | “Why AI?” decision gate | High | Local alternatives ignored | Include local process and nontechnical baseline | | Controlled deployment | High | Experiment may be unethical in high-stakes use | Risk-tier launch approval and simulation first | | Productization loop | High | Contract, service, and trust expectations differ | Localize delivery, support, consent, and remedy | | Verticalization | Moderate | Industry labels hide regional practice | Research local workflow, law, language, power | | User obsession | Moderate | Paying user may not be harmed stakeholder | Map users, subjects, bystanders, workers, communities | | Growth metrics | Low to moderate | Connectivity, device, income, and norms distort measures | Segment access and avoid punitive engagement goals | | AI-PM team model | Moderate | Assumes well-funded specialist roles | Combine responsibilities with explicit accountability | | Ethical principles | High as a start | Universal language can hide local priorities | Participatory review and jurisdiction-specific rules | ### Accessibility audit | Layer | Finding | Risk | Required repair | |---|---|---|---| | EPUB structure | Headings and chapter order extract cleanly | Early Access omissions break expected completeness | Publish clear completeness metadata | | Images | Most technical figures have short alt text; repeated Discord QR image and handwritten productization loop do not | Screen-reader users lose content or purpose | Descriptive alt text and equivalent text | | Figure captions | Several captions are marked `aria-hidden` in extracted HTML | Caption may not reach assistive tech | Keep captions in accessibility tree | | Code/results | Screenshots carry results but no full code, dataset, or text equivalent | Low vision and screen-reader barriers; no reproduction | Provide code blocks, tables, and downloadable notebook | | Language | Conversational and approachable | Jargon density grows; some ableist wording (“dumb system,” “lazy”) | Glossary, plain-language summaries, respectful terms | | Product advice | Mentions accessible design but not a test method | Teams may treat access as aspiration | Include WCAG, assistive-tech, keyboard, zoom, captions, cognitive load, and disabled-user testing | | Mental-health case | Conversational agent framed as supportive | Crisis, deception, privacy, and dependency risks | Clinical governance, clear nonhuman identity, crisis routing, consent, and independent evaluation | **Reading-level estimate:** Main prose is around grades 10–12 in concept load, with college-level sections on model metrics, architecture, regulation, and strategy. Tables, acronyms, long paragraphs, and unexplained business language raise the practical level. A beginner can follow the narrative but should use a metric glossary and current technical references. ### Supplementary perspectives - NIST's AI RMF and Generative AI Profile for lifecycle risk and documented actions. - W3C accessibility standards and disabled-user research for product requirements. - Indigenous data sovereignty and community-governance work for collective rights. - Labor research on data labeling, moderation, creative work, and automation. - Primary model cards, dataset documentation, incident databases, and regulatory guidance. - Domain-specific standards for health, employment, education, finance, and public services. ## Prompt 10 — Notes pipeline All note candidates remain in this single file. Suggested filenames and links are proposals only; no adjacent notes were created. ### Fleeting notes - The book's best idea may be that model behavior is only one dependency in a product system. - “Why AI?” should be a signed gate, not a workshop question that disappears. - Every metric needs a harmed-by-error row. - A fictional case can teach sequence, but invented success metrics corrupt evidence habits. - The unfinished Chapter 12 accidentally demonstrates why production state and source version matter. - Productization without exit planning can turn repeatability into vendor lock-in. - Psychological safety needs evidence in dissent, incident reporting, and response—not slogans. - Accessibility is missing precisely where the book says AI products are special. ### Literature notes | Proposed filename | Note | Attribution | |---|---|---| | `AI products are model-data-service systems.md` | AI-product quality depends on the interaction of model, data, interface, infrastructure, people, policy, and feedback. | Bratsis operating spine; EP-0003, EP-0019 | | `Why AI is a product gate.md` | Require a non-AI baseline and a measurable advantage before approving AI complexity. | Bratsis plus analysis; EP-0020 | | `A high score is not a readiness decision.md` | One score cannot reveal leakage, subgroup harm, stability, user value, or operational cost. | Analysis of EP-0006, EP-0021 | | `Productization bounds an uncertain service.md` | Scope, package, delivery, quality, support, and feedback turn repeated service work into a product candidate. | Bratsis; EP-0018 | | `North-star metrics need harm guardrails.md` | A single value measure should sit beside access, fairness, safety, privacy, and cost limits. | Bratsis plus analysis; EP-0025 | | `Monitoring is an accountability loop.md` | Monitoring matters only when a threshold triggers an owner, action, notice, fallback, and review. | Analysis of EP-0007, EP-0027 | | `Affected people exceed active users.md` | AI discovery must include subjects, bystanders, workers, communities, and people denied service. | Analysis of EP-0008, EP-0023 | | `Case-study provenance changes what it can prove.md` | A fictional scenario can test reasoning but cannot support efficacy, adoption, or causal claims. | Analysis of EP-0017, EP-0028 | ### Permanent notes 1. **[[AI product readiness is a system claim]]** — A model is not ready because one metric is high. Readiness is a bounded claim about data, model, interface, operations, users, harms, cost, rollback, and accountable ownership. Confidence: high. 2. **[[Use a non-AI baseline before funding an AI feature]]** — The right comparison is not “AI or nothing,” but AI against rules, search, workflow redesign, ordinary software, and human service. Confidence: high. 3. **[[Metrics encode the cost of being wrong]]** — Precision, recall, specificity, latency, and cost matter only in relation to who bears false positives, false negatives, delay, exclusion, and review burden. Confidence: high. 4. **[[Continuous monitoring without response is observability theater]]** — A dashboard has no governance value unless thresholds connect to authority and action. Confidence: high. 5. **[[Productization is a promise of repeatability]]** — A packaged AI service promises bounded inputs, outputs, quality, delivery, support, and change management; the promise must include uncertainty and failure. Confidence: moderate-high. 6. **[[A fictional case cannot validate a framework]]** — Use fictional cases to rehearse decisions, never to claim outcomes. Confidence: very high. 7. **[[Accessibility belongs in the AI lifecycle]]** — Data collection, evaluation, interface design, explanation, fallback, and appeal all create disability-access requirements. Confidence: high. 8. **[[Source version is part of evidence quality]]** — Early Access state, missing chapters, stale tools, and editorial remnants directly limit what a technical book can establish. Confidence: very high. ### Structure notes - **Book MOC:** [[AI Product Manager's Handbook — Work the Book]] → foundations; native-product lifecycle; productization; verticals; design; metrics; management; missing integration/career arc. - **Theme MOC:** [[Responsible AI product management]] → problem gate; data; evaluation; accessibility; deployment; monitoring; incident response; redress; retirement. - **Project MOC:** [[AI feature decision template]] → non-AI baseline; risk tier; RACI; measures; launch gate; rollback; review date. - **Unresolved MOC:** [[AI PM evidence questions]] → reproducibility; case provenance; current law; vendor drift; subgroup outcomes; total cost. ### Attribution audit result No system conclusion is presented as Bratsis's statement. EP identifiers are system-selected source anchors, not reader highlights. Akeira/Waterbear stays labeled illustrative. Later policy, standards, vendor, and publisher facts are linked to current primary or publisher sources. ## Prompt 11 — Teaching pack ### Audience and objectives **Audience:** Product managers, technical writers, designers, engineers, analysts, and risk/accessibility partners who have basic software-product knowledge. By the end, learners can: 1. Explain why an AI product is a model-data-service system. 2. Apply a “Why AI?” gate and compare non-AI alternatives. 3. Distinguish accuracy, precision, recall, specificity, and F1. 4. Design a lifecycle from discovery through retirement and incident response. 5. Separate evidence, inference, promotional cases, and fictional examples. 6. Add accessibility, fairness, privacy, security, and redress measures to a product plan. 7. Build a value metric with harm guardrails. ### Prerequisites and vocabulary Prerequisites: product discovery, software release basics, elementary probability, and the idea of train/validation/test data. Key terms: AI, ML, model, feature, threshold, overfitting, drift, MLOps, productization, precision, recall, north-star metric, guardrail, rollback, redress. ### Lesson sequence | Lesson | Focus | Activity | Critical distinction | |---:|---|---|---| | 1 | What makes an AI product | Label a sample architecture | Model vs product system | | 2 | Why AI? | Compare AI, rules, search, workflow, human service | Capability vs value | | 3 | Data and evaluation | Compute confusion-matrix metrics | Accuracy vs cost of error | | 4 | Deployment and monitoring | Draft shadow/canary/rollback plan | Observation vs response | | 5 | Productization and economics | Bound scope and SLA | Repeatability vs bespoke service | | 6 | Design and access | Stakeholder and barrier map | Active user vs affected person | | 7 | Governance | Apply NIST Govern/Map/Measure/Manage | Principle vs owned control | | 8 | Evidence clinic | Grade book examples and Akeira | Illustration vs proof | ### Discussion and misconception checks - If a model scores 0.99, what else must you know before launch? - When can a canary test itself be unethical? - Who is harmed by a false positive, false negative, or unavailable fallback? - What evidence would show that personalization needs more data? - Can an AI product be “native” when all model capability comes from an API? - Which parts of a fictional case remain useful? Misconceptions to test: - “AI” means ML. - More data is always better. - A high test score proves generalization. - Explainability is either fully possible or impossible. - MLOps and AIOps are the same. - User count proves product quality. - Accessibility means clear writing only. - Ethics belongs to one specialist. ### Case activities and assessment 1. **Ether model audit:** Identify missing data, time-split, preprocessing, baseline, leakage, subgroup, and external-validation evidence. 2. **Akeira launch review:** Treat it as a fictional mental-health product. Define clinical, privacy, crisis, disability, consent, and anthropomorphism controls. 3. **Metric correction:** Rewrite the book's “classification accuracy” list with distinct formulas and error costs. 4. **Tool drift check:** Replace Google Optimize, which [closed September 30, 2023](https://support.google.com/analytics/answer/12979939?hl=en), with capability requirements rather than a brand list. Assessment options: decision memo, risk register, metric rationale, launch checklist, evidence-grade defense, or 10-minute teach-back. Grade source separation, decision logic, access, and response design—not enthusiasm for AI. ### Facilitation cautions and adaptations - Don't teach the book's disputed metric definitions as settled. - Don't use fictional adoption numbers as outcomes. - Avoid asking learners to disclose personal mental-health, discrimination, or employment experiences. - Provide text equivalents for diagrams, captioned media, keyboard-accessible materials, plain-language summaries, advance vocabulary, flexible participation, and breaks. - Invite region-specific law and cultural examples; don't treat US startup practice as universal. ## Prompt 12 — Action, PARA, experiments, and drift monitoring ### Action inventory **Take** - Add a signed “Why AI?” section and non-AI baseline to every AI feature brief. - Require data, model, interface, access, security, policy, cost, fallback, and retirement owners. - Tie every monitored signal to a threshold, response, notice, and review. - Use separate user-value and harm/access guardrail metrics. - Label real, composite, fictional, vendor, and promotional cases at first use and every outcome claim. **Consider** - A reusable AI product decision template based on NIST Govern/Map/Measure/Manage. - Model/data cards, incident drills, affected-community review, and red-team exercises proportional to risk. - Productizing repeated internal AI work only after demand and delivery variance are measured. **Avoid** - Choosing AI because competitors use it. - Calling funding, press, or downloads proof of efficacy. - Launching high-stakes features on aggregate accuracy alone. - Collecting data without a necessity, consent, retention, and deletion case. - Treating accessibility, ethics, privacy, or security as a final checklist. - Reusing named vendor/tool advice without a current check. **Need more evidence** - Whether the finished edition corrects technical issues. - The reproducibility and validity of the Ether-price comparison. - Independent outcomes for named commercial and public-interest products. - Role-performance evidence for the proposed AI-PM skill hierarchy. ### PARA routing | Bucket | Material | Why | |---|---|---| | Projects | AI feature decision template; one pilot audit | Has a defined output and review date | | Areas | Product management; accessibility; responsible AI; technical communication | Continuing responsibility | | Resources | This consolidated note; metric glossary; NIST/ISO/EU links | Reference material | | Archives | Early Access source provenance and checksum | Preserve evidence state without treating it as current | ### Small reversible experiments | Experiment | Claims | Hypothesis | Action | Measure | Duration | Stop condition | Review | |---|---|---|---|---|---|---|---| | Why-AI gate | C-002 | A non-AI baseline improves decision clarity | Apply to one proposed feature | Options considered; decision confidence; risk/cost delta | One decision cycle | Team can't define comparable outcome | 2026-09-30 | | Metric harm map | C-017, C-021 | Mapping error costs changes metric choice | Add harmed-by-FP/FN rows to one metric brief | Changed threshold/metric; uncovered stakeholder | Two weeks | No accountable owner for harms | 2026-09-15 | | Monitoring-action link | C-006 | Alerts with owners reduce unresolved risk | Map five signals to thresholds and runbook | Ownership gaps; drill response; time to decision | Four weeks | Drill exposes unsafe production dependency | 2026-10-01 | | Accessibility gate | C-019 | Early disabled-user review finds issues before build | Test concept/prototype with access criteria | Barriers found and resolved; test coverage | One prototype cycle | No consented, appropriate testing path | 2026-10-15 | | Evidence-lane audit | C-009, C-024 | Provenance labels reduce overclaiming | Label cases in one presentation | Unsupported claims corrected | One document | Legal/privacy concern in source material | 2026-09-15 | | Productization test | C-013 | Repeated work has enough common scope to package | Compare three deliveries | Shared steps, variance, margin, support load | Six weeks | Bespoke needs dominate or quality drops | 2026-11-15 | ### Decision rules - **Adopt:** The experiment improves a defined decision or outcome without breaching access, safety, privacy, or cost guardrails. - **Adapt:** Value appears, but one population, workflow, metric, or owner needs a narrower design. - **Test again:** Evidence is promising but dependent, sample-limited, or operationally incomplete. - **Reject:** The non-AI baseline matches value with lower risk/cost, or a critical safeguard has no owner. - **Defer:** The source, legal basis, data rights, accessibility path, or rollback mechanism is missing. ### Drift monitor | Topic | What can change | Sources | Frequency | Reassess by / trigger | |---|---|---|---|---| | Book completeness | Edition text and corrections | Packt finished edition | Once | Before teaching or citing missing sections | | US policy | Orders, agency rules, enforcement | Federal Register, NIST, agencies | Quarterly | 2026-11-24 or policy change | | EU AI Act | Guidance, standards, high-risk dates | European Commission | Quarterly | 2026-11-24 or scope change | | Standards | AI RMF revision, ISO guidance | NIST, ISO | Semiannual | 2027-02-24 | | Model capability | Versions, modalities, limits, system cards | Provider primary docs | Before each procurement/release | Any model or API change | | Tool availability | Closures, pricing, ownership, APIs | Vendor primary docs | Before recommendation | Product closure or contract change | | Metrics | Task and population fit | Primary technical docs, internal evals | Every release | Distribution or threshold change | | Accessibility | Standards and user barriers | W3C, platform docs, user research | Every design milestone | New flow, modality, or audience | | Costs | Inference, review, storage, incidents, compliance | Bills and internal logs | Monthly | Budget variance >10% | **Likely misuse:** A team may copy the book's vocabulary and stage names while skipping the costly parts: independent evaluation, access research, data rights, security, fallback, incident response, and the option not to use AI. ## Prompt 13 — Visual thinking pack Only visuals that clarify this book are included. A timeline, stakeholder map, systems map, causal loop, comparison matrix, and action map would repeat the structures below without enough added value, so they are omitted. ### Visual 1 — Available-source book map - **Purpose:** Show actual coverage and the missing Early Access arc. - **Nodes:** Chapters 1–5; Chapters 6–12; planned Chapters 13–20. - **Connections:** foundation → native product → integration/career. - **Uncertainty:** final content is not inferred. - **Sources:** EP-0001, EP-0028. - **Layout:** left-to-right, with missing cluster dashed. ```mermaid flowchart LR A["Chs. 1–5\nFoundations + market"] --> B["Chs. 6–12\nAI-native product"] B -. "absent from snapshot" .-> C["Chs. 13–20\nIntegration + AI-PM career"] style C stroke-dasharray: 5 5,fill:#fff4d6 ``` ### Visual 2 — AI product system and feedback loop - **Purpose:** Replace a model-only mental picture with the book's strongest system view. - **Nodes:** problem, people, data, model, interface, operations, measures, governance, feedback. - **Connections:** delivery loop plus governance across every node. - **Uncertainties:** organizational ownership varies. - **Claims:** C-001, C-003, C-006, C-019, C-023. - **Sources:** EP-0003, EP-0007, EP-0019, EP-0024, EP-0027. ```mermaid flowchart LR P["Problem + affected people"] --> D["Data"] --> M["Model"] --> I["Interface + workflow"] --> O["Operations"] O --> V["Value + harm measures"] --> F["Feedback + incidents"] --> P G["Governance: owners, rights, access, security, fallback, retirement"] --- P G --- D G --- M G --- I G --- O G --- V ``` ### Visual 3 — Decision tree: should this use AI? - **Purpose:** Turn “Why AI?” into an approval gate. - **Nodes:** outcome, baseline, measurable advantage, data rights, risk controls, ownership, experiment, stop. - **Connections:** yes/no branches. - **Uncertainties:** risk tolerance and jurisdiction vary. - **Claims:** C-002, C-004, C-008, C-012, C-018. - **Sources:** EP-0004, EP-0005, EP-0020, EP-0023. ```mermaid flowchart TD A["Define user outcome and affected people"] --> B{"Can rules, search, workflow change, or human service meet it?"} B -- Yes --> C["Prefer simpler baseline or compare directly"] B -- No --> D{"Does AI show a measurable advantage?"} D -- No --> X["Stop or defer"] D -- Yes --> E{"Lawful fit-for-purpose data and access path?"} E -- No --> X E -- Yes --> F{"Owners for risk, security, fallback, appeal, retirement?"} F -- No --> X F -- Yes --> G["Run bounded, reversible evaluation"] G --> H{"Value clears guardrails?"} H -- No --> X H -- Yes --> I["Stage launch, monitor, and reassess"] ``` ### Visual 4 — Evidence-strength matrix - **Purpose:** Separate useful operating guidance from weak proof. - **Axes:** directness/reproducibility (low → high) and independence (low → high). - **Place:** NIST/EU/ISO primary text high-high for their own claims; book lifecycle frameworks medium-low; company promotional cases low-low; fictional Akeira scenario no empirical placement; reproducible product experiments can move high-high. - **Claims:** C-003, C-009, C-018, C-024, C-025. - **Layout:** 2×2 Excalidraw canvas. Use circles for evidence, squares for frameworks, and a separate striped “teaching only” box for fictional cases. ### Visual 5 — Metric choice by error cost - **Purpose:** Correct the book's metric conflation. - **Nodes:** false-positive cost, false-negative cost, class balance, threshold, accuracy, precision, recall, specificity, F1, calibration, subgroup outcomes. - **Connections:** error costs and population determine metric/threshold; metrics feed launch and monitoring decisions. - **Uncertainties:** no metric alone captures welfare. - **Claims:** C-017, C-021. - **Source:** EP-0021 plus [Google's primary metric definitions](https://developers.google.com/machine-learning/crash-course/classification/accuracy-precision-recall). - **Layout:** Excalidraw decision canvas with “Who pays for each error?” centered above the metric boxes. ## Prompts used this session | Session field | Result | |---|---| | Session date | 2026-08-24 | | Executed | Prompts 0–13 | | Skipped | None | | Remaining | None for the available source | | Working note | This single consolidated file | | Session status | Complete with source limitations | | Source of truth | User-pasted prompt; local Early Access EPUB for book evidence | ## Session completion check - **Prompts run:** 0–13. - **Usable outputs:** All prompts produced usable output. - **Partial failures:** Prompt 0 found no exact Readwise document. Prompts 2, 6, 7, 8, 10, 11, 12, and 13 are limited by missing Chapters 13–20. No missing content was simulated. - **Missing source material:** Finished-edition preface, Chapters 13–20, conclusion/back matter, and any corrections made after this Early Access snapshot. - **Assumptions introduced:** The newest downloaded EPUB was the intended book; the reading purpose was practical evaluation for product and technical-communication work; user priors and DEIB goals were not supplied. - **Contradictions flagged:** The source first says every model has hyperparameter tuning and later says not all do; it conflates classification metrics; it treats AIOps and MLOps as interchangeable; the roadmap and available chapter set do not match; Chapter 12 ends with production notes and no closure. - **Attribution:** Author, third-party, illustrative-case, and analysis lanes remain separate. - **Next prompt:** None. The next responsible source step is to obtain the finished second edition before analyzing the absent chapters. ## Bottom line Use this book for the questions it teaches a PM to ask: Why AI? What data? Which people and harms? How will we deploy, observe, respond, and learn? What value do we measure? Who owns the decision? Do not use this Early Access snapshot as a current model, tool, legal, metric, or finished-edition authority. Its operating spine is useful; its technical examples and proof standards need correction and stronger primary evidence.