How We Teach AI Models  🔗 **Source:** [How We Teach AI Models](https://twitter.com/leerob/status/2080467752897146898/?rw_tt_thread=True) > [!note]+ Document Notes > tags: [model-training, reinforcement-learning, ai-evaluation, model-specifications, agent-memory] > author: "Lee Robinson" > source: "https://x.com/leeerob/status/2014651429009101109" > reference: "01kya53zhyh08qrjejce9ecmtg" > date created: 2026-07-24 > > # 🏷️ Title: How we teach AI models > <!-- deep-read:sections start --> ## 🧭 Table of Contents Summary; main content and notes; references; paraphrase; newsletter; action items; hypotheses; methodology; questions; future research; implications; learning, accessibility, DEIB, logic, DARVO, conclusion; refactor appendix; template library. ## 🔗 References - Source: https://x.com/leeerob/status/2014651429009101109 - Reader document: 01kya53zhyh08qrjejce9ecmtg ## 📝 Summary AI systems become useful through layers: pretraining supplies broad knowledge, supervised fine-tuning demonstrates desired answers, and reinforcement learning rewards behavior that meets a model specification. Held-out evaluations test whether behavior generalizes. Current conversational memory is normally stored outside model weights in rules, skills, and retrieved context. ## 📖 Main Content The source compares a model to a colleague who needs knowledge, examples, practice, and clear expectations. A model specification states behavioral goals. Supervised examples teach response patterns; reinforcement learning supplies outcome feedback. Objective benchmarks work when answers are known, while open-ended work often uses rubric-based model judges. The author separates intelligence from alignment and notes that live chats do not retrain the model. ### 🗒️ Notes - Claim: Model specifications externalize desired behavior so teams can inspect and revise it. - Mechanism: Held-out evaluations separate generalization from memorized examples. - Verification focus: Current agent memory often lives outside model weights in rules, skills, and retrieved context. - Scope: This analysis covers the captured source body. It does not treat linked claims as independently verified unless the source supplies direct evidence. ## 📚 References - Primary source: https://x.com/leeerob/status/2014651429009101109 - Follow linked primary documents before using numerical, legal, medical, or policy claims in publication. ## 🗣️ Paraphrase Models first absorb broad patterns, then copy good demonstrations, then practice against reward signals. Tests must stay outside training data. A capable model can still be unhelpful if its behavior does not match the product’s rules. Persistent agent behavior therefore depends on external instructions and memory stores. ## 📰 Newsletter **Why this matters for WordPress:** AI systems become useful through layers: pretraining supplies broad knowledge, supervised fine-tuning demonstrates desired answers, and reinforcement learning rewards behavior that meets a model specification. Held-out evaluations test whether behavior generalizes. Current conversational memory is normally stored outside model weights in rules, skills, and retrieved context. For teams building open web software, the useful move is to convert the source’s central claim into a small test, publish limits, and keep the people affected by the decision in the review loop. [Read the source](https://x.com/leeerob/status/2014651429009101109). ## ✅ Action Items - DevRel: turn claims into testable demos; OSPO: track governance and dependency effects; Community: invite affected groups into review; WordPress — Core: assess platform impact; Core Performance: measure cost and latency; Core AI: define model-specific tests; Core Program Team: coordinate milestones; Mobile: test small-screen use; Playground: build a reproducible demo; Community: gather contributor feedback; Training: create a lesson; Marketing: state limits; Support: collect failure patterns; Design: review workflows; Documentation: record behavior and caveats; Photos: not applicable unless visual assets are needed; TV: consider a short explainer; Polyglots: plan translation context; Accessibility: test WCAG 2.2 AA; Meta: review site and API effects; CLI: expose repeatable checks where relevant; Hosting: test deployment constraints; Tide: add automated quality checks where applicable; Openverse: review open-asset implications; Themes: test presentation effects; Plugins: publish compatibility guidance. The WPOCC: consider community-governance effects. FAIR: note implications for federated open-source collaboration. ## 🧪 Hypotheses 1. If model specifications externalize desired behavior so teams can inspect and revise it., teams will observe a measurable change in decision quality or failure rate. 2. If teams test the mechanism in a bounded pilot, results will differ by task complexity and user expertise. 3. Publishing limits beside outcomes will reduce overgeneralization in later decisions. ## 📊 Methodology Use a preregistered comparison with a baseline, clear unit of analysis, task-level outcomes, failure categories, cost and time measures, and interviews with affected users. Preserve raw inputs and report null results. For a short announcement, first obtain primary documentation and reproduce the stated behavior. ## ❓ Questions - Which claim would change the decision if it proved false? - What is the correct unit: task, job, team, organization, or ecosystem? - Which groups bear costs that the source does not measure? - What evidence would distinguish adoption from dependable outcomes? ## 🔮 Future Research Replicate across organizations and user groups; compare short- and long-term effects; measure distributional impact; test alternative explanations; and track whether early gains survive operational use. ## 💡 Implications Practice: use bounded trials. Policy: require traceable evidence for high-impact claims. Security: inspect failure paths and dependencies. Education: teach evaluation, not tool loyalty. Funding: tie continued spending to measured outcomes and public limits. ## 🧠 Learning Styles Assessment - Visual: add a claim-to-evidence diagram. - Aural: discuss the argument and counterargument aloud. - Verbal: rewrite the thesis in plain language. - Kinesthetic: run the proposed pilot. - Logical: map premises, mechanism, evidence, and conclusion. - Social: compare interpretations in a small group. - Solitary: keep a prediction and error log. - Naturalistic: examine resource and environmental effects. - Reading level: approximately grade 10–12; technical or institutional terms need definitions. ## 📊 Accessibility Assessment - Perceivable: provide alt text, captions, sufficient contrast, and text equivalents (WCAG 1). - Operable: support keyboard access, visible focus, and adequate target sizes (WCAG 2). - Understandable: use clear headings, definitions, and consistent navigation (WCAG 3). - Compatible: use semantic markup, valid names, roles, and values (WCAG 4). - Cognitive: shorten dense passages and expose the argument structure. - Motion and media: avoid forced motion; provide controls and transcripts. - Testing: combine automated checks with keyboard, screen-reader, zoom, reflow, and cognitive review against WCAG 2.2 AA. ## 🌍 DEIB Assessment - Cultural breadth: identify the source’s geographic and institutional scope. - Inclusive language: separate groups from stereotypes and avoid universal claims. - Neurodiversity support: provide summaries, headings, examples, and predictable structure. - Representation: seek evidence from people affected by the proposed change. - Psychological safety: invite dissent without requiring public disclosure of harm. ## ✅ Logical Validity The layered training explanation is coherent and distinguishes capability, evaluation, and alignment. It simplifies contested details such as data quality, reward hacking, judge bias, and post-deployment adaptation. The colleague analogy is explanatory, not proof. - Strong reasoning: the source offers a clear thesis and at least one mechanism. - Fallacy check: watch for hasty generalization, selection bias, false cause, motive attribution, and analogy used as proof. - Fix: add comparison data, definitions, uncertainty ranges, and counterevidence. - Tone: explanatory and persuasive. ## 🛡️ DARVO Analysis No complete DARVO sequence is established by the captured source. Where conflict appears, separate denial, criticism, and claims of harm rather than assigning a pattern from rhetoric alone. ## 🎯 Conclusion Model specifications externalize desired behavior so teams can inspect and revise it. Next, test the mechanism against a baseline and preserve the source’s stated limits. ## 🧱 Refactor Appendix ### Literature Split Pad - Source claim: AI systems become useful through layers: pretraining supplies broad knowledge, supervised fine-tuning demonstrates desired answers, and reinforcement learning rewards behavior that meets a model specification. Held-out evaluations test whether behavior generalizes. Current conversational memory is normally stored outside model weights in rules, skills, and retrieved context. - Evidence state: captured source analyzed; linked claims need primary-source checks. - Cross-links: [[model-training]] [[reinforcement-learning]] [[ai-evaluation]] [[model-specifications]] [[agent-memory]] #### Atomic Split Pad - Model specifications externalize desired behavior so teams can inspect and revise it. - Held-out evaluations separate generalization from memorized examples. - Current agent memory often lives outside model weights in rules, skills, and retrieved context. #### Fleeting Queue - What observation would falsify the central claim? - Which existing vault note should receive each atom? - Revisit after new evidence or a material product change. <!-- deep-read:sections end --> 📄 **Full text:** [[Full Document Contents/Tweets/How We Teach AI Models.md|How We Teach AI Models by Lee Robinson]] ## Highlights - The process is surprisingly human and easy to understand, even if you don’t have a machine learning background. ([View Highlight](https://read.readwise.io/read/01kya5qf7m4hnxmc0b4qr9sys6)) - Note: Central claim captured during deep read. - Tags: model-training, reinforcement-learning - AI models, increasingly used through agents, are similar to a helpful colleague. What traits do the best people you’ve worked with share? ([View Highlight](https://read.readwise.io/read/01kya5qg57nxd02qe32wtryq3p)) - Note: Supporting mechanism or evidence captured during deep read. - Tags: model-training, reinforcement-learning