Is Microsoft’s EngThrive framework immune to Goodhart’s Law?  🔗 **Source:** [Is Microsoft’s EngThrive framework immune to Goodhart’s Law?](https://leaddev.com/reporting/is-microsofts-engthrive-framework-immune-to-goodharts-law) > [!note]+ Document Notes > tags: [engineering-metrics, productivity-measurement, goodharts-law, microsoft-research, developer-experience, metrics-gaming] > author: Chris Stokel-Walker > source: LeadDev > reference: https://leaddev.com/reporting/is-microsofts-engthrive-framework-immune-to-goodharts-law > date created: {{date:YYYY-MM-DD}} {{time:HH:mm}} > > # 🏷️ Title: Is Microsoft's EngThrive framework immune to Goodhart's Law? > <!-- deep-read:sections start --> ## 🧭 Table of Contents - [📝 Summary](#-summary) - [📖 Main Content](#-main-content) - [🗒️ Notes](#️-notes) - [📚 References](#-references) - [🗣️ Paraphrase](#️-paraphrase) - [📰 Newsletter](#-newsletter) - [✅ Action Items](#-action-items) - [🧪 Hypotheses](#-hypotheses) - [📊 Methodology](#-methodology) - [❓ Questions](#-questions) - [🔮 Future Research](#-future-research) - [💡 Implications](#-implications) - [🧠 Learning Styles Assessment](#-learning-styles-assessment) - [📊 Accessibility Assessment](#-accessibility-assessment) - [🌍 DEIB Assessment](#-deib-assessment) - [✅ Logical Validity](#-logical-validity) - [🛡️ DARVO Analysis](#-darvo-analysis) - [🎯 Conclusion](#-conclusion) ## 📝 Summary - Microsoft Research developed EngThrive, a productivity measurement framework built around Speed, Ease, Quality, and Thriving dimensions that claims to minimize Goodhart's Law effects. - The system uses "gaming alignment" - deliberately choosing metrics where attempting to game them produces beneficial outcomes rather than trying to prevent gaming entirely. - EngThrive focuses on team and organizational assessment rather than individual performance evaluation, deployed across tens of thousands of Microsoft developers. - Expert David Manheim acknowledges the framework's innovation but questions whether any metric system can truly escape Goodhart's Law under sufficient optimization pressure. - The framework showed promising results: Time-to-First-PR experiment increased speed 30% while new hires produced 23% more pull requests in their first year. ## 📖 Main Content Microsoft's EngThrive framework represents a novel approach to engineering productivity measurement that acknowledges the inevitability of metric gaming while attempting to channel it constructively. The system is built around three core dimensions plus a guardrail: **Speed** measures elapsed calendar time for tasks like pull request completion. **Ease** captures developer friction through time measurements and surveys. **Quality** assesses long-term reliability by tracking quality-related disruptions. **Thriving** serves as a guardrail measuring overall team health. The key innovation is "gaming alignment" - deliberately selecting metrics where cheating produces desired outcomes. The Time-to-First-PR experiment exemplifies this: managers assigned trivial first commits (fixing typos, flipping test variables) to 4,000 developers. While this seems like obvious gaming, it forced new hires to set up development environments, learn review processes, and understand team workflows. Results showed 30% faster first commits and 23% more pull requests over the first year. Brian Houck, the framework's lead researcher, argues this approach works because actions that game the metrics align with beneficial behaviors. Breaking pull requests into smaller chunks games PR throughput but improves reviewability and testing. The system includes unconventional measurements like "Bad Developer Days" (counting context switching, build failures, incidents) and discovered 52% of Microsoft developer days qualified as "bad," with developers experiencing 3+ bad days weekly being three times more likely to quit. Crucially, EngThrive explicitly avoids individual performance assessment, focusing on team and organizational health to reduce optimization pressure that could break the metrics. ## 🗒️ Notes - Goodhart's Law: "When a measure becomes a target, it stops being a good measure" - Gaming alignment principle: Choose metrics where cheating produces desired outcomes - Time-to-First-PR experiment results: 30% speed increase, 23% more PRs in first year - 52% of Microsoft developer days classified as "bad" - Framework deployed across "tens of thousands" of Microsoft developers - System explicitly designed for team/org assessment, not individual performance - David Manheim's critique: Any metric can be gamed under sufficient pressure - EngThrive includes Net Satisfaction and Bad Developer Days metrics most orgs don't measure ## 📚 References - [Microsoft EngThrive research paper](https://leaddev.com/reporting/is-microsofts-engthrive-framework-immune-to-goodharts-law) - [David Manheim - Association for Long Term Existence and Resilience (ALTER)](https://leaddev.com/reporting/is-microsofts-engthrive-framework-immune-to-goodharts-law) - [Technion – Israel Institute of Technology](https://leaddev.com/reporting/is-microsofts-engthrive-framework-immune-to-goodharts-law) ## 🗣️ Paraphrase Microsoft researchers believe they've cracked the code on engineering metrics by embracing rather than fighting gaming behavior. Their EngThrive system intentionally chooses measurements where attempting to cheat actually produces the results you want. When they told managers to assign trivial first commits to new hires, it looked like obvious metric manipulation but actually accelerated real onboarding by forcing essential setup tasks. The key insight is that some gaming behaviors align perfectly with desired outcomes - like breaking large pull requests into smaller, more reviewable chunks to inflate PR counts. However, experts remain skeptical that any metric system can truly escape Goodhart's Law when organizations apply enough optimization pressure. ## 📰 Newsletter **Microsoft's Gaming-Friendly Engineering Metrics: Revolutionary or Risky?** Hey WordPress builders! 👋 This week's fascinating read explores Microsoft's bold new approach to engineering productivity metrics. Instead of trying to prevent developers from gaming their measurements, they're actively encouraging it - but only when the gaming produces beneficial outcomes. Their "Time-to-First-PR" experiment is brilliant: encourage managers to assign trivial first commits (fixing typos, etc.) to new hires. Sure, it games the metric, but it forces essential onboarding steps like environment setup and learning team workflows. Result? 30% faster first commits AND 23% more contributions over the first year. **Key takeaways for WordPress teams:** - Consider metrics where "cheating" actually helps (smaller PRs = better reviews) - Measure team health, not individual performance - Track "bad developer days" (context switching, build failures, incidents) - Focus on developer satisfaction alongside traditional velocity metrics **Worth exploring:** How might this apply to WordPress contribution metrics? Could we design measurements where gaming behaviors actually improve the contributor experience? The framework is open for adaptation beyond Microsoft - something to consider as our community evolves its approach to measuring and improving developer productivity. ## ✅ Action Items #### DevRel - Developer experience: Evaluate current onboarding metrics for gaming alignment opportunities, design first-contribution workflows that force essential setup tasks, measure time-to-first-meaningful-contribution rather than just time-to-first-commit. - Education: Create workshops on metric design principles, develop case studies of beneficial gaming behaviors, design curriculum around productivity measurement best practices. - Community management: Implement Bad Developer Days tracking for community contributors, survey contributor satisfaction alongside traditional metrics, establish team health dashboards rather than individual performance tracking. #### OSPO - Developer Support: Apply gaming alignment principles to contribution metrics, design policies that encourage beneficial gaming behaviors, establish contributor onboarding workflows inspired by Time-to-First-PR experiment. - Strategic Alignment: Develop KPIs that align gaming behaviors with desired outcomes, create dashboards focused on team/project health rather than individual metrics, establish satisfaction surveys for open source contributors. #### Community - Contributors: Design first-issue workflows that force essential environment setup, create mentorship programs that gamify beneficial learning behaviors, track and reduce "bad contributor days" caused by tooling friction. - Maintainers: Implement pull request size incentives that align with review efficiency, establish team health metrics rather than individual contribution tracking, develop satisfaction surveys for maintainer experience. #### WordPress - Core: Apply gaming alignment to contribution metrics, design first-time contributor workflows that force essential setup tasks, implement Bad Developer Days tracking for core contributors. - **Core Program Team:** Develop cross-team health metrics rather than individual performance measures, establish contributor satisfaction surveys, create dashboards that track team friction and context switching. - Community: Design meetup success metrics where gaming produces beneficial community outcomes, establish WordCamp health measurements beyond attendance numbers. - Training: Create lesson plans on healthy metric design, develop workshops on productivity measurement, establish learning path metrics that encourage beneficial completion gaming. ## 🧪 Hypotheses - Gaming alignment principles can be successfully applied to open source contribution metrics without degrading software quality. - Time-to-first-contribution workflows that force essential setup tasks will improve long-term contributor retention in WordPress projects. - Bad Developer Days tracking will reveal significant friction sources that traditional velocity metrics miss in distributed teams. - Team-focused metrics produce better long-term outcomes than individual performance measures in collaborative software development. - Encouraging beneficial gaming behaviors leads to more sustainable productivity improvements than attempting to prevent all gaming. ## 📊 Methodology - Controlled experiments comparing gaming-aligned metrics vs. traditional measurements across WordPress contributor cohorts. - Longitudinal studies tracking contributor satisfaction and retention under different metric frameworks. - Cross-sectional analysis of Bad Developer Days across different WordPress project types and team structures. - A/B testing of onboarding workflows with deliberate gaming alignment vs. standard approaches. - Qualitative interviews with contributors about their gaming behaviors and motivations. ## ❓ Questions - How can open source projects implement gaming alignment without compromising code quality standards? - What constitutes a "Bad Contributor Day" in distributed, volunteer-driven development environments? - Which WordPress contribution behaviors appear to be gaming but actually produce beneficial outcomes? - How do satisfaction-based metrics perform in volunteer communities versus paid development teams? - What safeguards prevent gaming alignment from degrading into purely performative contributions? ## 🔮 Future Research - Adaptation studies of EngThrive principles in open source and volunteer-driven development contexts. - Cross-cultural analysis of gaming behaviors and their outcomes in different developer communities. - Long-term impact studies of gaming-aligned metrics on software quality and team sustainability. - Development of specialized metrics for distributed, asynchronous development workflows. - Investigation of AI-assisted development impacts on traditional productivity measurements. ## 💡 Implications - **Practice:** Development teams may need to redesign metric systems to encourage rather than prevent beneficial gaming behaviors. - **Policy:** Open source governance structures should consider gaming alignment when establishing contribution guidelines and recognition systems. - **Security:** Gaming-aligned metrics require careful monitoring to ensure beneficial behaviors don't mask security or quality degradation. - **Education:** Developer education should include training on healthy metric design and recognition of beneficial gaming patterns. - **Funding:** Grant and sponsorship programs may need new frameworks for measuring impact that align gaming with desired outcomes. ## 🧠 Learning Styles Assessment - Reading level estimate: Flesch-Kincaid Grade Level approximately 12-13 (college level). #### 👀 Visual learners - Present: Framework diagram showing Speed/Ease/Quality/Thriving dimensions. - Improve: Before/after workflow diagrams for Time-to-First-PR experiment, visualization of Bad Developer Days impact on retention, charts showing gaming behaviors vs. beneficial outcomes. #### 🔊 Aural learners - Present: Quoted explanations from researchers and experts. - Improve: Audio summary of key gaming alignment principles, podcast-style discussion of Goodhart's Law implications, recorded case study walkthrough. #### 🗣️ Verbal learners - Present: Clear definitions of gaming alignment, Goodhart's Law explained with examples. - Improve: Glossary of metric terminology, analogies comparing traditional vs. gaming-aligned approaches, plain-language summaries of research findings. #### 🧩 Kinesthetic learners - Present: Specific experiment description with actionable steps. - Improve: Hands-on workshop designing gaming-aligned metrics, interactive exercises identifying beneficial gaming behaviors, step-by-step metric implementation guides. #### 🧠 Logical learners - Present: Structured framework with clear dimensions, experimental results with specific percentages. - Improve: Decision trees for metric selection, cause-and-effect diagrams of gaming behaviors, logical frameworks for evaluating metric health. #### 🧑‍🤝‍🧑 Social learners - Present: Expert commentary and researcher interviews. - Improve: Discussion prompts about metric gaming experiences, collaborative exercises designing team metrics, peer review of metric proposals. #### 🧘‍♀️ Solitary learners - Present: Individual reflection on metric gaming experiences. - Improve: Self-assessment tools for current metric health, personal journaling prompts about productivity measurements, independent case study analysis. #### 🌿 Naturalistic learners - Present: Ecosystem metaphor of healthy vs. unhealthy gaming behaviors. - Improve: Organic growth analogies for metric evolution, sustainability frameworks for long-term metric health, natural selection parallels in metric adoption. ## 📊 Accessibility Assessment #### Screen reader semantics - Present: Clear headings and logical structure. - Improve: Add ARIA landmarks for main content sections, ensure all metric examples have descriptive alt text, provide table headers for any data comparisons. #### Keyboard navigation - Present: Standard web navigation patterns. - Improve: Add skip links to main sections, ensure focus indicators on interactive elements, provide keyboard shortcuts for section navigation. #### Contrast and readability - Present: Standard text contrast on white background. - Improve: Verify 4.5:1 contrast ratio for all text, add high-contrast mode support, ensure sufficient line spacing for readability. #### Text resizing and reflow - Present: Responsive design accommodates basic resizing. - Improve: Test at 200% zoom to ensure no content clipping, implement prefers-reduced-motion for any animations, support browser text size overrides. #### Media and captions - Present: Text-based content with minimal media requirements. - Improve: Add captions for any embedded videos, provide transcripts for audio content, include alt text for framework diagrams. #### Forms and interactions - Present: Standard web form elements. - Improve: Add proper labels for all form fields, implement inline validation with accessible error messages, ensure sufficient target size for touch interfaces. ## 🌍 DEIB Assessment #### Cultural breadth - Present: US-focused case study with some international expert perspective. - Improve: Include examples from non-Western development cultures, address how gaming behaviors vary across cultures, provide multi-regional case studies. #### Inclusive language - Present: Gender-neutral technical language, clearly defined jargon. - Improve: Add plain-language definitions for all technical terms, avoid cultural idioms, expand glossary for international audiences. #### Neurodiversity support - Present: Well-structured content with clear headings. - Improve: Add content warnings for discussion of performance measurement anxiety, provide executive summaries for each section, use consistent visual patterns throughout. #### Representation - Present: Diverse expert voices including international researchers. - Improve: Include perspectives from underrepresented groups in tech, address how metric gaming affects different developer populations, ensure inclusive imagery in any visual materials. #### Psychological safety - Present: Respectful discussion of metric limitations and gaming behaviors. - Improve: Address measurement anxiety explicitly, provide guidance on creating psychologically safe metric environments, include resources for teams struggling with performance measurement. ## ✅ Logical Validity #### Strong reasoning - **Inductive reasoning about gaming alignment (Time-to-First-PR experiment):** The article presents specific experimental evidence where encouraging metric gaming produced beneficial outcomes, then extrapolates this principle to other metrics. Strong because it's based on empirical data from 4,000 developers. - **Causal reasoning about onboarding benefits:** The connection between trivial first commits and improved long-term contribution is well-supported by participant interviews explaining the learning mechanisms. - **Analogical reasoning about traditional metric failures:** Drawing parallels between lines-of-code gaming and other productivity measures effectively illustrates the pervasiveness of Goodhart's Law. #### Fallacies - **Hasty generalization:** The article may overgeneralize from Microsoft's specific context to all engineering organizations without sufficient evidence that gaming alignment works across different cultures, team sizes, or project types. - **Cherry picking:** The focus on successful Time-to-First-PR results may not represent the full range of gaming alignment attempts, potentially overlooking failures or negative outcomes. - **Appeal to authority:** Heavy reliance on expert opinion (particularly David Manheim) without presenting counter-perspectives or critical analysis of the framework's limitations. #### Tone of voice - **Objective and analytical:** The article maintains journalistic balance while presenting both supportive evidence and expert skepticism. - **Cautiously optimistic:** Language like "claims to minimize" and "isn't so sure" indicates healthy skepticism about revolutionary claims. ## 🛡️ DARVO Analysis This article does not exhibit DARVO patterns. It presents a balanced analysis of Microsoft's claims while including expert skepticism and acknowledging limitations. The discussion of gaming behaviors is framed as a systemic design challenge rather than individual blame, and there's no attempt to reverse victim/offender dynamics around metric gaming. ## 🎯 Conclusion Microsoft's EngThrive framework represents an innovative approach to engineering metrics by embracing rather than fighting gaming behaviors. While the gaming alignment principle shows promise in early experiments, expert skepticism about truly escaping Goodhart's Law remains valid. The framework's focus on team health over individual performance and inclusion of developer satisfaction metrics offers valuable lessons for any organization struggling with productivity measurement. For WordPress and open source communities, the key insight may be designing contribution workflows where beneficial gaming behaviors naturally emerge while maintaining focus on long-term community health over short-term individual metrics. ## 🧱 Refactor Appendix ### 📚 Literature Split Pad ### Gaming alignment produces beneficial developer behaviors > [!quote] [Is Microsoft's EngThrive framework immune to Goodhart's Law?](https://leaddev.com/reporting/is-microsofts-engthrive-framework-immune-to-goodharts-law) by [[Chris Stokel-Walker]] via [[LeadDev]]. Published May 21, 2026. Accessed {{date:YYYY-MM-DD}}. > "What we did is try to choose metrics where it's, like, 'Great, try to cheat the system. That's a good thing'." Gaming alignment deliberately channels gaming behaviors toward desired outcomes rather than preventing them. [view highlight](https://leaddev.com/reporting/is-microsofts-engthrive-framework-immune-to-goodharts-law) #xref #articleseed ID:rw_[MISSING] ### Team-focused metrics reduce optimization pressure risks > [!quote] [Is Microsoft's EngThrive framework immune to Goodhart's Law?](https://leaddev.com/reporting/is-microsofts-engthrive-framework-immune-to-goodharts-law) > "We never use these metrics for assessing individual-level performance. This is really about holding engineering leaders accountable." Individual performance measurement creates destructive gaming incentives that team-level metrics avoid. [view highlight](https://leaddev.com/reporting/is-microsofts-engthrive-framework-immune-to-goodharts-law) #xref ID:rw_[MISSING] ### Bad Developer Days predict turnover better than velocity > [!quote] [Is Microsoft's EngThrive framework immune to Goodhart's Law?](https://leaddev.com/reporting/is-microsofts-engthrive-framework-immune-to-goodharts-law) > "Some 52% of all developer days at Microsoft qualified as 'bad', and developers experiencing three or more bad days a week were three times more likely to quit." Developer experience friction metrics may be stronger predictors of retention than traditional productivity measures. [view highlight](https://leaddev.com/reporting/is-microsofts-engthrive-framework-immune-to-goodharts-law) #xref ID:rw_[MISSING] --- ### 🔖 Atomic Split Pad (Zettel Candidates) #### Gaming alignment requires outcome-behavior convergence - Beneficial gaming occurs when cheating behaviors naturally align with desired long-term outcomes. - Time-to-First-PR experiment demonstrates this principle in practice. - Links: [[Metric-Design]] [[Goodharts-Law]] [[Developer-Onboarding]] #### Team metrics avoid individual optimization pressure - Individual performance measurement creates perverse gaming incentives that team-level assessment avoids. - Reduces metric fragility under optimization pressure while maintaining accountability. - Links: [[Performance-Management]] [[Metric-Gaming]] [[Team-Health]] #### Developer friction predicts retention more than velocity - Bad Developer Days framework captures context switching, build failures, and compliance overhead. - Three or more bad days weekly correlates with 3x higher turnover risk. - Links: [[Developer-Experience]] [[Retention-Prediction]] [[Team-Productivity]] --- ### ✏️ Fleeting Queue (Extract by Selection) - [F1] Test gaming alignment with WordPress first-contribution workflows — context/Time-to-First-PR #fleeting #inbox - [F2] Survey Bad Contributor Days in open source vs. Bad Developer Days — context/retention-prediction #fleeting #inbox - [F3] Design team health metrics for distributed volunteer communities — context/metric-design #fleeting #inbox <!-- deep-read:sections end --> ## Highlights - Test gaming alignment with WordPress first-contribution workflows — context/Time-to-First-PR - Note: ✏️ Fleeting capture #atom/fleeting/f1 - Survey Bad Contributor Days in open source vs. Bad Developer Days — context/retention-prediction - Note: ✏️ Fleeting capture #atom/fleeting/f2 - Design team health metrics for distributed volunteer communities — context/metric-design - Note: ✏️ Fleeting capture #atom/fleeting/f3 - ** The article may overgeneralize from Microsoft's specific context to all engineering organizations without sufficient evidence that gaming alignment works across different cultures, team sizes, or project types. - Note: 🔍 Fallacy: Hasty Generalization #fallacy/hasty-generalization - ** Heavy reliance on expert opinion (particularly David Manheim) without presenting counter-perspectives or critical analysis of the framework's limitations. - Note: 🔍 Fallacy: Appeal to Authority #fallacy/appeal-to-authority ## New Highlights added 2026-05-21 at 10:39 AM - “What we did is try to choose metrics where it’s, like, ‘Great, try to cheat the system. That’s a good thing’,” says Houck, who recently left Microsoft to join Atlassian as a distinguished scientist. The principle, which the paper calls “gaming alignment”, is to pick metrics where the act of fiddling the numbers produces the outcome you actually wanted. ([View Highlight](https://read.readwise.io/read/01ks5dz1mrwne5185fcf9e04fg)) - Note: Gaming alignment produces beneficial developer behaviors - Tags: atom/literature/h1 - Some 52% of all developer days at Microsoft qualified as “bad”, according to the study, and developers experiencing three or more bad days a week were three times more likely to quit. “They measure net satisfaction, which most places don’t,” says Manheim, crediting the work. ([View Highlight](https://read.readwise.io/read/01ks5dz9z6hcn9q45ay8xhbwy7)) - Note: Bad Developer Days predict turnover better than velocity - Tags: atom/literature/h3 - EngThrive is designed to assess organizations and teams, but not individual engineers. “We never use these metrics for assessing individual-level performance,” says Houck. “It isn’t, and should never be, about holding individual developers accountable. This is really about holding engineering leaders accountable.” ([View Highlight](https://read.readwise.io/read/01ks5dz7azbdn55548q7aw2grt)) - Note: Team-focused metrics reduce optimization pressure risks - Tags: atom/literature/h2 - The lesson, he says, is to keep metrics on a dashboard rather than linking them to pay decisions. “Using this as a dashboard to pay attention to what’s happening is not putting tremendous optimization pressure,” he says. “If your metric is at all fragile, pushing too hard can break it.” ([View Highlight](https://read.readwise.io/read/01ks5dzcgc5q2j97w482dgkbc3)) - Note: Team metrics avoid individual optimization pressure - Tags: atom/zettel/z2 ## New Highlights added 2026-05-21 at 11:38 AM - Distinguished scientist Brian Houck and his co-authors unveiled EngThrive, a measurement and improvement system organized around three dimensions (Speed, Ease and Quality), with a fourth, Thriving, acting as a key guardrail. • **Speed** captures how quickly developers can convert intent into features, and is measured in units of “elapsed calendar time” for things like pull request completion time. • **Ease** tries to calculate the degree to which developers can work without unnecessary friction or interruption and is measured through “developer time” and surveys on perceived ease of delivery. • **Quality** assesses whether the engineering work produces long-standing and reliable outcomes, and is measured by the frequency and cost of quality-related disruptions. ([View Highlight](https://read.readwise.io/read/01ks5grks2jw2g5vpgbasybr62))