Zack Koppert – Can you measure InnerSource #InnerSourceSummit2025

Zack Koppert – Can you measure InnerSource? #InnerSourceSummit2025 🔗 **Source:** [Zack Koppert – Can you measure InnerSource? #InnerSourceSummit2025](https://www.youtube.com/watch?v=ao9JLzX7yK4) > [!note]+ Document Notes > tags: [innersource, measurement, open-source, github, devrel] > author: Zack Koppert > source: InnerSource Summit 2025 > reference: https://www.youtube.com/watch?v=ao9JLzX7yK4 > date created: 2024-12-19 10:30 > > # 🏷️ Title: Can you measure InnerSource? - Zack Koppert >  ## 🧰 Action Inbox - [H1] Define inner source terms clearly with stakeholders — establish what constitutes contributions, projects, and success #measurement - [H2] Try the Measure Innersource tool on GitHub repositories — github.com/github/measure-innersource #tools - [H3] Create org data JSON mapping team structures to enable measurement — 30-minute setup process #implementation ## 🧭 Table of Contents - [📝 Summary](#-summary) - [📖 Main Content](#-main-content) - [🗒️ Notes](#️-notes) - [📚 References](#-references) - [🗣️ Paraphrase](#️-paraphrase) - [📰 Newsletter](#-newsletter) - [✅ Action Items](#-action-items) - [🧪 Hypotheses](#-hypotheses) - [📊 Methodology](#-methodology) - [❓ Questions](#-questions) - [🔮 Future Research](#-future-research) - [💡 Implications](#-implications) - [🧠 Learning Styles Assessment](#-learning-styles-assessment) - [📊 Accessibility Assessment](#-accessibility-assessment) - [🌍 DEIB Assessment](#-deib-assessment) - [✅ Logical Validity](#-logical-validity) - [🛡️ DARVO Analysis](#-darvo-analysis) - [🎯 Conclusion](#-conclusion) ### 🔗 References - [Measure Innersource Tool](https://github.com/github/measure-innersource) - [InnerSource Commons](https://innersourcecommons.org) ## 📝 Summary - GitHub's Zack Koppert presents a framework for measuring inner source programs after receiving conditional CEO approval requiring measurable outcomes - Why it matters: Inner source programs frequently get cancelled when unable to demonstrate ROI and tangible value to stakeholders - Key stakeholders: Engineering managers, open source program offices, executives approving inner source initiatives - Risks and tradeoffs: Complex measurement spanning code, culture, collaboration; requires defining unclear terms; timing sensitivity with organizational changes - Bottom line takeaway: Three-part measurement framework (cross-team participation, contribution volume, cross-repository impact) with open source tooling can demonstrate inner source ROI ## 📖 Main Content Inner source measurement faces genuine complexity beyond typical software metrics like uptime or error rates. The challenge spans multiple dimensions: determining what constitutes inner source contributions (code reviews vs. discussions), tracking organizational context at contribution time (same people on different teams vs. same team), and defining success metrics (is 5% external contribution good or bad?). Koppert argues measurement is essential for program sustainability, having witnessed multiple inner source programs cancelled due to inability to demonstrate value. Organizations that successfully measure impact receive continued funding through compelling stories backed by data. The solution involves three core measurements: cross-team participation (is inner source happening?), contribution volume (commits, PRs, issues by external teams), and cross-repository impact (identifying high-value shared assets and collaboration champions). This enables ROI calculations - for example, 35% inner source ratio means 35% of development work shared across team boundaries, reducing duplication and maintenance costs. ## 🗒️ Notes #### ⏱️ [00:00](https://youtu.be/ao9JLzX7yK4?t=0) — [H1] Introduction and experience background - Key idea: Senior engineering manager at GitHub with extensive inner source program experience - Evidence: "I've worked with many inner source programs at previous jobs, either starting at them or working at them" #### ⏱️ [01:30](https://youtu.be/ao9JLzX7yK4?t=90) — [H2] The measurement challenge from CEO approval - Key idea: Got conditional approval for inner source program requiring measurable success metrics - Evidence: "Figure out how to measure if it's working. I walked out of that meeting like I said very excited about the approval but also terrified" #### ⏱️ [03:45](https://youtu.be/ao9JLzX7yK4?t=225) — [H1] Three core problems making measurement difficult - Key idea: Complexity, timing, and definition problems create genuine measurement challenges - Evidence: "measuring inner source is genuinely difficult...not just a little bit not technically tricky it is genuinely complex" #### ⏱️ [04:15](https://youtu.be/ao9JLzX7yK4?t=255) — [H2] Complexity problem - spans multiple dimensions - Key idea: Inner source encompasses code, culture, collaboration, knowledge sharing, and business outcomes - Evidence: "inner source isn't just about code it's about culture collaboration knowledge sharing business outcomes" #### ⏱️ [05:30](https://youtu.be/ao9JLzX7yK4?t=330) — [H2] Timing problem - organizational context matters - Key idea: Must measure with awareness of team structures at time of contribution - Evidence: "measurement of when inner source is happening needs to happen with an awareness of the current team structures" #### ⏱️ [07:00](https://youtu.be/ao9JLzX7yK4?t=420) — [H2] Definition problem - unclear terms need clarification - Key idea: Must define what constitutes inner source contributions, projects, and success - Evidence: "what about when somebody reviews code instead of generates code? Is that inner sourcing?" #### ⏱️ [09:30](https://youtu.be/ao9JLzX7yK4?t=570) — [H1] Stakes of not measuring - program cancellation - Key idea: Programs lose funding when unable to demonstrate ROI and tangible value - Evidence: "I've worked with numerous different organizations...that have been cancelled...because stakeholders couldn't see the tangible value" #### ⏱️ [11:00](https://youtu.be/ao9JLzX7yK4?t=660) — [H1] Three-part measurement framework introduction - Key idea: Successful measurement comes down to three core areas through practical tooling - Evidence: "I've built a tool...called measure inner source...successful inner source measurement comes down to three core areas" #### ⏱️ [11:30](https://youtu.be/ao9JLzX7yK4?t=690) — [H2] Measurement 1 - Cross team participation - Key idea: Answers fundamental question "is inner source actually happening" - Evidence: "automatically identifies contributors to repositories, maps them to teams using your organizational structure" #### ⏱️ [13:15](https://youtu.be/ao9JLzX7yK4?t=795) — [H2] Measurement 2 - Contribution volume - Key idea: Complete picture through counting commits, pull requests, and issues - Evidence: "The tool breaks down contributions by individual contributors so that you can see patterns" #### ⏱️ [15:00](https://youtu.be/ao9JLzX7yK4?t=900) — [H2] Measurement 3 - Cross repository impact - Key idea: Identifies high-value shared assets and collaboration champions for business value - Evidence: "run it across multiple repositories to identify which projects attract the most inner source collaboration" #### ⏱️ [17:00](https://youtu.be/ao9JLzX7yK4?t=1020) — [H1] Common measurement pitfalls to avoid - Key idea: Three dangerous mistakes - gititis, vanity metrics, and measurement death spiral - Evidence: "High commit volume might indicate inefficient development practices or even unhealthy collaboration" #### ⏱️ [19:00](https://youtu.be/ao9JLzX7yK4?t=1140) — [H2] Tool implementation - getting started - Key idea: Open source tool at github.com/github/measure-innersource with 30-minute setup - Evidence: "You could sit down in 30 minutes, put together a quick...organizational data.json" #### ⏱️ [20:30](https://youtu.be/ao9JLzX7yK4?t=1230) — [H1] Challenge and conclusion - Key idea: Inner source measurement demonstrates value exchange rather than just agreeing on ideals - Evidence: "inner source measurement isn't about agreeing on ideals. It's about demonstrating the values or the value exchange that occurs" ## 📚 References - [Measure Innersource GitHub Repository](https://github.com/github/measure-innersource) - [InnerSource Commons Community](https://innersourcecommons.org) - [GitHub Inner Source Resources](https://github.com) ## 🗣️ Paraphrase - Inner source programs need measurable outcomes to survive executive scrutiny and continued funding, but measurement is genuinely complex due to cultural and organizational factors beyond simple code metrics - A three-part framework measuring cross-team participation, contribution volume, and cross-repository impact can demonstrate ROI by showing percentage of work shared across team boundaries - Common measurement mistakes include focusing on raw git activity, vanity metrics without context, and over-measuring without taking action based on insights ## 📰 Newsletter **Measuring Inner Source Success: A Framework for Demonstrating ROI** Ever struggled to show executives the value of your inner source program? GitHub's Zack Koppert faced this exact challenge when his CEO granted conditional approval: "Figure out how to measure if it's working." **The Challenge**: Inner source spans code, culture, collaboration, and business outcomes - making it far more complex to measure than traditional software metrics like uptime or error rates. **The Solution**: A three-part measurement framework: 1. **Cross-team participation** - Is inner source actually happening? 2. **Contribution volume** - What's the depth and quality of collaboration? 3. **Cross-repository impact** - Which projects are high-value shared assets? **Practical Tool**: The open-source [Measure Innersource](https://github.com/github/measure-innersource) tool automates these calculations, requiring just a 30-minute setup with your organizational structure data. **Key Insight**: Organizations that measure inner source impact receive continued funding through compelling stories backed by data. A 35% inner source ratio means 35% of development work is shared across team boundaries - directly demonstrating reduced duplication and waste. Resources: [Inner Source Commons](https://innersourcecommons.org/), [GitHub Inner Source Guide](https://github.com), [Measure Innersource Tool](https://github.com/github/measure-innersource) ## ✅ Action Items #### OSPO - Policy & Governance: Define inner source contribution types, project categories, and success metrics with stakeholders - Software Inventory: Map organizational team structures to enable cross-team contribution tracking - Developer Support: Implement Measure Innersource tool for quarterly ROI reporting to executives - Community Engagement: Identify collaboration champions and high-value shared assets through cross-repository analysis - Strategic Alignment: Create KPIs based on inner source ratio targets and contribution volume goals #### DevRel - Developer advocacy: Use inner source measurement data in conference talks and case studies - Education: Create workshops on inner source measurement methodology and tooling - Developer experience: Improve documentation for repositories with low inner source ratios - Developer marketing: Highlight collaboration champions and shared asset success stories - Community management: Foster cross-team participation through measured feedback loops #### Community - Maintainers: Support projects identified as high-value shared assets with governance templates - Contributors: Create onboarding programs targeting teams with low inner source participation - Sponsors: Provide transparent ROI reports showing percentage of work shared across boundaries - Ecosystem: Coordinate measurement standards across inner source programs ## 🧪 Hypotheses - Organizations measuring inner source impact receive more sustainable funding than those relying solely on anecdotal evidence - Inner source ratios above 30% indicate meaningful cross-team collaboration and reduced development duplication - Cross-repository measurement reveals high-value shared assets more effectively than single-project analysis - Teams contributing to external repositories become collaboration champions spreading inner source practices ### 📊 Methodology - Longitudinal study comparing funding retention rates between measured vs unmeasured inner source programs - Cross-organizational analysis of inner source ratios and their correlation with development efficiency metrics - Qualitative research on executive decision-making factors for inner source program continuation ### ❓ Questions - What inner source ratio threshold indicates genuine cross-team collaboration versus casual engagement? - How do organizational structure changes impact the accuracy of historical inner source measurements? - Which contribution types (commits, reviews, issues) best predict long-term inner source program success? ### 🔮 Future Research - Automated organizational structure mapping integration with HR systems for real-time measurement - Machine learning models predicting inner source program sustainability based on early measurement patterns - Cross-industry benchmarking of inner source ratios and their correlation with business outcomes ### 💡 Implications - Practice: Engineering teams need clear definitions of inner source contributions before implementing measurement - Policy: Organizations should establish inner source ratio targets and regular measurement cadences for executive reporting - Security: Cross-team contribution tracking requires careful handling of organizational structure data and access controls - Education: Inner source advocates need training on ROI calculation and executive communication of measurement results - Funding: Programs demonstrating measurable impact through data-backed stories receive more sustainable resource allocation ### 🧠 Learning Styles Assessment Reading level estimate: 12th grade (technical presentation with business context) #### 👀 Visual learners - Present: Framework diagrams, measurement examples, tool screenshots in presentation - Improve: Organizational chart examples, inner source ratio visualizations, ROI calculation flowcharts #### 🔊 Aural learners - Present: Full conference presentation video with clear narration - Improve: Podcast-style summary discussing measurement challenges and solutions #### 🗣️ Verbal learners - Present: Clear definitions of inner source terms, concrete examples, analogies to business metrics - Improve: Glossary of measurement terminology, story-driven case studies #### 🧩 Kinesthetic learners - Present: Hands-on tool implementation, 30-minute setup process, GitHub action workflow - Improve: Step-by-step measurement checklist, interactive organizational mapping exercise #### 🧠 Logical learners - Present: Three-part framework structure, cause-and-effect relationships, ROI calculations - Improve: Decision trees for measurement implementation, measurement validity criteria #### 🧑‍🤝‍🧑 Social learners - Present: Conference presentation format, community discussion opportunities - Improve: Cohort implementation guides, peer review of measurement definitions #### 🧘‍♀️ Solitary learners - Present: Open-source tool for independent implementation, self-paced setup process - Improve: Reflection prompts on organizational measurement needs, self-assessment worksheets #### 🌿 Naturalistic learners - Present: Ecosystem thinking about organizational collaboration - Improve: Organic growth analogies, collaboration network analysis ### 📊 Accessibility Assessment #### Screen reader semantics - Present: Clear heading structure, descriptive link text in references - Improve: Alt text for measurement framework diagrams, table headers for data examples #### Keyboard navigation - Present: Standard video controls, logical tab order in referenced tools - Improve: Skip links for long transcript sections, keyboard shortcuts for timestamp navigation #### Contrast and readability - Present: High contrast text, readable font sizes in presentation - Improve: Ensure 4.5:1 minimum contrast in measurement visualization examples #### Text resizing and reflow - Present: Responsive formatting for various screen sizes - Improve: Prevent clipping of measurement data tables at 200% zoom #### Media and captions - Present: Full transcript provided, clear audio quality - Improve: Downloadable transcript, audio descriptions for visual elements #### Forms and interactions - Present: Clear GitHub repository links, accessible tool interfaces - Improve: Form labels in organizational data JSON examples ### 🌍 DEIB Assessment #### Cultural breadth - Present: GitHub-focused examples, enterprise organizational context - Improve: Multi-industry case studies, different organizational structure examples #### Inclusive language - Present: Gender-neutral terms, defined technical jargon - Improve: Plain-language summaries of measurement concepts, avoid business idioms #### Neurodiversity support - Present: Chunked content with clear sections, consistent structure - Improve: Appropriate heading frequency, summary boxes for key concepts #### Representation - Present: Focus on technical content over individual representation - Improve: Diverse organizational examples, varied team structure scenarios #### Psychological safety - Present: Respectful framing of measurement challenges, non-judgmental tone - Improve: Content warnings for program cancellation discussions ### ✅ Logical Validity #### Strong reasoning - Deductive strength: Framework --- ## Full Transcript [00:06] Hello everyone. Uh, as Yuki said, uh, my [00:09] name is Zach Copern, a senior software, [00:12] uh, engineering manager at GitHub. Um, [00:14] I've worked with many inner source [00:16] programs at previous jobs, either [00:18] starting at them or working at them. Um, [00:20] also, uh, currently at GitHub um, [00:23] working on inner source things as well [00:25] as with many GitHub customers. Um, so [00:28] want to share that experience with you. [00:29] so that you know uh what perspective I'm [00:31] talking from today. [00:35] Um and speaking of today, I wanted to [00:38] talk with you about the question uh that [00:40] has puzzled myself and many others in [00:42] the inner source community. Uh can you [00:44] actually measure inner source? Um I uh [00:48] also make no claim to know everything [00:50] about this. Um but I have some I I do [00:52] claim to be somebody who has uh tried [00:55] again and again and want to share what [00:57] I've learned with you. [01:00] So let's get to it. Can you measure [01:02] inner source? Uh here's the agenda for [01:05] what we'll cover today. Uh we're going [01:07] to talk about the problem of measuring [01:09] inner source. Why it's so hard to solve. [01:12] Uh what's at stake if we don't solve it. [01:15] Uh what we're leaving on the table to [01:17] say it another way. Um the a solution [01:20] framework that I've come up with um that [01:23] uh has uh helped me make some progress. [01:26] uh and also some tricks that I've [01:28] learned uh pitfalls that I think that [01:30] you should avoid and tools. [01:35] So let's start with the problem. Uh when [01:39] I first pitched an inner source program [01:41] at a previous company, I I was able to [01:43] convey the the need for inner source by [01:46] demonstrating how much rework we were [01:48] doing over and over again across [01:50] departments. Uh it wasn't good for [01:52] engineers. It didn't feel good that we [01:54] were doing the same work as other folks. [01:55] and uh didn't feel good uh on the [01:58] management side because uh it it's [02:00] waste. Um it turns out the CEO didn't [02:03] like rework and waste when I talked to [02:05] him about it. So we did get a [02:07] conditional approval which is very [02:08] exciting. The condition was this. Figure [02:12] out how to measure if it's working. [02:15] I walked out of that meeting like I said [02:17] very excited about the approval but also [02:20] terrified. I had no idea how to measure [02:24] if uh inner source was working. Uh what [02:27] was good enough for working? What does [02:29] working look like? Um yeah, any of that [02:32] sound familiar? Um I know that this [02:34] story isn't unique in talking to so many [02:38] um uh different GitHub customers as well [02:40] as uh other um fellow innersource [02:43] advocates here in the inner source [02:45] commons. Um I've heard from many others [02:48] in fact that um that face the exact same [02:51] challenge of uh they think that this is [02:54] a good idea and they would like us to [02:55] measure how successful it uh our steps [02:57] towards it are going to be. Um, so I'd [02:59] like to show you how I've been [03:01] approaching this problem and the [03:03] framework that um, I've been working on [03:05] and how you might be able to use it as [03:07] well to start uh, measuring inner source [03:09] or change uh, the way perhaps that [03:11] you're currently measuring uh, inner [03:13] source um, and thinking about what [03:16] success looks like. [03:19] So [03:21] let me start with a confession. Uh, [03:23] measuring inner source is genuinely [03:27] difficult. um not not just a little bit [03:31] not technically tricky it is genuinely [03:34] complex um and I think that there are [03:37] three core reasons uh why uh first the [03:40] complexity problem of it is uh inner [03:43] source isn't just about code it's about [03:47] culture collaboration knowledge sharing [03:51] uh business outcomes and uh like we've [03:54] heard from uh other talks about all of [03:56] those different aspects Right. Um unlike [04:00] traditional software metrics where you [04:01] might measure uptime which is an exact [04:04] number, error rates which are also an [04:07] exact number uh or maybe bug counts in [04:09] in the software. Um all very [04:12] quantifiable. Um inner source success [04:16] spans across multiple dimensions that [04:18] are not easily quantified sometimes and [04:22] that are also inherently interwoven [04:25] together. [04:27] So second uh is the timing problem. Uh [04:31] measurement of when inner source is [04:34] happening needs to happen with an [04:36] awareness of the current team [04:38] structures. Uh think about like u [04:40] organizational reporting charts some [04:42] folks call them um at at the point in [04:46] time when the contribution is made. Uh [04:48] so let me illustrate to you why that's [04:51] important uh if that's not um something [04:53] that you've ran into before. So imagine [04:55] that uh two people uh are working on the [04:59] same project uh on the same team um uh [05:04] working against something that's on [05:05] their team's road map. Uh product A [05:07] we'll call it. Um that wouldn't [05:10] necessarily be considered inner source [05:12] if we were just creating features on [05:14] product A together both assigned to that [05:16] thing. Um, however, take those same two [05:19] people, put them on different teams [05:22] developing different products in [05:23] different parts of the organization who [05:26] both need to be able to come up with uh [05:29] a button or a widget and and they need [05:32] to maybe use those for two different [05:33] things or on like I said across two [05:35] different products. Um and they [05:37] collaborate together across [05:39] organizational structure to uh develop [05:43] some code that they can both use, [05:44] develop some designs they could both [05:46] use. Um and in reality um those [05:51] individuals uh may change over time, [05:54] right? So uh at the time of contribution [05:57] they may have been on different teams. [05:58] Fast forward 10 years later and they may [06:01] be on the same team. And so if you try [06:03] to reme-measure that inner source, the [06:05] context of where they were in the [06:07] organization is lost. Um so if it wasn't [06:11] clear, that second example was an [06:12] example of what most would consider [06:14] inner source. Um so all that to say that [06:17] measuring inner source activity requires [06:19] context about the team structures at the [06:22] time of contribution. [06:24] And if that's not tricky enough for you [06:27] yet, there is still a third problem that [06:29] makes this difficult. Uh in order to [06:32] measure inner source, we have to define [06:34] the relevant terms. Uh which means you [06:37] need to be clear on what an inner source [06:39] contribution is. Uh thinking about like [06:42] code and documentation. [06:45] Most folks agree that those fit in [06:47] categories that can be inner sourced. Um [06:50] what about when somebody reviews code [06:53] instead of generates code? Is is that [06:55] inner sourcing? Uh maybe they're on a [06:57] different team. What about if somebody [06:59] contributes a thought or has a [07:01] discussion with you? Can that be [07:04] considered an inner source contribution? [07:08] It gets tricky. Uh so defining your [07:10] terms is really important specifically [07:12] with an inner source contribution and [07:14] what that means. Um what can be [07:18] considered an inner source uh project or [07:20] not as well if you're going to start [07:21] labeling uh projects as inner source or [07:24] not. So code bases uh libraries data [07:27] sets uh again there's gray area in these [07:31] definitions in that what about uh using [07:33] the same platform or same tools or same [07:37] configuration for those tools what [07:40] amount of sharing and what subject of [07:42] what you're sharing uh constitutes inner [07:45] source. uh try to be really clear on [07:47] these terms so that we know what to [07:49] include um when we try to measure inner [07:52] source otherwise if we were to measure [07:54] inner source without defining these [07:56] terms we might be measuring smaller or [07:58] larger than someone else's definition so [08:02] extremely important um and then uh not [08:05] only the subject the inner source [08:07] contribution or the inner source project [08:08] that you're talking about but uh what [08:11] about what success looks like right so [08:14] if we're going to start measuring [08:15] something what are we measuring it [08:16] against and and what is the outcome that [08:19] we want? Well, if we want inner source [08:21] practices to be flourishing um and we're [08:24] working to quantify that into numbers, [08:26] um what does good look like? Uh what if [08:29] I told you that uh an inner source um [08:32] project had uh 5% of its contributions [08:37] were from external to the team that owns [08:39] it. They were considered inner source [08:41] contributions uh by one definition. [08:43] Would that be good? Would that be bad? [08:46] What about 50%. [08:48] Or 90%. [08:51] Is there too much? Right? Um, so all of [08:55] these things need to be clearly defined [08:57] and they're best defined in the [08:59] organization that you're trying to [09:00] measure them at. Getting around the [09:02] table with your stakeholders and saying, [09:04] "We want to define these terms because [09:06] it's important that we measure all of [09:08] what we're trying to increase here." [09:13] So if you can come through and get a [09:16] definition, next is what's at stake, [09:20] right? Um we've established that the [09:21] problem's hard to solve. We have to [09:23] define terms and goals. Uh do we still [09:26] want to do this? Is it worth it to [09:27] measure inner source or should we say [09:29] it's unquantifiable? Uh to answer that, [09:32] we need to know what's at stake if we [09:34] don't measure it. Um and this is where [09:37] it gets personal for me. Uh I've worked [09:40] with numerous different organizations in [09:42] in with inner source programs that have [09:45] been cancelled um or significantly [09:48] reduced because uh stakeholders couldn't [09:51] see the the tangible value that was [09:53] being delivered by dedicating resources, [09:57] time, energy, money, funds towards uh [10:01] furthering inner source within the [10:02] organization. [10:04] the problem seemed clear, but it wasn't [10:07] clear that we were solving the problem [10:09] or how well we were solving the problem. [10:12] When you can't demonstrate your return [10:14] on investment, uh to use a common term, [10:17] um programs lose funding and support. Uh [10:21] that's not like a measurement problem [10:22] anymore. That becomes a sustainability [10:25] problem for Inner Source in your [10:26] organization. Um so the good news there [10:30] is good news uh is that I've also [10:33] supported organizations that do measure [10:36] their impact on uh furthering inner [10:38] source practices in their organization [10:40] and that they've also been more [10:42] successful in receiving continued [10:44] funding. They're demonstrating their uh [10:46] return on investment through uh both [10:49] anecdotal stories that they can tell and [10:52] some measured data um in order to have [10:56] an impactful and compelling story um and [10:59] data to back it up. [11:02] So the question that I ask that some of [11:04] you might be asking as well is how do we [11:07] become one of those success stories? How [11:09] do we lean towards um creating [11:12] impactful, compelling uh stories that [11:15] are backed up with data to demonstrate [11:18] enough um for us to dedicate ourselves [11:22] and focus energy on furthering inner [11:24] source in an organization? So, let's [11:27] take a look. [11:30] Um, [11:33] I've built a uh tool uh to help with [11:36] this uh called measure inner source um [11:40] to to attempt to solve the measurement [11:42] problem. Um through using it across [11:45] multiple organizations, I've learned uh [11:47] that successful inner source measurement [11:49] comes down to three core areas. So, let [11:52] me show you what exactly this tool [11:54] measures and how you can use it. [11:59] There are three measurements. Um this [12:02] wouldn't be a conference presentation if [12:04] it weren't three points. So there you [12:05] go. Uh measurement number one is uh [12:08] cross team participation. [12:11] The tool measures cross team [12:12] participation uh which answers the [12:14] fundamental question is inner source [12:17] actually happening. [12:20] The measure inner source tool [12:22] automatically identifies contributors to [12:25] repositories, maps them to teams using [12:28] your organizational structure and then [12:30] categorizes contributions as either from [12:34] the team that uh owns this uh repository [12:38] or external to the team determining that [12:41] or assuming that external contributions [12:44] from outside of the uh nuclear team that [12:47] owns it are considered inner source. Um [12:50] and then it also calculates your inner [12:52] source ratio and then reports all of [12:55] this in a a markdown or or JSON format. [12:58] So um in terms of the inner source ratio [13:01] to be clear that's the percentage of [13:02] work done by external teams. [13:06] So uh for example in one report I [13:09] generated we discovered that while a [13:11] repository had 25 total contributors [13:14] only eight were from the owning team. [13:16] That's a 68% uh participation rate uh of [13:20] inner source participation. So the the [13:23] tool shows exactly who those external [13:24] contributors were and which teams they [13:27] came from demonstrating inner source was [13:30] occurring answering our core question [13:32] for this measurement is inner source [13:33] actually happening. Measurement number [13:35] two is contribution volume. [13:38] The tool gives you a complete picture of [13:40] inner source activity by counting [13:42] commits, code changes and improvements, [13:44] pull requests, feature additions, and [13:46] bug fixes. Uh, as well as issues, bug [13:49] reports, and feature requests. Working [13:52] on expanding it to also GitHub [13:54] discussions. Uh, but it's not there yet. [13:57] Still, still evolving. [14:00] Here's what makes this powerful. The [14:02] tool breaks down contributions by [14:04] individual contributors so that you can [14:06] see patterns. Are your inner source [14:08] contributors making substantial code [14:10] contributions or are they primarily [14:12] filing issues? Are certain external [14:14] teams contributing more than other [14:16] teams? This granular data helps you [14:18] understand the quality, the depth and [14:21] the uniqueness of your cross team [14:24] collaboration. So one example uh is I [14:27] ran this on a repository that showed 23 [14:30] total inner source contributions 15 [14:33] commits five pull requests and three [14:35] issues coming from four different teams [14:38] across the organization. So that tells a [14:41] valuable story about meaningful [14:43] technical contribution rather than [14:45] casual flyby engagement uh because of uh [14:49] those qualities right commits and pull [14:51] requests rather than only issues and [14:54] discussions. [14:56] Uh finally measurement three that's [14:58] cross repository impact. Uh so here's [15:02] where the tool becomes really powerful [15:03] for demonstrating business value. uh run [15:06] it across multiple repositories to [15:08] identify which projects attract the most [15:10] inner source collaboration. These can be [15:13] categorized as your high value shared [15:15] assets from a business perspective. [15:18] Which teams are the most active as inner [15:21] source contributor? These are your [15:24] collaboration champions, right? The ones [15:26] who not only hold the ideals but also um [15:29] collaborate in practice. [15:31] Which repositories have low inner source [15:34] ratios? [15:35] These could be opportunities for [15:37] improvement or uh be identifying [15:41] something that uh serves a single [15:42] purpose uh for a single product and [15:45] that's okay too. Remember this is all [15:47] about how you define your success [15:48] metrics, right? We don't have to have [15:50] everything must be inner source. Um but [15:53] we want to look at uh furthering inner [15:55] source practices and uh increasing the [15:58] quality of our inner source. All of [16:00] these things lead to your return on [16:02] investment calculation that I was [16:04] talking about before. So let's say you [16:06] run the tool on 20 repositories, find an [16:08] average of a 35% inner source ratio that [16:12] that pops out in the report for you. Uh [16:14] that means that 35% of your development [16:16] work is being shared across team [16:18] boundaries. Woohoo. That's great. Uh [16:21] that's reduced duplication, shared [16:23] maintenance costs, and faster innovation [16:25] through that collaboration. Imagine [16:28] taking that number down to zero. [16:32] That's demonstrating no inner source, no [16:34] shared assets. [16:37] That means that uh that illustrates that [16:39] we're duplicating effort and wasting [16:40] resources if we believe that there is [16:43] anything that could be shared. [16:45] So now before I show you how to [16:47] implement this, let me save you some uh [16:50] from some of the mistakes that I see um [16:53] organizations make all the time. And by [16:55] that I mean uh you know mistakes that [16:58] I've made in the past. [17:01] So some common pitfalls what not to do. [17:04] Um so the three most dangerous uh [17:07] measurement mistakes that um I've seen [17:09] others make and chiefly I've made myself [17:11] is um the first one I call it gitis. Um [17:16] uh folks saying uh oh we have 10,000 [17:18] commits this quarter. inner sources [17:20] working um and uh git activity is um not [17:26] always able to be uh categorized or [17:29] comprehensive on its own. It's also a [17:31] lagging indicator at best can often be [17:34] misleading. High commit volume might [17:37] indicate um inefficient development [17:39] practices or or even unhealthy uh [17:43] collaboration if somebody is um you know [17:46] let's say changing something um [17:48] twiddling bits some folks call it um and [17:52] trying to get something to work on their [17:53] own without seeking collaboration with [17:55] somebody else right just one example [17:59] um mistake number two is vanity metrics [18:02] so being able to say the total number of [18:04] reposit tories or the total contributors [18:07] without context. Uh that's like [18:10] measuring your company's success by [18:11] counting how many meetings you have. [18:13] It's not tied directly to the meaningful [18:16] outcomes. And then mistake number three [18:19] is uh the the measurement death spiral. [18:22] Overmeasuring and then overmeasuring and [18:25] overmeasuring and and underacting. Um, I [18:28] I've been guilty of this before that by [18:30] the time I get the measurements I'm [18:31] looking for for the month that it's time [18:33] to take the next measurements for the [18:34] next month to see how they've changed. [18:36] Um, and and never setting aside time to [18:38] act. [18:41] So um I have one question that I ask [18:44] myself about metrics when I'm collecting [18:45] them um as sort of a litmus test and [18:49] that's um based on the data that we've [18:51] collected what should we do differently [18:53] next month and then do that um and then [18:56] measure again and if that means that [18:58] you're skipping months or weeks or [19:00] however whatever your cadence for [19:02] measuring is that's fine. Um we want to [19:05] measure to see if we've improved not [19:06] just to measure to measure. Um so um let [19:11] me talk a little bit more about the um [19:13] the tool and how you could implement it [19:16] as well. [19:18] Uh so the tool like I said is called [19:19] Measure Innersource. It's at [19:21] github.com/githubmeasure-inource. [19:25] It's a mouthful. Um so but that's what [19:28] it is. It's completely free and open [19:30] source. It can either run locally or as [19:32] a GitHub action. Um it can um [19:36] automatically calculate those inner [19:38] source ratios that we've been talking [19:39] about and generates that markdown [19:41] report. Um it also uh handles large [19:44] repositories as well. If you're [19:45] concerned about oh we have a we have a [19:47] monor repo or something like that. Um [19:49] there's some cool algorithms in there [19:51] that that help with that. Um so what you [19:54] need to get started essentially is a [19:57] file mapping your team's structure. Uh [20:00] if that sounds daunting, if you're a [20:01] part of a large organization, don't [20:02] worry. You don't have to um like write [20:05] down who everyone's manager is across [20:07] the whole organization and get that in [20:09] into the tool. Um you can start with [20:12] just the uh contributors that show up in [20:14] a repository and work from there. [20:16] There's also uh another path path which [20:18] is um taking your data out of your [20:20] people management software like workday [20:23] or uh or Microsoft and um getting that [20:26] into a JSON format that represents your [20:28] organizational structure at the time. [20:30] Then the tool can use that to see where [20:32] folks uh land on which team they land. [20:35] So it can detect um if inner source is [20:38] occurring or not. So essentially uh you [20:41] could sit down in 30 minutes, put [20:43] together a quick um uh the files called [20:46] the org data or organizational [20:48] data.json. [20:49] Uh put that together uh copy the open [20:52] source workflow file or download the [20:55] repository and run it locally and then [20:58] uh you're off to the races. You'll uh [21:00] it'll generate your first um uh report [21:03] inner source report for you. [21:06] So before you leave today, uh I want to [21:08] leave you with two questions and a [21:10] challenge. [21:17] Would running this tool and getting this [21:18] information um help uh convey the value [21:22] of inner source of your inner source [21:23] program that you're running? Would [21:26] measuring inner source in this way help [21:27] you make better decisions to improve [21:29] your inner source program? If either of [21:31] those are yes, then my challenge is find [21:34] some time on your calendar to take a [21:36] look at the tool or another tool that [21:38] you found. Um, and I challenge you that [21:41] because um, it getting started for me [21:43] was um, almost as hard as creating this [21:46] tool. Um, and I know that it can be for [21:48] others. Um, so wanted to leave you with [21:51] that challenge. And [21:55] here's the bottom line. What I want you [21:56] to remember uh the problem of measuring [21:58] inner source is genuinely hard. It [22:00] requires defining terms, understanding [22:02] context, focusing on meaningful [22:04] outcomes, but it's absolutely worth the [22:07] effort to demonstrate and increase the [22:10] value of the inner source that's [22:12] happening in your practice. Uh inner [22:14] source measurement isn't about agreeing [22:16] on ideals. It's about demonstrating the [22:20] values uh or the the value exchange that [22:22] occurs. [22:24] So, I appreciate you guys for being [22:26] here. Uh, thank you for giving me your [22:28] precious time. Um, I'd love to hear [22:30] about your inner source measurement [22:31] journeys and attempt to answer any [22:33] questions that you might have.  ## 📺 Video ![](https://www.youtube.com/watch?v=ao9JLzX7yK4) --- # 📝 Full Notes (Deep-Read Analysis) # 🏷️ Title: Can you measure InnerSource? - Zack Koppert ## 🧰 Action Inbox - [H1] Define inner source terms clearly with stakeholders — establish what constitutes contributions, projects, and success #measurement - [H2] Try the Measure Innersource tool on GitHub repositories — github.com/github/measure-innersource #tools - [H3] Create org data JSON mapping team structures to enable measurement — 30-minute setup process #implementation ## 🧭 Table of Contents - [📝 Summary](#-summary) - [📖 Main Content](#-main-content) - [🗒️ Notes](#️-notes) - [📚 References](#-references) - [🗣️ Paraphrase](#️-paraphrase) - [📰 Newsletter](#-newsletter) - [✅ Action Items](#-action-items) - [🧪 Hypotheses](#-hypotheses) - [📊 Methodology](#-methodology) - [❓ Questions](#-questions) - [🔮 Future Research](#-future-research) - [💡 Implications](#-implications) - [🧠 Learning Styles Assessment](#-learning-styles-assessment) - [📊 Accessibility Assessment](#-accessibility-assessment) - [🌍 DEIB Assessment](#-deib-assessment) - [✅ Logical Validity](#-logical-validity) - [🛡️ DARVO Analysis](#-darvo-analysis) - [🎯 Conclusion](#-conclusion) ### 🔗 References - [Measure Innersource Tool](https://github.com/github/measure-innersource) - [InnerSource Commons](https://innersourcecommons.org) ## 📝 Summary - GitHub's Zack Koppert presents a framework for measuring inner source programs after receiving conditional CEO approval requiring measurable outcomes - Why it matters: Inner source programs frequently get cancelled when unable to demonstrate ROI and tangible value to stakeholders - Key stakeholders: Engineering managers, open source program offices, executives approving inner source initiatives - Risks and tradeoffs: Complex measurement spanning code, culture, collaboration; requires defining unclear terms; timing sensitivity with organizational changes - Bottom line takeaway: Three-part measurement framework (cross-team participation, contribution volume, cross-repository impact) with open source tooling can demonstrate inner source ROI ## 📖 Main Content Inner source measurement faces genuine complexity beyond typical software metrics like uptime or error rates. The challenge spans multiple dimensions: determining what constitutes inner source contributions (code reviews vs. discussions), tracking organizational context at contribution time (same people on different teams vs. same team), and defining success metrics (is 5% external contribution good or bad?). Koppert argues measurement is essential for program sustainability, having witnessed multiple inner source programs cancelled due to inability to demonstrate value. Organizations that successfully measure impact receive continued funding through compelling stories backed by data. The solution involves three core measurements: cross-team participation (is inner source happening?), contribution volume (commits, PRs, issues by external teams), and cross-repository impact (identifying high-value shared assets and collaboration champions). This enables ROI calculations - for example, 35% inner source ratio means 35% of development work shared across team boundaries, reducing duplication and maintenance costs. ## 🗒️ Notes #### ⏱️ [00:00](https://youtu.be/ao9JLzX7yK4?t=0) — [H1] Introduction and experience background - Key idea: Senior engineering manager at GitHub with extensive inner source program experience - Evidence: "I've worked with many inner source programs at previous jobs, either starting at them or working at them" #### ⏱️ [01:30](https://youtu.be/ao9JLzX7yK4?t=90) — [H2] The measurement challenge from CEO approval - Key idea: Got conditional approval for inner source program requiring measurable success metrics - Evidence: "Figure out how to measure if it's working. I walked out of that meeting like I said very excited about the approval but also terrified" #### ⏱️ [03:45](https://youtu.be/ao9JLzX7yK4?t=225) — [H1] Three core problems making measurement difficult - Key idea: Complexity, timing, and definition problems create genuine measurement challenges - Evidence: "measuring inner source is genuinely difficult...not just a little bit not technically tricky it is genuinely complex" #### ⏱️ [04:15](https://youtu.be/ao9JLzX7yK4?t=255) — [H2] Complexity problem - spans multiple dimensions - Key idea: Inner source encompasses code, culture, collaboration, knowledge sharing, and business outcomes - Evidence: "inner source isn't just about code it's about culture collaboration knowledge sharing business outcomes" #### ⏱️ [05:30](https://youtu.be/ao9JLzX7yK4?t=330) — [H2] Timing problem - organizational context matters - Key idea: Must measure with awareness of team structures at time of contribution - Evidence: "measurement of when inner source is happening needs to happen with an awareness of the current team structures" #### ⏱️ [07:00](https://youtu.be/ao9JLzX7yK4?t=420) — [H2] Definition problem - unclear terms need clarification - Key idea: Must define what constitutes inner source contributions, projects, and success - Evidence: "what about when somebody reviews code instead of generates code? Is that inner sourcing?" #### ⏱️ [09:30](https://youtu.be/ao9JLzX7yK4?t=570) — [H1] Stakes of not measuring - program cancellation - Key idea: Programs lose funding when unable to demonstrate ROI and tangible value - Evidence: "I've worked with numerous different organizations...that have been cancelled...because stakeholders couldn't see the tangible value" #### ⏱️ [11:00](https://youtu.be/ao9JLzX7yK4?t=660) — [H1] Three-part measurement framework introduction - Key idea: Successful measurement comes down to three core areas through practical tooling - Evidence: "I've built a tool...called measure inner source...successful inner source measurement comes down to three core areas" #### ⏱️ [11:30](https://youtu.be/ao9JLzX7yK4?t=690) — [H2] Measurement 1 - Cross team participation - Key idea: Answers fundamental question "is inner source actually happening" - Evidence: "automatically identifies contributors to repositories, maps them to teams using your organizational structure" #### ⏱️ [13:15](https://youtu.be/ao9JLzX7yK4?t=795) — [H2] Measurement 2 - Contribution volume - Key idea: Complete picture through counting commits, pull requests, and issues - Evidence: "The tool breaks down contributions by individual contributors so that you can see patterns" #### ⏱️ [15:00](https://youtu.be/ao9JLzX7yK4?t=900) — [H2] Measurement 3 - Cross repository impact - Key idea: Identifies high-value shared assets and collaboration champions for business value - Evidence: "run it across multiple repositories to identify which projects attract the most inner source collaboration" #### ⏱️ [17:00](https://youtu.be/ao9JLzX7yK4?t=1020) — [H1] Common measurement pitfalls to avoid - Key idea: Three dangerous mistakes - gititis, vanity metrics, and measurement death spiral - Evidence: "High commit volume might indicate inefficient development practices or even unhealthy collaboration" #### ⏱️ [19:00](https://youtu.be/ao9JLzX7yK4?t=1140) — [H2] Tool implementation - getting started - Key idea: Open source tool at github.com/github/measure-innersource with 30-minute setup - Evidence: "You could sit down in 30 minutes, put together a quick...organizational data.json" #### ⏱️ [20:30](https://youtu.be/ao9JLzX7yK4?t=1230) — [H1] Challenge and conclusion - Key idea: Inner source measurement demonstrates value exchange rather than just agreeing on ideals - Evidence: "inner source measurement isn't about agreeing on ideals. It's about demonstrating the values or the value exchange that occurs" ## 📚 References - [Measure Innersource GitHub Repository](https://github.com/github/measure-innersource) - [InnerSource Commons Community](https://innersourcecommons.org) - [GitHub Inner Source Resources](https://github.com) ## 🗣️ Paraphrase - Inner source programs need measurable outcomes to survive executive scrutiny and continued funding, but measurement is genuinely complex due to cultural and organizational factors beyond simple code metrics - A three-part framework measuring cross-team participation, contribution volume, and cross-repository impact can demonstrate ROI by showing percentage of work shared across team boundaries - Common measurement mistakes include focusing on raw git activity, vanity metrics without context, and over-measuring without taking action based on insights ## 📰 Newsletter **Measuring Inner Source Success: A Framework for Demonstrating ROI** Ever struggled to show executives the value of your inner source program? GitHub's Zack Koppert faced this exact challenge when his CEO granted conditional approval: "Figure out how to measure if it's working." **The Challenge**: Inner source spans code, culture, collaboration, and business outcomes - making it far more complex to measure than traditional software metrics like uptime or error rates. **The Solution**: A three-part measurement framework: 1. **Cross-team participation** - Is inner source actually happening? 2. **Contribution volume** - What's the depth and quality of collaboration? 3. **Cross-repository impact** - Which projects are high-value shared assets? **Practical Tool**: The open-source [Measure Innersource](https://github.com/github/measure-innersource) tool automates these calculations, requiring just a 30-minute setup with your organizational structure data. **Key Insight**: Organizations that measure inner source impact receive continued funding through compelling stories backed by data. A 35% inner source ratio means 35% of development work is shared across team boundaries - directly demonstrating reduced duplication and waste. Resources: [Inner Source Commons](https://innersourcecommons.org/), [GitHub Inner Source Guide](https://github.com), [Measure Innersource Tool](https://github.com/github/measure-innersource) ## ✅ Action Items #### OSPO - Policy & Governance: Define inner source contribution types, project categories, and success metrics with stakeholders - Software Inventory: Map organizational team structures to enable cross-team contribution tracking - Developer Support: Implement Measure Innersource tool for quarterly ROI reporting to executives - Community Engagement: Identify collaboration champions and high-value shared assets through cross-repository analysis - Strategic Alignment: Create KPIs based on inner source ratio targets and contribution volume goals #### DevRel - Developer advocacy: Use inner source measurement data in conference talks and case studies - Education: Create workshops on inner source measurement methodology and tooling - Developer experience: Improve documentation for repositories with low inner source ratios - Developer marketing: Highlight collaboration champions and shared asset success stories - Community management: Foster cross-team participation through measured feedback loops #### Community - Maintainers: Support projects identified as high-value shared assets with governance templates - Contributors: Create onboarding programs targeting teams with low inner source participation - Sponsors: Provide transparent ROI reports showing percentage of work shared across boundaries - Ecosystem: Coordinate measurement standards across inner source programs ## 🧪 Hypotheses - Organizations measuring inner source impact receive more sustainable funding than those relying solely on anecdotal evidence - Inner source ratios above 30% indicate meaningful cross-team collaboration and reduced development duplication - Cross-repository measurement reveals high-value shared assets more effectively than single-project analysis - Teams contributing to external repositories become collaboration champions spreading inner source practices ### 📊 Methodology - Longitudinal study comparing funding retention rates between measured vs unmeasured inner source programs - Cross-organizational analysis of inner source ratios and their correlation with development efficiency metrics - Qualitative research on executive decision-making factors for inner source program continuation ### ❓ Questions - What inner source ratio threshold indicates genuine cross-team collaboration versus casual engagement? - How do organizational structure changes impact the accuracy of historical inner source measurements? - Which contribution types (commits, reviews, issues) best predict long-term inner source program success? ### 🔮 Future Research - Automated organizational structure mapping integration with HR systems for real-time measurement - Machine learning models predicting inner source program sustainability based on early measurement patterns - Cross-industry benchmarking of inner source ratios and their correlation with business outcomes ### 💡 Implications - Practice: Engineering teams need clear definitions of inner source contributions before implementing measurement - Policy: Organizations should establish inner source ratio targets and regular measurement cadences for executive reporting - Security: Cross-team contribution tracking requires careful handling of organizational structure data and access controls - Education: Inner source advocates need training on ROI calculation and executive communication of measurement results - Funding: Programs demonstrating measurable impact through data-backed stories receive more sustainable resource allocation ### 🧠 Learning Styles Assessment Reading level estimate: 12th grade (technical presentation with business context) #### 👀 Visual learners - Present: Framework diagrams, measurement examples, tool screenshots in presentation - Improve: Organizational chart examples, inner source ratio visualizations, ROI calculation flowcharts #### 🔊 Aural learners - Present: Full conference presentation video with clear narration - Improve: Podcast-style summary discussing measurement challenges and solutions #### 🗣️ Verbal learners - Present: Clear definitions of inner source terms, concrete examples, analogies to business metrics - Improve: Glossary of measurement terminology, story-driven case studies #### 🧩 Kinesthetic learners - Present: Hands-on tool implementation, 30-minute setup process, GitHub action workflow - Improve: Step-by-step measurement checklist, interactive organizational mapping exercise #### 🧠 Logical learners - Present: Three-part framework structure, cause-and-effect relationships, ROI calculations - Improve: Decision trees for measurement implementation, measurement validity criteria #### 🧑‍🤝‍🧑 Social learners - Present: Conference presentation format, community discussion opportunities - Improve: Cohort implementation guides, peer review of measurement definitions #### 🧘‍♀️ Solitary learners - Present: Open-source tool for independent implementation, self-paced setup process - Improve: Reflection prompts on organizational measurement needs, self-assessment worksheets #### 🌿 Naturalistic learners - Present: Ecosystem thinking about organizational collaboration - Improve: Organic growth analogies, collaboration network analysis ### 📊 Accessibility Assessment #### Screen reader semantics - Present: Clear heading structure, descriptive link text in references - Improve: Alt text for measurement framework diagrams, table headers for data examples #### Keyboard navigation - Present: Standard video controls, logical tab order in referenced tools - Improve: Skip links for long transcript sections, keyboard shortcuts for timestamp navigation #### Contrast and readability - Present: High contrast text, readable font sizes in presentation - Improve: Ensure 4.5:1 minimum contrast in measurement visualization examples #### Text resizing and reflow - Present: Responsive formatting for various screen sizes - Improve: Prevent clipping of measurement data tables at 200% zoom #### Media and captions - Present: Full transcript provided, clear audio quality - Improve: Downloadable transcript, audio descriptions for visual elements #### Forms and interactions - Present: Clear GitHub repository links, accessible tool interfaces - Improve: Form labels in organizational data JSON examples ### 🌍 DEIB Assessment #### Cultural breadth - Present: GitHub-focused examples, enterprise organizational context - Improve: Multi-industry case studies, different organizational structure examples #### Inclusive language - Present: Gender-neutral terms, defined technical jargon - Improve: Plain-language summaries of measurement concepts, avoid business idioms #### Neurodiversity support - Present: Chunked content with clear sections, consistent structure - Improve: Appropriate heading frequency, summary boxes for key concepts #### Representation - Present: Focus on technical content over individual representation - Improve: Diverse organizational examples, varied team structure scenarios #### Psychological safety - Present: Respectful framing of measurement challenges, non-judgmental tone - Improve: Content warnings for program cancellation discussions ### ✅ Logical Validity #### Strong reasoning - Deductive strength: Framework --- # 📄 Full Transcript *Auto-generated from YouTube captions · timestamps roughly every 30 seconds · [watch on YouTube](https://www.youtube.com/watch?v=ao9JLzX7yK4)* **[00:06]** Hello everyone. Uh, as Yuki said, uh, my name is Zach Copern, a senior software, uh, engineering manager at GitHub. Um, I've worked with many inner source programs at previous jobs, either starting at them or working at them. Um, also, uh, currently at GitHub um, working on inner source things as well as with many GitHub customers. Um, so want to share that experience with you. so that you know uh what perspective I'm talking from today. Um and speaking of today, I wanted to **[00:38]** talk with you about the question uh that has puzzled myself and many others in the inner source community. Uh can you actually measure inner source? Um I uh also make no claim to know everything about this. Um but I have some I I do claim to be somebody who has uh tried again and again and want to share what I've learned with you. So let's get to it. Can you measure inner source? Uh here's the agenda for what we'll cover today. Uh we're going to talk about the problem of measuring **[01:09]** inner source. Why it's so hard to solve. Uh what's at stake if we don't solve it. Uh what we're leaving on the table to say it another way. Um the a solution framework that I've come up with um that uh has uh helped me make some progress. uh and also some tricks that I've learned uh pitfalls that I think that you should avoid and tools. So let's start with the problem. Uh when **[01:39]** I first pitched an inner source program at a previous company, I I was able to convey the the need for inner source by demonstrating how much rework we were doing over and over again across departments. Uh it wasn't good for engineers. It didn't feel good that we were doing the same work as other folks. and uh didn't feel good uh on the management side because uh it it's waste. Um it turns out the CEO didn't like rework and waste when I talked to him about it. So we did get a conditional approval which is very exciting. The condition was this. Figure **[02:12]** out how to measure if it's working. I walked out of that meeting like I said very excited about the approval but also terrified. I had no idea how to measure if uh inner source was working. Uh what was good enough for working? What does working look like? Um yeah, any of that sound familiar? Um I know that this story isn't unique in talking to so many um uh different GitHub customers as well as uh other um fellow innersource **[02:43]** advocates here in the inner source commons. Um I've heard from many others in fact that um that face the exact same challenge of uh they think that this is a good idea and they would like us to measure how successful it uh our steps towards it are going to be. Um, so I'd like to show you how I've been approaching this problem and the framework that um, I've been working on and how you might be able to use it as well to start uh, measuring inner source or change uh, the way perhaps that you're currently measuring uh, inner **[03:13]** source um, and thinking about what success looks like. So let me start with a confession. Uh, measuring inner source is genuinely difficult. um not not just a little bit not technically tricky it is genuinely complex um and I think that there are three core reasons uh why uh first the complexity problem of it is uh inner **[03:43]** source isn't just about code it's about culture collaboration knowledge sharing uh business outcomes and uh like we've heard from uh other talks about all of those different aspects Right. Um unlike traditional software metrics where you might measure uptime which is an exact number, error rates which are also an exact number uh or maybe bug counts in in the software. Um all very quantifiable. Um inner source success **[04:16]** spans across multiple dimensions that are not easily quantified sometimes and that are also inherently interwoven together. So second uh is the timing problem. Uh measurement of when inner source is happening needs to happen with an awareness of the current team structures. Uh think about like u organizational reporting charts some folks call them um at at the point in time when the contribution is made. Uh **[04:48]** so let me illustrate to you why that's important uh if that's not um something that you've ran into before. So imagine that uh two people uh are working on the same project uh on the same team um uh working against something that's on their team's road map. Uh product A we'll call it. Um that wouldn't necessarily be considered inner source if we were just creating features on product A together both assigned to that thing. Um, however, take those same two **[05:19]** people, put them on different teams developing different products in different parts of the organization who both need to be able to come up with uh a button or a widget and and they need to maybe use those for two different things or on like I said across two different products. Um and they collaborate together across organizational structure to uh develop some code that they can both use, develop some designs they could both use. Um and in reality um those **[05:51]** individuals uh may change over time, right? So uh at the time of contribution they may have been on different teams. Fast forward 10 years later and they may be on the same team. And so if you try to reme-measure that inner source, the context of where they were in the organization is lost. Um so if it wasn't clear, that second example was an example of what most would consider inner source. Um so all that to say that measuring inner source activity requires context about the team structures at the **[06:22]** time of contribution. And if that's not tricky enough for you yet, there is still a third problem that makes this difficult. Uh in order to measure inner source, we have to define the relevant terms. Uh which means you need to be clear on what an inner source contribution is. Uh thinking about like code and documentation. Most folks agree that those fit in categories that can be inner sourced. Um what about when somebody reviews code **[06:53]** instead of generates code? Is is that inner sourcing? Uh maybe they're on a different team. What about if somebody contributes a thought or has a discussion with you? Can that be considered an inner source contribution? It gets tricky. Uh so defining your terms is really important specifically with an inner source contribution and what that means. Um what can be considered an inner source uh project or not as well if you're going to start labeling uh projects as inner source or **[07:24]** not. So code bases uh libraries data sets uh again there's gray area in these definitions in that what about uh using the same platform or same tools or same configuration for those tools what amount of sharing and what subject of what you're sharing uh constitutes inner source. uh try to be really clear on these terms so that we know what to include um when we try to measure inner source otherwise if we were to measure **[07:54]** inner source without defining these terms we might be measuring smaller or larger than someone else's definition so extremely important um and then uh not only the subject the inner source contribution or the inner source project that you're talking about but uh what about what success looks like right so if we're going to start measuring something what are we measuring it against and and what is the outcome that we want? Well, if we want inner source practices to be flourishing um and we're working to quantify that into numbers, **[08:26]** um what does good look like? Uh what if I told you that uh an inner source um project had uh 5% of its contributions were from external to the team that owns it. They were considered inner source contributions uh by one definition. Would that be good? Would that be bad? What about 50%. Or 90%. Is there too much? Right? Um, so all of these things need to be clearly defined **[08:57]** and they're best defined in the organization that you're trying to measure them at. Getting around the table with your stakeholders and saying, "We want to define these terms because it's important that we measure all of what we're trying to increase here." So if you can come through and get a definition, next is what's at stake, right? Um we've established that the problem's hard to solve. We have to define terms and goals. Uh do we still want to do this? Is it worth it to **[09:27]** measure inner source or should we say it's unquantifiable? Uh to answer that, we need to know what's at stake if we don't measure it. Um and this is where it gets personal for me. Uh I've worked with numerous different organizations in in with inner source programs that have been cancelled um or significantly reduced because uh stakeholders couldn't see the the tangible value that was being delivered by dedicating resources, time, energy, money, funds towards uh **[10:01]** furthering inner source within the organization. the problem seemed clear, but it wasn't clear that we were solving the problem or how well we were solving the problem. When you can't demonstrate your return on investment, uh to use a common term, um programs lose funding and support. Uh that's not like a measurement problem anymore. That becomes a sustainability problem for Inner Source in your organization. Um so the good news there is good news uh is that I've also **[10:33]** supported organizations that do measure their impact on uh furthering inner source practices in their organization and that they've also been more successful in receiving continued funding. They're demonstrating their uh return on investment through uh both anecdotal stories that they can tell and some measured data um in order to have an impactful and compelling story um and data to back it up. So the question that I ask that some of **[11:04]** you might be asking as well is how do we become one of those success stories? How do we lean towards um creating impactful, compelling uh stories that are backed up with data to demonstrate enough um for us to dedicate ourselves and focus energy on furthering inner source in an organization? So, let's take a look. Um, I've built a uh tool uh to help with **[11:36]** this uh called measure inner source um to to attempt to solve the measurement problem. Um through using it across multiple organizations, I've learned uh that successful inner source measurement comes down to three core areas. So, let me show you what exactly this tool measures and how you can use it. There are three measurements. Um this wouldn't be a conference presentation if it weren't three points. So there you go. Uh measurement number one is uh **[12:08]** cross team participation. The tool measures cross team participation uh which answers the fundamental question is inner source actually happening. The measure inner source tool automatically identifies contributors to repositories, maps them to teams using your organizational structure and then categorizes contributions as either from the team that uh owns this uh repository **[12:38]** or external to the team determining that or assuming that external contributions from outside of the uh nuclear team that owns it are considered inner source. Um and then it also calculates your inner source ratio and then reports all of this in a a markdown or or JSON format. So um in terms of the inner source ratio to be clear that's the percentage of work done by external teams. So uh for example in one report I **[13:09]** generated we discovered that while a repository had 25 total contributors only eight were from the owning team. That's a 68% uh participation rate uh of inner source participation. So the the tool shows exactly who those external contributors were and which teams they came from demonstrating inner source was occurring answering our core question for this measurement is inner source actually happening. Measurement number two is contribution volume. The tool gives you a complete picture of **[13:40]** inner source activity by counting commits, code changes and improvements, pull requests, feature additions, and bug fixes. Uh, as well as issues, bug reports, and feature requests. Working on expanding it to also GitHub discussions. Uh, but it's not there yet. Still, still evolving. Here's what makes this powerful. The tool breaks down contributions by individual contributors so that you can see patterns. Are your inner source contributors making substantial code contributions or are they primarily **[14:12]** filing issues? Are certain external teams contributing more than other teams? This granular data helps you understand the quality, the depth and the uniqueness of your cross team collaboration. So one example uh is I ran this on a repository that showed 23 total inner source contributions 15 commits five pull requests and three issues coming from four different teams across the organization. So that tells a valuable story about meaningful **[14:43]** technical contribution rather than casual flyby engagement uh because of uh those qualities right commits and pull requests rather than only issues and discussions. Uh finally measurement three that's cross repository impact. Uh so here's where the tool becomes really powerful for demonstrating business value. uh run it across multiple repositories to identify which projects attract the most inner source collaboration. These can be **[15:13]** categorized as your high value shared assets from a business perspective. Which teams are the most active as inner source contributor? These are your collaboration champions, right? The ones who not only hold the ideals but also um collaborate in practice. Which repositories have low inner source ratios? These could be opportunities for improvement or uh be identifying something that uh serves a single purpose uh for a single product and **[15:45]** that's okay too. Remember this is all about how you define your success metrics, right? We don't have to have everything must be inner source. Um but we want to look at uh furthering inner source practices and uh increasing the quality of our inner source. All of these things lead to your return on investment calculation that I was talking about before. So let's say you run the tool on 20 repositories, find an average of a 35% inner source ratio that that pops out in the report for you. Uh that means that 35% of your development **[16:16]** work is being shared across team boundaries. Woohoo. That's great. Uh that's reduced duplication, shared maintenance costs, and faster innovation through that collaboration. Imagine taking that number down to zero. That's demonstrating no inner source, no shared assets. That means that uh that illustrates that we're duplicating effort and wasting resources if we believe that there is anything that could be shared. So now before I show you how to **[16:47]** implement this, let me save you some uh from some of the mistakes that I see um organizations make all the time. And by that I mean uh you know mistakes that I've made in the past. So some common pitfalls what not to do. Um so the three most dangerous uh measurement mistakes that um I've seen others make and chiefly I've made myself is um the first one I call it gitis. Um uh folks saying uh oh we have 10,000 **[17:18]** commits this quarter. inner sources working um and uh git activity is um not always able to be uh categorized or comprehensive on its own. It's also a lagging indicator at best can often be misleading. High commit volume might indicate um inefficient development practices or or even unhealthy uh collaboration if somebody is um you know let's say changing something um **[17:48]** twiddling bits some folks call it um and trying to get something to work on their own without seeking collaboration with somebody else right just one example um mistake number two is vanity metrics so being able to say the total number of reposit tories or the total contributors without context. Uh that's like measuring your company's success by counting how many meetings you have. It's not tied directly to the meaningful outcomes. And then mistake number three **[18:19]** is uh the the measurement death spiral. Overmeasuring and then overmeasuring and overmeasuring and and underacting. Um, I I've been guilty of this before that by the time I get the measurements I'm looking for for the month that it's time to take the next measurements for the next month to see how they've changed. Um, and and never setting aside time to act. So um I have one question that I ask myself about metrics when I'm collecting them um as sort of a litmus test and that's um based on the data that we've **[18:51]** collected what should we do differently next month and then do that um and then measure again and if that means that you're skipping months or weeks or however whatever your cadence for measuring is that's fine. Um we want to measure to see if we've improved not just to measure to measure. Um so um let me talk a little bit more about the um the tool and how you could implement it as well. Uh so the tool like I said is called Measure Innersource. It's at github.com/githubmeasure-inource. **[19:25]** It's a mouthful. Um so but that's what it is. It's completely free and open source. It can either run locally or as a GitHub action. Um it can um automatically calculate those inner source ratios that we've been talking about and generates that markdown report. Um it also uh handles large repositories as well. If you're concerned about oh we have a we have a monor repo or something like that. Um there's some cool algorithms in there that that help with that. Um so what you need to get started essentially is a **[19:57]** file mapping your team's structure. Uh if that sounds daunting, if you're a part of a large organization, don't worry. You don't have to um like write down who everyone's manager is across the whole organization and get that in into the tool. Um you can start with just the uh contributors that show up in a repository and work from there. There's also uh another path path which is um taking your data out of your people management software like workday or uh or Microsoft and um getting that into a JSON format that represents your **[20:28]** organizational structure at the time. Then the tool can use that to see where folks uh land on which team they land. So it can detect um if inner source is occurring or not. So essentially uh you could sit down in 30 minutes, put together a quick um uh the files called the org data or organizational data.json. Uh put that together uh copy the open source workflow file or download the repository and run it locally and then uh you're off to the races. You'll uh **[21:00]** it'll generate your first um uh report inner source report for you. So before you leave today, uh I want to leave you with two questions and a challenge. Would running this tool and getting this information um help uh convey the value of inner source of your inner source program that you're running? Would measuring inner source in this way help you make better decisions to improve your inner source program? If either of **[21:31]** those are yes, then my challenge is find some time on your calendar to take a look at the tool or another tool that you found. Um, and I challenge you that because um, it getting started for me was um, almost as hard as creating this tool. Um, and I know that it can be for others. Um, so wanted to leave you with that challenge. And here's the bottom line. What I want you to remember uh the problem of measuring inner source is genuinely hard. It requires defining terms, understanding **[22:02]** context, focusing on meaningful outcomes, but it's absolutely worth the effort to demonstrate and increase the value of the inner source that's happening in your practice. Uh inner source measurement isn't about agreeing on ideals. It's about demonstrating the values uh or the the value exchange that occurs. So, I appreciate you guys for being here. Uh, thank you for giving me your precious time. Um, I'd love to hear about your inner source measurement journeys and attempt to answer any **[22:33]** questions that you might have.