ANTHOLOGY — Maintaining Maintainers  ## Metadata - Author: [[The Changelog: Software Development, Open Source]] - Full Title: ANTHOLOGY — Maintaining Maintainers - Category: #podcasts - URL: https://share.snipd.com/episode/61c25ff2-bcf8-4b62-98f1-75f0e62967ca ## Document Notes ## Highlights - "Chaos": Community Health Analytics for Open Source Software Chaos is a community health analytics for open source software. The project has two pieces of software within the chaos projects: Auger and GrimarLab. "I like to look at trends over time," says co-founder Chris Lister. Transcript: Jerod Santo Help me out with the rest. Dawn Foster Community health analytics for open source software. So chaos with two S's. Jerod Santo Two S's. Chaos. Chaos. Chaos. Dawn Foster Yeah, so we basically, I'll give you an overview of what chaos is. We are a project and we're focused on kind of two things. We're focused on metrics, so defining metrics so that we can be, when we talk about a certain metric, that we can be consistent about what it is and have a definition that we can point people To. And we say, when we're talking about, you know, numbers of lines of code, that's what this means. If we're talking about, you know, the bus factor, which is, you know, how many people you have contributing to a project, that we measure that kind of the same way. So we do metrics definitions, and then we do software. So we have two pieces of software within the chaos projects. We have Augur and Grimoire Lab, and those are both, they're basically software projects that go out and they gather a bunch of data from various sources. So GitHub, obviously, Slack, other things that you can, basically anything with an API that you can get access to the data from, and allow people to analyze that using software. Very cool. Jerod Santo So do you have some sort of a score, or how do you quantify health? Dawn Foster That is an excellent question. Thank you. No, we don't have a score. Okay. And I am anti- health scores. Okay. So what, what I like to look at when I'm looking at project health is I like to look at trends. So, you know, are you closing more of your pull requests? Is your pull request backlog getting bigger or smaller? Are you responding to pull requests and issues more, more quickly, or is it taking you more time? So I like to look at trends over time. And I like to look at metrics in the context of projects because individual projects have certain ways of working and certain things that impact the metrics. And unless you're part of the project, you don't know. So, for example, if I work on a project and it's, we're cutting a huge release that has a bunch of breaking changes, there's probably going to be some weird things in the metrics associated With that. So, pull requests are going to get in the backlog while you get everything together for the release, for example. I was talking to a friend at Google, Sophia Vargas, and she does a lot of analysis on things like Kubernetes. And some of the metrics that she was looking at it made just no sense because the way Kubernetes works is you've got bots that do all the things, right? So like, you have bots that respond to things automatically, the bots close the issues automatically after a certain amount, you know, they go stale, they close them. So there's all this like bot activity that she was looking at data and she's like, this makes no sense. And she went and talked to some people and they were like, oh yeah, because that's the bots. That's what they do. Jerod Santo It's normal to them. Dawn Foster But unless you understand that, you can't interpret. It doesn't tell you anything about the health of the project unless you understand what's going on within the project. Adam Stacoviak Yeah. So ([Time 0:38:06](https://share.snipd.com/snip/64e798e3-2ad0-4dec-a5c1-bd70a0c46feb)) - "Chaos": Community Health Analytics for Open Source Software • Chaos is a project focused on defining consistent metrics and developing software for community health analytics in open source software. • The Chaos project includes two software pieces: Auger and GrimarLab, which gather data from various sources for analysis. • Rather than using health scores, Chaos looks at trends over time and considers metrics in the context of individual projects. • Understanding the specific workings of a project is crucial for interpreting metrics and assessing project health. Transcript: Jerod Santo Help me out with the rest. Dawn Foster Community health analytics for open source software. So chaos with two S's. Jerod Santo Two S's. Chaos. Chaos. Chaos. Dawn Foster Yeah, so we basically, I'll give you an overview of what chaos is. We are a project and we're focused on kind of two things. We're focused on metrics, so defining metrics so that we can be, when we talk about a certain metric, that we can be consistent about what it is and have a definition that we can point people To. And we say, when we're talking about, you know, numbers of lines of code, that's what this means. If we're talking about, you know, the bus factor, which is, you know, how many people you have contributing to a project, that we measure that kind of the same way. So we do metrics definitions, and then we do software. So we have two pieces of software within the chaos projects. We have Augur and Grimoire Lab, and those are both, they're basically software projects that go out and they gather a bunch of data from various sources. So GitHub, obviously, Slack, other things that you can, basically anything with an API that you can get access to the data from, and allow people to analyze that using software. Very cool. Jerod Santo So do you have some sort of a score, or how do you quantify health? Dawn Foster That is an excellent question. Thank you. No, we don't have a score. Okay. And I am anti- health scores. Okay. So what, what I like to look at when I'm looking at project health is I like to look at trends. So, you know, are you closing more of your pull requests? Is your pull request backlog getting bigger or smaller? Are you responding to pull requests and issues more, more quickly, or is it taking you more time? So I like to look at trends over time. And I like to look at metrics in the context of projects because individual projects have certain ways of working and certain things that impact the metrics. And unless you're part of the project, you don't know. So, for example, if I work on a project and it's, we're cutting a huge release that has a bunch of breaking changes, there's probably going to be some weird things in the metrics associated With that. So, pull requests are going to get in the backlog while you get everything together for the release, for example. I was talking to a friend at Google, Sophia Vargas, and she does a lot of analysis on things like Kubernetes. And some of the metrics that she was looking at it made just no sense because the way Kubernetes works is you've got bots that do all the things, right? So like, you have bots that respond to things automatically, the bots close the issues automatically after a certain amount, you know, they go stale, they close them. So there's all this like bot activity that she was looking at data and she's like, this makes no sense. And she went and talked to some people and they were like, oh yeah, because that's the bots. That's what they do. Jerod Santo It's normal to them. Dawn Foster But unless you understand that, you can't interpret. It doesn't tell you anything about the health of the project unless you understand what's going on within the project. Adam Stacoviak Yeah. So ([Time 0:38:06](https://share.snipd.com/snip/30d60b3f-2e10-4bfb-9465-802b6e8b0aff)) - The Limitations of Scoring Project Health • Every project is different, so comparing their metrics can be challenging. • Metrics should be interpreted in light of other factors specific to each project. • Timing and location can impact project metrics. • Metrics can provide insight into project health. Transcript: Adam Stacoviak Specifically, why did you not score? Why did you not establish maladaptive, healthy, a score of 50? Why the pushback against the scoring? Is it too concrete? Do you need to be a bit more ambiguous in terms of that true health? No, it's because every project's different. Dawn Foster So how do you compare a Kubernetes and give that like a, like any, any algorithm that you could put together that would score something like Kubernetes, um, and then compare it to a project That has like, like two contributors and, you know, 10 pull requests a month, like any metric that you could, you could score would give you wildly different results. Cause those are very different sizes of projects and they have different automation they have different like release schedules every project's different so I want the project themselves To think about what do these metrics mean to me for my project and interpret it in light of the other stuff that's going on with their with their project like you know like a release window For example or you know kubecon comes up and you see you see a drop across the board on like cncf projects like the week leading up to KubeCon where everyone's writing their talks and during KubeCon. And then, you know, you see it go back up. Right. So there's lots of stuff that can impact that. If it's a mostly European-based project, you see a big dip in July. Adam Stacoviak Yeah. Dawn Foster Because we're all on vacation. Adam Stacoviak Does it answer the question, are we healthy or not? Is that, what is it, what's question specifically that it answers? Community health. Yeah, like is it, because like you can score health and say we are healthy or we're healthy-ish, and it can be specific to your repo, and I can understand why, you know, if it's a European Team, why July might be less so. And it's not like, even as an OSPO, I might be like, are my projects healthy or are they less healthy? And if it says less healthy, oh, because it's July. And that makes sense. Speaker 6 Yeah. Dawn Foster I mean, ([Time 0:50:41](https://share.snipd.com/snip/54f0f09a-35d1-4ff1-bbf7-ea4d32a32a48))