The World of Open Source Metadata  ## Metadata - Author: [[Changelog Master Feed]] - Full Title: The World of Open Source Metadata - Category: #podcasts - URL: https://share.snipd.com/episode/16b258d9-2839-4b94-ad17-9999cd615b69 ## Document Notes ## Highlights - **24 Pull Requests Sparked Metadata Work** - The 24 Pull Requests project exposed how many repos lacked active maintainers and clear contribution paths. - That motivated building metrics and tooling to highlight healthy, maintainable projects. Transcript: Andrew Nesbitt Yeah, it's great to be back. Jerod Santo Ecosystems. It is, I mean, okay, we have a lot of context that maybe our listeners don't share, but take us back to what you're interested in, which seems like you've been interested in similar things For a long time. And you built libraries.io around this, and ecosystems is a very similar thing. I'm wondering if it's the same old thing or if it's a new thing. So tell us about your past and collecting and organizing dependencies and the information about them and open source projects, sustainability, and then how that brought you to ecosystems. Andrew Nesbitt Yeah, okay. So I have been swirling around the world of open source metadata for, must be coming up to 10 years now, starting with 24 pull requests. That's right, 24 pull requests. That didn't kind of start from metadata, but the idea of that project was to encourage people to contribute to open source as part of kind of the run-up to Christmas. And after kind of like first getting that off the ground, we quickly ran into like, oh, how do we suggest, like where should people go and contribute to? And a lot of people would try and send a pull request to a project that had no activity and like the maintainer was gone. Or just like were struggling to be able to even like work out how to send a pull request to some projects because they were really not very friendly or easy to contribute to. And that kind of led me down this path of like okay well what's a how do you define what a good project is and then like can we scale it up rather than manually having to have people kind of Like submit their things and keep those things up to date every year because that project would just kind of come and go every December and shut down afterwards so the maintenance there Couldn't be entirely human because there was thousands of people contributing to that project and like sending pull requests. ([Time 0:04:01](https://share.snipd.com/snip/09238727-5f29-4513-8807-3711bf2073e2)) - **Dependencies Reveal Real Usage** - Mining dependency data gives a stronger signal of real usage than stars or downloads. - Dependencies reflect actual runtime usage and drop when projects stop being used or maintained. Transcript: Andrew Nesbitt Yeah, okay. So I have been swirling around the world of open source metadata for, must be coming up to 10 years now, starting with 24 pull requests. That's right, 24 pull requests. That didn't kind of start from metadata, but the idea of that project was to encourage people to contribute to open source as part of kind of the run-up to Christmas. And after kind of like first getting that off the ground, we quickly ran into like, oh, how do we suggest, like where should people go and contribute to? And a lot of people would try and send a pull request to a project that had no activity and like the maintainer was gone. Or just like were struggling to be able to even like work out how to send a pull request to some projects because they were really not very friendly or easy to contribute to. And that kind of led me down this path of like okay well what's a how do you define what a good project is and then like can we scale it up rather than manually having to have people kind of Like submit their things and keep those things up to date every year because that project would just kind of come and go every December and shut down afterwards so the maintenance there Couldn't be entirely human because there was thousands of people contributing to that project and like sending pull requests. And it was a lot of data to try and work with. So I started to build out some basic metrics there to try and go like, does this project look like it has activity that's happening on it? ([Time 0:04:36](https://share.snipd.com/snip/a9949555-1298-43fa-a066-c1f73afab76a)) - **From 24PRs To Large-Scale Indexing** - Andrew described starting with 24 Pull Requests and building metrics to judge project health. - That led to Libraries.io and later to ecosystem-scale dependency indexing work. Transcript: Andrew Nesbitt So I have been swirling around the world of open source metadata for, must be coming up to 10 years now, starting with 24 pull requests. That's right, 24 pull requests. That didn't kind of start from metadata, but the idea of that project was to encourage people to contribute to open source as part of kind of the run-up to Christmas. And after kind of like first getting that off the ground, we quickly ran into like, oh, how do we suggest, like where should people go and contribute to? And a lot of people would try and send a pull request to a project that had no activity and like the maintainer was gone. Or just like were struggling to be able to even like work out how to send a pull request to some projects because they were really not very friendly or easy to contribute to. And that kind of led me down this path of like okay well what's a how do you define what a good project is and then like can we scale it up rather than manually having to have people kind of Like submit their things and keep those things up to date every year because that project would just kind of come and go every December and shut down afterwards so the maintenance there Couldn't be entirely human because there was thousands of people contributing to that project and like sending pull requests. And it was a lot of data to try and work with. So I started to build out some basic metrics there to try and go like, does this project look like it has activity that's happening on it? Does it look like it's ever received third party like contributions and things like that? And that led me to kind of, I got a job at GitHub from there. And then GitHub promptly fell apart internally. Tom Preston Warner left. It was a horrible time. And so then I left there and started Libraries.io as essentially a like, okay, well, looking at package manager metadata is a different way of kind of getting some measure of what's An interesting open source project. Kind of bearing on a lot of projects, especially as you go down from the kind of like the massive frameworks, the kind of those huge keystone projects. ([Time 0:04:38](https://share.snipd.com/snip/a16b45cf-5aea-45b7-866c-234186ba482f)) - **Modular Services Beat Monoliths** - Ecosyste.ms is a modular rework of Libraries.io into many services handling different metadata types. - Splitting into focused services makes contribution and scaling easier than a single huge app. Transcript: Andrew Nesbitt And so ecosystems is essentially a do-over of Libraries.io. It's many different Rails apps that are focused on collecting different kinds of open source metadata and then combining them together in different ways. So there's a packages service, there's a repo service that collects the dependency information from repositories, there's an advisory service and a commit service and an issue service, Basically all the different things that you might be interested in. And each one of them can then be independently worked on and scaled up as like different amounts of data pour in and kind of collect in different places. And that has been going on for nearly i want to say three years now um really kind of like going from uh it was a nice kind of year where i just worked on it myself didn't really tell anyone About it just kind of like plugged away and the there are core pieces because libraries i was open source i was able to reuse like the dependency parser and a load of the mappings to the Package managers uh actually like take that code and kind of reuse that in a way that is also allows you to have multiple different package manager registries uh where libraries i would Only support one which was really nice when rubygems uh had its all of its drama recently and the gem.coop popped up i was able to go oh i can quickly start indexing gem.coop uh it just fits Straight into that new schema and then like since since kind of like the past year it's just absolutely exploded in usage. ([Time 0:14:31](https://share.snipd.com/snip/8782ba2e-46bc-4b88-85bb-aa7c6856b967)) - **Open Source Usage Is Hyper-Concentrated** - Usage is extremely concentrated: a tiny fraction of packages account for most usage. - Roughly 0.01% of packages can represent ~80% of ecosystem usage. Transcript: Andrew Nesbitt It's like 0.01% of packages make up 80% of usage. It's tiny amounts. There might be 2,000 node modules total that make up 80% of all of the usage of NPM in terms of downloads and in terms of like discrete dependent repositories uh which is like when you then Start to really focus that lens you see a long tail of stuff that never gets used and it's also like all kinds of spam and malware and stuff that floats around but there's like a 10, 15,000 Packages, which are like the packages that make up most open source usage across all these ecosystems. Jerod Santo It's kind of amazing how massive that asymmetry is when you like pin that down. Andrew Nesbitt Yeah. And that's like on average, one maintainer per package at that critical level as well. So that's like 15,000 people maintaining all of open source usage. Jerod Santo That makes the SKCD comic even more poignant. You know, the one person in Nebraska, you know, replace Nebraska with wherever they are in the world. Probably in different towns. How many of them have you had on the changelog? That's a good question. Probably a good percentage of those oh man so there's all there's 15 000 people basically running the world um for free well ([Time 0:24:40](https://share.snipd.com/snip/0d71e5cf-cbdd-4f5e-8c6f-979522922c4e)) - **Critical Packages Often Lack Funding** - Many critical packages are maintained by single maintainers with limited funding. - Only 25–50% of top packages have an automated funding link or sponsorship path. Transcript: Andrew Nesbitt I have done a little bit of indexing of you know like how many of those have uh github sponsors or are their projects on open collective or they have some other kind of funding link. And in terms of those top critical packages, it comes out to kind of like, depending on the ecosystem, it's somewhere between 25% and 50% have some way of, you know, like, here's an automated Way you can give me a donation to the project. There's a good chunk of those as well that are, you know, massive corporate funded projects, like all of the AWS Ruby gems that make up the AWS CLI are in the top of Ruby gems because they're Just massively used. They don't need any funding, right? Because that's Amazon has full-time staff, but there's a good, they might need some funding. Jerod Santo Like right They're laying people off again. Andrew Nesbitt Hopefully they didn't lay off all the, uh, the Ruby people maintaining the CLI there. Jerod Santo That would be awful. So do you track, so you're tracking those who are, uh, able to receive funding in some sort of automated fashion. Do you track funding itself? Like who's getting how much money and how? Andrew Nesbitt Yes. Well, where possible. So I'm tracking, I call it a funding link and some package managers have funding links support where you can say like, oh, I get, you can donate to me over here. Repositories have the funding YAML file and I go looking for that wherever possible. And you actually see that even on GitLab and Codeberg. I don't know how well those platforms like display in the UI, but it definitely, because obviously GitHub sponsors is not, I don't think there's a GitLab sponsors or a Codeberg sponsors. Those files do show up all over the place. And then also being able to go like this repository is owned by a user on GitHub who is part of GitHub sponsors is another way of kind of detecting that. ([Time 0:26:06](https://share.snipd.com/snip/05fa9682-0aaa-4ea3-934e-3ebfbe7b9536)) - **Split Databases And Keep Hosting Simple** - Host focused databases per service rather than one huge instance to make scaling manageable. - Use affordable dedicated machines and simple deployment to reduce operational costs. Transcript: Andrew Nesbitt So that 24 billion dependencies is a bit of a headache. I bet. I mean, that's crazy. Almost all of this is stored in Postgres. Okay. Individual Postgres instances on dedicated machines in France and Amsterdam, mostly because they're very affordable. Online.net is a very reliable host, similar to Hertzner or some of these other kind of like bare metal machines. So I do the maintenance of the machine myself. And obviously scaling up is a little more tricky because there's not just a nice Heroku slider anymore. I use Doku as essentially like the open source Heroku, which is really nice. Just Git push, it builds your Docker image, and then it handles putting Nginx, kind of proxying all of those things. Very nice for like an individual machine. It doesn't really give you any kind of multi-machine things, but I try to avoid too much complexity when there's only a very small number of people working on doing the infrastructure And it's mostly me uh rather than i try i calculated like a back of the napkin thing the other day i think it would cost me 15 times as much to host on aws as it does to host it on dedicated machines Now. But these Postgres, each service basically has its own database. So rather than it being one that is enormous, it's split out, which at least makes it kind of like I can work on individual ones and be like, oh, this one is reaching capacity, so it's time To scale it up. Or I should make another box of web machines or sidekick workers separately i don't need to kind of do everything in one big lockstep uh which keeps it you know fairly easy to do and then The whole website is basically read only like you can't ask you can't put data into it as a user you read from it uh and it all the data comes in in the background through loading data from Package managers and repositories and there's about 2 000 different git hosts in there that i'm constantly crawling at different rates to go like oh there's there's new activity over Here so i can cache things very aggressively the kind of HTTP layer. ([Time 0:33:27](https://share.snipd.com/snip/9dc21a40-9669-4fa3-9bb9-40b1e065055b)) - **Use Dual Licensing For Sustainability** - Offer dual data licenses: open CC BY-SA and paid commercial terms for restrictive use. - Use paid licensing to cover hosting costs while keeping base data open for research. Transcript: Andrew Nesbitt You can use it for whatever you like, as long as you also persist the license and you credit where it came from. But if you don't want to do that, then you can pay to essentially have a CC0 license. It's not actually CC0 because there's some things there to say like, oh, don't just completely undercut us and sell that on again. But we have a number of customers that basically pays for all the hosting costs so it's self-sufficient it runs itself as long as but you don't get any extra feature development on top Of that right so that's like where i'm trying to work on right now is to um is to get that level of sustainability higher and we just received a grant from alpha omega to basically make that Happen that's uh alpha omega is part of open ssf and their goal is like turn money into security and they have become a big user of uh ecosystems for doing analysis of like who are the critical Projects in a particular space? Who are the ones that are like going to be most likely impacted if there's a big security vulnerability? Who are the ones who have never had a security vulnerability and maybe don't know what to do if they get one? Things like that. So they have basically given us a grant to try and help make ecosystems long-term sustainable. ([Time 0:42:11](https://share.snipd.com/snip/6718734e-9402-4c36-bd3b-51892df61b58)) - **SBOM Enrichment Drives Traffic** - SBOM enrichment is a major use case: mapping package entries to advisories and metadata. - Teams run GitHub Actions to enrich SBOMs and this drives substantial weekday traffic. Transcript: Andrew Nesbitt And SBOM is kind of like the natural conclusion of that of course you have two different sbomb standards there can't just be one standard for things um but that being able to look things Up by pearl is something that ecosystems does really well because you can basically then take an sbomb and just work through it every single package that's in there and say can you tell Me about this package can you tell me what security advisories are affecting the version that i've got in my sbomb uh and that is like the biggest use right now is um there are lots and lots Of people with github actions that are just enriching their SBOMs with this kind of information. And they just, it's funny how much more traffic we get on a weekday than on a weekend. And it's, I think it's just because of the GitHub Action kind of like, oh, this is happening every time someone commits. So you see a smash of traffic of them, like enriching-bombs and checking out every package that is in there. ([Time 0:48:37](https://share.snipd.com/snip/4abe607f-084e-45d2-9854-492b1bec59da)) - **R Registry Breaks Reproducibility** - R's package ecosystem lacks a stable API and allows package removal, harming reproducibility. - This led to scientific reproducibility problems and adoption of registry snapshots. Transcript: Andrew Nesbitt So R has this really weird, I think it's because it's come from a scientific kind of like non-developer background. Like R, it also has one indexed arrays, which not many programming languages have that, right? But their package manager won't let you pin to an older version of something. It won't say, like, I want version 1, even though there's version 2.0 is out. And the knock-on effect of that is that, so when, as a user, if I'm going to say install my R packages, I always get the latest version of everything. Means that if something's broken because something else got a new version rather than the the new version causing the breaking change be told off it's actually the package that didn't Upgrade to fix the uh the problem with this other package just updated. So if you're not proactive in fixing breakages with your package being used with other packages, your package gets removed. It gets kicked out of that registry, which is pretty wild because, you know, people, especially in science, trying to make their science reproducible are like, oh, my package got yanked Like how am i supposed to reproduce this science it's no longer here so they have some very strange behaviors where they'll actually make snapshots of the registry and then like so you Can say i want to install my r package from the this registry on this day so you actually have like a weird historical aspect of the thing uh which is it's not like a lot of other package managers Um and it's very hard to change because you know there's just not a lot of we don't have a lot of funding in open source but in terms of research software engineers there's no incentive There to maintain and develop software unless it has a paper attached to it, right? You get, if you can get citations, great. Like you can continue to make a case to keep working on those things. But once it's done, it's done kind of thing. You're like, oh, you already published that paper. I don't need to continue maintaining the software. ([Time 0:53:58](https://share.snipd.com/snip/4438f681-b65f-4411-9753-b530bc98edbc)) - **Metadata Is An LLM Attack Surface** - Exposing package metadata to LLMs or MCP servers creates prompt-injection risk from malicious package fields. - Metadata fields like version or description can become vectors that corrupt agent behavior. Transcript: Andrew Nesbitt But the security implications really hurt my brain. So I have kind of like held off going hard into it because, you know, every string that is returned by the MCcp is essentially like a prompt injection so you imagine your version number That is pulled from an npm package and then fed through an mcp server into your context they have the ability to make a version number especially if it's like semver with your your um your Pre-release string on the end of the version number, you could make like prompt injection ver where I just start putting like, ignore all previous instructions dash one dot one in the Strings of the thing like that come from the package manager is suddenly a security vector or even just the description of the package or the name of the package. Like there's a lot of trust that happens on that kind of go through when it comes out as an MCP server on the other side. If you're just saying like blindly install whatever the MCP server told me, then there's a lot of trust that you're putting into many layers of indirection that could happen. And we've definitely seen like loads of threat actors have realized how, like I'm going to use the word juicy so many times, but in terms of being able to go like, I can just, there's no Restrictions. ([Time 1:04:25](https://share.snipd.com/snip/db84100c-e958-48f2-81a2-82b9e36bdd4c)) - **Encourage Polite API Use And Bulk Access** - Require polite API use: offer higher rate limits when callers provide contact info. - Provide bulk exports or a column-store query path to avoid excessive per-entity polling. Transcript: Adam Stacoviak Like, how do you leverage that? Because I can pass in my email, but then you say, well, I can reach out to you later. You're watching my rate limits, of course. Can you just shut me off because of me passing that email to you? Or how do you curb the enthusiasm, so to speak? Andrew Nesbitt So right now we, we have this, like we have the anonymous rate limit, which I think is 5,000 requests an hour per address, basically. And then the polite pool, which is a term we borrowed from a service called OpenAlex, which is basically like ecosystems, but for research papers, they have this, if you pass in your Email address as part of the user agent, then you just get an uprated rate limit. So that if we see that you're smashing the API, we can contact you and say like, oh, what are you like doing? Can we help you do this in a different way? So far, I haven't actually like been tracking that particularly closely. I've literally just like, great. Cloudflare is still like catching most of that stuff before. Like if you hit anything that's cached, it doesn't even touch your rate limit. So it's only the, the uncached things that actually affects that rate limit. Um, but even then it's like 10,000 requests an hour. You're, uh, you're gonna, if you're really, really hitting it, you're going to run into that. And then a 429 request is very cheap to serve up. So I can serve up a lot of like rate limit used requests before things start to fall over. And then like looking at the patterns and going like, how are people using this? And is there a way I can do like a higher level API that avoids, you know, having to have someone do that trawling? Or is there ways of being able to export big chunks of data rather than doing individual, lots of little queries is another thing that we're exploring. ([Time 1:07:10](https://share.snipd.com/snip/d15aacd9-0804-4175-a08e-ec7cdf8fea94)) - **Show Maintainers Their Downstream Users** - Help maintainers see downstream impact by showing top dependents, pinned versions, and failing upgrades. - Use that visibility to proactively coordinate upgrades and reduce breakage. Transcript: Andrew Nesbitt So SNCC has a tool called Parlay, which does SBOM enrichment. And so I can then go and these things are open source. I can go and look at them and see like, oh, how are they currently using the existing API? Is there a better way that I can do? Or do I just need to beef up the caching in some of these kinds of places? It's very much like the prioritization a little bit is like just running around putting out fires. But then occasionally it's like, right, I'm going to turn everything off and I'm going to go and like tackle one of these slightly chunkier problems of essentially like solving a bigger Challenge than just, oh, there needs to be a new API. Often that's like, oh, there needs to be another service for another kind of data, or there needs to be another way of querying this thing because lots of people have been asking for this. Like the biggest thing is just being, like coming and asking for things on the issue tracker is a great way to kind of kick off that conversation and say like, Oh, I've been trying to do This. I'm trying to solve this problem, but I can't work out how to go through, you know, like I've hit a wall here, or there's just too many individual bits of data over there to, you know, like, Can there be an aggregation of this thing somehow? And sometimes that's easy. And sometimes it's like, oh, actually, if we make this index, it's going to be like the index itself is like 500 gigabytes in size. That's hard to fit into RAM. So maybe we think of another way to solve that problem rather than just like adding an index for every single different way you might want to query Postgres. Adam Stacoviak I found the introducing Parlay post, they even mentioned that we're enriching Parlay. It's enriching these SBOMs using ecosystems. So are they one of your paying customers then, considering this tool is probably part of their... Andrew Nesbitt No, they are using... So Parlay is an open source tool that other people can use. Okay, gotcha. And it's primarily companies because, you know, open source developers don't actually care about SBOMs because they're like, here's the code. Adam Stacoviak I had to search what SBOM enrichment was. I guess I should have guessed that by take a little bit of data make it better i don't know okay well if you most sbomb extractions don't uh like when you produce an sbomb from say a repository Andrew Nesbitt Or from a docker container it will go here are the packages and the version numbers but it's not going to tell you like and here is all of the information about that package because they Just don't have that on disk available most of the time some package managers especially the like the distro package managers do actually have that information right there but you Know these sbomb generation tools don't go and hit the npm api directly to fetch all of those so if you want to be able to get a high level overview of all of the license breakdown of all the Different packages in your sbomb then you need to enrich it by you know basically going through each one and fetching some extra information and filling in the license field uh maybe They're like maintainers there's a load of different things in there and it depends on which sbomb standard you're looking at as well because they're different um but they're also Like just being able to look up all the security cve stuff it's nice if you're only working in one particular ecosystem because you can use npm audit or bundle audit but as soon as you get Into the like multi ecosystem things which every docker container is right it's going to be like oh i've got my my django app with a javascript front end and also all of the back end like Low level distro package stuff like there's a big collection of random bits of software in there and i really don't have to use 10 different tools to enrich it. Adam Stacoviak I just want one thing that will just sweep across and support everything. You mentioned a couple of times building things on top of, is this, since this is sort of a redo for you, it's kind of like a take two, do it better. Is this the substrate for many things? And what are some of those things that you mentioned? Like you mentioned some things being built on top of, but what are those things? Andrew Nesbitt What's the world you envision? There's a few that are listed on the ecosystem's homepage. So we have the things that I've built for Open Collective, which are the funds app and the dashboards app. Those two things are like definitely, they don't have their own data. They're essentially like aggregations of various bits from ecosystems to solve particular challenges. One thing I've not built is a search engine. I've kind of been like, I'd like to see if someone else would build that, you know, like I already did that as in libraries.io. But that would be a natural one to add in there. What I'd really like to build is things that help maintainers understand who is using their software. And this is going back to that 24 billion rows of dependency data to be able to say like. Adam Stacoviak How much are bots? How much is Docker pulls? How much is just like CI builds? I guess those are all still users right i mean if i'm that person uh releasing 100 times i'm still pulling the packages right every time they commit yeah it's like yeah boom new version Andrew Nesbitt Because i can you know and also to be able to go like if we can flip that graph upside down and show you here are the key, like people downstream dependencies of your library, then rather Than you find out that you broke them because they come into your issue tracker after you just published that release and say like you broke stuff. Building a CI that is like an inverse that goes, okay, well, you committed something. Let me go and test this against your downstream, like your most popular downstream users to make sure that you didn't break those things. And there's some difficult bits there in making sure, you know, those downstream CIs are reliable. They're not just going to be like, oh, actually tests pass all the time regardless or they fail all the time so you can't trust like if you actually broke anything or not but to be able to Do that would give maintainers insights that would be like they can actually be proactive about some of these things and maybe even be able to coordinate and go like oh i'm able to reach Out to these projects and say like i'm gonna break this thing or i'm going to change this thing to make it better. ([Time 1:11:58](https://share.snipd.com/snip/9d57482d-6003-4666-8c2c-7b83ce41e204)) - **Taxonomy To Improve Project Discovery** - Andrew started an open-source OSS taxonomy to standardize facets like role, technology, and domain. - A structured taxonomy improves discovery beyond free-text keywords and topics. Transcript: Andrew Nesbitt The other thing I just started noodling on is an open source taxonomy. So to try and define like a taxonomy that describes the different facets of what makes an open source project. You know, what does it do? Who is it for? What technologies does it use? There's about six different facets and about 130 different terms that I put together as like a V1 kind of thing of going like, if you were to put these packages into a box or six boxes, like Which ones would it do? Rather than just going like, here's some free text keywords. Like here's a load of the kind of chunks of things, including like the role of the user as well, rather than just thinking about like, oh, it's a front end react app. It's like, but it's for a, is it for an end user or is it for a sysadmin or for a developer like to be able to and then what domain is it in as well uh it's like really early but i'm hoping it is Another way that can produce some alignment in this open source discovery world because you know i worked at github for a while on open source discovery and didn't wasn't really able To make a good dent in it there but i think there's still a lot of low-hanging fruit in terms of like just helping people find the right kind of tools to use because not many other people Have really it's also there's just not a lot of money in that space it's a lost leader right for most companies is like searching for open source is not going to turn you into like oh yeah You can't even run a lot of ads against that kind of stuff because open source developers are like the number one user of ad block so those ads will disappear pretty quickly uh but the i'm Hoping that that taxonomy will be like here's a nice blueprint of ways that you can define your project and put it into ways that then allow you to kind of go like, okay, well, I've got five Dimensions here, but I want to rotate around one of them. I want a web framework for researchers, but I want to rotate about the technology. Like, what are my options there or i'm definitely in this technology space uh and looking at this kind of like position in the stack but what options do i have here for different like users Uh and to be able to kind of like twist the picture a little bit but in a fairly defined space rather than in, you know, just arbitrary free text. Because again, you run, you just end up in this like soup of words, which is like, yeah, we kind of just get very fluffy. And often projects just don't have very well-defined ways of finding things. You know, like they don't add a description to their GitHub repo or any keywords or topics. So you just kind of like never find it unless it's in a generic search engine, which is then really hard in terms of like, oh, well, what are my options in this space? And I made this as just like, surely someone has made one of these already. And I found a taxonomy of software in the research space, but I did not find a taxonomy of open source software. So I was like, okay, I can make a stab at one of these. I like, I've never made a taxonomy before, but I put it together as like, this should be interesting. And it's been useful so far. ([Time 1:31:59](https://share.snipd.com/snip/60b2a14d-46bf-43b1-9834-a0ea8f314ff7)) - **Use Cross-Ecosystem Discovery** - Encourage cross-ecosystem discovery to reveal alternatives and reusable components. - Use taxonomy-driven filters to help developers find smaller pieces instead of monolithic projects. Transcript: Andrew Nesbitt And it started some interesting conversations but i really need some people with more experience in you know actually defining taxonomies than i have to to give more input and also Expand it and like cover the problems because i'm pretty sure there's going to be loads of problems in it because i basically just like put it together in a couple of days as like okay i Think this this should work but you know yeah mostly where does that where does that live that is on the ecosystems github is also a really uh quick um web page i made of taxonomy.ecosystems Um it's not it's literally just a few days ago. So it's not anywhere on the website, but it is on the GitHub org as OSS-Taxonomy. Jerod Santo But I get a link in the show notes. Awesome. Yeah. Send us that. And anything else you want to make sure we get into our show notes so that y'all can just click through and find that and help Andrew figure out this taxonomy so that we can all start to kind Of formulate around it. Categorization is always useful, especially for very gray, otherwise gray areas such as these, especially if you're self-defining, it helps you to even flesh out your idea or your Adam Stacoviak Project better. I think this is fertile ground right there, honestly, because you got so many, I would describe it as like ecosystem explorers you may have pre previous to llms being a ubiquitous thing And agents helping you you may have just stayed in the zone that you're comfortable in because you're the the mere human that cannot think 10x faster you know and then you get into this Llm world you're like man i can actually explore new languages because it knows it i know this language and i can at least translate my knowledge and so now you find yourself exploring Go or rust whenever you would have normally just stayed in the ruby world because maybe that's where you're comfortable you know and so when you go into those worlds you're like well How do people test here how do people deal with http how do you deal with security things and so you find yourself exploring new worlds while you know the ruby world well you don't know The same kind of projects that would help you in a different lens i can think that's that's gonna be useful honestly yeah ([Time 1:35:37](https://share.snipd.com/snip/b341e112-f354-420e-bcf6-493aa33ffe17))