CHAOSS Funding Impact WG, Feb 11 2026 ![rw-book-cover](https://img.youtube.com/vi/Pfm_-PfGUnE/maxresdefault.jpg) 🔗 **Source:** [CHAOSS Funding Impact WG, Feb 11 2026](https://www.youtube.com/watch?v=Pfm_-PfGUnE) Okay, welcome everyone to the February 11th funding impact measurement working group. Uh, as always, the working group is under the chaos code of conduct. So, keep that in mind and please be kind to each other. Uh, we have a few things on the agenda today. Um, starting so we have a we have a co-chair's update and then uh, Alexandra is going to talk about um some of some research. So that'll be interesting. And then we have a couple of other agenda items if we if we have time um towards the end about FASDM and then the practitioner funding impact measurement practitioner guide. Um with that I think uh Paul did you want to did you want to take over and facilitate since you've got the the first bet? >> Yep. Happily uh we'll do. Um so we have an update from our side um which is I think a combination of let's say some sad news but also some very good news. Um so um we've seen a bit of a change in uh the formation of our co-chairs. Um Kalian Osborne has uh left because he started a new position um that will take up a lot of his time. Uh and so Dawn and I um uh were left with uh thinking about who we'd like to join um uh as another co-chair. Um and we invited Adrian who's with us today um who is very kindly accepted to join which we're very thrilled about and excited about. Um it's obviously a very natural fit um given uh Adrian's position at uh LF uh as director of quantitative research and also past involvement um presenting work on multiplier effects uh with NGI comments. Um I'll give over the um uh the space to Adrian in just a second to say a couple of words. But before that, I wanted to share um a message from Kon uh who very kindly recorded a little farewell um which means that we're going to be in the interesting position of playing a recording from Jity on Zoom that is itself going to be recorded. So we're going to have several layers of sort of abstraction here and let's see how the video quality holds out. Um but yeah, I'll share my screen and uh let Kian say a few words and then I'll hand over to you Adrian. Um okay, hope you can all hear this. >> We are not hearing it at all. >> Oh no, sorry. Bear with me. Maybe what I'll do is I might uh we'll see if I can post this in the chat somehow. >> Yes, sure. >> That might be better. Sorry about that. >> No worries. >> Okay. Well, in that case, I'll hand over to you Adrian. Apologies for the technical difficulties. >> No worries. No worries. Um just as the first thing, thank you so much Don and Paul for um thinking of me and it's it is really a great opportunity. I was selling to them um already that um this working group has given me a lot of motivation actually and and was very beneficial to the work that I'm doing at NGI comments. So thank you for that and circuit and Alexandra both um were very active members of this and um helped a lot even until now. So now now I'm very excited to continue this as co-chair and hopefully collaborate with all of you um invite interesting speakers and discuss much more about the impact of funding. >> Hi Jer. >> Very cool. Thank you for that Adrian and yeah welcome aboard uh from everyone. Um wonderful. Um I think with that um uh we will turn to Alexandros um to present uh his latest uh research project. Um I think for those of you who were with us last year, you'll remember that Alexandros presented um last autumn. Uh and as a bit of a followup, so with that, I'll hand over to you Alexandros. >> Thank you very much. Um let's see. to make the screen share work. Does that work? >> Yep. We're seeing you slides. >> All right. Um yeah. Uh so thanks uh first uh giving me the opportunity to talk about this uh second piece uh of research I've conducted or I've submitted in end of end of January which um uh hopefully is uh relevant to that group. So um yeah the title of that uh work is why authors and maintainers link or don't link their Piper libraries to good repositories and donation platforms. And as we focus on uh the funding side or uh the donation platform side of this paper uh also the presentation um kind of focuses on that part um if you want to kind of read up the other parts. I also linked the paper um also provided um the replication package in case you want to um dig into the the the pipeline we used for analyzing um these answers. So uh where does this uh research come from? Um so as Paul correctly said this is a follow-up on a quantitative analysis I've uh presented in October November somewhere in autumn correct uh so where we analyzed um to what extent pi libraries uh link to donation platforms on pi and github and um basically uh yeah some um suspicious patterns um camea came to our mind and we wanted to basically understand what's going on there. So um the paper is basically um following up on some of these insights you get from these information here in the figures and the quote. Um so um first we basically wanted to understand uh why people use meaning the gray and the orange bar which is actually or which is actually pretty small. So we wanted to understand why they do link to donation platforms uh but also uh understand in my opinion even more important the big blue part why they don't link to donation platforms with all the benefits uh we are aware of. So these are basically the first two questions as we will see uh on the next slide. And then here in this um next um graph uh noting that this is a lock scale we saw that basically most of the links um were found on uh GitHub exclusively. uh yeah uh kind of raising the question uh why maintainers basically do not provide the information on pipi as well given the information that they already link their uh their uh github repository so they did the work linking their github repository but did not uh basically make this follow-up step to link their donation platform which is then provided on github so we we we were not sure about this part So we wanted to understand this yeah huge dominance of GitHub. And then the last um question arised from the fact that um yeah quite a quite a lot of these GitHub sponsors URLs which were either linked on Piper or on uh GitHub themselves were outdated. um also raising the question why people basically uh uh yeah don't um use their time or uh update uh these URLs uh at all. um that came or that led to the foreign problem and research questions. As I mentioned before, you might wonder why do these research questions start with three and four and not with one and two. And this is basically uh because I copied that from the paper. Um meaning if you go back to the paper and you try to map what you have seen in the presentation to the uh paper content, uh I wanted to basically uh stay consistent there. Uh so um if you read about the research questions targeting um the repository URLs they would start uh with the numbering of one and two. So first question as I mentioned is about um the question why they do link to donation platforms. Second question was then about uh the question of this dominance of GitHub meaning uh why they did not add a URL on pipey but exclusively on GitHub. Uh third question was about um the GitHub sponsor URLs being outdated. And the last question in my opinion one of the most important ones here is why they are not linking these information to provide you some insights about the methodology uh we applied here. Um so as you probably uh expected uh we needed to uh get the contact information of um maintainers and authors of Pi packages meaning uh we at the point in time it was it started in June 24. We basically crawled all the packages available on the Python package index and then extracted the email addresses um from these packages uh via an official endpoint which are uh voluntarily provided by the uh author and maintainers of these packages. What we then did is basically um in a sequential um pattern to create two independent surveys targeting these authors and maintainers regarding the topics of repository URLs and donation platform URLs. uh where both of these surveys basically um base their questions on these quantitative each on the quantitative results. Um as we've uh seen before, these surveys were were then published um between the dates. Um you can read here and in total um for both of these surveys we got um yeah over 1,400 um responses uh to these surveys um which break down um to for the repository u survey we um contacted random randomly 50,000 of these um maintainers um where yeah almost 6,000 were uh basically not deliverable. Um then leading to almost 1,400 uh responses um leading to a response rate of 3.1% which is pretty good in these um yeah qualitative research area. Um we also asked basically uh these maintainers if uh we would be allowed to uh follow up for further uh research. And this is then um where we sequentially uh uh contacted the maintainers regarding the donation platform survey uh where we um had the allowance of 570 people to recont them. uh were only two bounds and we got 67 responses um which if you um um consider that in isolation is a response rate of 11.8%. Um if you look at these numbers from the perspective of this uh working group, it's of course a pity that uh we contacted the maintenance in uh that um order and not the other way around because for uh the first um survey we got way more responses as we were allowed to um yeah basically recont to contact way more um Python maintainers. Um, regarding the survey in general, we use open text questions uh to basically capture maintainers motivations and barriers beyond um predefined answer options. So we wanted to kind of um evoke the creativity of these maintainers and not of list the things uh the the research had already in mind or we would come up um as options. Um moving moving further to the uh basically the analysis and the pipeline we used to analyze all these um answers. So uh we created a LLM based topic modeling pipeline using a local llama 3.3 instance with 70 billion um parameters. Um we used batch prompting to extract these topics um with at the end um a topic reduplication step um to kind of control granularity of these resulting uh topics. Uh we evaluated the whole approach um by repeating um the pipeline 30 times and then me measuring basically the extracted topics on based on lexical similarity but also semantic similarity um in order to identify basically instances where the wording has slightly changed in the topic but uh the meaning has stayed the same. Um and then in the end as for um these LLM based topic modeling approaches um these traditional evaluation um mechanisms don't work anymore. Uh we also applied or validated our approach uh with uh 23 human experts um on multiple uh dimensions where we basically asked them about interpretability, relevance, granularity and the duplication of uh the topic modeling of the yeah topics basically identified from our approach. So uh maybe to give you a quick insight about the statistics of the survey answers. So um for each of the survey we considered in the paper we had basically four questions. Um what is pretty interesting that even though um basically the number of responses vary significantly between the two surveys, the mean character length of the answers stayed basically on average the same and was also within this traditional tweet limit. um which is um referred to um as working well with for short um phrase uh shortterm um topic modeling approaches um identified in uh yeah related work. Um what is or what what is to note here as we focus on the donation platform URL um that there are questions especially the second and the third um which are not answered that much but as I mentioned before I think it is especially important to understand why people do not link these um URLs and um this is basically the most answered question as well. So um even though uh maybe the the the results from question two and three should be uh uh yeah basically you used with caution um the most important um answers to the question four are there uh in a in a in a pretty decent number. What is basically the outcome of this whole uh topic modeling pipeline? So um you can think about that in the in the following uh manner. So you basically get all the answers then um you basically pro provide the LLM uh with a question a prompt which you can uh read in the paper if you want and then basically um by uh by by using batches of the answers um the discovered topics as you can see them on the right are identified. And this is more or less uh the direct uh uh output uh of the pipeline. So I think the only thing which uh comes differently is that there are underscores um in the in the discovery topics um uh if you directly consume them from the pipeline. But this is basically the output you get and then you basically can uh can can have these uh results for each um question separately. um uh because of time I would not go through each of the questions and each of the topic now separately but rather discuss our uh takeaways and implications for this working group on an aggregated level in the end um to also give you some insights about uh the robustness and the human evaluation results um we uh analyze in the paper um wanted to to to show build these two tables. So what we can clearly see is that uh we uh basically create a pipeline um which is pretty robust again against um the execution of multiple runs. So as I said we executed um each of each pipeline with the two surveys 30 times and um what what was always achieved that um the coziness similarity meaning the semantic similarity um was higher than the jacket similarity meaning that there are instances where you have some small se uh uh syntactical changes in the identified topics. But um the semantics actually did not change drastically. So uh even uh if you execute uh these these these p this this pipeline 30 times for uh both um surveys you get almost like a 90% um of the same results. Um what is also pretty promising about the approach is um the the the quality the human experts evaluated the topics on. Um and what is basically uh important to see here is this red marked line where uh we first ask the human raers um if basically the identified topic is um is uh interpretable interpretable is fitting the question and is basically on the on the right granularity meaning um yeah so meaning uh it's not too specific and actually 77% for the or 75% for the donation platform URL survey agreed that um 75% of these uh topics basically um met all these three uh quality criteria and actually only uh a low number of um instances were defined as uninterpretable or not fitting the question or being too specific. There were some doubt about um topics being uh duplicate duplicates uh which is uh basically indicated by this last number. What is pretty good um that if you basically um calculate the interrator agreement between all these human raers, you get like a a moderate um agreement between these uh humans. uh yeah which is uh which makes our approach uh and the results um pretty confident. Moving uh to the last part which is probably uh yeah the most important um slide for uh this group. So what were the key learnings if we focus on um the answers and the topics we extracted from the donation platform survey. So um if we start with the first point the motivations to link these donation platforms are mixed. So it ranges from altruistic reasons supporting OSS and of course um the uh basically desire to um get uh direct financial incentives or support. Um but um we as I said before we need to be a bit um skeptical or we we need to uh take these results a bit with caution as the AD ab adaption is pretty rare and as um we applied random sampling basically when when uh selecting the the authors and maintainers uh targeted for participation only a small minority in the end does include include these uh donation platform links meaning that these motivations we got here might not not be fully representative. Um what we saw that donation platforms are often added only on GitHub due to higher visibility and referral strategy. So apparently uh the value on GitHub um for the maintainers are higher to place it there compared to Pi. Um apparently there is um like technical difficult difficulty, lack of knowledge and yeah simple oversight as reasons which prevent them to add these links. Also on pipi um there is um skepticism towards monetization um employer compensation or organizational policies or also simply they don't meet these things um which limit the abduction meaning now uh moving over to this part um what are reasons basically why maintainers don't add these links and what we also saw that donation links are missing or outdated due to unclear guidance and limited feedback when these links break. Um, in my opinion, I would particularly give this paper a read because it is pretty interesting how the results from the repository URL survey kind of overlap partly uh with the results from the donation platform URL survey for example. When looking at these metadata quality issues where uh yeah, some of these topics are basically identified in both of these surveys. what are the implications? So um we as suggest she made that improving the visibility guidance and kind of implementing lightweight validation on pi could lower the friction and normalize the donation platform meta data without pressuring maintainers. Uh I would like to give uh one note or one closing note uh regarding uh these suggestions. Um I've recently listened to the podcast to a podcast of Andrew Nesbbit talking about um his research basically on um package metadata also in his working group and I am aware uh that basically these development life cycles of the pack package registry or package managers are uh pretty slow but I think it's still valuable to know these uh informations. how basically for example these um outdated URLs could be solved by basically hinting um the the maintainers by an email or any notification uh basically how such functionalities maybe could um help um these um systems in the future. Um yeah maybe as a small outlook uh what might be interesting um as future contribution to this uh working group I uh quickly mentioned it I think in the last presentation already so I'm working on a impact related study to say so so uh what I plan to do is basically identifying a subset of libraries which potentially maximizes the impact on the ecosystem um if theoretically being supported whatever an impact means uh in that situation um I will uh I will I will or can read about talk about that uh in the future and then what I basically plan to do is break down um these subsets who basically a maintainer set um who are responsible for these libraries and then kind of reconnect these libraries also to metadata results I've already talked talked about um which are basically responsible for reachability and so portability of these maintainers. So maybe quick outlook which uh uh will or hopefully will uh come in the future. Um yeah, this is basically just one source um where uh the Curan paper uh basically is based on this um last research. And now I yeah want to thank you for your attention and I'm happy to answer your questions. >> Amazing Alexandra, thank you very much for that. It's um also really nice to see a kind of development of research that or to see how the research that you presented last year is continuing and um without speaking for others in the group, it's really great to see it develop and kind of build on that. So, thank you for coming back as it were. Um I uh have a couple of questions myself, but um I wanted to open it up to the floor if anyone else wants to jump in. I think I saw a few in the chat. Um, does anyone want to start us off? >> Uh, Gerard, please. >> Hi. Um, it's, um, uh, first of all, thank you, Alex, for your research. Um, it's quite, um, an interesting topic. Um but uh I was just having here some thoughts about if you have any metrics on so what percentage of the pipi projects that you search were actually having their repos uh pointing to GitHub and what percentage of the active repos were actually the GitHub um repos they were pointing to that is uh you had reported some stale repos so that maybe the project have had changed uh from using GitHub into something else and they forgot to update the pi record. Um, and uh, this is important because there's a a question about the migration. >> Sorry, I'm not hearing you anymore. Uh, maybe I don't know if it's on my side. >> No, I also lost his audio. >> Okay. >> Hi again. >> Now, now, now you're back. >> Is it Do you hear me? >> Yes. >> No. No. >> Yeah. Yeah. Yeah. Yeah. Yeah. We can hear you. >> We can hear you. It it was just a question that might influence the results is that uh maybe the pi records are not updated and they're still pointing to an old uh repo of the code and that creates a sort of an error into your uh analysis afterwards. So um that is your question finished or do you want to kind of continue? >> Actually yes it's finished. This is mainly the question. >> Okay so regarding the first part um if I have an analysis how many of um the packages do link to GitHub repositories and also uh not outdated ones. there is basically um a publication only on this part which which kind of uh motivates the survey for the repository URL which I didn't talk now in this in in in this presentation but this is part of the paper. So there is basically um if I if I jump back to to to to to to these to these um to these figures, there is basically um a similar paper or a paper using similar methodologies um like the one for the donation platform URLs which analyzed at to what extent Piper libraries will link to um GitHub repositories. We did not include in the selection of maintainers um whether uh package would be actively maintained or not. However you measure that. So it's basically um taking the emails of the maintainers and then having these 180,000 emails. We basically took randomly a third um of these emails and um basically tried to kind of approach so we we didn't want to approach only actively maintained ones because um I think unmaintained or partly maintained libraries are also a significant part of ecosystem. So we wanted to kind of get a broad picture of um maintainers or participants for our >> Okay, that was not really my question. U question is that >> if the project at some point had changed and so you were looking at an unmaintained repo although there was an active one elsewhere outside of GitHub. >> Mhm. It's a question about the quality of data in the pi records. >> Yeah. Um so no we so we would have not catched if uh there if basically the repo which is linked is not the the the real one uh which is where the actual development is taking place. No we would not have able to catch that. >> Just so that I can focus one more thing. So your um source of data was the was pi and not the actual project pages if they existed. >> Exactly. We >> started No, no problem. Just so it was clear. >> Okay. >> Thank you for that. >> Zan, do do you want to jump in with your question? Um and then maybe at 22 we'll do any other business and just use the last two minutes in this section. >> Uh yeah sure. Um well my question is that did we collect any information about how much money they raise the ones that have the link that's not in the research right? uh just my when we read about the articles uh on this uh maintainers articles about especially the ones that have a story about burnout that they usually mention that they raise very little money etc. So was curious about that and then the other thing is that I was also reading the paper that you mentioned page rank to understand sort of the criticality of the packages. What was that method? Did we talk about it? You said that even the 1% of the packages have very little uh uh links and then you came up that with a page rank. What what is that? Sorry, I I I think it is not part of that paper. I think it was the >> maybe somewhere else. >> Yeah. So I I posted I posted another paper in another working group. Um but I think like here in this in here in in this paper I did not use the page rank for any selection of libraries to be honest. >> Okay. Um yeah, I'll I'll check the paper again. So that was my second question. if there's a source for that. >> Uh it would be interesting to check that. >> Yeah. So maybe maybe I'm I'm completely confused now, but I'm pretty sure that in this paper I did not use any any any page rank. If I did I maybe we we get in touch um outside but I'm I'm pretty sure it was it was not used in that paper and maybe you read another work of mine where I did use uh the page rank for for for for basically pack package filtering. Okay, I'll I'll write it in the chat. >> Yeah, perfect. Thank you. >> Great. Um well, first off, thank you Alexandra for presenting that. Um I might actually ping you with a couple of follow-up questions I have myself, but uh in the interest of time, um I think we'll uh use the remaining five minutes to just um wrap up. Um I might actually hand over to Dawn if you wanted to talk about the um practitioner guide. Um please feel free. >> Yeah, absolutely. So we we've basically been working on a funding impact measurement practitioner guide. So this is something that the chaos project has been doing and these are guides for practitioners to um help them understand some aspect of open source open source projects uh using metrics. So the the practitioner guides cover a pretty wide range of topics. And so what we did was we took the paper, the academic paper that um Paul Kalin, I along with Merkco presented at the OFA symposium um a year and a half ago, the one that was in Boston. And so I basically took that paper and pulled out some of the key points and rewrote it for less of an academic audience and more of a practitioner audience. So So we have this practitioner guide that's work in progress. So, what I would ask of you now is if you'd like to uh give it a read and give us any feedback for ways that we can improve the guide, that would be that would be helpful. I would encourage you to look at a couple of the other guides because it's it's kind of a very specific format. So, it um it might help you to understand what the practitioner guides are are supposed to look like, but um uh but yeah, so it' be really good to get your feedback on that. >> Great. Definitely. Um I think with that we've just got a couple of reminders um to wrap up. Um for those who aren't aware, there's the uh Linux Foundation Open Source Summit in North America. Um that's happening in May uh 18th uh to the 20th in Minneapolis. Um the call for papers um has been extended to February 15th. Um so if anyone felt like uh applying might be a very good opportunity to highlight some of the research that we talk about in this working group. Um do feel free um otherwise I think that's it from our side. Any other business? Anything anyone wants to share? No. Okay. Wonderful. In that case um yeah thank you all for being here. Thank you again Alexandros for presenting. Um, thanks for the great conversation and also special thanks to Adrian for joining us as new co-chair which we're looking forward to very much. Cool. All right, everyone take care. See you next time everybody. Thank you. Byebye. Byebye., CHAOSS Funding Impact WG, Feb 11 2026, YouTube, 01kp8hpqwrbxsbrzqw26a56wmz.