The Linux Foundation in the Age of AI  ## Metadata - Author: [[The New Stack Podcast]] - Full Title: The Linux Foundation in the Age of AI - Category: #podcasts - URL: https://share.snipd.com/episode/d3cd8296-5e9f-487c-a940-012cbd70c96d ## Document Notes ## Highlights - **AI Stack Is Layered And Open-Source Driven** - The AI ecosystem has distinct layers: training, inference, and emerging agent stacks built on open source. - Open source tooling (like PyTorch) underpins training while open-weight models and inference projects push open AI forward. Transcript: Jim Zemlin Question you know the the ai stack that has been emerging around a few different vectors um obviously the thing that got everybody's attention about ai and it's kind of crazy to think That it was it wasn't that long ago, is when frontier models came out, starting with ChachiPT, and people started realizing how incredible and transformative that technology was. And so for a long time in the very short span of AI's relevance in tech, there was so much emphasis on training frontier models. And sort of what it was based historically on machine learning and creating ML models. But now we're training these huge models. And so you had sort of the training stack from an open source perspective, with PyTorch and TensorFlow and other technologies. The frontier models, you know, kind of early on weren't really open source, but the training stack, for sure, entirely open source. And it's really PyTorch, which is a Linux foundation project that has emerged as the key pillar of training frontiering models. And then you sort of had our deep seek moment this year where all of a sudden we have these open weight models still trained using that open source training stack. But now you really have in the open model realm, you know, sort of performance equivalent to proprietary models. And you, you know, see a lot of these emerging from China. Now in the US, OpenAI has responded. I think we're going to see more open weight models coming from various US-based companies here in Europe. ([Time 0:00:50](https://share.snipd.com/snip/5a4b83aa-11fb-4d28-9ce4-57414d0f28e2)) - **Inference And Agents Are Becoming First-Class** - An inference stack and agentic layer are now emerging with open-source projects enabling deployment and reasoning. - Most of this runs on cloud infrastructure and often in Kubernetes environments. Transcript: Jim Zemlin What has then started happening is you see the emergence of AI reasoning and inference and an inference stack emerging as well. I should mention that most of this runs on the cloud and most of it runs in a Kubernetes world. But in the inference stack, you have things like VLLM, LLMD, Ray, open source ecosystems emerging that are really, you know, enabling that world as well. You know, and there are proprietary applications and UX and things that are built on top of those. But really, if you look at kind of the core inference tech, it's all open source as well. And then now we're sort of moving into the agent world. So now you have an emerging agentic. I'm not sure what term the industry is going to, you know, is it agents? Is it agentic AI or whatever? Agentic stack. But there's an emerging agentic stack built on open source and open protocols, you know, things like MCP, A2A, you know, Cisco has opened a bunch of technology in a group of projects Called Agency. And so, you know, we're really seeing that world emerge. ([Time 0:03:17](https://share.snipd.com/snip/d3f67b61-1af5-4d99-a564-818c4346af48)) - **Avoid Forcing One Big AI Foundation Now** - Don't rush to force a single umbrella foundation for AI while the technology and communities are still forming. - Let projects grow organically and provide neutral governance only where it helps without stifling innovation. Transcript: Jim Zemlin I get asked a lot, should there be one big AI foundation, open source initiative to rule them all? I mean, I guess that would be nice and convenient in terms of being able to pool resources at a large scale, having kind of a few bigger events, so maybe a little less travel for a lot of the Folks who are working on this technology. But, you know, it's really hard to predetermine this stuff as the technology is emerging. And I think the important thing to do is a few things. One, when really cool technology in the open source community is emerging, my number one rule is don't screw it up. Like, you know, MCP as a project and a community kind of knocking it out of the park. And what you want to do is for any of those kind of projects, be a place where folks are confident that can come and neutrally govern these together, but you don't want to screw it up. You want to make sure that the developers who are working on this stuff are continuing to innovate in the way that they already are. The second thing you want to think about is, you know, sometimes maybe things can be too big, you know, and sometimes things can be too small, depending on what the goals are, you know, To create sort of a ubiquitous Kubernetes to move the world from sort of virtual machines to container-based orchestration. That required a lot of resources. It required changing the mindset of, you know, the entire world in terms of how they deploy cloud applications. And so having a big organization where the world could kind of come together in this new modality of container-based infrastructure, back when people hadn't even thought of telemetry And things that we have today in a way that the Cloud Native Computing Foundation has made flourish. I think balancing big versus small agility is an important thing. And so what we try to do is be a little bit, you know, hands off and like let things grow. But eventually, if, is a utility in having a larger collective working on moving the entire ecosystem towards a perspective on agent technology that doesn't get in the way of organic Innovation, we want to support that. ([Time 0:06:43](https://share.snipd.com/snip/0c64ae48-8840-4969-9fc5-561f1d5a09c9)) - **Protect Emerging Open Projects' Momentum** - When promising open-source projects emerge, protect their communities and governance rather than breaking what works. - Give developers confidence to continue innovating without heavy-handed interference. Transcript: Jim Zemlin One, when really cool technology in the open source community is emerging, my number one rule is don't screw it up. Like, you know, MCP as a project and a community kind of knocking it out of the park. And what you want to do is for any of those kind of projects, be a place where folks are confident that can come and neutrally govern these together, but you don't want to screw it up. You want to make sure that the developers who are working on this stuff are continuing to innovate in the way that they already are. ([Time 0:07:22](https://share.snipd.com/snip/abc4534d-d1d9-48e9-b043-cef90e2e4759)) - **Frontier Models Demand Huge Resources** - Building frontier models requires enormous capital and large curated datasets that open groups struggle to match. - Open-weight models are valuable pragmatically even if full end-to-end openness (including data) isn't yet feasible. Transcript: Jim Zemlin Is kind of a long answer to this this question because i think it's a complex topic. The first thing I think about when I look at what the big labs are doing around frontier models is first that it's sort of the sport of kings, for lack of a better term. Which starts with an incredible amount of capital expense that is required to stand up the computing infrastructure and networking and so forth to actually do that work. Labs about a collective, I don't know what it is right now, but well over $100 billion, a couple hundred billion dollars in CapEx. In China, you see similar figures around just an enormous amount of capital expense standing up compute infrastructure to build these frontier models. For an open source collective project consortium, that's a big number. The Linux Foundation is sort of the largest organization of our type, and we're definitely not in the billions at this point. Allen Institute recently announced some government funding in the United States. I think it was around 150 million U.S. Dollars towards creating fully open frontier model. But, you know, you compare that to the billions that are being spent in these frontier model developments, it's pretty impressive. Like, when you're building a frontier model, you not only need all this CapEx, but you also need big data sets. Another thing that is pretty tricky for an open source effort to be able to put together. You have things like Common Crawl and you have a set of open data sources, but those data sets are not big enough compared to a lot of the work that's going on in the big frontier labs. And then you get sort of to the actual training and the weights, which are open, and then the fact that those open weight models are freely available to people to use. I think that's all a good thing. I think that the open community would love to see all the way back to open data, everything being completely open. And who wouldn't want that? But I think pragmatically speaking, given capital expenses, the access to data, you don't want to look at the open weights models and say, well, you're not pure enough and look a gift Horse in the mouth to some extent. So I do think that the open weight model trend important and good. I do think we will eventually see, you know, end-to open models, open data all the way through. ([Time 0:10:20](https://share.snipd.com/snip/7144d4c9-469d-49e5-90f5-b4ceadf6c5fa)) - **Label Model Openness Clearly** - Use nuanced labels to describe model openness so users know exactly what they're getting. - The Linux Foundation's Model Openness Framework defines levels so developers can understand restrictions and access. Transcript: Jim Zemlin In fact, in anticipation of that, the Linux Foundation created a framework called the Model Openness Framework. This is a way to sort of have a more nuanced view of what an open model is. From level one being the most open to level three being really more open weights. And when I talk to developers on this topic, what they tell me is, you know, I just want a model I can use, download and play with uh they're not in the ai world as much of sort of you know like Everything has to be open um maybe eventually they'll get there but what they do want and this is the important part is to know what i'm getting right right you know which parts open which Parts not there restrictions? Are there not? And that's kind of what the model open framework tries to do, which is not to define, you know, what should be more or less open, but to say like, OK, this is the highest level, this is the Second highest level so that people know what they're getting. ([Time 0:13:33](https://share.snipd.com/snip/453f113d-b623-4399-bf86-783957436a9d)) - **China's Long Road To Open-Source Strength** - China embraced open source early to kickstart a domestic software ecosystem and trained many developers through collaborative projects. - That long-term engagement helped China produce major open-weight models and organic innovation before the DeepSeek moment. Transcript: Jim Zemlin Going back and forth from the US to China for 20 plus years, you know, in the early days of the software industry in China itself, the biggest criticism of the world was that there's just A lot of pirated software in China, which was true at the time, like back in the 90s and then early 2000s to some extent. And when China moved to enter the World Trade Organization, they needed to make sure that their intellectual property was sort of shored up and get their intellectual property act Together, so to speak. And that's when China started really embracing open source, because here was this way for them to be able to share software, to kickstart a domestic software ecosystem in a way that Was pretty incredible. We don't just see that in China. We see that all over the world. We see that in the US, right, in terms of all industries leveraging open source to create interesting software and innovation. And so pretty quickly, folks in China started realizing like, wow, this open source is great. We're going to build on top of it. And they did all sorts of things to kind of create different Linux operating systems at the time or red flag Linux and all these different Linux operating systems. But what really started happening that was pretty interesting is, you know, government policy was created to sort of be very favorable to open source, pretty light touch. In telecommunications really started a wholesale, not only using open source, but bringing developers from those organizations into the open source community to learn alongside, You know, the greatest developers in the world, you know, whether it's Linus Torvalds in operating systems or some of the core maintainers of Kubernetes that came out of Google and Microsoft and Amazon and other places. And eventually, they were peers to a lot of those elite developers. And what happened, it was long before DeepSeek, is you sort of had the Chinese software industry reach sort of an escape velocity where they weren't just kind of taking open source and Using it domestically, kind of modifying it. They were creating their own organic innovation inside of China with, you know, net new open source projects coming out of that. ([Time 0:15:00](https://share.snipd.com/snip/4a9a6cf3-4ad2-48c3-a78f-e322330a274d)) - **Open Source Faces Growing Regulatory Scrutiny** - Regulators are increasingly focused on open source because it's ubiquitous, so maintainers must consider compliance. - The Linux Foundation builds tools like the Baseline Project to help projects assess and meet regulatory requirements. Transcript: Jim Zemlin So, you know, there was an initial concern around some of the cybersecurity policy that came out of the CRA, Cyber Resiliency Act, out of the European Union. Fortunately, the open source community was able to influence and educate regulators on that in order to sort of reduce the negative impact that it could have had on open source technology. I think that the open source community and downstream users of implementers of open source technology are starting to get their arms around how to deal with that. One of the things the Linux Foundation is working on is a project we call the baseline project. So this is a tool comes out of our open source security foundation that looks at open source technology and projects and sort of does a risk assessment of how, you know, is this in line With regulatory compliance around the world, not just the CRA, but other regulatory regimes around the world. Which are getting more and more complicated. Sure. And I think that, you know, I sort of have talked in the past about open source has become so successful, so ubiquitous, that it's not actually that surprising that regulators have taken Notice. You know, the CRA doesn't just apply to open source software, it's a proprietary software as well, but because open source is such a ubiquitous building block of any modern technology, Which it's no surprise that regulatory compliance is something that the open source community has to pay attention to. The important thing to do is make that regulatory compliance as seamless and straightforward as possible so that you don't disrupt the rapid organic innovation that happens in the Open source. That's the Linux Foundation's goal, is to make sure that we understand these policies, that we build tools that can help maintainers and developers within our communities to comply With regulations as seamlessly. And hopefully for a maintainer, you know, they don't really need to do anything at all because we're helping them with that. ([Time 0:18:52](https://share.snipd.com/snip/6161f03b-763b-448d-91bb-dd6af13d6424)) - **Use AI To Multiply Developer Productivity** - Treat AI as a productivity multiplier and equip developers with trustworthy AI tools across the development lifecycle. - Provide AI-assisted tooling for documentation, testing, and security to increase co-development efficiency. Transcript: Jim Zemlin You know, the joke is sort of, I'm flipping the question, which is, you know, everyone talks about how important open source is to building frontier models you know, rolling out inference Technology and agents. But the question is, like, I think everyone has to think about is how is AI technology going to impact our industry? And you see all these headlines about, you know, I think Mark Benioff, the CEO of Salesforce said, you know, we're not going to hire any more software developers this year. AI is taking care of all of that. You see kind of a lot of doomsayer headlines about how recent college graduates are not able to get jobs because they've been replaced by AI in the software world. So you sort of have to ask yourself, are there going to be more or less software developers because of AI? What are the ramifications of tools that can help automate tasks that have typically been very tedious for open source developers? And how do we embrace those in a positive way? And so, yeah, we have to think about that. I would say my opinion on whether or not there's going to be more or less open source developers in the world is probably going to be more. I think the developers that I speak with see AI technology as a productivity tool, as a force multiplier. And that doesn't mean that there's going to be fewer software engineers. That means that we're going to have a lot more interesting software out there because these developers are able to work much more efficiently using AI technology. One of the things that we're working on inside the Linux Foundation is at every stage of the software co-development process, what interesting AI tools are out there, whether it's To help automate documentation or testing or whatever, security review, how can we provide those AI tools in a trustworthy way to our communities, uh, so that they can become more efficient. ([Time 0:21:33](https://share.snipd.com/snip/8a469b62-5f69-4fad-97b2-4f8e5613884f)) - **Maintainers Drowned By AI-Generated Noise** - Maintainers are swamped by low-quality AI-generated bug reports and PRs, adding to pre-existing overload. - Zemlin suggests using AI to filter this noise and reduce the burden on maintainers. Transcript: Jim Zemlin I was talking to some of the linux kernel developers about this uh yesterday they so a lot of the ai uh for software development doesn't cover C, maybe as well as some of the abstracted Programming languages. So the kernel team gets a few of these a week, where it's obviously AI-generated code or feedback or like I found a bug. Yeah, yeah. But it's not enough to be like incredibly distracted. In some of the more abstract program language communities, you know, like they get a lot of this, you know, where people probably good natured are using AI tools where they think they're Finding a bug, where they trust something that's actually a hallucination. And you are getting a tremendous increase in, you know, people, you know, submitting issues, to filing bugs and so forth that are not really genuine, I think it's a real problem for a Lot of maintainers. This is on top of maintainers that were already kind of overwhelmed pre-AI because maybe their particular MPM package, for example, got super popular. And, you know, people, again, in a good natured way would be, you know, submitting, you know, feature requests, bug fix requests, and so forth. And the maintainers all sound like, oh my God, this is overwhelming. So it would be interesting to look at how we can, AI to also root out some of that DDoS that comes in. Seems to be the answer these days. We'll use AI. Sometimes the answer for problems created by technology is more technology. And so in this case, that might be true. And more standards. ([Time 0:24:15](https://share.snipd.com/snip/c43b76ce-2918-4909-9fbb-d8472aedc1c5)) - **Using AI For Research, Not Final Writing** - Zemlin uses AI for research and structuring his writing but not for the creative act of writing itself. - The Linux Foundation also uses AI for practical tasks like creating project logos and boosting organizational productivity. Transcript: Jim Zemlin Software development at all. First of all, I don't do much software development at all. But I do a lot of writing. And so I write in a weekly newsletter to the Linux Foundation staff and our board of directors. And I sort of want to cover all the news around open source that's relevant to the work that we're doing and provide sort of a take on why, you know, this particular aspect of the open source World is important to understand, whether it's around intellectual property or some really cool new open source project and in a particular industry vertical. And so I do a lot of writing. And I find AI tools to be pretty darn good for helping with that, but not in the actual writing itself. Good, I'm glad to hear you say that. Not in the actual. I think that's, and I'm a bit of a snob on this one, so I will say that AI writing is exactly what you would expect it to be, which is pretty predictable, decent, but not great. And it's just, you can spot it a mile away, right? For sure. What it is very good at is structuring your thoughts, doing research. I think all of the major, whether it's Gemini or OpenAI or Cloud, the research tools are really, really good. You didn't mention any open source ones. No, I think the open source ones are great as well. I use them all. And it's clearly good at going out and allowing you to kind of consider different perspectives, collect your thoughts, and it makes me a far more efficient writer. And so in that context, I'm using it. Inside the Linux Foundation, we use it for all kinds of stuff. We have a lot of new projects that come in, and every project would like a logo, for example. And so creating the newest logo for our open source projects is something that used to take quite a bit of time. Now it takes really no time at all. Just all kinds of tasks that we do. ([Time 0:26:15](https://share.snipd.com/snip/4b824a07-e5dc-4c72-b196-95b0b17ca0fa))