Reinventing Python Tooling With Rust  ## Metadata - Author: [[Changelog Master Feed]] - Full Title: Reinventing Python Tooling With Rust - Category: #podcasts - URL: https://share.snipd.com/episode/d38584ea-39e2-4f7a-ba47-c510b4cfaf75 ## Document Notes ## Highlights - **One Tool As A Trusted Entrypoint** - Astral models UV after Rust's Cargo to give a single, trusted entrypoint for Python workflows. - Combining functionality into one tool reduces cognitive load and improves developer confidence. Transcript: Charlie Marsh So UV is our Python package manager, Python tool chain manager. We kind of view it as the one thing you install that then gives you everything you need to be productive with Python. And there's a couple aspects that I think make it unique. One is, as with a lot of the things we build, it's really focused on performance. So we think a lot about how do we make things very, very fast, things like way faster than you thought they could be is sort of the way that we try to view it ourselves. But we're also trying to just take a lot of the complexity out of building with Python. I think like Python packaging has for a very long time been a thing that people have complained a lot about and often for good reasons, like they're users running into problems, people Having trouble trying to understand why things are so complicated or things aren't working the way they want. And so a lot of what we've tried to do with UV is cut through a lot of that complexity. In some cases, by bringing more functionality together. So you can just install UV and then it can manage things that previously you might've needed to distribute across a bunch of different tools. So maybe before you had to learn four or five tools, and now we say, hey, here's UV. It can do all that stuff for you and then partly also by coming up with some of our own kind of like workflows and apis that we've put into uv that we think make things uh basically make it Easier to get things right for users yeah we model uv a lot off after um we're very inspired by like how rust does tooling so uv is a lot of ways modeled after cargo which is like rust package Manager and with rust it kind of feels like you install this thing and then everything you do is like very high confidence like you kind of like know how to like install dependencies and Run code and test code and like that's what we want to get to with with uv is we want to give people kind of this very high confidence experience of working with python everything for russ Adam Stacoviak Really revolves around cargo right like cargo does pretty much all the heavy lifting so it's it's linting it's doing builds it's doing all the things yeah and it's i mean russ and um like Charlie Marsh Other new programming languages it has this like kind of second mover or i don't know about second mover but it has this late mover effect where they get to learn a lot about like what makes A programming ecosystem nice to work in. And so they got to do like it's very different position from when Python was being created. And it was like decades ago and everything kind of evolved very organically. And it wasn't really clear, like how serious things would be or like what was going to change. Like packaging kind of just like emerged as this like organic property of like people needing to share code and like distribute code. And for us, it's really different. It's like, okay, we're building a new programming language. Let's learn from all these things that people have built. And so in Rust, like cargo is very much like a blessed tool. It's like you install Rust with Rust up and then you use Cargo for everything. And in a lot of places, Cargo is actually kind of like a front end to other tools. Like Rust actually has a formatter called Rust Format. But you never really think about that as a user. Like as a user, you just run Cargo Format and that actually runs Rust Format. So Rust becomes kind of this like focal point for or sorry cargo becomes this focal point for how you work with rust and all of that design was very intentional so for us it's a little different Because we're like coming into this ecosystem that's decades old and like absolutely enormous which is like the python right system and we're trying to both like meet people where They are in a lot of ways and be like, Hey, we want to give you better tools that like don't require you to completely rethink like how you work. But also we sort of think about how do we build towards a very different experience and give people something that is very different for people who want to embrace kind of a different Way of working. ([Time 0:04:32](https://share.snipd.com/snip/a0a66530-7631-4245-b0df-249dfce06c74)) - **Ruff Began As A Rust Learning Side Project** - Ruff started as a Rust side project where Charlie wanted to learn the language and shipped a usable linter incrementally. - That early open-source traction convinced him there was room to expand into a full Python tooling company. Transcript: Charlie Marsh Totally. So, uh, like I said, we started with rough. So we started building this like static analysis tooling and, you know, I started that just started as an open source project. And eventually I turned it into, into this company. And, um, you know, when I was looking at what we want to be as a company, it's like, okay, we want to be like the Python tooling company, let's say. Well, if we want to be the Python tooling company, I think we have to like take on the hard problems in Python. And for me, so for me, it was like, okay, we have to do something in packaging because every time you talk to people about using Python, they have this groan, which comes from installing Python or installing packages or setting up the environment. And for me, it was like a lot of people, I mean, something that's intimidating is like a lot of people have actually tried this. Like there are lots of tools and that's because like a lot of people have tried to do different takes on it. And so for me, it was a little bit of like, we have to do this both to prove that we can. And because it seems like the most important tooling problem in the ecosystem. So if we're going to try and like, really like lift up the ecosystem in some way, like this is the thing that we have to go after. And it was, you know, we thought a lot about, we think about this with everything we build, but it's like, why, what is like the insight or like, why can we build something? Why would we succeed here? Whereas other people, I, I, I don't think it's that other people have failed, but it's like, why would we, if the space is really fragmented and some users are still having problems, Like what are we going to do differently that's going to make the, it's going to overcome that fragmentation or like overcome those problems? And I think part of it is just that um we had the resources and like the ambition to try and like do the whole stack because if you look at a lot of these other tools they kind of build on other Pieces and they have to like cut the cord somewhere and so for us it was okay we're gonna do packaging we're actually gonna do like the whole stack like everything from parsing dependency Specifiers, version specifiers, through to resolution, through to installation, through to managing Python itself, the Pythons you install and all the versions, through to actually Building those Pythons for you. We're going to do the entire stack. And so that, I think, is where a lot of the effectiveness of... The reason that we're able to do things differently, like a good amount comes from that, which is we kind of did, we were like, we're gonna do the whole stack and like, everything's going To be aware of everything else. And that lets us build experiences that kind of work better together and are like more automatic. Like when you go into a project, you install UV and you run UV sync. Like we can do everything from, we figure out what version of Python you need. We go install the prebuilt version of Python that we built. We put it in the right place. We resolve all your dependencies. We put them in a lock file. We create the virtual environment. We install everything into the environment. And then we run the command that you gave us in that environment. So like all of that goes away and that comes from being willing to say okay we're actually going to do like we're going to try to do like the whole stack um and i think that's where a lot of It came from i you know i think the other piece is just making good decisions about like where to be pragmatic and where to be like dog sort of dogmatic um like where to say this is something We really believe that we're going to like do differently and and being otherwise being other areas being willing to say okay we we need to do this for compatibility or like just it'll Break too many users and just like on the margin trying to make good decisions around behaviors is very hard i think we've gotten some i think we've gotten a lot of them right but some we've Gotten wrong and some we've like changed over time but like a lot of ultimately that too you know a decent amount of this comes from um having the resources to like work on this stuff full-time Um like being able to kind of rally people uh including investors and like bring in investors and say like this is an ecosystem that's like really worth working on and like we can do something Like we think really special and different here and so being able to bring people in full time and really pay you know put in all the engineering investment to build this thing the community Investment to like spend all this time in the issue tracker and understand like what's going well and what's not and fix things for people and be really close with the community and like Try to iterate really quickly so you know i think i think ultimately it comes from being able to have that level of ambition of like we're going to do something really different um and Jerod Santo Then uh executing that on that in a way that i think has been really effective so did you have that goal in mind when you started talking to investors because like you said you needed n years To do this or however many years you think it is or months months or years probably years right how long it took to build uv is that the question well even like to be able to bite off the entire Thing like that's what your goal is right it's like everything and so um if i'm a full-time developer python developer you know some place and i have this like you know what i'm going to Solve packaging for Python. I got to do that nights and weekends. Maybe I cut back my hours and, and work on it. Maybe I convince my boss I can work out at 20, my 20% time. But like you said, no, we're going to, we're going to do it all. We're going to do it right. Ground up to a certain extent. We're going to even installing Python through our tools. And so to do that, you raise money, right? Like that was because you need to have, you need multiple people for multiple years to actually get that done. And so was that your first step was like, I want to do this and I'm going to go raise money to do this, or I'm going to, I'm going to convince people, how did that all play out? Charlie Marsh Yeah. Yeah. So the first step was, um, I started working on the tools before I raised any money or started a company. So like I, I was actually, I'd left my last job. I was at a computational biology company I was in charge of a lot of all the sort of like software infrastructure, data infrastructure, machine learning infrastructure. We wrote a lot of Python. And I was kind of like figuring out what I wanted to do next. And I did, I was like looking to start a company, but I didn't really think it would be this. This was kind of like my side projects where I actually, I kind of like wanted, I wanted to learn rust. And so I started building this cause I was like, I think this could be cool, you know, et cetera, et cetera. And, um, and that was rough at the time. Yeah. Yeah. And, um, that project then really started to take off and I kind of realized there was an opportunity to take these similar ideas and extend them to other parts of the tool chain. Like we could build a package manager. At the time, it was just a linter. And it was like we could build a format or we could build a package manager. Now we're also building like a type checker and a language server. Like we're trying to build all this stuff. And I saw there was this opportunity. And, you know, I think a couple of things that come to mind. Is like very very important everywhere but especially in this context to find ways to demonstrate like incremental value and being able to get things out to users incrementally like Rough was a really good project for that because it was a linter so it's like a set of rules right and a set of... ([Time 0:17:06](https://share.snipd.com/snip/c8cf3e09-609a-4ecb-a8e1-a2cb62337454)) - **Build The Whole Stack To Automate Workflows** - Controlling the entire stack enables more seamless experiences because components can share knowledge. - Doing everything from version resolution to environment creation lets UV automate and reduce friction for users. Transcript: Charlie Marsh We're going to do the entire stack. And so that, I think, is where a lot of the effectiveness of... The reason that we're able to do things differently, like a good amount comes from that, which is we kind of did, we were like, we're gonna do the whole stack and like, everything's going To be aware of everything else. And that lets us build experiences that kind of work better together and are like more automatic. Like when you go into a project, you install UV and you run UV sync. Like we can do everything from, we figure out what version of Python you need. We go install the prebuilt version of Python that we built. We put it in the right place. We resolve all your dependencies. We put them in a lock file. We create the virtual environment. We install everything into the environment. And then we run the command that you gave us in that environment. ([Time 0:19:19](https://share.snipd.com/snip/8ec3e306-bd58-438f-911b-0bb5bfd41162)) - **Validate With Incremental Public Milestones** - Focus features around concrete user workflows that can be shipped incrementally rather than finishing a massive tool in private. - Use small public milestones to gather feedback and prove product-market fit. Transcript: Charlie Marsh So we were able to ship, like the first version I shipped was like not very future complete, but people could actually use it. And over time we could like extend and grow it. So it was usable very quickly and it grew quickly. And then we kind of expanded the scope of what it could do. It was something we could ship very incrementally um the formatter was maybe a good example of something that's not like this at all like a formatter is not useful until it's done like It has to be finished like it's not useful if a formatter can format like function definitions but nothing else a third of your code yeah so that was like a much harder a bigger challenge Where it's like we actually had to iterate not in private but like we didn't have like a useful release for like a long time. Like we were like, it was all in public, but it was like, no one was using it until it was done. And then when we got to the package manager, we took, we thought about that a lot. And the thing that we did, we actually built the entire, the first, like I started working on it in like October, let's say of, I think 2023. And then we did the first release in February. So it was only a couple months and it was like three of us working on it. And the first release, we, we said, we actually found really good ways to kind of cut like what went into that first release. Like the first release was just a PIP compatible CLI. So all it did was like uv pip install um instead of pip install and uv vm to create virtual like it was very the cli now does a bunch of other stuff like supports like installing python we Have like lock files like installing global tools like all this stuff none of that stuff was in the initial release and so for us it was like the release, how do we prove that we can do this? Let's ship this very well scoped. It just does pip install and like pip uninstall, right. And like creates virtual environments. And that's actually useful for a lot of people. Like that actually solves a lot. It's not like some people looked at that and were like, oh, it's just faster pip. That's not interesting. And like what we're trying to do is much bigger than that now, but that's what we started with. And so for me, it's like a lot of the focus is on how can we, let's think like super critically about like use cases and how can we get something out there as quickly as possible. It's like useful to people that we can actually start iterating on in public and with users. And that's kind of been like, that drives, I think, a lot of how we build things. It's like the type checker is similar. It's like the type checker kind of has to be done to be useful. And so it's much, it's actually harder to build and you have to be willing to put in a lot more investment over a longer period of time. But when you can find a tool that you can get out a small version of, that's actually useful, like just, just ruthlessly asking yourself, what would it take for someone to actually be Able to use this? ([Time 0:24:08](https://share.snipd.com/snip/ac6d3eb6-c4f0-4833-8847-8ba24a1dce23)) - **Ship Minimal, Useful Releases Fast** - Ship a small, useful version first and iterate in public to validate demand quickly. - Scope launches to the minimal user workflow that proves you can solve the problem and expand from there. Transcript: Charlie Marsh And the thing that we did, we actually built the entire, the first, like I started working on it in like October, let's say of, I think 2023. And then we did the first release in February. So it was only a couple months and it was like three of us working on it. And the first release, we, we said, we actually found really good ways to kind of cut like what went into that first release. Like the first release was just a PIP compatible CLI. So all it did was like uv pip install um instead of pip install and uv vm to create virtual like it was very the cli now does a bunch of other stuff like supports like installing python we Have like lock files like installing global tools like all this stuff none of that stuff was in the initial release and so for us it was like the release, how do we prove that we can do this? Let's ship this very well scoped. It just does pip install and like pip uninstall, right. And like creates virtual environments. And that's actually useful for a lot of people. Like that actually solves a lot. It's not like some people looked at that and were like, oh, it's just faster pip. That's not interesting. And like what we're trying to do is much bigger than that now, but that's what we started with. And so for me, it's like a lot of the focus is on how can we, let's think like super critically about like use cases and how can we get something out there as quickly as possible. It's like useful to people that we can actually start iterating on in public and with users. And that's kind of been like, that drives, I think, a lot of how we build things. It's like the type checker is similar. It's like the type checker kind of has to be done to be useful. And so it's much, it's actually harder to build and you have to be willing to put in a lot more investment over a longer period of time. But when you can find a tool that you can get out a small version of, that's actually useful, like just, just ruthlessly asking yourself, what would it take for someone to actually be Able to use this? And then getting to that and getting into an iteration loop, I think was really helpful because we kind of proved we could build this packaging stuff. And then from February to, I don't know, August or whatever, we built this whole other part of the CLI and released it. And then we had a huge new launch around that. ([Time 0:24:58](https://share.snipd.com/snip/4048d334-d636-45b4-84bf-af66e633d4a5)) - **Keep Environment Checks Instant** - Make runtime checks cheap so you can enforce correctness at command time without irritating users. - Optimize performance so commands like `uv run` can auto-sync environments every invocation. Transcript: Charlie Marsh Basically that your environment matches what your declared dependencies are. It will do that every time you run UV run. And the thing that I think is cool is like, that's kind of an experience you can only reasonably build if you have a really strong, uh, performant, like baseline. Because you can't, if that takes like, imagine that took like 10 seconds, then like you can't have 10-second overhead every time anyone tries to run a command. You'd be upset. Yeah. So for me, that's one of the cool things where it's like, okay, focusing on performance actually lets you build different experiences because you can build things that otherwise would Have been prohibitively slow before. And so, yeah, we didn't want to have this workflow where like you might run a command and you're actually your environment is like stale and you're like missing all the dependencies Or something. Like that happens all the time and with like NPM and node and stuff. And I'm like, I don't want that. Like I want the cargo version where like you do cargo run and it like takes care of resolving your dependencies, installing them and then runs the command with all the right stuff. ([Time 0:30:19](https://share.snipd.com/snip/3f02316b-0aea-4c90-8bef-89ef59d3eb29)) - **Global Cache Cuts Installs And Disk Use** - Using a global content-addressed cache and linking per-project avoids repeated downloads and saves disk space. - This design yields huge speedups and reduces duplicate native binary storage across projects. Transcript: Charlie Marsh So basically if you install a package once, we put it in the cache. If you then go to a different project and install the same package, we effectively just like simlink that package into your environment. It's not exactly a simlink, but the basic idea is for each package you install, we actually only keep like one copy, one real copy on disk. And all your projects just point to that, which makes installation incredibly like if you are installing something that's already been installed before. It's basically a no up. Itop. It just points the files to the right place. And that's something that PNPM and Bun do. And there are different tricks you can do on different file systems to make that faster or different depending on what's available. ([Time 0:34:54](https://share.snipd.com/snip/2029c7df-94f2-4fae-80af-3ba66b6b9fc5)) - **Borrowing Pays Off Over Time** - Rust's borrow-checker is initially steep but rewards you with safer, fewer memory bugs and confidence in large-scale tools. - Over time developers internalize ownership patterns and the borrow-checker becomes invisible. Transcript: Jerod Santo Bring more Pythonistas, but you're building them in Rust. So you must like Rust. Charlie Marsh Yeah. I mean, I, I would say I prefer writing Rust to writing Python. But it's just... Well, first of all, I write a lot more Rust than I do Python now. So I've just spent a lot more time in the ecosystem. Um, Rust is, I think, actually quite a hard programming language to learn. Um, because it, well, it depends a lot on your background. For me, it was a hard programming language to learn because my background was, um, you know, I'd done a lot of Python, but also a lot of TypeScript, like maybe like two years of Java professionally, Like a little bit of Go, kind of barely, and then like not really any Rust. So I didn't have like a systems programming background, and I wasn't really used to this idea of thinking about, in Rust you would call it like ownership but thinking about memory management. And so when I first started writing Rust at my last company, someone else introduced it to the project, a really great engineer. But I didn't really know it. And so every time I went in there, I was trying to get in and out as quickly as possible. And I was thinking in terms of Python and trying to map that to Rust. Was like why is it so hard to like do a comprehension over a hash map and um basically like nothing made sense to me and so it took me like working on rough and kind of building something from Scratch and kind of banging my head against those concepts so i think really understand like what the language is about and the thing is now rust has this this this mechanism called the Borrow checker, which is like the thing that, the thing that makes sure that you don't break its rules around how memory works. Um, and it's not really like writing Python or JavaScript where you can just like create things and like pass them around and do whatever you want. You have to follow certain rules that the compiler enforces. The thing is for me now, I don't really think about the borrow checker anymore. Cause I've written a lot of Rust. I think in terms of that kind of ownership, like the thing that makes it hard, eventually that kind of goes away in my experience. It's like, I don't, I just write my code now in the way that I kind of know what work for the borrow checker. And I don't really feel like I have a lot of overhead from it. So that's like the thing that makes Rust hard. I think at least when you're beginning is the borrow checker. And that kind of dissipates over time. And then it just becomes a lot easier. I mean, there are still plenty of things I complain about. Like I would like compile times to be faster, for example, et cetera, et cetera. But I do find that the way I think about programming now is better to Rust now. ([Time 0:39:19](https://share.snipd.com/snip/1487b4f4-b61f-4392-b0ae-224d9fa25fb7)) - **Combine Rust With Algorithmic Tuning** - Use Rust for a faster baseline and then profile to find high-impact algorithmic improvements. - Optimize data representation (e.g., encode common versions into a single integer) to save repeated work and memory. Transcript: Charlie Marsh It's a great question. I think it's pretty nuanced and will be hard to put numbers on. So like, I guess the way I think about it is like, we get some baseline from like being written in Rust. That's like probably faster than like the baseline that we would get from being in Python, but rough, I just, even as rough as an example has actually gotten like significantly faster Over time and it was always written in Rust. Right. So like, there's like a big, there's a huge Delta that you can get even within just like being Rust. Um, you know, like an example is like at one point we rewrote our parser, like the thing that takes Python source code and turns it into a syntax tree that we can analyze. And like, you know, the initial version we use was based on something called a parser generator, where you like, you kind of like write out your grammar, and then it generates the code For you to a certain degree, which can be really handy if you're working with something, which you're building a parser, because often you can describe a parser in terms of the grammar. Like, okay, you have the def is a keyword followed by the function name, blah, blah, blah. Right. And we ended up rewriting it to use, I guess what would be called like a handwritten parser. So we got rid of the parser generator and we kind of like wrote it out ourselves and it became like way faster and like several times faster it made rough as a whole like like 30 or 40 faster And so that's like the same that's all within rust right but it's like about thinking hard about um how you're that how you're doing things and um you know it also varies a lot like even If you look at rough and uv because in uv we're doing way more io and so a lot of being fast not all but a good chunk of being fast is trying to be smart about how you do io because we're either Like downloading things from the network or like writing things to disk or like reading things from the cache and writing them somewhere else. And so like I talked about the caching trick earlier, like you could also do that in Python. Um, it would probably be, the program would probably be slower if you wrote the exact same program in Rust and Python for a bunch of reasons. Um, but you could also do that to write a faster version, you know, in Python. Um, in rough, that tends to be less true. Like in rough, there is IO, like we have to read the files, but like beyond that, it's a lot more, um, I guess, like a compiler, like it's kind of like has to parse all these, all this source Code and then figure out how to efficiently like traverse it and collect diagnostics and report them back. Jerod Santo So you know i think rust like gives you a better baseline faster baseline how much better and twice as good three times as good i don't know it depends what you're doing but yeah same python Program well yeah yeah a python based linter and a rust based linter doing the exact same linting same exact same stuff i mean obviously there's a rough numbers no pun not intended yeah Charlie Marsh I don't know probably like probably like somewhere around an order of magnitude is okay so that's significant okay keep going yeah for sure um but yeah i think the other thing that i've Come to appreciate with rust is like it kind of like gives you the tools that you need to like optimize further and think really hard like like allocating memory is like pretty expensive. And so in relative terms, and so like a lot of what we think about when we try to optimize things is actually like, how can we allocate less memory or allocate memory less frequently or Whatever. And like in Python, it's actually just pretty hard to have control over that because like, you're not, you don't really have any control over like the alligator and like where memory Is getting created or destroyed it's definitely not staring you in the face like it is no but in rust yeah you're kind of thinking like we did this whole thing where we i went deep on i gave A talk at your rust where i like went through the exact design here but like when you run uv um we parse a lot of versions. Like it sounds like a silly thing, but just like package versions, like for a very complex resolution, that code might run like 10 million times or something. Cause like just parsing versions. And so we saw that showed up in a flame graph and like when we were profiling and we redesigned how we represented versions and we came up with the scheme whereby like 95 of versions, something Like that can be represented in a single U64 integer. Like we encoded them as an integer. So it's like, okay, the first eight bytes are the major version number, right? The next eight bytes are the minor version, et cetera, et cetera. And that actually had like a big, like a very measurable speed improvement. And like, that's not really something that you could think that would be intuitive to do or very easy to do in Python. ([Time 0:50:11](https://share.snipd.com/snip/d9349169-fca4-492a-a640-7ab136ed54b8)) - **PYX: A Hosted, Curated Registry** - Astral will offer PYX as a hosted registry for companies needing curated artifacts and GPU-aware builds. - The registry aims to complement open-source UV rather than replace public PyPI. Transcript: Jerod Santo Bit a little. So surely you want to avoid some of the problems that they've had. Um, and I'm curious the design of PYX and, you know, the ambitions here and how you're going to go about building similar, a registry that's commercial. Charlie Marsh Yeah. I mean, it's a little bit different because like we're, you know, at least right now we're like, we're now focused on this being like a public registry. This is a tool that's designed for companies. Like we work with lots of companies that either already pay for registries and we think we can build something a lot better or need to adopt some sort of internal registry as they grow Or like have problems that we think we can solve with the registry that we actually couldn't solve like just with the client um and so for us it's like how do we go and help more of those users And like fit the needs of those companies while continuing to build out like the open source so you know commercially like for us we're not looking to like charge money for like ruff for Uv like we view that as our like open source tooling, which we want to remain free forever, very permissively licensed. And what we're trying to do instead is kind of build these paid services that are complimentary and are sort of a natural evolution if you're already using our open source tool. So like I said, we talked to tons of companies who use UV, they buy registries. So we're going to go offer them a registry and we think it's going to be better in like a variety of different ways than the things that are out there already so like a lot of why we built this Thing and are building this thing comes from like i said or as i alluded to like problems that users have brought us that we can't really solve like in the open source, like maybe people Will come and they're using some private registry and it behaves in a way that's like incorrect. And we're like, okay, we actually can't help you anymore because like, that's a problem with like the software. And so just being able to offer them something different, I think is one manifestation of that. But the other is like, sometimes people want to install packages that are like broken in different ways, or maybe there aren't builds available for like their, their Python version, Or maybe there aren't builds available for their GPU or something like that. And in that case, again, that's not something that we can solve with the package manager. Cause to solve that problem, we actually need like a server that has like artifacts that we can manage and so again it's like trying looking at these problems where we've spent a bunch Of time in the issue tracker trying to help people and ultimately concluded that there's only so much we can do and saying well what if we had our own registry our own server couldn't we Jerod Santo Do something pretty different that makes tons of sense is there a world that you can envision in which astral would want to host a public registry to make UV better in some sort of way that Charlie Marsh You can't do it as just a client? Maybe. I guess anything's possible. It's not like in our current mission for the problems we're trying to solve here. And I think, I guess what I would prefer to see happen is like PyPI, which is sort of the Python equivalent of NPM. It's like a public, it's the public registry that people use by default. It's run by the Python Software Foundation. So it's owned by a nonprofit. I want to make sure that that has longevity and continues to evolve and be stable because I think it's a really, like, I'm happy for that to be kind of like the public system of record and For people publishing packages and the way that, you know, the way that most people are maybe installing packages. But for us, like we mirror that stuff in, for example. So like you can use PYX to install things from PyPI. And we're not necessarily focused on like the public serving we're more focused on what's the experience we can build around the raw artifact storage um and are there are also are there Like other things that we can expand to over time um that aren't a registry but are related to hosting and serving artifacts um like we've also thought about like code execution for example Um, um, like maybe you should be able to execute code through, you know, through our tooling and maybe that would have some tie-ins with the registry too. So for us, it's not like, you know, we're not like a registry company. Like we're like a company that's trying to build this whole Python developer experience. And the registry is kind of like the first extension into building something beyond like a command line tool. So it's like a big, thing for us. ([Time 0:55:40](https://share.snipd.com/snip/ed29eb20-6aa1-48ed-8fd6-b3128b5bdc05)) - **GPU Packaging Needs Hardware Metadata** - GPU complexity arises because packaging standards lack hardware metadata like CUDA versions. - A registry plus client that share hardware-aware contracts can cure much of the ML packaging pain. Transcript: Charlie Marsh Yeah, so it's a problem for a couple of reasons. So in Python, Python has actually very good support for building and distributing native code, like native binaries. And it's part of why it's been such a success. It's both a result of and why it's been a big part of data science. If you think about installing NumPy or something, most of that is not Python. It's like a compiled binary that's like native code that's compiled for your machine. And when you go to install NumPy, you don't actually have to build that from source. You don't have to download the compiler and compile everything yourself. What happens is when NumPy does a release, they publish... I promise this will get back to GPUs. But when NumPy does a release, they publish builds for Linux, Windows, Mac OS, Mac OS Arm, like M1, M2, like Mac OS Intel, x86, all the different Python versions. So they pre-build these things and they put them on the registry. And the package manager knows how to look at your machine and figure out like which version of which NumPy build is right for your machine. So that's like, that's like central to Python. It's where a lot of complexity comes from in packaging, but it's also kind of a superpower because you can build like UV, you can install UV, you can pip install UV. And that's just a Rust binary that we basically turn into a Python package and publish. And you don't have to build it from source. Python standards let us do that. GPUs make things harder in part because of some of the gaps in the standards. In the standards, there's no way to express that I just built PyTorch and it was built for CUDA 12.8. Or I just built PyTorch and it was built for CUDA 12.6. Or I just built PyTorch and it was built for AMD RockM. These are just different kinds of GPUs. There's no way to actually express that. So when they build PyTorch, the PyTorch team builds PyTorch for a bunch of different architectures, and they basically have to hack in the way that they do this. They have to just find ways to encode it that aren't really codified by standards, which leads to a lot of complexity. There's no way in PIP, for example, let's say that you have an NVIDIA GPU on your machine. There's no way to do PIP install Torch and get the right Torch version based on your GPU. There's nothing in standards that would really allow them to do that. So that's kind of the problem that we want to solve, which is like... There's actually a second... What they do is they publish the different version of Torch. Each architecture basically gets its own registry. Wow. So they create an index for CUDA 12.8, an index for CUDA 12.6. And on the 12.8 index, they publish the builds that they built for CUDA12.8. And each library also solves this in a different way. So like JAX, which is like a Google library for working with GPUs, does like something totally different. So there's been a lot of experimentation around trying to make this work, but it ends up being pretty difficult. And there's sort of like a second order problem, which is that there are packages that build against PyTorch. And so it's not just like PyTorch needs a GPU. There are also other packages like VLLM, extremely popular piece of software for actually serving language models. Like if you actually want to do like, like run an endpoint that like does runs a GPU to like do predictions and give back data, you would use VLLM for that. That has PyTorch as a dependency. And when they build VLLM, they not only have to encode the GPU version, but also the PyTorch version. So it gets very complicated very quickly. And that's kind of like the complexity that we're trying to tame with the registry. So like GPU aware means a few things. One, it means that UV, the client, can actually figure out like what GPU you have on your machine. And then it can map that to basically the right endpoints in PYX. So in PYX, we have like curated distributions based on the hardware. And we go through and we not only take PyTorch, but we also pre-build a lot of other stuff that people commonly use with PyTorch. And we make sure all the versions are compatible and all the metadata is correct. ([Time 1:01:58](https://share.snipd.com/snip/c77715fc-ac18-493e-8092-d18bedea5d33)) - **Treat Support As Brand Building** - Treat every user interaction as a branding opportunity by being responsive and respectful in issues and support. - Invest time in clear, succinct public messaging to earn trust and compound brand value. Transcript: Charlie Marsh Definitely. I mean, I think like, I don't really explicitly think in terms of like developer marketing, but like when I first started, um, like when I first released UV, I was trying to just explain To people as quickly as possible, as succinctly as possible, why they should care about this project. And so the read me for me, like the top of the fold of the read me had to capture that. It had to capture like, why is this interesting? And I remember when I did the launch too, I had like a little graph that was just like a benchmark graph of like UV rough versus like a bunch of other things. And like, I think that graph was like very important for like conveying to people the significance of like what's happening. So like for, for, for communicating to developers or communicating to anyone, really, it doesn't matter if they're like developers or not. It's like, you have to assume that you have basically like no, like you get like no attention. Like if you write a blog post for example you have to be thinking in terms of uh most people they might read the headline and they might read the tldr and they might look at the one image at The top but they probably won't read any of the text some people will and it's important to care a lot about what it says but like you have to be thinking in terms of how do i explain to people Why they should care about this as quickly as possible so like even just by thinking about that and productionizing it, I think as an engineer, you'll probably be doing more than a lot Of small... Also, by the way, caveat before we get into the actual interesting stuff, you definitely don't have to care about any of this if your goal is not to make your project very popular. You can just build stuff and publish it and not care about this at all. I think that's totally cool but if your goal is to get people to use your thing and care and then follow along you know i think this stuff matters a lot a lot of how i think about brand too like It's very holistic so like when people come to our repo and like they file an issue or like they ask a question in discord or something like i would always view those moments as like i have A moment i have an opportunity here to like make a friend or like win a fan or something or win someone who's going to support the project and so like you try to compound that over time and So when people would come in first of all when i was starting the project and anyone would come found issue i would just be so excited that they cared at all and i would kind of just focus On how do i give these people a great experience um and over time it becomes um even if i'm going to say no to what they're asking for how do i make sure that they have a good experience like As in they feel heard and respected and they understand why i said no and we've just focused on that a lot like we try to be really responsive in the open source and we try to like give people A good experience. And it takes a very long time for that to have an effect. But like compounding over years, I think it's had a huge effect on like our open source community and like how people view the project. And you have to take a very long term view towards a lot of these things. But again, that all connects to brand. Brand is not... It is the visual identity of the company, but also it's like, what do people associate with it? And I want people to associate with us what I hope is true, which is we want to be good, responsible, approachable, open source maintainers who are demonstrating responsible stewardship For these projects and that people can trust us um it takes a lot to keep this up to like doing the open source like even just trying to be really responsive in the open source is like a huge Investment of our time um like we could probably have the whole company like just maintain the open source and build nothing new and that would still be like full-time work for us but It it's also like with everything we release, we want to like maintain the quality bar that we have. You know, I think maybe actually a good example of something that we've tried to put into our brand is like, we try to fix things really quickly. And so even if we release something that's broken, we'll fix it really quickly. And that actually, I think has become a really helpful part of our brand. Because if something's broken, we fix it quickly. And then it gives people more trust that like, if something breaks, we will fix it quickly. If we ship something that's not finished, you know, we'll fix it quickly. So I think I just take kind of a pretty long-term view towards a lot of these things. And I try and think really hard about like, if I were a user, what would I need to hear? ([Time 1:30:06](https://share.snipd.com/snip/ad3fbe44-34da-4601-beb1-0e3042f16d3c))