Reinventing Python Tooling With Rust  ## Metadata - Author: [[The Changelog: Software Development, Open Source]] - Full Title: Reinventing Python Tooling With Rust - Category: #podcasts - URL: https://share.snipd.com/episode/242d0bd9-7a68-4581-95ba-c43175929c77 ## Document Notes ## Highlights - **One Installer, Complete Python Workflow** - Astral designs UV as a single installer that gives you everything needed to be productive with Python. - They model UV after Rust's Cargo to provide a unified, confident developer workflow. Transcript: Charlie Marsh Yeah. No, thanks. It's I honestly, I feel pretty lucky because I don't really spend a lot of my time thinking about how to get, you know, on top of the stack overflow developer survey. I get, I get to just spend my time like the thing and talking to users in the issue tracker and trying to make it better. And then it just keeps growing. So it's kind of like a dream job in a lot of ways for me. So UV is our Python package manager, Python tool chain manager. We kind of view it as the one thing you install that then gives you everything you need to be productive with Python. And there's a couple aspects that I think make it unique. One is, as with a lot of the things we build, it's really focused on performance. So we think a lot about how do we make things very, very fast, things like way faster than you thought they could be is sort of the way that we try to view it ourselves. But we're also trying to just take a lot of the complexity out of building with Python. I think like Python packaging has for a very long time been a thing that people have complained a lot about and often for good reasons, like they're users running into problems, people Having trouble trying to understand why things are so complicated or things aren't working the way they want. And so a lot of what we've tried to do with UV is cut through a lot of that complexity. In some cases, by bringing more functionality together. ([Time 0:04:10](https://share.snipd.com/snip/ffa9d2e5-cfa6-485e-b1d6-842b62efb4fe)) - **Ruff Laid The Groundwork For Astral** - Ruff began as a Rust linter side project while Charlie learned Rust, then grew into a company foundation. - That early open source traction convinced him packaging was the next vital problem to solve. Transcript: Charlie Marsh So we started building this like static analysis tooling and, you know, I started that just started as an open source project. And eventually I turned it into, into this company. And, um, you know, when I was looking at what we want to be as a company, it's like, okay, we want to be like the Python tooling company, let's say. Well, if we want to be the Python tooling company, I think we have to like take on the hard problems in Python. And for me, so for me, it was like, okay, we have to do something in packaging because every time you talk to people about using Python, they have this groan, which comes from installing Python or installing packages or setting up the environment. And for me, it was like a lot of people, I mean, something that's intimidating is like a lot of people have actually tried this. Like there are lots of tools and that's because like a lot of people have tried to do different takes on it. And so for me, it was a little bit of like, we have to do this both to prove that we can. And because it seems like the most important tooling problem in the ecosystem. So if we're going to try and like, really like lift up the ecosystem in some way, like this is the thing that we have to go after. And it was, you know, we thought a lot about, we think about this with everything we build, but it's like, why, what is like the insight or like, why can we build something? Why would we succeed here? ([Time 0:17:10](https://share.snipd.com/snip/2ef0f86f-8b85-47a1-a6b4-ba0b759d4535)) - **End-To-End Control Enables Better UX** - Building the entire packaging stack lets Astral make components aware of each other and automate workflows. - That end-to-end control enables features like auto-installing the correct Python and creating consistent environments. Transcript: Charlie Marsh We're going to do the entire stack. And so that, I think, is where a lot of the effectiveness of... The reason that we're able to do things differently, like a good amount comes from that, which is we kind of did, we were like, we're gonna do the whole stack and like, everything's going To be aware of everything else. And that lets us build experiences that kind of work better together and are like more automatic. Like when you go into a project, you install UV and you run UV sync. Like we can do everything from, we figure out what version of Python you need. We go install the prebuilt version of Python that we built. We put it in the right place. We resolve all your dependencies. We put them in a lock file. We create the virtual environment. We install everything into the environment. And then we run the command that you gave us in that environment. ([Time 0:19:19](https://share.snipd.com/snip/e3f14077-67c4-4079-ac3d-a320d952992d)) - **Iterate Publicly From A Focused First Release** - Release incrementally and scope the first version to real user needs, not every planned feature. - Use early, narrow releases to validate demand and iterate publicly with users. Transcript: Charlie Marsh And the thing that we did, we actually built the entire, the first, like I started working on it in like October, let's say of, I think 2023. And then we did the first release in February. So it was only a couple months and it was like three of us working on it. And the first release, we, we said, we actually found really good ways to kind of cut like what went into that first release. Like the first release was just a PIP compatible CLI. So all it did was like uv pip install um instead of pip install and uv vm to create virtual like it was very the cli now does a bunch of other stuff like supports like installing python we Have like lock files like installing global tools like all this stuff none of that stuff was in the initial release and so for us it was like the release, how do we prove that we can do this? Let's ship this very well scoped. It just does pip install and like pip uninstall, right. And like creates virtual environments. And that's actually useful for a lot of people. Like that actually solves a lot. It's not like some people looked at that and were like, oh, it's just faster pip. That's not interesting. And like what we're trying to do is much bigger than that now, but that's what we started with. And so for me, it's like a lot of the focus is on how can we, let's think like super critically about like use cases and how can we get something out there as quickly as possible. It's like useful to people that we can actually start iterating on in public and with users. And that's kind of been like, that drives, I think, a lot of how we build things. It's like the type checker is similar. It's like the type checker kind of has to be done to be useful. And so it's much, it's actually harder to build and you have to be willing to put in a lot more investment over a longer period of time. But when you can find a tool that you can get out a small version of, that's actually useful, like just, just ruthlessly asking yourself, what would it take for someone to actually be Able to use this? And then getting to that and getting into an iteration loop, I think was really helpful because we kind of proved we could build this packaging stuff. And then from February to, I don't know, August or whatever, we built this whole other part of the CLI and released it. And then we had a huge new launch around that. ([Time 0:24:58](https://share.snipd.com/snip/c3c66c76-1e2f-43cd-8116-4c5463c370ad)) - **Ship Small To Prove Product Fit** - Ship a narrowly scoped, useful first release to prove product viability and gather user feedback. - Iterate publicly from that small, working base to expand functionality over time. Transcript: Charlie Marsh And the thing that we did, we actually built the entire, the first, like I started working on it in like October, let's say of, I think 2023. And then we did the first release in February. So it was only a couple months and it was like three of us working on it. And the first release, we, we said, we actually found really good ways to kind of cut like what went into that first release. Like the first release was just a PIP compatible CLI. So all it did was like uv pip install um instead of pip install and uv vm to create virtual like it was very the cli now does a bunch of other stuff like supports like installing python we Have like lock files like installing global tools like all this stuff none of that stuff was in the initial release and so for us it was like the release, how do we prove that we can do this? Let's ship this very well scoped. It just does pip install and like pip uninstall, right. And like creates virtual environments. And that's actually useful for a lot of people. Like that actually solves a lot. It's not like some people looked at that and were like, oh, it's just faster pip. That's not interesting. And like what we're trying to do is much bigger than that now, but that's what we started with. ([Time 0:24:58](https://share.snipd.com/snip/77184a6b-6015-4515-9ebe-900e01bdb40d)) - **Make Performance A Design Enabler** - Prioritize performance to enable new developer experiences that would be too slow otherwise. - Optimize common fast-paths so features like UV run have negligible overhead for users. Transcript: Charlie Marsh So like UV run, when you run that command, it will make sure that your dependencies are in sync with the lock file, make sure that your lock file is in sync with your environment. Basically that your environment matches what your declared dependencies are. It will do that every time you run UV run. And the thing that I think is cool is like, that's kind of an experience you can only reasonably build if you have a really strong, uh, performant, like baseline. Because you can't, if that takes like, imagine that took like 10 seconds, then like you can't have 10-second overhead every time anyone tries to run a command. You'd be upset. Yeah. So for me, that's one of the cool things where it's like, okay, focusing on performance actually lets you build different experiences because you can build things that otherwise would Have been prohibitively slow before. ([Time 0:30:08](https://share.snipd.com/snip/a6f8798a-d72b-4166-a5f5-5236eef848ab)) - **Global Cache Cuts Time And Disk Use** - UV uses a global cache and links packages into projects to make installs near-instant and save disk space. - That approach mirrors pnpm/bun and dramatically reduces duplicate large native wheels like PyTorch. Transcript: Charlie Marsh So basically if you install a package once, we put it in the cache. If you then go to a different project and install the same package, we effectively just like simlink that package into your environment. It's not exactly a simlink, but the basic idea is for each package you install, we actually only keep like one copy, one real copy on disk. And all your projects just point to that, which makes installation incredibly like if you are installing something that's already been installed before. It's basically a no up. Itop. It just points the files to the right place. And that's something that PNPM and Bun do. And there are different tricks you can do on different file systems to make that faster or different depending on what's available. ([Time 0:34:54](https://share.snipd.com/snip/e329fee9-6193-4162-88ee-adcfdcb4b600)) - **Rust's Ownership Enables Safer Tooling** - Charlie prefers writing Rust and credits its ownership model for safer, faster tooling. - The borrow checker is hard initially but yields confidence and fewer memory-related bugs in production. Transcript: Charlie Marsh Yeah. I mean, I, I would say I prefer writing Rust to writing Python. But it's just... Well, first of all, I write a lot more Rust than I do Python now. So I've just spent a lot more time in the ecosystem. Um, Rust is, I think, actually quite a hard programming language to learn. Um, because it, well, it depends a lot on your background. For me, it was a hard programming language to learn because my background was, um, you know, I'd done a lot of Python, but also a lot of TypeScript, like maybe like two years of Java professionally, Like a little bit of Go, kind of barely, and then like not really any Rust. So I didn't have like a systems programming background, and I wasn't really used to this idea of thinking about, in Rust you would call it like ownership but thinking about memory management. And so when I first started writing Rust at my last company, someone else introduced it to the project, a really great engineer. But I didn't really know it. And so every time I went in there, I was trying to get in and out as quickly as possible. And I was thinking in terms of Python and trying to map that to Rust. Was like why is it so hard to like do a comprehension over a hash map and um basically like nothing made sense to me and so it took me like working on rough and kind of building something from Scratch and kind of banging my head against those concepts so i think really understand like what the language is about and the thing is now rust has this this this mechanism called the Borrow checker, which is like the thing that, the thing that makes sure that you don't break its rules around how memory works. Um, and it's not really like writing Python or JavaScript where you can just like create things and like pass them around and do whatever you want. You have to follow certain rules that the compiler enforces. The thing is for me now, I don't really think about the borrow checker anymore. Cause I've written a lot of Rust. I think in terms of that kind of ownership, like the thing that makes it hard, eventually that kind of goes away in my experience. It's like, I don't, I just write my code now in the way that I kind of know what work for the borrow checker. And I don't really feel like I have a lot of overhead from it. ([Time 0:39:23](https://share.snipd.com/snip/cb6d9707-45de-4fee-af46-7fa722a8d88c)) - **Profile, Then Optimize Representation** - Use a systems language and profile hotspots to squeeze orders-of-magnitude gains. - Revisit algorithms and representations (e.g., encode versions as integers) to cut repeated work. Transcript: Charlie Marsh It's a great question. I think it's pretty nuanced and will be hard to put numbers on. So like, I guess the way I think about it is like, we get some baseline from like being written in Rust. That's like probably faster than like the baseline that we would get from being in Python, but rough, I just, even as rough as an example has actually gotten like significantly faster Over time and it was always written in Rust. Right. So like, there's like a big, there's a huge Delta that you can get even within just like being Rust. Um, you know, like an example is like at one point we rewrote our parser, like the thing that takes Python source code and turns it into a syntax tree that we can analyze. And like, you know, the initial version we use was based on something called a parser generator, where you like, you kind of like write out your grammar, and then it generates the code For you to a certain degree, which can be really handy if you're working with something, which you're building a parser, because often you can describe a parser in terms of the grammar. Like, okay, you have the def is a keyword followed by the function name, blah, blah, blah. Right. And we ended up rewriting it to use, I guess what would be called like a handwritten parser. So we got rid of the parser generator and we kind of like wrote it out ourselves and it became like way faster and like several times faster it made rough as a whole like like 30 or 40 faster And so that's like the same that's all within rust right but it's like about thinking hard about um how you're that how you're doing things and um you know it also varies a lot like even If you look at rough and uv because in uv we're doing way more io and so a lot of being fast not all but a good chunk of being fast is trying to be smart about how you do io because we're either Like downloading things from the network or like writing things to disk or like reading things from the cache and writing them somewhere else. And so like I talked about the caching trick earlier, like you could also do that in Python. Um, it would probably be, the program would probably be slower if you wrote the exact same program in Rust and Python for a bunch of reasons. Um, but you could also do that to write a faster version, you know, in Python. Um, in rough, that tends to be less true. Like in rough, there is IO, like we have to read the files, but like beyond that, it's a lot more, um, I guess, like a compiler, like it's kind of like has to parse all these, all this source Code and then figure out how to efficiently like traverse it and collect diagnostics and report them back. Jerod Santo So you know i think rust like gives you a better baseline faster baseline how much better and twice as good three times as good i don't know it depends what you're doing but yeah same python Program well yeah yeah a python based linter and a rust based linter doing the exact same linting same exact same stuff i mean obviously there's a rough numbers no pun not intended yeah Charlie Marsh I don't know probably like probably like somewhere around an order of magnitude is okay so that's significant okay keep going yeah for sure um but yeah i think the other thing that i've Come to appreciate with rust is like it kind of like gives you the tools that you need to like optimize further and think really hard like like allocating memory is like pretty expensive. And so in relative terms, and so like a lot of what we think about when we try to optimize things is actually like, how can we allocate less memory or allocate memory less frequently or Whatever. And like in Python, it's actually just pretty hard to have control over that because like, you're not, you don't really have any control over like the alligator and like where memory Is getting created or destroyed it's definitely not staring you in the face like it is no but in rust yeah you're kind of thinking like we did this whole thing where we i went deep on i gave A talk at your rust where i like went through the exact design here but like when you run uv um we parse a lot of versions. Like it sounds like a silly thing, but just like package versions, like for a very complex resolution, that code might run like 10 million times or something. Cause like just parsing versions. And so we saw that showed up in a flame graph and like when we were profiling and we redesigned how we represented versions and we came up with the scheme whereby like 95 of versions, something Like that can be represented in a single U64 integer. Like we encoded them as an integer. So it's like, okay, the first eight bytes are the major version number, right? The next eight bytes are the minor version, et cetera, et cetera. And that actually had like a big, like a very measurable speed improvement. And like, that's not really something that you could think that would be intuitive to do or very easy to do in Python. Like in Rust, we have to think about how are things actually represented, for example. ([Time 0:50:11](https://share.snipd.com/snip/99e7788b-aa32-4537-b2dd-f94665ec6c8a)) - **Open Source Tools, Paid Hosting Complement** - Astral will sell hosted PYX registry services to companies while keeping core tools open source and free. - Paid hosting solves structural problems (artifact hosting, curated builds, compliance) open-source clients alone cannot. Transcript: Charlie Marsh Yeah. I mean, it's a little bit different because like we're, you know, at least right now we're like, we're now focused on this being like a public registry. This is a tool that's designed for companies. Like we work with lots of companies that either already pay for registries and we think we can build something a lot better or need to adopt some sort of internal registry as they grow Or like have problems that we think we can solve with the registry that we actually couldn't solve like just with the client um and so for us it's like how do we go and help more of those users And like fit the needs of those companies while continuing to build out like the open source so you know commercially like for us we're not looking to like charge money for like ruff for Uv like we view that as our like open source tooling, which we want to remain free forever, very permissively licensed. And what we're trying to do instead is kind of build these paid services that are complimentary and are sort of a natural evolution if you're already using our open source tool. So like I said, we talked to tons of companies who use UV, they buy registries. So we're going to go offer them a registry and we think it's going to be better in like a variety of different ways than the things that are out there already so like a lot of why we built this Thing and are building this thing comes from like i said or as i alluded to like problems that users have brought us that we can't really solve like in the open source, like maybe people Will come and they're using some private registry and it behaves in a way that's like incorrect. And we're like, okay, we actually can't help you anymore because like, that's a problem with like the software. And so just being able to offer them something different, I think is one manifestation of that. ([Time 0:55:54](https://share.snipd.com/snip/1abe9d17-84c8-4001-b752-ed702143b5d5)) - **GPU Awareness Needs Registry And Client** - GPU ecosystems lack standard metadata, so installs require curated registries and client-server coordination. - Astral's registry plus client detects hardware and serves matching curated builds to avoid user pain. Transcript: Charlie Marsh Yeah, so it's a problem for a couple of reasons. So in Python, Python has actually very good support for building and distributing native code, like native binaries. And it's part of why it's been such a success. It's both a result of and why it's been a big part of data science. If you think about installing NumPy or something, most of that is not Python. It's like a compiled binary that's like native code that's compiled for your machine. And when you go to install NumPy, you don't actually have to build that from source. You don't have to download the compiler and compile everything yourself. What happens is when NumPy does a release, they publish... I promise this will get back to GPUs. But when NumPy does a release, they publish builds for Linux, Windows, Mac OS, Mac OS Arm, like M1, M2, like Mac OS Intel, x86, all the different Python versions. So they pre-build these things and they put them on the registry. And the package manager knows how to look at your machine and figure out like which version of which NumPy build is right for your machine. So that's like, that's like central to Python. It's where a lot of complexity comes from in packaging, but it's also kind of a superpower because you can build like UV, you can install UV, you can pip install UV. And that's just a Rust binary that we basically turn into a Python package and publish. And you don't have to build it from source. Python standards let us do that. GPUs make things harder in part because of some of the gaps in the standards. In the standards, there's no way to express that I just built PyTorch and it was built for CUDA 12.8. Or I just built PyTorch and it was built for CUDA 12.6. Or I just built PyTorch and it was built for AMD RockM. These are just different kinds of GPUs. There's no way to actually express that. So when they build PyTorch, the PyTorch team builds PyTorch for a bunch of different architectures, and they basically have to hack in the way that they do this. They have to just find ways to encode it that aren't really codified by standards, which leads to a lot of complexity. There's no way in PIP, for example, let's say that you have an NVIDIA GPU on your machine. There's no way to do PIP install Torch and get the right Torch version based on your GPU. There's nothing in standards that would really allow them to do that. So that's kind of the problem that we want to solve, which is like... There's actually a second... What they do is they publish the different version of Torch. Each architecture basically gets its own registry. Wow. So they create an index for CUDA 12.8, an index for CUDA 12.6. And on the 12.8 index, they publish the builds that they built for CUDA12.8. And each library also solves this in a different way. So like JAX, which is like a Google library for working with GPUs, does like something totally different. So there's been a lot of experimentation around trying to make this work, but it ends up being pretty difficult. And there's sort of like a second order problem, which is that there are packages that build against PyTorch. And so it's not just like PyTorch needs a GPU. There are also other packages like VLLM, extremely popular piece of software for actually serving language models. Like if you actually want to do like, like run an endpoint that like does runs a GPU to like do predictions and give back data, you would use VLLM for that. That has PyTorch as a dependency. And when they build VLLM, they not only have to encode the GPU version, but also the PyTorch version. So it gets very complicated very quickly. And that's kind of like the complexity that we're trying to tame with the registry. So like GPU aware means a few things. One, it means that UV, the client, can actually figure out like what GPU you have on your machine. And then it can map that to basically the right endpoints in PYX. So in PYX, we have like curated distributions based on the hardware. And we go through and we not only take PyTorch, but we also pre-build a lot of other stuff that people commonly use with PyTorch. And we make sure all the versions are compatible and all the metadata is correct. ([Time 1:01:58](https://share.snipd.com/snip/50d3181e-091f-4840-9aee-ea891bdd0bff)) - **Use Support As Brand Building** - Treat every user interaction as an opportunity to build trust and brand loyalty. - Be responsive, fix issues quickly, and communicate clearly to compound goodwill over years. Transcript: Charlie Marsh Definitely. I mean, I think like, I don't really explicitly think in terms of like developer marketing, but like when I first started, um, like when I first released UV, I was trying to just explain To people as quickly as possible, as succinctly as possible, why they should care about this project. And so the read me for me, like the top of the fold of the read me had to capture that. It had to capture like, why is this interesting? And I remember when I did the launch too, I had like a little graph that was just like a benchmark graph of like UV rough versus like a bunch of other things. And like, I think that graph was like very important for like conveying to people the significance of like what's happening. So like for, for, for communicating to developers or communicating to anyone, really, it doesn't matter if they're like developers or not. It's like, you have to assume that you have basically like no, like you get like no attention. Like if you write a blog post for example you have to be thinking in terms of uh most people they might read the headline and they might read the tldr and they might look at the one image at The top but they probably won't read any of the text some people will and it's important to care a lot about what it says but like you have to be thinking in terms of how do i explain to people Why they should care about this as quickly as possible so like even just by thinking about that and productionizing it, I think as an engineer, you'll probably be doing more than a lot Of small... Also, by the way, caveat before we get into the actual interesting stuff, you definitely don't have to care about any of this if your goal is not to make your project very popular. You can just build stuff and publish it and not care about this at all. I think that's totally cool but if your goal is to get people to use your thing and care and then follow along you know i think this stuff matters a lot a lot of how i think about brand too like It's very holistic so like when people come to our repo and like they file an issue or like they ask a question in discord or something like i would always view those moments as like i have A moment i have an opportunity here to like make a friend or like win a fan or something or win someone who's going to support the project and so like you try to compound that over time and So when people would come in first of all when i was starting the project and anyone would come found issue i would just be so excited that they cared at all and i would kind of just focus On how do i give these people a great experience um and over time it becomes um even if i'm going to say no to what they're asking for how do i make sure that they have a good experience like As in they feel heard and respected and they understand why i said no and we've just focused on that a lot like we try to be really responsive in the open source and we try to like give people A good experience. And it takes a very long time for that to have an effect. But like compounding over years, I think it's had a huge effect on like our open source community and like how people view the project. And you have to take a very long term view towards a lot of these things. But again, that all connects to brand. Brand is not... It is the visual identity of the company, but also it's like, what do people associate with it? And I want people to associate with us what I hope is true, which is we want to be good, responsible, approachable, open source maintainers who are demonstrating responsible stewardship For these projects and that people can trust us um it takes a lot to keep this up to like doing the open source like even just trying to be really responsive in the open source is like a huge Investment of our time um like we could probably have the whole company like just maintain the open source and build nothing new and that would still be like full-time work for us but It it's also like with everything we release, we want to like maintain the quality bar that we have. You know, I think maybe actually a good example of something that we've tried to put into our brand is like, we try to fix things really quickly. And so even if we release something that's broken, we'll fix it really quickly. And that actually, I think has become a really helpful part of our brand. Because if something's broken, we fix it quickly. And then it gives people more trust that like, if something breaks, we will fix it quickly. If we ship something that's not finished, you know, we'll fix it quickly. So I think I just take kind of a pretty long-term view towards a lot of these things. And I try and think really hard about like, if I were a user, what would I need to hear? ([Time 1:30:06](https://share.snipd.com/snip/881948c3-665f-46d7-9ab6-df5692388556))