Setting Docker Hardened Images Free  ## Document Notes ## Highlights - **Make Secure Images The Default** - Docker recognized that minimizing image surface area and faster upstream patching reduce supply-chain risk significantly. - Hardened images combine minimal packages, faster patching, SBOMs, SALSA provenance, and signing to make images more secure by default. Transcript: Tushar Jain We can explain that too. Perfect. So let's start at the beginning, before you do that. I'm going to assume everyone knows Docker and Docker Hub. Everyone builds containers, you use images. Docker Hub is effectively upstream for open source container images. We get billions and billions of pulls per month. Everyone pulls from us. And these are the repositories or images of usable open source software. And not just upstream base images, but like, you know, I want MySQL on Debian. You get that from us. That works. Great. We've been doing this for a decade. But we basically keep up with upstream. As a result, images do have CVs. They've lost to CVs caused by multiple reasons. One, bloated stuff, bloated packages that are built for usability first. As a result, have many packages in them or just not patching fast enough. Hard-end images, and this is a concept that has started the industry even before we launched a product, which is let's first minimize this problem. What people started doing was we left scanners in production. They'll see when there's a CV. We'll go alert some teams. They'll have to go update and patch, and this is sort of the world I lived in. Instead, why do we need all these images? Can we first minimize them? Let's get minimal packages that are only what we need. Second, can we have someone patch these faster, and then we drive that so we release the burden on engineering teams. This is the movement that started. It's very natural for Docker to do this. And so we launched Docker images as a paid product early last year. So hardened images, base images, app images that are minimal, low to no CVs, backed by an SLA by us. When we launched, we had a limited catalog, and we've been aggressively growing that. Our vision was always, like Docker is like broad adoption, get tooling and content out to everyone. So vision was always, we need to make this accessible to everyone. And then for enterprises, we provide things enterprises care about, compliance, and we can cover what is an enterprise package. But for everyone out there, they should be able to a great starting point and a secure starting point. So that was a vision always. We just had to build up to that. So that's what we got to last year and launched that out there. ([Time 0:04:56](https://share.snipd.com/snip/eb0a66a4-e906-40c5-a11a-c5b227ea3f06)) - **Three Pillars Of Image Trust** - SBOMs, SALSA build provenance, and VEX disclosures form the foundational guarantees for trusted images. - Docker enforces SALSA-level reproducible builds and publishes VEX statements to reduce scanner noise while remaining transparent. Transcript: Tushar Jain Everyone gets an S-BOM, our build pipeline salsa is. Salsa. Adam Stacoviak That's how you say it, salsa? Salsa. Tushar Jain That's how you say it, salsa. It's fun. There you go. Adam Stacoviak I'm not going to trip over my acronym ability then and just do Salsa. Tushar Jain And so that is how we build these. So we have a Salsa level three build pipeline. We actually open sourced our builder for this. What's been interesting is to do this, we had to change how we build images. It's still using like Docker build underneath the covers but we moved away from Docker files to our own semantic layer, to our new build as well, I can explain that, and then our build System. It's been interesting as we've done this, supply chain security is a broad topic and secure content is one part of that. Securing your build system is another part of that that's key. So it's been interesting to have done this now. Lots of companies are interested in our build pipeline. So that's the next thing we're looking at is exposing that as technology to everyone. So everyone can have secure build pipelines. And we'll keep going down this route of how to basically see the supply chain is just critical. Not just for traditional container stuff, but we'll also see this with AI when we talk about that. Adam Stacoviak You said you moved away from Dockerfile? Tushar Jain Is that right? Concretely, I mean, it's still using the same technology on the covers, but just we built our own, we open source this too. We built our own YAML syntax here just to make builds more repeatable. Like in Dockerfiles, you can shell out, you can do stuff, remove that. So in this syntax, it's very repeatable, it's reproducible. So you can actually do these in a way where you meet salsa requirements. It's still a build kit underneath the covers. ([Time 0:08:28](https://share.snipd.com/snip/0d9ab3f6-b075-4cc7-913e-3a7d797c2c8a)) - **Use Multi-Stage Builds For Safer Production** - Use multi-stage builds and separate dev vs production images to keep production images minimal and safer. - If you need extra packages, add them via a hardened build pipeline so SBOMs and provenance remain intact. Transcript: Jerod Santo Just say i'm a working developer with a couple of servers out there in the wild and they're all dockerized maybe i got a postgres server and my base image is like debian or alpine or something Basic and then i apt get install postgres and i you know my docker file does all the things or whatever what do i gain by switching to a docker hardened image and what do i potentially lose Or what might i hit up against when I try to do that? Tushar Jain Yeah, so in terms of what you gain, so like two ways, we generally have two flavors of hot images, like a development image and a production image. In production, in development, you want stuff. Like you need a package manager, you need shell, you need debug, you know, all these things, right? You need the VZ debug. Cool, we'll give that to you, still minimal. Really, then we recommend a multi-stage build for your production images. You don't need that stuff necessarily, so minimize that. There are trade-offs here, primarily to do with usability and how you manage that. So first, the images you get from us, maybe you need a few more packages. We've customized build for that. You run that through a build pipeline, we'll add those in. Those are still hardened packages we're putting in and you still get all of S-BOM, Salsa, VX, all that carries forward. But, you know, if you've just built a project and doing it, a lot of times, a lot of people, migration is really easy. Sometimes you've done stuff where like, okay, I have to figure out what's my trade-off between usability and security and what am I managing there. And if I've built my system in a way where I can't put this up well, or I really depend on shell access in production, then those are trade-offs I'm making. And so those are typically the challenges that a number of projects can run into. For the most part, from a lot of our customers, we hear the vast majority of their projects are able to migrate easily to this. We are also looking at building an agent here to help do this. We've got initial versions of running internally. We use it internally. And then we'll start building that up. ([Time 0:13:29](https://share.snipd.com/snip/8f8aac37-f044-4677-857d-dd8b97b726bf)) - **Try Hardened Images First, Assess Trade-Offs** - Try replacing base images with Docker Hardened Images to lower risk; the migration effort is often small. - If you depend on shell access or debug tools in production, evaluate trade-offs and consider dev/prod splits or migration tooling. Transcript: Tushar Jain Yeah, maybe just as an anecdote first, we did a webinar, I want to say, a couple weeks ago, something, I forget when. That was pretty broadly attended. I think my favorite question from that was, so is there any reason I shouldn't use Docker Art Images? There really is like, no, you shouldn't. Jerod Santo There's really no reason why you wouldn't want to just have a hardened image, you know? There's really no reason, right? Tushar Jain This is part of the reason for opening this up, like, you should go do this. So early on, we had lots of good traction with customers and working with enterprise deals. Since we've open-sourced this, also to be fair, we open-sourced it right before the break, like I think December 16th or something is when our... Not open-source, but the free tier. That's when our launch was. But even then, we saw immediate interest and pickup, and so we're tracking open-source packages, adopting this dramatically. Like a much any tenant's gone to it. And then with customers, it's resonated. So like CISOs like it, head of platform like it, in part because, you know, we're seeing play out what we hoped, which is, okay, now someone on the team, typically someone has a mandate Of like, oh, I should go. And people care about this problem. The barrier for them to go adopt and try it out and see the benefit is low. It's basically zero. They can just do it. And then they're like, okay, now I want all these, you know, additional security guarantees. I just think, great, now we can have a conversation about the paid tier. So we've seen an uptake for sure. And there's this motion playing out where like, opusus adoption, top of funnel with companies, and then we start working through this. ([Time 0:15:49](https://share.snipd.com/snip/6c7aaf5c-caa3-42e2-b944-fb4f1202d36e)) - **Fast Team Formation And Rapid Launch** - Docker formed the Hardened Images team in early February and reached limited release within months, then pushed to GA by summer. - The decision to make the catalog free was finalized in November with a rapid four-week push to launch the free tier. Transcript: Tushar Jain Let's see. I'll try to jog my memory. So I think what I'll say is, so when you see a joint, I would say, I think, Feb of last year, that's when the site is rolling around, and the first team is like, yep, we're going to do this, We're going to launch it. So I think from that point, people with skill set, but they formed a team at that point. So, I don't know, early Feb, mid-Fe something like that. And we got it out into, um, like limited release, like early, uh, limited release. And I want to say three months where customers GA'd it, you know, within the next three months. So that got us to like summer ish. Then we grew, kept growing, growing, growing. And then I think the real, we knew we wanted to make it free. We weren't sure when, but I think the real thing was like, oh, we should work towards free. December, I want to say it was like, honestly, it was like maybe early November. When November was like, okay, we're doing this. Or like right around, yeah, it was close to Thanksgiving, it's whatever it was called somewhere on there. And so from then till there was like a, probably like a four week sprint. ([Time 0:23:40](https://share.snipd.com/snip/7483fa9f-1003-4b24-a79d-0217b1387941)) - **Treat Supply-Chain Security As Urgent** - Move quickly: Docker formed a focused team and shipped Hardened Images within months, iterating afterwards. - Treat supply-chain security as an urgent, cross-functional company priority rather than a slow optional project. Transcript: Tushar Jain Let's see. I'll try to jog my memory. So I think what I'll say is, so when you see a joint, I would say, I think, Feb of last year, that's when the site is rolling around, and the first team is like, yep, we're going to do this, We're going to launch it. So I think from that point, people with skill set, but they formed a team at that point. So, I don't know, early Feb, mid-Fe something like that. And we got it out into, um, like limited release, like early, uh, limited release. And I want to say three months where customers GA'd it, you know, within the next three months. So that got us to like summer ish. Then we grew, kept growing, growing, growing. And then I think the real, we knew we wanted to make it free. We weren't sure when, but I think the real thing was like, oh, we should work towards free. December, I want to say it was like, honestly, it was like maybe early November. When November was like, okay, we're doing this. Or like right around, yeah, it was close to Thanksgiving, it's whatever it was called somewhere on there. And so from then till there was like a, probably like a four week sprint. Jerod Santo That's all pretty good. It's all pretty impressive. You said you have a good team there. I mean that I was expecting longer. So I guess, you know, congrats to you and the team for really a pretty quick turnaround. Tushar Jain Yeah. Well, since day and age, you don't have time. Everything. Jerod Santo You don't have time. Yeah, you better get it done yesterday. Tushar Jain You better get everything done yesterday. ([Time 0:23:40](https://share.snipd.com/snip/0f2e4900-fbd4-4a68-b4d7-d26d0b581dcf)) - **Free Tier As A Long-Term Play** - Docker sees its role as a core part of the software supply chain and thus has both an opportunity and responsibility to lead security. - Making hardened images free drives broad adoption while paid tiers cover enterprise compliance and long-term support. Transcript: Adam Stacoviak Can you speak to the tension and what it took to sort of own the responsibility? Tushar Jain Yeah, absolutely. So, you know, these ideas go way back, right? It can go to when Google Digital started, right? And so like, a lot of the ideas go back. And so discussion has been there like, what else should we do here? What to build or how to manage this? I think there's been lots of like, in the past discussion of how should we do this? And then it's like, is there a big enough business here? Should we go after this? How do we think about this versus what else we're doing across the company, etc.? So this discussion has definitely been there for some time. And what's the best way to do it? I think a few things came together for us in Feb. One, we should change clarity of, yeah, we're doing this stuff, we're going to do it. Second, I'd say on the technical side, clear clarity on how we should do this. Like, no, the serpents were like, we're going to do this differently, and here's how. Even concretely, VEX statements are a thing the industry is adopting now, and we're helping in driving that. We've broken all the scanners of, like, this is why you should adopt it. It's a standard, but there's no universal adoption of here. And we're driving that forward and making that happen. So the tension is definitely there. But before that, I've been a year and a half. I'd say even before my time, it's been there. It's one of those topics that's been in the industry for a while. And then the real thing was like, no, we should do this. And that was both business clarity and second, I'd say, technical clarity on how to do this. And then on the speed, we built a bunch of stuff, but we get to leverage a lot of Docker underpinnings, right? We've got Pilkit here, we've got Docker Engine here, we've got Hub here. We get to leverage all of that for how we get to go drive this and make it happen. Yeah, I think the core part is just realizing this will come from like, we are a, not really, we are a core partpart of supply chain. ([Time 0:26:38](https://share.snipd.com/snip/b6694844-77f5-4b73-ba72-eb867ce09202)) - **Roadmap Beyond Base Images** - Docker will expand from hardened images into hardened system packages, language packages, and long-term enterprise patching. - The plan includes secure build pipelines, migration agents, and broader content coverage to harden the entire stack. Transcript: Tushar Jain Yeah, so a number of things. One, we're going to roll out hardened system packages. Also today, a lot of system packages that you want come from upstream repos. We're going to start offering our own hardened system packages, built from source. We'll patch ahead if and when needed. So we started doing that and that'll come out. We're also going to look at language packages. We'll attack that language by language and go into that and get those out. On the enterprise side, we'll look at long-term support. Typically, packages have LTS there after two years. You stop getting patches from upstream for three years. You can buy long-term support from us, so we can continue patching. And typically, for enterprises, for various reasons, they move slower. And so that's important there. So we expand. The way to think about this like expand the breadth and coverage of all the things, of all the content you would care about. Let's get that out. The next thing after that for us, I think, is a secure build pipeline. This is not another thing we're trying to look at. Duke is seeing all the interest here. And so we have to figure out how exactly we'll do that. We want to get this out so anyone who's building software should run on us and get the benefits of a SaaS-3 build pipeline and work on getting that out from there. And then I'd say last thing, and this is like I mentioned, parts of this we started is, well, I really want to get some agents out here that help you with either migration or help you with Understanding your state of affairs and get you how to like, how to get them secure. ([Time 0:37:55](https://share.snipd.com/snip/b02d1f8d-f118-4105-a3ec-8084dc0625f0)) - **Use Scout Scores To Vet Images** - Check the image Scout health score and vulnerability details before adoption to understand residual risks. - Favor images with A scores and follow up on any listed medium or high CVEs that may need attention. Transcript: Jerod Santo So I've been cruising your hardened images directory or catalog as you do. And I've been looking at a few of these and it's very cool. I have some questions around like the security summary so i'm looking at the php image based on debian 13 92 packages so that's pretty slim um seven tools included if you're in around The php image and it has one medium severity vulnerability 10 low severity vulnerabilities six unspecified severity vulnerabilities. I assume those are upstream vulnerabilities that you know about because you're not doing hard in packages. Like those things are just like you're patched up as far as you can go, but there's just known vulnerabilities. Is that what those mean? Tushar Jain Good question. So the lows and unknowns, since my step with the medium curious. I'm going to go look at it afterwards. Typically those, if there's any high, we work with upstream or we'd go ahead and do it. Medium should fall in that category too. I think for us, it should be something we go after soon. So I'll look at that one afterwards. But generally, high and critical, of course, and even mediums, we try to get ahead of and drive quickly. Jerod Santo So when you have, like, say there's this medium here and we don't know what it is, I can't seem to find if it lists what that is somewhere. I think that'd be a pretty cool addition. Tushar Jain It should be. If it isn't, yeah, it'd be a good addition. Yeah, that'd be a sweet addition. Jerod Santo I do see a full security details and it still shows the vulnerabilities list, but I can't seem to find it at the moment. Anyways, is that then, is that a known CVE against a package or against one of these 92 things that have been installed? Yes. But that doesn't necessarily mean that there is a patch or is there a patch that just hasn't been applied? Tushar Jain If there was a patch, we apply it really fast, like ours, like there's another patch, you know, or an unlikely case. Yeah, like not a patch. But even then, we typically try to go work at it and get a patch in place. That's what we do for high. So the medium, it depends a lot where we are on that. Jerod Santo Yeah, I mean, you got lots of packages, you got lots of images. It's probably a never-ending task. It's just to continuously be... Tushar Jain Yeah, so there's behind this, there's a machine running, right. Uh, of, uh, totally software and people. Jerod Santo So then the other question I have about is this scout health score, which maybe it's not as new to me. Is that new in general or just new to me? Tushar Jain Scout should be something we've had for some time. Um, scout is our own scanner that we've had, and it's, it's our own scanner that scans everything. And now we've just put it in here so uh you can see the health score that's there uh we've given it you know uh in hub package owners can see the health score for their packages that they're Publishing um and now here we've done it so anyone can see the health score packages we're publishing here yeah with the dhi yeah that's super cool so this one has an a score and it has all Jerod Santo The reasons like no high no high profile vulnerabilities no fixable critical or high vulnerability signed supply chain attestations no embedded secrets no embedded malware like On and on and on and i assume there's a score for every image you'll have on here there should be a score for every image and there should not be any score that's lower than an A. Tushar Jain And if there is, I'll follow up on that. Jerod Santo Well, if we have a, let's see, filter my scout score and just say, show me the ones that are B's or lower, and then get to work. Tushar Jain You mentioned one thing there, which is like, no unbearable secrets, etc. So that's another thing where it's not just for reducing the package playing CDs. We go through actually like, you know, list of stuff like what makes the thing secure. And it shows us that like there's no credentials in that none of this stuff. And keep in mind, we're getting lots of patches from upstream all the time. And we scan every single one of them. ([Time 0:39:34](https://share.snipd.com/snip/2d19ba87-1427-4685-9263-b034fece7201)) - **Principles Over One-Size Technicals** - The principles behind securing container builds (provenance, reproducibility, transparency) can likely be adapted beyond images to other package ecosystems. - Docker suggests extracting core principles first, then mapping specific technical components per domain. Transcript: Tushar Jain Is actually, I think the first thing is at least extracting the principles and the end goals, being very clear about that. We have some core principles that we applied. And I say that because the hows might differ depending on the domain of what's needed. And so then the second one, what can you extract at a technical level to that? I am not sure. I'm sure there's stuff here, but especially if I think about stuff outside container images land, then it's interesting and a little different. But the principles definitely do, and the approach does in terms of common things that you can pull out there. I think there are things here, for example, like the way we're building, yes, we made it for images and containers, but I suspect that if you sit down and look at it, core parts of that stuff, We can pull out and make it work for non-container stuff too, maybe. I'm winging it a bit here when I say that. But there's core parts of how do you do a build pipeline that should apply, I think, a little more generally too, as an example. There are parts of some of the AI agents we're running that can apply more broadly than just for, because they run at the code level to verify security of all the thousands of upstream PRs, patches we're getting, right? So there might be stuff like that. ([Time 0:50:20](https://share.snipd.com/snip/618f9c26-83e9-40fe-a7c0-cd4b6c5e3bcd)) - **Start New Projects With Hardened Defaults** - Start new projects with hardened images so secure defaults propagate across the ecosystem. - Influence documentation, templates, and agents to recommend Docker Hardened Images as the default starting point. Transcript: Tushar Jain One, I'd love to see this be the default starting point. What is needed to get in the place when I'm building something new? Why not start with Docker hot images? And what all is needed to achieve that? And I expect it's a mixture of technical and non-technical things that are needed there. One, it's like, you know, for someone who's starting, like, where did they learn how to start? How do we make this be the default easy path? For a lot of people, it's like, I'll copy what someone else did, or I'll just do whatever ChatGPT tells me to do. So, like, how do we go influence all these places and let this be a starting point for everyone? Because that should be a key thing. If I jump forward in years in the future, great. The next popular open source package starts with just DHI, because why wouldn't it? That's the thing you do. And so if you go achieve that. And the reason that matters to us, apart from, you know, the artistic goal or just the real goal of like, makes office security, like on ([Time 0:56:46](https://share.snipd.com/snip/945260fa-0d01-4ae8-9929-28120dda64d7)) - **Secure Runtime For Untrusted AI Workloads** - Docker is building a secure runtime and sandbox model for untrusted AI workloads with microVMs, network proxies, and credential injection. - The goal is to let coding agents run productively while enforcing guardrails, credential safety, and governance locally and in the cloud. Transcript: Tushar Jain That's, you know, so we've done all this, but like the AI stuff is clearly a big focus for us. And it sort of comes together in my mind. Maybe I'll talk about how we think about it. I just think, you know, how I think about this a bit. So look, Docker, you know, everyone uses it. It's a core part of the SCLC. We help code go from laptop to production. And we kind of solve for like, you know, the big growth of apps that happened over the last decade, right? Cloud-native apps. Everyone moves to the cloud. Everyone builds services. Containers are a way to do that and drive that. We solve all that with our packaging, with the content distribution, with the engine for building and running. Well, there's two big shifts happening now, right? Everyone is, the entire SDLC is changing. You're developing coding agents, but literally the entire SDLC, how you build, test, publish, run code is going to change and become AI first. And second, the kinds of applications you're writing are going to be agents now, the next growth of apps are agents. So I kind of see a very natural thing for us is like, you know, I think of Docker as our job is to help engineers and engineering team securely built and deliver software. And so we've done that over the last decade for the way things have worked. And now we're adapting that for AI. A core thing for us that we've had with Docker and Docker images and even security trust we talked about, but even more important with AI and agents, I think is trust. The thing is, if you're going to trust across this layer here for agents, that'll help people trust agents more. But see, today everyone's writing to the code, but no one's shipping to the code. In large part, because there's not core trust yet. Even when I've used an agent, how to let it run unfettered? Do I trust all this output? How do I test all of this? How do I get this out? How do I know what it's building with? So that's sort of a framework we have. And we're starting with a runtime environment. So we've got Docker, Docker Engine. This is how you build and package and run isolated environments with containers. Well, now we've got a slightly different thing with agents. So with coding agents, we think, I frankly think it's crazy. Everyone runs coding agents, flatten the machines, and then just like NPX installs MCP servers and runs them in the machine. Talk about supply chain risk. And so great. But to be clear, the productivity benefits here are crazy, right? What I want to do is I want to use an agent and let it run. I don't want to run a YOLO mode by default, go do everything, but just give me some security. So we're going to adapt our engine, build a new engine. The way I think it was, we're building a new runtime engine for untrusted workloads. It should be a place where you can go put in a coding agent. It needs a computer. It should be able to go run, do everything it can do with security guardrails. So by default, we'll have micro VMs here. So if you go look, we have our initial start of this style should have just come out. If you look at Docker sandboxes, you can do Docker sandbox run cloud, spins up cloud in a micro VM. Around that we have network proxies that are outside the VM. So when it tries to go outside of the network, we run through proxy layer. We have a credential layer here where you don't have to give it your credentials. But if it tries to talk to GitHub, we inject the credentials outside. So the agent doesn't need to know your secure content or data. You control files it has access to, control where it goes, but let it run. Then for MCP, the server we're adding our MCP security. We have a trusted registry of MCP servers. These are MCP servers that we've vetted. We run through security hardening here. We're also building DHI versions of these and a gateway that's plugged in. And the gateway is where we can start injecting a lot more security rules too. And so the vision is built towards a secure runtime for untrusted workloads, you know, folks and coding agents. And we'll make this work both locally and remotely and give you the same thing so you can be working locally, but we'll have a cloud that'll be coming out soon. And we actually have some early partners who are working on this already. But as a developer, I should just start in our Docker sandbox. I get all the property benefits. I run my agent unfettered. I get all the security benefits. I'll get cloud. And for enterprise, they can manage governance around all of this stuff. And then we tie supply chain security in here as we're doing this, create by default, build on the secure content we've talked about, get that in here, et cetera, et cetera. ([Time 1:00:20](https://share.snipd.com/snip/7e4931d3-c28a-4250-ba1a-27fa6131dfd2))