OpenTF for an Open Terraform  ## Metadata - Author: [[The Changelog: Software Development, Open Source]] - Full Title: OpenTF for an Open Terraform - Category: #podcasts - URL: https://share.snipd.com/episode/0b9434fe-592f-4088-af43-416dc477353a ## Document Notes ## Highlights - **Gruntwork's Contributions and Concerns** - Gruntwork contributed heavily to the Terraform ecosystem, including Terragrunt and the book *Terraform Up and Running*. - HashiCorp's license change introduced uncertainty about Gruntwork's compliance, leading them to explore alternative solutions. Transcript: Josh Padnick And what was frustrating about that announcement is, so I'm the co-founder and chief product officer of Grunt work, and we've contributed an enormous amount to the Terraform ecosystem. And we've produced TerraGrunt and my co-founder wrote the book Terraform Up and Running. We've published all these blog posts. And so when this announcement was made, it immediately introduced uncertainty about whether we would be compliant with the license. And I can keep going with the story, but ultimately we figured out what some of the challenges with that license were. And then we reached out to some friendly competitors, and we decided to band together to do something about it. ([Time 0:06:15](https://share.snipd.com/snip/d54d8a73-3212-4fc4-986e-1baa799e47d3)) - **Technology Choice as Investment** - Choosing a technology is like an investment, and developers invest time in learning tools like Terraform. - A business source license would have discouraged early adoption as it limits usage and control. Transcript: Josh Padnick I mean, absolutely not. When we pick a technology and its infancy, you are making an investment choice. It's almost like picking a stock. Like, when I think personally about how many thousands of hours of my career have been invested in Terraform, it's like I made this investment to learn it back in 2014, 2015, and that Investment has paid off well, and I'm really happy I made that investment. If at the time Terraform had been a business source license, I would have said, wait a minute, no one's going to use this because everything has to be through HashiCorp. So I'm just going to invest elsewhere in other technologies. I'm guessing most people would have thought some of the things. There's actually an interesting parallel. It's not an exact analogy, but HashiCorp released a policy as code language called Sentinel a few years ago. The technology was quite well done. It was well documented. It was really exciting when it came out, but it was not open source. In fact, it wasn't even usable or accessible without purchasing like Terraform cloud or Terraform enterprise. Somehow you had to pay money to get it. And so it never took off. Then an alternative open policy agent or OPA came on the scene. I don't have a personal opinion on that, which technically is better or worse, but it's just a fact that OPA has become the dominant policies code ([Time 0:09:06](https://share.snipd.com/snip/0135ef64-2b2a-4213-a976-a8201508e0c0)) - **License Ambiguity and Control** - The HashiCorp license's vagueness and dynamic nature create uncertainty for businesses in the Terraform ecosystem. - This uncertainty necessitates seeking HashiCorp's permission for almost any activity, hindering innovation and competition. Transcript: Josh Padnick And ultimately, what it all boils down to when you really think through the dynamic is you have to get HashiCorp's permission to do anything. If there's even a hint of ambiguity, the only realistic way to proceed as a business is to get their official sign off to say, yes, what you are doing is okay. Or what you are doing will require a license and here's what it will cost. So then if you play that thought out, then it's like, okay, from a business perspective, what is my world as a member of the Terraform ecosystem look like? I now need to think about when I'm choosing to invest in this tool, Terraform, and offer services on top of it, what happens if HashiCorp changes their mind in the future? What happens if they decide, no, no, no, we are competing? What happens if we license Terraform from them, but they decide they want to increase the price? What happens if they have new owners and the new owners have a different interpretation of that FAQ? If you think about presenting those scenarios to an investor or to an acquirer, those are non-starters for any startup. Like, it's this huge amount of risk that you'd have to be taking on. And the only practical way to eliminate that risk is to get HashiCorp's official sign off, which brings me to the final conclusion of this thought process, which is that I personally Do not want to be an active participant in an ecosystem where everything I do needs to be blessed by one party. ([Time 0:15:35](https://share.snipd.com/snip/4260213d-96b3-4dac-b22a-9394695ac818)) - **Terragrunt's Unintentional Competition** - Gruntwork's Terragrunt, while open-source, became a competitor to HashiCorp's paid products. - This was because Terragrunt's features made it a preferable alternative, inadvertently hindering HashiCorp's business. Transcript: Adam Stacoviak You consider Groundwork a competitor to HashiCorp? Josh Padnick You know, that is such an interesting question because we were like co-opetition for a while. On the one hand, Groundwork probably did more to promote Terraform than almost any other company other than HashiCorp. I mean, that feels hyperbolic when I say that. But we published all these blog posts. We wrote TerraTest, the official, or not, I guess it's not official, the most popular testing framework for Terraform. We created TerraGrunt, which is a way to use Terraform at scale, also open source, and sits on top of Terraform. Like I said earlier, my co-founder. He wrote the book Terraform Up and Running. Our whole business model is working with teams who want to adopt Terraform and giving them best practices and getting them up and running with all the things that they need to do. We call it DevOps Foundations. So in that sense, we were in every way supporting Terraform. But what was interesting about all of that is because we wrote TerraGrunt, and because we felt TerraGrunt was the better way to launch your infrastructure and to manage it. It's not obvious when you start, but when you start to think through like, eh, how do I manage the blast radius? And when do I do about global variables? And how do I set module default values? And how should I organize my folder structure? That's where TerraGrunt starts to become useful. So we did almost no marketing of it. It took off as a project. And the problem with TerraGrunt, from HashiCorp's perspective, is you can't use their commercial products with TerraGrunt. So Terraform Cloud and Terraform Enterprise, they don't support TerraGrunt. So this really strange dynamic emerged where if a customer made the decision to adopt TerraGrunt, they immediately opted out of using Terraform Cloud or Terraform Enterprise. And therefore, our humble open source tool, which we really have not done much to monetize and all candor, was now considered a competitor by HashiCorp's sales team to Terraform Cloud And Terraform Enterprise. ([Time 0:17:19](https://share.snipd.com/snip/7a92c421-59bb-4baf-b14c-eb29f73fddf0)) - **Resource Allocation** - Despite HashiCorp's size, a small team maintains Terraform. - The OpenTF community has committed significantly more full-time employees to the project. Transcript: Josh Padnick That's a really excellent point. In fact, that has probably been the biggest objection that we've heard in the community is, hey, Terraform is huge. Hashacorp is huge. How can a ragtag team of different companies possibly have enough resources to really deliver here? And what's so interesting about that is if you actually look at the GitHub contributor history for Terraform core or maybe the GitHub contributor history for the Terraform AWS provider, Which happens to be maintained by Hashacorp, most Terraform providers. So like what you guys were calling plugins earlier, the technical term for that in the Terraform world is provider. Jerod Santo Most Terraform providers are about that. Oh, yes. I saw an opportunity to be pedantic. Hey, I know I called that because Dave, the CEO called a plug-in. Oh, really? She said that. Okay. I was just following suit. So the CEO, current CEO of Hashacorp was calling them plug-ins. I'm like, okay, I'll follow suit. I didn't know they were called providers. Yeah. Speaker 7 Well, see, business guy is CEO. That's what happens. We'll get to the leads there. Speaker 4 We're not there yet. Josh Padnick So Hashacorp, if you look at the GitHub contributor history of Terraform core and let's say the Terraform AWS provider, as best we can tell, there are anywhere from six to 10 people contributing Which are employed by Hashacorp, which is not to say they're necessarily full-time on it. ([Time 0:31:50](https://share.snipd.com/snip/11bf78dc-17ad-4e50-9261-202c89a83fad)) - **OpenTF's Vision and Roadmap** - OpenTF prioritizes backward compatibility with HashiCorp Terraform, including bug fixes. - They aim to be a drop-in replacement initially and then introduce community-driven innovations through an RFC process. Transcript: Josh Padnick That's a great point. So one of the, there are a few kind of core tenets of our vision. And one of them is, of course, a clear, unambiguous open source license. Another one is backward compatibility. And what that specifically means for us is that any bug fixes that are reported either to us or that we see to Terraform, we would like to implement fixes for. I should also say it's extremely important to the group to remain compliant with Hashacorp license. So we cannot just go to their repo and copy code improvements just because they're available. We are still working with our lawyers to figure out what information can we legally glean from what happens there and where do we need to just not even look. So like legal compliance is something that's really important to us. Anyway, what are your point about compatibility? Speaker 5 So that's bug fixes. Josh Padnick Then for new features that Hashi Terraform releases, like I just said, we cannot just copy their implementation. But our understanding is that we can copy the interface and we can do our own implementation and make it interface compatible. So to the greatest extent possible, we want to make OpenTF a drop-in replacement for Hashi Terraform. And then once we feel like we've really achieved that expectation where people feel like, hey, this project is pretty stable. Speaker 5 I can use this. I'm happy with it. Josh Padnick The other kind of big thread that we have is a community-driven RFC process for whatever improvements people want to see in OpenTF that could never make their way into Hashacorp. So one example would be those encrypted state files, like I mentioned before. So Hashacorp never wanted to merge that PR. And now this is an opportunity for OpenTF to figure out the right way to do encryption on state files, what all the potential pitfalls could be, go through a proper RFC process, get that Merged. ([Time 0:35:35](https://share.snipd.com/snip/8a8c1673-5d84-4e27-af9b-a9ce0f4570bc)) - **Registry Access** - HashiCorp's registry update restricts access to OpenTF, creating noise and confusion. - OpenTF plans to establish a mirror registry with potentially improved UX, working around HashiCorp's limitations. Transcript: Josh Padnick Think it also hosts policies and one other thing. And they just announced earlier today, I don't know if they announced it, but they updated the terms of service and some folks in OpenTF pointed it out, that only the Terraform binary Can access the Terraform Registry managed by HashiCorp. And so what they're trying to do is say, if you're using OpenTF, you can't use the Terraform Registry. And what's so interesting about that is it is mostly, in my opinion, designed to create noise and confusion. Because if you look at the technical details of what's happening with the provider registry, the Terraform Registry, first let's take a look at providers. It turns out that if HashiCorp is hosting a non-HashiCorp-maintained Terraform provider, like Azure, I think is not maintained by HashiCorp, then that is actually downloaded from Terraform through GitHub, rather. So like you connect to the registry, they do a 301 redirect and they send you to GitHub. And you're literally just downloading a binary that is a GitHub release asset. It's just that you have to be going through the Terraform, HashiCorp Terraform Registry to do that. So the only real impact that this announcement has is that you can't download a provider that's managed by HashiCorp directly, like the Terraform AWS provider. And there's a trivially easy solution to work around that, which is OpenTF will put up some kind of mirror registry that hosts their MPL-licensed and go release assets. And then you just download it from there. So it's like trivial technical workarounds, but it just creates noise. And then for Terraform modules, that's also just a HTTP 301 redirect to GitHub. And the language that they have in the terms of service update is very aggressive. Like you can't copy this content, you can't do anything, even though they are not even the owners of the content in many cases. Like they host modules or providers created by others that are just made available for there. So I found that to be disingenuous. And on the OpenTF side, we will take our resources and we will create a parallel registry that hopefully has a better UX and works just fine with OpenTF and will probably work just fine For Terraform as well. ([Time 0:44:02](https://share.snipd.com/snip/302b0605-a3c4-4a7b-8e47-7d0b29ee4410)) - **Jenkins/Hudson Parallel** - Josh Padnick draws a parallel between Jenkins/Hudson and Terraform/OpenTF. - Oracle's acquisition and proprietary shift with Hudson led to the successful Jenkins fork, suggesting a possible outcome for OpenTF. Transcript: Josh Padnick A parallel that we've often referenced internally is Jenkins versus Hudson. So I don't know if you guys remember, but like when I first came on to the DevOps world, this was like 2012, 2013, 2014, I was looking at CI systems and I discovered Jenkins. And I thought, like, oh, this is this is cool for the time. Now Jenkins is in a different place. But it was only in like the deep bowels of the documentation that I saw a reference to Hudson. Speaker 5 And then I went to the website and I was like, oh, this clearly looks like the lesser project. Josh Padnick And then I kind of forgot about it. And only now have I started thinking about the fact that what happened is Oracle bought Hudson or somehow acquired the rights to Hudson and changed it from an open source project to a Proprietary project. The ([Time 0:54:32](https://share.snipd.com/snip/f547140b-bf38-4936-9d2e-ead189dbfd43)) - **Joining OpenTF** - To join OpenTF, explore job postings from companies explicitly committing full-time employees. - Look for open-source engineer roles and consider multiple companies. Transcript: Jerod Santo Somebody's come to this podcast for the first time or one of the first times, they're kind of hearing the depth of the information behind this. And obviously, they probably heard about this change because information travels quickly, but they're fired up in positive ways, maybe some negative ways to think, like, you know What, if they're well funded to quote you, which you said before, and they're hiring, what's that process? Do you have a page you can go to? Is it these other companies that are hiring folks? Like, how is employing folks to work at OpenTF or on OpenTF? Actually, like, you have a call to action, where can people go? Josh Padnick You know, that is a great question. And it makes me realize we should have that call to action right on OpenTF org. And I will propose that to the consortium after this podcast. Okay. As far as what to do right now, here's the thought process. You could see the small list of companies that have explicitly committed full-time employees. You can go to their careers websites, they have, they've named the positions something like open source engineer, and look at the companies to see which one you like the best, or maybe Apply to more than one, and apply for the role and begin the conversation. There is enormous enthusiasm for the project. So if you're someone who's passionate about writing open source and about maintaining a key part of the modern infrastructure stack as open source, there's about 20 great opportunities Speaker 5 For you. ([Time 1:09:33](https://share.snipd.com/snip/da2a37b8-5972-41b4-8ccd-dd6f65abe367)) - **Preventing Future Rug Pulls** - To prevent another "rug pull," OpenTF should adopt a permanent open-source pledge. - This commitment aligns with joining foundations like CNCF, ensuring long-term community trust. Transcript: Adam Stacoviak Notes. The priority link will be there for you to get involved in the conversation. My only last question is like, how do we stop another rug poll? You know, like, are you guys thinking about this? Because what if OpenTF becomes like OpenAI, you know, and the open open and name only? What do you guys change your mind and decide you don't want to be open TF? You want to be close to you? Have you thought about that? Like, can you put that into your bylaws or something? Like, we're never going to undo this. Josh Padnick We've done exactly that. So one of the requirements of joining Cloud Native Computing Foundation, Linux Foundation, is you have to commit for all time to being open source and we're prepared to make that commitment. There's also a venture capital firm I saw funded by the CEO and founder of GitLab, Sid C. Brandy, I think his name is. It's called like OpenCore Ventures or something like that. And that he published on Hacker News, a blog post promoting this idea of like an open pledge or open core pledge where you're essentially making a public pledge to always keep your project Permanently open source. So I think it's a good question. We're entering an era of open source where we're still figuring out what the social contract is with an open source project, its community and its maintainers. I always thought that contract was once open source, always open source. Unless there's some really malicious thing going on and then you'll just target that one actor. But I feel like this business source license thing is a way of thinking that is new to me and is not comfortable to me. Even if it were on a project that I didn't have a vested interest in, I would not want to participate in a project that had a business source license. So I guess we'll see how things play out. What I am hopeful for is that open source becomes something where it is a clearly understood part of a company's business model, whether it's for lead generation, recruiting or being A free tier. And companies know the game that they're playing and they don't reframe that game as we're being taken advantage of. But instead they can say, well, this is what the dynamic is when you do open source. Competitors can and will use it against us. But we have a privileged position as the creators of this tool. And here are the many ways in which we're going to leverage that to build an amazing thriving business. That's ([Time 1:14:10](https://share.snipd.com/snip/b98047dd-19d4-4bab-9be2-abc6d05ea28e))