The Architecture Behind Laravel Cloud With Chris Fidao  ## Metadata - Author: [[The Laravel Podcast]] - Full Title: The Architecture Behind Laravel Cloud With Chris Fidao - Category: #podcasts - URL: https://share.snipd.com/episode/5bc9cd78-1a6c-42ae-a2bf-0db414b77e0f ## Document Notes ## Highlights - **From Content Creator To Cloud Engineer** - Chris Fidao joined Laravel Cloud after years of creating server-focused content and tools for PHP developers. - Taylor and others recruited him because his Service For Hackers work made him known as the "server guy." Transcript: Chris Fidao Yeah. Hi. Like you said, I'm Chris. I am on the cloud team at Laravel. So that means we, I and the team, of course, have built up the cloud product. And right now, over time, I've transitioned, you know, we were talking about this earlier, transitioned from basically being an infrastructure engineer, doing Terraform, building Infrastructure, all that fun stuff. And now I'm the team lead for the infrastructure cloud team. I'm going to say team a million times, which means I'm effectively an engineering manager now. So my day-to now is managing people instead of actually doing the code. Although, you know, it's a little of everything. Matt Stauffer I have a lot of questions for you about that. But before I get to those questions about what you do day-to ([Time 0:00:41](https://share.snipd.com/snip/31dfb20b-ff1e-48e3-8c78-6db888d0aabe)) - **Why Kubernetes Over Serverless** - The team rejected a Lambda-style serverless approach because it didn't fit Laravel's PHP workload patterns and cost model. - They chose Kubernetes to manage many workloads reliably and leverage its large ecosystem and orchestration features. Transcript: Chris Fidao Because I came in specifically to help build cloud and that kind of stuff. So I was pretty early after funding. So even before Andre started, I think by then he like had confirmed he was joining Laravel, but I hadn't even started yet. So like Tom, the CEO was there, then me and then Andre started. So I was there really early, right after funding and figuring out how we're going to do stuff. So I got to help work with Joe on how to, you know, Joe Dixon on how to like figure out what we're going to do with cloud. At first we were looking at Vercel's model of using serverless, you know, Lambda. And we ended up not doing that for just a bunch of reasons. We can get into that if it's interesting. We ditched that and then we were like, okay, so if we're going to use not serverless, then we're doing servers and then what do we do to manage servers kind of the next obvious thing is kubernetes Just because it's so you know it has a huge ecosystem basically because it has a huge ecosystem like there's nothing it can't do effectively and it helps with our model of you know how Many workloads can we put into a server reliably and all that kind of fun stuff kubernetes just has answers to that kind of stuff, right? So Kubernetes, it runs containers and it answers the question of, if I have a crap load of servers, how do I figure out what server to put a workload on and manage putting a workload there And run it and connect it to the internet and all that stuff. ([Time 0:07:20](https://share.snipd.com/snip/2335b3db-380e-4580-b412-189952aae95b)) - **Automate Complex Deploys With Operators** - Use Kubernetes operators to programmatically orchestrate complex deployment steps like running migrations during deploys. - Prefer code-driven operators over pure YAML when you need interactive orchestration and rollback capabilities. Transcript: Chris Fidao So essentially, no, it's mostly YAML. It's all YAML all the way down. And the YAML tells Kubernetes to do stuff. Matt Stauffer And that's what I have experience with. But I was like, I haven't gone to your level of Kubernetes. So I was just like, is there a world that I don't know about other than just the YAML and YAML and YAML? Okay, got it. Chris Fidao There is a fun thing where we do, and it's called an operator, and you can program against Kubernetes API. And essentially, the way to think about that is that you're generating, you're not actually generating YAML at all. You're actually making API calls to the Kubernetes API. But you can think about it as code that generates YAML that tells Kubernetes what to do. It's like the YAML in the end is getting converted to like something that makes API calls anyway. Matt Stauffer Got it. Okay. Chris Fidao In this way, you get to programmatically interact with Kubernetes stuff. So you can do fancier things than, as you can imagine with code, right? So like YAML can tell Kubernetes to do something like spin up this container, run it, blah, blah, blah. Code can be more interactive and say, this is the status of the world. You know, do this task. Like an example of that is when you deploy an application on cloud, you might want to run a task right before you, the code is released. So that might be like a migrate command, right? ([Time 0:09:26](https://share.snipd.com/snip/8ca5c113-dbee-441a-8d9c-a9f618892bd2)) - **Hidden Costs And Limits Of Lambda** - Serverless (Lambda) suffers from poor visibility and a hard 15-minute execution limit that clashes with typical PHP workloads. - Lambda's architecture forces AWS to price for burst capacity, making it an expensive compute option for Laravel Cloud. Transcript: Chris Fidao Yeah, no problems because we didn't even get as far as deciding to use it and try it out. Well, we do know problems because we have vapor as an example. And the biggest problem is visibility into what's going on into serverless is hard, just by nature of serverless. You can't SSH into anything and see what's up. But just by nature of how they built Lambda, also the 15-minute limit of running a Lambda is pretty killer, depending on your queue jobs and that kind of stuff. So AWS has their specific use cases if they want people to use Lambda for, and things that take longer than 15 minutes is not one of those use cases. Not on that list. Unless you do fancy things like step functions or whatever. You can chunk jobs into different whatever. Who cares? It doesn't matter. It doesn't work for Laravel as easily. So that's one factor, just one. Other factor, it's expensive. ([Time 0:11:32](https://share.snipd.com/snip/e8748870-a064-4d3c-9154-b9aa2bb6fa32)) - **Factor Cost Per Customer Into Design** - When building a paid platform, always consider cost-per-customer for infrastructure decisions and optimize accordingly. - Evaluate trade-offs like monitoring density or managed services that increase aggregate compute and billing at scale. Transcript: Chris Fidao So I can relate that to people and try to like basically effectively communicate like why decisions were being made to, you know, other people on the team. I don't know. I guess I'll just throw out an example. So one example is that like we have to be very conscientious of cost because every decision, not every, but many decisions we make might include or might increase our cost per customer's Workload. And that's different from a lot of people coming in from other places where maybe they have a product they're working on or maybe they're a platform engineer who make an internal development Tool. And it's like a cost per developer isn't a big deal, but we have to charge for our product. If we do some kind of infrastructure addition that has a definite cost per customer workload that runs you know at scale that makes a huge amount of cost and then we either have to charge A customer for it or figure out a way to like you know change that so like like those kind of mindset shifts or something like i help people along with when we're um well i mean just as a manager I guess yeah ([Time 0:16:42](https://share.snipd.com/snip/50b30e08-f0a4-4a8f-a12e-d9de3a386478)) - **Tracking Work With A Slack Check-In** - Chris advocated bringing back a Slack check-in app to record individual contributions and visibility for managers. - They resurrected Taylor's Beep app to give managers evidence of work and celebrate accomplishments. Transcript: Chris Fidao I advocated for that a bunch. In the end, we resurrected this Beep app that Taylor made years ago. Really? And that's like, and it turned into like a Slack check-in app. And so that came to be, thanks a lot more to Joe Dixon than me, although I was advocating it for it behind the scenes. So that kind of thing certainly has been around. ([Time 0:21:43](https://share.snipd.com/snip/f54c45ab-5918-45ab-8f6d-8fb61419a56d)) - **Plan Work In Six-Week Cycles** - Run work in fixed cycles (they use six-week Shape Up cycles) and coordinate leads to plan scope and priorities. - Use cool-downs and regular syncs with product and engineering leads to keep projects on track. Transcript: Chris Fidao We have cool down cycle, that kind of thing. So a lot of management stuff is helping with the product managers and then Joe and Jeremy and myself who are kind of like leads of teams. Joe's like the overall tech lead. And then we have Jeremy who's the lead of like the web app side. Myself, the lead of the infrastructure side, all in cloud. Get together and do a lot of management of what projects are going to be coming up the next cycle, how the projects are going currently, and all that kind of fun stuff. So there's a lot of those type meetings and there's meetings with partners. There's meetings with potential customers, usually kind of like leaning towards enterprise or large team side of things. ([Time 0:22:43](https://share.snipd.com/snip/999b2062-ab00-4a83-8120-1f66d9bf7e34)) - **Kubernetes Can Manage Cloud Services Too** - Kubernetes can provision cloud resources via operators like Crossplane, letting infrastructure be declared and managed through GitOps. - This lets the web app request an AWS RDS instance and have Kubernetes orchestrate its creation and status callbacks. Transcript: Chris Fidao That's more of just in the weeds. Matt Stauffer As being responsible for code that spins up people's actual servers that run their actual websites is a lot more stressful than just being responsible for your own code. I feel like you're at multiple levels. Like I'm like, if I push out a new feature and I make a mistake, like one client may, you know, one user might be like, Hey, you know, this PDF wasn't where I want it. ([Time 0:27:06](https://share.snipd.com/snip/4cb624d7-c6f4-4931-935b-887f99b28628)) - **Processes Reduce Infrastructure Risk** - High-responsibility infrastructure requires processes: Git-reviewed changes, GitOps deployment, regional canaries, and staging environments. - Treat infrastructure as code and add automated tests and staged rollouts to reduce catastrophic outages. Transcript: Chris Fidao So it kind of just like happens. But like, so it's all process. It's making a process to make mistakes less likely to happen. Some of it's forced on you by like SOC 2 or other compliance things, which is good because there's forcing function to make you sure you have this process. So for example, there is that everything that might land in production has to have a review behind it. So that's usually like a GitHub review that updates a GitHub repository that then is read by something that adjusts itself because it sees the code and the repository has been updated. You know, GitOps, right? So Git changes, something reads that change, applies that change to your infrastructure. And then there's other fun things like we can control changes down to like the region and sometimes down to like the Kubernetes cluster that runs in a region. So you might roll out a change in one region, make sure it's working fine, and then slowly roll it out to other regions. You know, that kind of process too. In addition to having development, staging, production environments, and all kind of fun stuff. So like, you know, all the touch points you have before. Infrastructure can have, a lot of it is code. Therefore, a lot of tests can be written. So those test suites for, you know, these operators I keep talking about and all this ancillary kind of software, just kind of like in tooling you make along the way. ([Time 0:27:54](https://share.snipd.com/snip/810ccb65-2497-4b8c-94f4-d737caf2dca0)) - **Foreign Keys Are A Scale Trade-Off** - At scale, common defaults like foreign key constraints can cause deadlocks and migration pain, requiring different trade-offs. - Start with constraints as defaults, then remove them if scale forces transaction and locking problems that hurt availability. Transcript: Chris Fidao OK, the answer is not truly. And that's a good thing, because it means people listening don't have to worry about like it's just it's more. It's not different. Like things happen at scale. You have new issues at scale. Big example of that is their database. So we've we've run bad migrations before and they're not bad migrations they are migrations that took the database a lot it was a lot harder on the database than we thought because it Was huge tables the tables are very busy combination of foreign key constraints being used which i just freaking hate but like people love foreign constraints but they caused some Issues so some of those issues are when you run migrations they don't run a smoothie because there's more deadlocks and transaction issues and that kind of stuff so at scale you run into Those kind of issues but i don't have a great answer to like we we do this process but i never did this exact process before because you know this scale requires that there's no like magic Or we can do this at scale but nowhere else type things that that at least that we're using um we try to keep things as simple as possible though so i you know there could be things that come Up on the horizon as we as we get larger but really it's like just a lot of testing and a lot of that testing is either automated or manual you know through our environments that kind of thing Having a set process making sure the process is followed and i mean writing a lot of tests honestly like the the web application uh the laravel web location is like very heavily tested Like extremely to the point of being annoying, which is good. Overall, the code quality is very good. Thanks to that. A combination of, you know, test coverage, syntax or style guide, you know, all that kind of stuff, typing, type tests, that kind of fun thing. Infrastructure side, there's like so many projects that it's not all like 100% test coverage because like there's like one repository for the web application. There's like 50 repositories for infrastructure because we just have all these kind of like weird concerns where it doesn't make sense to throw it all into one Git repo. It's kind of just the nature of infrastructure. ([Time 0:30:05](https://share.snipd.com/snip/32fd0d5a-34aa-4ec8-9ee8-bee34ff776bd))