MCP on Code Mode  🔗 **Source:** [MCP on Code Mode](https://share.snipd.com/episode/17aa8c28-ad0e-41d9-915d-2c7313e36f0c) ## Highlights - **Server Side Code Mode Scales APIs** - Cloudflare's server-side Code Mode lets an MCP server expose thousands of API endpoints using just a small context by having models write and execute code on the server. - They expose search and execute tools: the model writes TypeScript to search the OpenAPI spec and call Cloudflare endpoints, then run that code in a sandboxed dynamic worker loader. Transcript: Matt Carey So running code mode inside an MCP server. So the model doesn't need to, the agent doesn't need to call tools. The agent can just write code that acts upon the tools that we have on our, like in our backend. And then all of that code is executed super safely, securely on dynamic workers on our server side. And so your agent that calls the tool can just write code and yeah, it all gets executed on the server. And it meant that we could put our whole Cloudflare API, all 2,500 odd endpoints. There was a number, but the number keeps changing. So every time I remember the number is, it's out of date. But yeah, so around two and a half thousand endpoints, we can put all of that behind one MCP server that actually works, that fills a thousand tokens of context. And there was a lot going around about like, oh, fixing MCP. It's like, I don't think we're fixing MCP. We're just using MCP to the best of its capabilities. It was a really well-designed protocol, I believe. I think it continues to be well-iterated on. ([Time 0:07:40](https://share.snipd.com/snip/ceb356b9-60c3-4804-9914-d75119cb5bd2)) - **Let Models Write Code Not Tools** - Let models write code (TypeScript/Python/Bash) instead of adding many discrete tools; code composes APIs without bloating model context. - Run that model-written code in a sandboxed interpreter or dynamic worker loader for safety and controlled outbound access. Transcript: Matt Carey Yeah, so when LLMs first came out, they just produced text, right? And then at some point, I can't quite remember exactly when it was, but I'm pretty sure it was after the big chat GPT moment, function calling became popular. And I remember the first model that could do function calling really well was I think GPT-4. And GPT-4, you could ask it like, what's the weather in London? And it would reply, call weather function, param location equals London. And you could take that, that structured piece of information, and you could plug that into some JavaScript, some Python, some code. You could call a third party API, a weather API with London as the argument, the city argument, and you could get a result. And that result, whatever it was, you would pass back into the model as the tool result or as the function core result. And then the model continue generating and make it all nice and pretty for you. And that was like how LLMs performed actions in the outside world. And we, we, I guess like rightly or wrongly assumed that each individual function would be like a piece, like a, would be like a hand that the model could use in the outside world. And then they were renamed to tools after a while. And that makes more sense. Like each function was a tool that the model could use in the outside world. But there is like a problem where as you try to get these agents to do more and more things, you add more and more tools. And then at some point you start filling the initial context window of the model. So for instance, like the GitHub MCP server is always one that's used and they've done loads of work on it. So like I don't know, not throwing any shade, but initially when it came out, it was like 15,000 tokens or something. And now I think it's a little bit less and they do some stuff to dynamically add tools or not. But if you're filling the context window with 20,000 tokens initially, before you've even given the model your task, the models of yesteryear, like GPT-4, they had much smaller context Windows. And so you were filling them very quickly. And even now, the foundational models, the best ones, even though they maybe have a million tokens context window, 200k to a million for the normal ones, they do start losing power around The 50k mark, like all of them. And this is quite well documented. And so you really don't want to be filling the context window too much. And you see in code code, you'll have like compaction step. That's triggered because you've filled a context window. So say you've added like, you've added like 20 tools or whatever, you've like got a really chunky context window, but now like that's only 20 things the model can do, um, or the agent can Do. Imagine you want to have like proper personal AI or something that can actually try and automate your job. Like imagine how many individual functions you do in your job. It's way more than 20. It's probably more than a hundred. It's probably near a few hundred. So like you can't really automate anything unless you have the ability to add all of those functions into a model. So in the summer, Kenton and Sunil, who are my colleagues of mine, worked out that you could, or they wrote a really great blog post, the CodeMode blog post, the original one, an amazing Name from Kenton, like really, really stunning. It's like, I think it caught on a zeitgeist. It's like, you could just use CodeMode. And the idea was falling back to an idea that had been around for a while. Like I'm pretty sure Hugging Face did a research paper on Code Act a while ago. But the idea was that models should just write code. Like AI has been trained on so much code, we should just write code. And if we write code, the code can interact with the functions that we want to use. And so, CodeMode generated a TypeScript API, or TypeScript SDK, really, for the underlying functions, the molecule call. And then the model just wrote code to compose those SDK calls. And I guess the good innovation for this, the reason why it's, I think, slowly taking off and we're going to see much more adoption this year is the advent of something called a dynamic Worker loader which is a cloudflare primitive but other people have started building similar things um if not the same um but like this is a primitive that allows you to execute a sandbox Worker as a string like so from a string from some code that's a string you can just be like eval this new function this except it runs on a separate host in a fully sandboxed environment In a va isolate and what does this mean like traditionally people got very very scared when you say give me code and i'm going to execute it on my machine because there are so many different Ways that you can mess someone up by doing that. You can like out of memory them, you can access M variables, you can do loads of stuff that's really hard to protect against. And this sandbox is very particular sandbox. That's not a full VM. It's just, it's just a VA isolate, allows you to spin up like billions of these little scripts almost instantaneously if you wanted to. Like at Cloudflare scale, at global scale. And just run these pieces of code. Like very, very safely and securely. You can even like restrict the outgoing fetch. It's called a global outbound. And you can say, I only want the outgoing fetch to be able to access example.com or mysass.com, or I don't want to access anything. Just run code that's entirely constrained in this host. And this is really cool because it allows this code act idea, this code mode idea, to really take off because the model can just write code. It doesn't matter if it's prompt injected or if it's trying to be, I don't know, trying to be adversarial. Like it's running, the code is running in a super safe environment and that's all good. And that meant that the original code mode blog post had this code execution happening in the agent. And that was like the smart thing to do, right? The proper thing to do is like you make the agent do code execution and then you become like a code agent. And that's what happens. But this relies on every, or on the agent that wants to do it, actually shipping code execution in the agent. And like, it turns out that that is also quite tough. ([Time 0:11:58](https://share.snipd.com/snip/3e460277-9409-4c68-8052-9c2aebcf79c3)) - **Sandbox Model Code With Outbound ACLs** - Use a VA isolate or equivalent sandbox for executing model-written code to prevent resource abuse and restrict network fetches. - Enforce outbound ACLs (allowlist domains) so generated code cannot access arbitrary external resources. Transcript: Matt Carey You can like out of memory them, you can access M variables, you can do loads of stuff that's really hard to protect against. And this sandbox is very particular sandbox. That's not a full VM. It's just, it's just a VA isolate, allows you to spin up like billions of these little scripts almost instantaneously if you wanted to. Like at Cloudflare scale, at global scale. And just run these pieces of code. Like very, very safely and securely. You can even like restrict the outgoing fetch. It's called a global outbound. And you can say, I only want the outgoing fetch to be able to access example.com or mysass.com, or I don't want to access anything. Just run code that's entirely constrained in this host. ([Time 0:16:59](https://share.snipd.com/snip/a8fd6747-bd66-44fd-b1e1-28acc1111f23)) - **Two Tools Replace Thousands Of Endpoints** - Mapping every single API endpoint to one tool quickly exhausts a model's context window and forces cherry-picking. - Cloudflare instead exposes just search and execute tools and lets the model programmatically find the right endpoint in the OpenAPI spec. Transcript: Matt Carey And they'd be like, oh, we want to get all issues or post an issue or like delete an issue. Like a one-to Yeah, like one-to mapping. And sometimes you might do like a one-to mapping a little bit if you had like a particular workflow that was very common on your website, you might like create a tool for that workflow. If there was a very common workflow that customers do. But you're kind of restricted to around 10 to 20 tools, like maybe up to 25. I know Cursor has a 40 tool limit max for supplied MCP servers. So theoretically you could fill that, but like, it's getting pretty large. It's getting pretty large. And you're still not covering anywhere near, you're still cherry picking like for big platforms, like the Cloudflare platform, like GitHub, for instance, I use GitHub, but in Cloudflare, But like you can imagine any large API, you're not covering anywhere near the full amount of the breadth of the API. But when you put code mode in front of it, you're now just asking the model to write code over that API. And so the Cloudflare MCP server exports just two tools, a search tool and an execute tool. So this is combining a similar idea to tool search, which is present inside Claude Code, as also present inside Cursor, I think, where they will search for the right tool on demand, and Then they'll load the right tool, depending on the user intent. So we do that, but we do that on the server side. So there's search and execute, but the critical thing that no one else does is for search, we let the model write code to search over the Cloudflare OpenAPI spec. Code. There's no search function or anything. The model just goes like spec.paths and then filters like super naively. ([Time 0:20:17](https://share.snipd.com/snip/b3929a67-070f-4abe-96ea-ce0a1cb280e4)) - **Scope Prompts And Protect Your Remotes** - When using coding agents, prefer constrained, intentional prompts and keep the model in-distribution for best results. - Use local safeguards (Matt's Zaggy git wrapper) to prevent agents from force-pushing or overwriting remote repos. Transcript: Matt Carey So my workflow with Claw has actually changed quite a lot over the past. I'm sure everyone's has. It's changed a huge amount over the past. It's constantly changing, yeah. Yeah, I would say. A year and a half ago, I was big into Cursor. I really liked it. And I was using it mostly for the tab model, I'd say, about a year and a half ago and then January last year I was like now the agent model is the future um I need to like work out how I can just Prompt like how can I just prompt things into existence like what guardrails do I have to put on things do I how do I can I make the feedback loop how can I have tests that have good patterns And all this sort of good stuff? To, like, build basically my code bases for agents. And this was about January last year. And then when Claude Code came out, I was like, right, no, this is definitely the future. Now I'm just going to have an IDE open when I want to actually visually look at code. When I want to review, I'll open an IDE. Otherwise, just straight in the terminal. Let's just chat. Let's build something. And I tend to run everything on dangerously skipped permissions, like 100% of the time. I have some like sandboxing on my laptop. It's a custom thing that I'm not like don't super want to talk about um but i am curious what you want to talk about though yeah yeah i'm not gonna talk about hugely i have okay so i have my Own version of git that runs on my that runs on my machine and it's just aliased um and it just stops like the model doing like stuff that i don't want it to do so it stops it force force pushing To branches overwriting stuff even because i have admin permissions on a lot of repos so to me this was like the base level of thing that i didn't want to happen i didn't want um anyone to Be i didn't want any agent running on my laptop to be able to overwrite a remote repo. That's like base level. My own laptop OS, I don't mind breaking stuff. I don't mind breaking anything locally, but I don't want to break anything externally. And so I have a few aliases like that where I've just completely overwritten the Git internally. And my Git wrapper is called Zaggy. It's public on GitHub. I built it in Zig actually. It's super fun to build using libgit2. And it just like has a bunch of these protections out of the box. So I really don't mind running stuff in dangerously skip permissions. I tend to do it all the time. ([Time 0:28:22](https://share.snipd.com/snip/6b27bf3e-333a-45d2-8c81-948522bde321)) - **Granola Transformed Matt's Doctor Father's Workflow** - Matt gave his doctor father Granola and saw it transform his notes workflow by auto-summarizing consultations. - The doctor now uses Granola to reduce time spent on paperwork and improve consultation quality. Transcript: Matt Carey Slightly on this point i recently got my dad using granola and he's a doctor and he he's kind of fed up with he writes so many notes and like granola has completely changed his whole workflow Oh i bet um he like sees people on zoom all the time. And he used to see, he sees people in person as well. Now he just starts Granola, writes up all of his meeting notes. And it's like, it's just, it's been, it's been like transformational for him. Just that like basic summarization. Like Granola is a great product. Don't get me wrong, stunning product. But like, it is not a complicated workflow and it's like completely changed his like quality like how long he spends doing his consultations um so yeah i guess like shout out there like There are there are some small things you can try there are some i'm a fan i'm actually i'm a paying user of granola so yeah yeah oh wow they should pay me Come on, Granola. Adam Stacoviak Pay me. Yeah. I love Granola. It's amazing. ([Time 1:12:01](https://share.snipd.com/snip/25ac4552-18d1-4d04-9fdd-33d4ab900ec6)) - **Memory Needs Flexible Provider APIs** - 'Memory' is multifaceted: short-term session compaction, on-demand loading of skills, long-term vectorized facts, and continual learning all need different APIs. - A provider-based storage model lets developers choose durable objects, Postgres, or vector stores without locking them in. Transcript: Adam Stacoviak We're off of our personal soapboxes about how AI has changed our life and how it's taken some away, how we can't stop thinking about it and how we prompt, et cetera, et cetera. But take me into the world of, I guess, next few agents, MCP, where does memory fit in? Matt Carey Yeah, so memory, it's like such a loaded term. It's such a loaded term. So it is quite hard to know where to start. Essentially, I want a way for my agents to remember a conversation that we're having right now. And be able to refer back to context that I gave previously in the chat. But also to remember conversations over time. Um, and also to be like very programmable. So I work on SDKs, like developers are going to program with my SDKs. It's like, how do we, how do we build something that's mega customizable to like the next new, um, the next new trend? Like for instance, skills, like skills are just a markdown file that's loaded into a context on demand by an agent. How can we support that in a memory system that can also support compaction of sessions, can also support continual learning, can support like the migration of a session to long term Storage so an agent can like search over it over time. I guess I'm just trying to work out the shape of those apis right now there's some really good examples like maybe not examples but there's some really good inspiration um on the in the Typescript uh world at the moment uh like letter is very very cool letter ai master just to name like a couple they all have some some cool memory stuff. And I know there are, there are some, there are some really cool memory startups that are actually like doing managed memory, like super memory. Like I just like shout those guys out as like really good inspiration for what we're trying to do. What we're trying to do is not trying to replace anything like that. Uh, but it's like how, like Cloudflare has some really cool storage primitives. How can we let developers best use those storage primitives in the function of making a better agent? And I realize they're all questions rather than answers, and I don't have a huge amount of answers, so I'll probably keep my powder dry on that one. ([Time 1:19:00](https://share.snipd.com/snip/4128aba4-e04c-41a0-97c1-43c6b00f080c)) - **Generate SDKs For Clearer Agent Permissions** - To scale agent capabilities, generate an SDK or CLI and have the model call that instead of discrete function tools. - Prefer typed SDKs (TypeScript) to expose features and permissions clearly to models and enable safer code generation. Transcript: Matt Carey I guess like the first thing would be to go and have a look at the blog post or dump it in your coding agent so it's like blog.cloudflare i think forward slash code dash mode dash mcp and we'll Hopefully pop it in the show notes i think that's like if you dump that in your coding agent or or just like have a read i think it's a decent read. It'll give you a lot of insight. But the main thing is try not to let your AI do individual discrete actions. Try to write an SDK, write a CLI. A CLI is also code mode in some way because a model is writing. If it's writing bash, as far as i'm concerned it's writing code you know like i prefer to write typescript or python but if it has to write bash then go to town so like let the model write Code and get out of its way just like let it roll it'll be fine let it roll yeah and then if you want it to be like secure and you want to deploy it properly, then have a look at deployment options For building or using like a sandboxed interpreter or some type of sandbox, whether it's like a VM, whether it's, I know Pydantic released something pretty cool called Monty, which Is like a Python interpreter built entirely for code mode. That's pretty cool. Or dynamic worker loaders, like the Cloudflare option for running JavaScript. Like really like, yeah, sure, it's fun. You can run code locally, just eval it. But if you want it to be safe, secure, and I don't know, deployed properly, then have a look at some of those options. But really, yeah, the main takeaway is just let the model write code, dude. Let the model write code. Yeah. Adam Stacoviak I assume you prefer TypeScript over Bash. I don't mind Bash. Bash is the limi franca of agents these days. TypeScript is the second best, I think, for the agents, but they're really fun with Bash. Matt Carey I just let it rip. I like Bash. I just think there's an easier permissions model if you're generating a TypeScript SDK. Because the first thing, the disclosure of features is easier. It's much harder to generate. ([Time 1:45:20](https://share.snipd.com/snip/ba4b0ce6-55d9-48ad-a064-fa6eb7375a13)) - **Make CLIs Agent Friendly With --agent** - Add agent-aware flags to CLIs (e.g., --agent) that output machine-friendly Markdown describing usage so coding agents can parse and call commands. - Provide non-interactive alternatives to interactive CLIs to support agent automation. Transcript: Matt Carey Like basically you can have like this, like anti corruption layer, this ACL that sits around the, the, the model, the code execution. ([Time 1:48:44](https://share.snipd.com/snip/cfa538c8-045e-451a-bb46-6bb4264df894))