A New Direction for AI Developer Tooling  ## Metadata - Author: [[The Changelog: Software Development, Open Source]] - Full Title: A New Direction for AI Developer Tooling - Category: #podcasts - URL: https://share.snipd.com/episode/0cc8ee17-fe4e-48ca-bf4e-ac3f3f5bed0d ## Document Notes ## Highlights - **Context Compacting Causes Fragile Agent Memory** - Agents and models prune or compact conversation context to extend useful history, but this can remove tool outputs or earlier data. - You can provoke context loss or surprising behavior by re-invoking or tricking tools, revealing fragility in long-lived chats. Transcript: José Valim So right now, like they, they do the auto compactation of the chat and context engineering is all the rage right now right so uh right but like they do the automatic compactation of the Of of the the conversation which is summarizing it but something that they do is also that when the context is getting filled they just prune the tools output from beginning. So there is like some files or some searches or commands you ran at the very beginning of the conversation. They prune that, and that also allows them to go for long without having to summarize stuff. Because when you summarize, there's always a chance that you are losing some data. And something that is really something that I do it a lot is that you can actually have conversations with the agent about this stuff. So you can like, so people, so a lot of people say like, oh, like there is the agent doesn't tell me like, oh, I'm using this agent and this agent doesn't tell me which tools you have available. But you can always just ask the agent, like which tools you have available. Look invoke so something that i do is like invoke like echo with uh uh with you know a a hundred thousand times three times so i can force it to fill in the context name like what you see now On that tool that was invoked and then it's it's like oh the two output disappeared it's like it's surprised that stuff like itself like it now says this thing. And you can, you can trick it like to crash very easily as well. ([Time 0:09:59](https://share.snipd.com/snip/37ed42ed-1e38-4f38-a0e3-dc84d825b4d4)) - **Run Agents In The Real Browser Context** - Tidewave runs a coding agent tightly integrated with web frameworks and the browser to close verification loops. - Running the agent in your local browser lets it interact with the real app, DOM, templates, and verify behavior directly. Transcript: José Valim So TideWave is a coding agent for full stack web applications. So I'm going to summarize it. We can jump into it later, but the idea is to have a coding agent that is tightly integrated with your web framework. So we understand like what is on the DOM and how that maps to a template, it can coordinate the browser. So it gives it a really strong verification loop. So as you ask it to build features, it can verify that features work. You can interact with the actual web page and ask changes on the page instead of asking for changes on the code. I have like this whole idea that I think we should run coding agents on top of what we produce. So if I'm working on a library, what I produce is like API docs, fine, run that in an editor. But if I'm building a web application, I want to run the coding agent in the actual browser because I want it to understand what I produce, right? And I want to be able to interact with what I produce because if I can't do that, we are doing boring translation work all the time. I like looking what happens in the screen, go to the editor, ask it to change things, right? And then the agent says, I'm done. You reload the page. There's an exception. You have to copy and paste the exception back to the agent. You don't want to do this boring stuff. And I say, the data science folks, they were the first ones to notice that because they were the first ones to put the coding agents inside notebooks. They're like, okay, let's run this thing inside a notebook because if it understands my variables, if it understands my cells, they're going to be more productive. But nobody caught up to that trend, right? We kind of regressed. We first put in the editor and then we put it in the command line, right? Like, you know, so we should be going up, right? So that's TideWave. ([Time 0:19:26](https://share.snipd.com/snip/b1b9453a-9de6-4fee-9cc8-1c8247f3f94c)) - **Discard Bad Context And Restart Chats** - When an agent produces imperfect code, start a fresh chat and give concise new instructions instead of trying to repair the broken context. - Discard noisy context to avoid compounding errors and guide the agent with targeted constraints. Transcript: José Valim And then the other thing that I like, I tell people to do, which works really well for me is to, I don't correct the agent. I just like, if it does something wrong or if it's like 70, 80% good, I just go and finish it. That's fine. Adam Stacoviak You code it for it. José Valim It depends. So I do two things. So imagine that I ask it, does this thing? And then I leave, I come back like, oh, this sucks. I'm not asking it like, oh, you were supposed to do this instead. Because often when it does something wrong, it's there in the context. It has a really... Adam Stacoviak It's going to keep getting wrong. José Valim Yeah. And then when it fixes, it doesn't fix everything. So when it does something wrong, I usually go and like, I start a new chat. I just discard everything, right? No, nobody's going to be upset. I just discard everything. I was like, okay, start again, but do this, this, and don't do that. I add a little bit more of context and then if necessary, I start again. So you start fresh with additional little warnings or instructions. ([Time 0:24:00](https://share.snipd.com/snip/f3f88c9e-fd08-45dc-bfad-2ff1a1712e44)) - **Weekend Prototyping In YOLO Mode** - José used agents for weekend prototypes, letting them run YOLO-style while he checked progress intermittently. - He experimented embedding TideWave in extensions, games, and mobile sims to quickly validate ideas without full manual builds. Transcript: José Valim I also use coding agents for things that I am not reviewing, particularly for like prototypes. And that part has been really fun because if you're working on a product, you have ideas of, wait, what could this project, which directions we could go in the future. You would like think about it, put on some notes, right? And then maybe if you're lucky in two, three months, like somebody from the team can take a look at it, give feedback, right? And now with agents, you can just say, okay, go for it, implement this thing, right? And so as I was saying, like I have this idea that coding agents should run on top of the thing that we produce, right? And we talked about TideWave web that works for our applications. I talk about notebooks. Well, but if I'm working on a game, right, I want to have TideWave running in the game engine. If I am building a mobile app, it needs to know about the mobile device, simulators, right? And all this kind of stuff. So I was able to, like, I think for four or five weekends straight, like what I would do during the weekend is to come to the computer from time to time, see if the agent was working, and just Have it build like a different proof of concept of embedding TideWave somewhere completely different. ([Time 0:29:40](https://share.snipd.com/snip/0c4ab195-f784-456b-8c4d-396569f19f6a)) - **Shared Browser Session Avoids Test Fragility** - Running TideWave in your actual development browser avoids creating separate test sessions that miss local state like logged-in users or dev-only fixtures. - That shared session makes debugging and iterative fixes immediate because the agent operates on what you already see and have configured. Transcript: José Valim So over the last month you had, we already had like, let's talk a little bit about the browser design. So like we already had for some time, the Playwright MCP. So somebody may be listening to this and say, well, I can use VS Code with Copilot and install the Playwright MCP. Adam Stacoviak Recently, Chrome DevTools MCP. Chrome. José Valim Yeah. They, they released there. And I think yesterday cursor browse came out. What's that? It's just controlling Chrome. It is like the, it's like the playwright puppeteer MCP. It's just built in. Right. And the issue with those tools is that it is a separate browser session is not the one that you are developing. So imagine, for example, that you are working on a project manager and what you need to do is that you need to implement a feature for transferring a project between two organizations, Right? So in order for you to implement this feature, you need to create a user, create two organizations, probably make sure that the user is admin on both organizations, create the project, And then you can transfer it. And then a lot of the times the MCP is going to get stuck just in this process. A lot of the times the MCP cannot create an account because creating an account requires sending something to an email that the MCP doesn't have. So now we start writing like those back doors for tests. So there's a big amount of work, right? And then the fact that we run in the browser, we are literally running in your browser session. Because when you're going to develop the feature, you open up the... You are already logged in your development version. You go to the page already. And when you're going to validate that the feature works, you already have all that set up. So because it's running there in your session, everything that you do for development, the agent can do. And the agent is going to verify the things in front of you, not in a separate session. And then you can actually have a back and forth. Like if you're using the MCP, imagine like the agent's like, okay, let me test that it works. And then the MCP with the separate browser is running. And then you see a bug right when the thing is testing. How you're going to debug that? Because it's a separate browser. How are you going to click things and say, hey, you would have to go around, say, around this page, maybe there is a bug here. With TideWave, it is your browser. I think that's the most important thing. You can stop the testing, go with the TideWave inspector, there is a bug here, fix it. ([Time 0:38:21](https://share.snipd.com/snip/f1ce988b-8ac3-46bd-82c2-c7ea5625d5d4)) - **Give Agents Framework-Level Context** - Use framework integration so the agent can map DOM elements back to templates or components and open the exact file in your editor. - Provide inspectors and page-aware chat prompts so requests can be UI-driven and precise. Transcript: José Valim We understand, like when you inspect like a DOM element, we know the DOM element and send it to the agent, but we also know where in the template or which React component that thing came From. And we send that to the agent. So we don't have to do the manual working of figuring that out and do it to the agent. When there's an error page, we detect the error page of all the web frameworks we support, automatically feed that to the agent. So it's really meant to be like, look, it's like you, the agent, the browser, the web framework, like in a shared context, kind of everybody can see what the other is doing because otherwise It's it becomes your responsibility you are the ones who are like getting information from all those places and passing around sounds pretty cool man and that's not pretty cool uh but Jerod Santo I'm passing a lot of that stuff i mean the fact that it has you know that i mean that's something i just uh i've always wanted i guess i mean you're going back and forth like. It's better to do it right there real time. I haven't played with it to know the UX really of that. Like when you're filling with, let's say, is it a button? Maybe it's not working properly. What is the back and forth with the experience? Can you speak to it? Can you type to it? Like what are some of the interfaces you can think of? José Valim Right. So I think there are three ways that you are interacting with it. So one is the usual chat prompt, right? And with the difference that we know what is the page that you're currently looking at. So you can talk to the page in the sense that, so for example, imagine that you just boot up your dev instance, your database is empty. You can go to a page that is listing all the podcasts, like for changelog and can say, oh, this page is empty. Add some podcasts. It knows which page it is at. So it can find information from the controller or from the live view and then say, okay, that's the data I need to, like, it gives you an entry point. So that's the chat. It has the context of the page. The other one is the inspector. It's like the browser inspector, but so you can click it and then you can mouse over elements. We show the DOM element. We also show like which templates on which Phoenix template it came from. And then you can like click it to open your editor or you can click it to ask the agent to do something. ([Time 0:41:07](https://share.snipd.com/snip/2a3e33c9-c776-49c1-9708-de077c4133de)) - **Agent Implemented Autoplay And Verified Itself** - José had TideWave implement autoplaying videos and then watched the agent write JS that scrolled and validated playback across videos. - The agent reloaded, scrolled, and ran JS to confirm each video played correctly, demonstrating an automated verification loop. Transcript: José Valim So for example, one of the coolest features that, uh, that we use TideWave to implement, like if you go to tidewave.ai today, we have videos in the, in the homepage. And I, so I added like the, the YouTube URLs or I thought YouTube, like the, the URLs for the, I added the video tags, right. And then I wanted to make it. So as I was scrolling through the page, video started to autoplay so i asked a thought tidewave to implement this which it can do it's like it's a it's a it's a straightforward feature You know i can't do it but i assume it's a straightforward straightforward feature for me i didn't look at the code but i'm sure it was pretty easy yeah so it implemented the thing, right? It implemented the thing. And then in order to test that it worked, it actually like reloaded the page. So this Tidewave, it wrote JavaScript code to reload the page and scroll to the first video. And then it runs on JavaScript to validate that the first video was playing, but not the other two, then it automatically scrolled a little bit more. So the second video started playing and then it ran some JavaScript to make sure that the second video was playing and not the other two. ([Time 0:44:20](https://share.snipd.com/snip/eac643d2-6217-4bd8-b339-2968925df7e6)) - **Prefer Code-Execution Tools Over Many MCPs** - Instead of many narrow MCPs, give agents a small set of powerful tools that can execute code in the app and page contexts. - Let those tools run code and use existing app connections (DB, auth) rather than duplicating via external MCPs. Transcript: José Valim I actually have a hot take in here that it's not a, it's not a, it's not a unique hot take, but, so to answer your question, which is going to kind of reveal the hot take is that. He's just teasing it. He was not going to reveal this. Adam Stacoviak He's just teasing the hot take. Stop setting it up, Jose, and give us the hot take. José Valim So like almost all of our APIs is write code. Oh, you can execute code in the context of the web application. You can execute code in the context of the web page. That's it. Because like we are doing all this dance. Like, oh, I said about the database, right? Like, oh, I'm going to have an MCP for the database. No, my web application already knows how to talk to the database. Just use that. Oh, I want to have an MCP to talk to GitHub. And I'm like, well, I already have like, I already logged in on GitHub in the browser. I already have the GitHub command line. Use that, right? Like for coding agents, we are even going as far as adding MCPs for documentation. And then I'm like, why am I going to a separate website to get documentation? You are a coding agent. The code is in your machine. And usually with the code, you have documentation. Why don't you use the documentation that is there already on your machine with the exact version that you're using? Because sometimes you go to the remote server and then we get the documentation for Phoenix 1.8, but we are still on 1.7, right? So for me, like the answer for, oh, they are like too many, like the context thing is like, I'm going to have just a small amount of tools. And what those tools are going to do is that they can run code, right? And I'll let them do whatever they need to do. So trying to keep the amount of tools minimal and powerful. And like this, this take that, you know, like, oh, MCPs are too much, you probably just need code, right? ([Time 0:59:22](https://share.snipd.com/snip/bfaf043e-eb4d-4453-bcad-1b78152bde4b)) - **Make Agent Tools Sustainable And Predictable** - Charge for a productized agent to sustain rapid iteration, multi-framework support, and ongoing model integration. - Offer predictable subscription choices (e.g., GitHub Copilot) instead of unpredictable per-token billing to lower adoption friction. Transcript: José Valim So it is a paid product we consider a little bit but then i realize well everybody this is an ai thing it's going it's a very rapidly changing landscape so if you want to be able to keep up And feel invested on this and continue improving it and also support different kinds of frameworks, we need to find a sustainable way of doing that. So yeah, we'll see where the launch was pretty good. We got a lot of people excited, but it also pointed out like, so today's like bring your own API key. And the feature that people ask the most for is cloud code support. So being able to bring like codecs and cloud code. And yeah, let's see. Right now i i like to i think in the email ([Time 1:10:55](https://share.snipd.com/snip/747ac456-6536-4c45-a3d9-29949a638944)) - **Tune Prompts And Edits Per Model** - Tailor prompts and tooling per model because different models obey prompts and edit formats differently. - Accept that maintaining model-specific prompts and edit handlers is painful but necessary for reliable agent behavior. Transcript: José Valim And it was very curious because we would say, like, hey, implement this feature. It would do all those things. And then at the end, done. And then we would ask something and it would say, good. And then we realized that the prompt we had for Entropic that was saying, be concise, GPT-5 was actually listening to that prompt and it was being concise. That's why I was just saying, done, good. It was not doing any fluff or anything. And that's when you realize that you actually have to come with, if you're building a coding agent like I am, you have to actually build a prompt per model. And now GPT-5 codecs came up with its own prompt that it's different from the GPT-5. And then not only, so just doing that, like fine-tuning the prompt per model, that's a pain. That's boring work. That's not something I want to do. And then it gets the other thing, which is then they have the tools. And at this point, those coding models, they are becoming so important for those companies that they're actually fine tuning, like how it should send edits to a file. They are fine-tuned the models for that. So when the GPT-5 codex model came out, they also said like, look, this model is best at sending these kind of diffs and edits over the wire. So now I have to implement the specific editing tools per model that I support. ([Time 1:22:54](https://share.snipd.com/snip/8bcf7bba-0a8e-41af-9fe7-86c9b48fcb5c))