Flowing With Agents  ## Metadata - Author: [[Changelog Master Feed]] - Full Title: Flowing With Agents - Category: #podcasts - URL: https://share.snipd.com/episode/83ce712e-69d1-4a98-965e-35255800e545 ## Document Notes ## Highlights - **Model Selection Should Be Invisible** - AMP abstracts model selection so users don't pick specific LLMs manually. - The product picks the best model per subtask to optimize latency and competency. Transcript: Beyang Liu Yeah, I don't think that it takes too much explanation. AMP is a coding agent. So most people, I would imagine your audience, know what that is at this point. But if you've been living under a rock for the past six months, a coding agent is essentially an AI-powered program that takes natural language instructions and then goes and gathers Context and then modifies the code following your instruction. And so it's much more kind of like high level. You kind of describe what you want. You figure out how to instruct it. And then it does and figures out how to do most of the editing for you. It runs the tests, runs compilers, it can correct itself. And the idea is that we want to enable programmers to operate at a higher level. So dictating what the architecture should be, what the key interfaces should be, what the UI should look like, and not as much have to be in the weeds of every single line of code that is Written. And then AMP in particular, among the landscape of coding agents, is distinguished by the fact that I think we're the only coding agent that I've come across that has this approach of, So we're a multi-model. We use multiple LLMs, which is not distinctive. There's a lot of coding agents that use a variety of underlying large language models and small language models now. But I think our approach is different in that we don't have a model selector. It's not like a model harness. So an AMP user doesn't think in terms of like, oh, let me use Claude Sonnet 4 today, or let me use GPT-5 today. We view that as basically like an implementation detail. So we'll figure out what the models are good at, and the contract to the end user is an agentic experience that hopefully just works well for your use cases. So there's no toggling on different specific models. ([Time 0:02:40](https://share.snipd.com/snip/c7b0dc9c-b3ab-44af-9023-6326047b0f8c)) - **Always Review Generated Code** - Read and review agent-generated code; don't trust it blind. - Keep doing human code review to catch subtle bugs and complexity the agent may introduce. Transcript: Beyang Liu Yeah, I feel like, so like the committing to Git, I think that's, I mean, at least for like AMP, that's largely solved now. It's been a while since I've seen like a class of error. Adam Stacoviak I'm blaming somebody else. Beyang Liu Okay, I think those sorts of things, I have very high confidence that it could just do now. But I get what you're saying. There is this spectrum where some things that you, a human, would consider difficult, it can just almost one shot. Whereas other things where a human, it'd be like a two-line change or whatnot, and it will just, we would call it like doom looping, where it's just like iterating over and over again And can't figure it out, even though you're saying like, okay, now try it like this. Try it later. You're trying to like nudge it, and it just like just won't get it. There's certainly that phenomenon that's still at play. And it's almost at the point where it's like a lot of this is it reduces to the underlying model intelligence. And so I think the proper way to view this is, part of it is, it's like, okay, there's some improvements that we need to make the product. In terms of how we present the answer to the user and help people prompt in a way that gets you to the sweet spot of the model. But there's also, users have to view this as a skill set they need to acquire as well. So the more you use agent decoding tools, the more you develop an intuition for what it can do and can't do. And so you can almost tell from talking to a person how much they've used these tools in terms of what they express their frustrations as. A lot of newbie users, especially the ones that are from the onset skeptical for whatever reason, they will say, oh, I asked you to do this thing that ought to be very easy, but it fell flat On its face. And then you ask to see what their prompt looked like, and it's like five words. It's almost like at that point, what do you expect to do? It's not a mind reader. There's almost like an information, you put a certain amount of bits in and from that the model has to tease out what your intention was. It can certainly accept instructions that are five words long, but if it's only five words then it's going to be very much based what its prior behavior wants to do. If you have a five-word prompt to create a simple React app or a simple game or whatnot, it could probably do that, but it's not going to be probably what you intended, unless what you intended Is the median React app or the median JavaScript game that's represented in its training set. Whereas the more expert users, they still have complaints, but it's oftentimes around very specific things. It's like, okay, I know what it's good at. I'm not complaining about the fact that it can't do these things anymore. I can still do those things, it's fine. It's more around like, hey, I would love to be getting more out of this tool, but there's certain bottlenecks that are kind of constraining how much I can use it. I think one very important bottleneck that still remains is code review. I think most expert users of coding agents realize that they're very powerful and they want to be using them heavily, but you can't trust them fully. You still got to read through the code that they emit and understand it, because otherwise the slop will creep in and they'll make subtle changes that if you don't catch and are not wary Of will add complexity and nuanced bugs, subtle bugs to the code. And so this process of the human understanding the code that's been generated and ensuring that it doesn't do anything very incorrect or not according to their intentions, that's An important part of the process. ([Time 0:10:31](https://share.snipd.com/snip/2f25e1ad-839d-4142-84ac-71bccf621273)) - **Learn Prompting As A Skill** - Invest time to learn prompting; it's a skill like any other. - Develop intuition for what agents can and cannot do to get reliable results. Transcript: Beyang Liu And so you can almost tell from talking to a person how much they've used these tools in terms of what they express their frustrations as. A lot of newbie users, especially the ones that are from the onset skeptical for whatever reason, they will say, oh, I asked you to do this thing that ought to be very easy, but it fell flat On its face. And then you ask to see what their prompt looked like, and it's like five words. It's almost like at that point, what do you expect to do? It's not a mind reader. There's almost like an information, you put a certain amount of bits in and from that the model has to tease out what your intention was. It can certainly accept instructions that are five words long, but if it's only five words then it's going to be very much based what its prior behavior wants to do. If you have a five-word prompt to create a simple React app or a simple game or whatnot, it could probably do that, but it's not going to be probably what you intended, unless what you intended Is the median React app or the median JavaScript game that's represented in its training set. Whereas the more expert users, they still have complaints, but it's oftentimes around very specific things. It's like, okay, I know what it's good at. I'm not complaining about the fact that it can't do these things anymore. I can still do those things, it's fine. It's more around like, hey, I would love to be getting more out of this tool, but there's certain bottlenecks that are kind of constraining how much I can use it. ([Time 0:11:56](https://share.snipd.com/snip/bb842724-662d-4caf-933f-0224bd634362)) - **Agent Workflow Is A For-Loop** - Agents operate as a for-loop around an LLM that issues tool calls and receives results. - The loop continues until no more tool calls remain and then the agent returns a final response. Transcript: Beyang Liu So I'd say there's no weird rocket science here. I'd say at the core, at a very high level, AMP operates the same way that every other agent does, which is, the way I'd describe it, it's a for loop wrapping an agentic LLM. So you take an LLM like Claude or GPT-5 or any of the other agentic LLMs that are now online, you put that in a for loop, and what the loop looks like is you take the user input, you feed it in The model, the model will come back with some combination of tool calls and written response, like thinking things out or responding to the user query. And the tool calls are just text. They describe, like, hey, you told me that you had access to these tools. Maybe it's grep, maybe it's a read file tool, maybe it's a tool to edit a file. I want to invoke the tool of these arguments. And then our application logic goes and executes the tool call. So given the spec that we get from the model, we execute the tool, we get the response, and then that response gets fed back into the next iteration of the loop. And that just keeps looping until there are no more tool calls, at which point the model generates a final response. ([Time 0:19:37](https://share.snipd.com/snip/6a19e2cb-531d-458a-8224-c0460572edb5)) - **Use Sub-Agents For Specialized Tasks** - Sub-agents are nested agent loops specialized for tasks like code search. - Those sub-agents let AMP delegate targeted work and compose capabilities reliably. Transcript: Beyang Liu So sub-agents are a special class of tools where you call a tool, but underneath the hood it's just its own nested for loop. So it's its own agentic loop doing a more targeted task. So you could have just like an agent calling itself in sort of like a recursive fashion, or might call a domain-specific agent. Like we have an agent that's dedicated to code-based search, which has been optimized for finding and locating relevant context within the repository. And so there's a lot of tuning that we do to make sure that all those pieces operate well together, that we can kind of eliminate the dumb class of errors, like, you know, oh, I can't SSH Into this thing, or I can't read this file. The fill in the gaps for what blocks the agent from getting further, or what helps it course-correct itself when it doesn't immediately do the right thing off the bat. And then there's this client-server architecture that we have, where part of it always has to be on the server because the best models are still server-bound right now because local Models have not yet gotten to the point where they're fast enough to run locally. But in addition to that, AMP also has a server-side store for all the threads. So we call the agentic interactions that people have with AMP threads. ([Time 0:21:29](https://share.snipd.com/snip/674520a3-86a0-4417-9e8a-d53c7cad451e)) - **Save Threads As Learning Artifacts** - Keep agent session transcripts (threads) to learn and share best practices. - Use server-synced threads so teammates can inspect interactions and improve prompting. Transcript: Beyang Liu So every session you have with AMP to accomplish a task is a thread. And all those threads get synced to the server. And the benefit of that is that you can go on AMP and see the threads of all your teammates and how they're using AMP, which is really helpful because coding agents, using them effectively, In our view, is a high-ceiling skill. Like you said before, it's not trivial. There's certain things, especially if you're a newbie, where you expect it to be able to do something, especially if you've come in consuming a lot of the hype around AI. You're like, oh, it's magic. Then it doesn't do the thing, it doesn't read your mind. It's really helpful to have this team-wide view of how other people are using this tool to great effect, because then you can go and learn what their best practices are. So that's another big component of the AMP architecture, is we do have the server-side component that stores a thread information. So you can share with your teammates, and also you can go back and revisit previous threads in case you're like, oh, what was that thing that I was learning about a couple days back? Let me go up and look at that thread and see what it actually did there. ([Time 0:22:56](https://share.snipd.com/snip/fbb9e997-1b0a-47a4-b3a6-a3c93840d33d)) - **Define Roles And Store Prompts** - Create reusable role templates and an agents.md to steer agent behavior. - Store role prompts and style guides so agents behave consistently across sessions. Transcript: Adam Stacoviak Like know i've i've come to this uh you know my usage pattern has gotten to this point where i feel like i need to give i need to define roles similar to how you build out a team i have a linux Platform engineer or a you know a go style enforcer who understands really good go idiomatic go, and then a style guide for my Go code. And then I have a Go-style reviewer, because you were talking about code review before. And so I will give them different roles, and I feel like that works for me. But as part of getting there, it was condensing more and more of what I would kind of keep as this massive prompt, and I would just sort of move that larger prompt for me into a history file Of sorts, where this is a role, assume this role, here's your task, and then go forth with that new role. And I feel like that gives it some really good context versus some magical prompt I've been car-culting again and again and again. So I've learned a skill set, if you can call it that, to better be an agent babysitter. Beyang Liu Yeah, I think, you know, I talked to a lot of users and a lot of people have some variant of that, where it's almost like they've built up a library of prompts for specific task types. I think a lot of people have built up some sort of template for listing out. It's like a combination of like, here's how to structure your plan. First of all, generate a plan at all. If you want to do a longer running task, generate a plan. Here's a couple pointers about how to structure that. ([Time 0:30:40](https://share.snipd.com/snip/28d93dd3-40eb-425d-ab22-ee477acb9a78)) - **Plan Then Execute In New Threads** - Front-load detailed plans for complex tasks and then execute them in separate threads. - Use a planning thread to produce a plan and feed it as input to focused implementation threads. Transcript: Beyang Liu Yeah, I think, you know, I talked to a lot of users and a lot of people have some variant of that, where it's almost like they've built up a library of prompts for specific task types. I think a lot of people have built up some sort of template for listing out. It's like a combination of like, here's how to structure your plan. First of all, generate a plan at all. If you want to do a longer running task, generate a plan. Here's a couple pointers about how to structure that. And then maybe there's a couple pointers about the code base as well. We have introduced this thing called agents.md. I think it's pretty standard now across many coding agents. And now there's finally a standard. I think more and more people are hopping on to this using the agents.md file as a standard for context. So AMP will consume that, but a lot of people add additional context beyond that that are maybe not specific to the code base, but specific to their personal, the things that they personally Feel they want the agent to do. And so they'll copy and paste that in to sort of like steer the agent to do and behave the way they want it to behave, especially over these like longer running tasks. Adam Stacoviak But how do you feel about the user experience, like the way a user interacts? So you said you interact with a lot of different users who have different ways and all that good stuff. I've been working on a, since I've been building my skills to be an agent babysitter i've kind of come to this idea i'm calling it agent flow it's an internal name to my own brain i'm a team Of one you know i got a team right here at changel but the things i'm working on just sort of like fun tools that solve my own itch and my own problems and so my interactions is not team-based. So I'm not leveraging your threads and sharing those threads with other people. It's simply just a team of one doing it. And so I've come up with this idea of agent flow. I would say the protocol is called document-driven development, and the implementation is called agent flow. How do I get agents to understand how I want to flow with it? You know, we vibe and we flow and so I'm like, okay, agent flow sounds kind of cool. I don't know that's trademarked or not, but I like it. Okay. Um, I've kind of given it, given you a glimpse of it where I define roles. I obviously leverage your agent file. I think Claude has its own version of it. And so, you know, maybe you can centralize it to just simply agent versus Claude.md, But essentially, how do I want my agent to know about my code base, to know about the things, the tasks I'm giving it? And obviously, which role to take on, which hat to put on, so to speak. And maybe a style guide of sorts that represents how do we write Go code or how to write Rust code? Like what is the blessed idiomatic way of doing that? And then I think the additive to that that really sort of solidifies agent flow for me, and you sort of mentioned it was this, how do you give it its list? How do you give it its direction? And I just borrowed it from the great language of Python. They have PEPs, Python Enhancement Proposals, and they have a whole system around drafting PEPs. And so I took the same acronym, PEP, and I just said that this is a project enhancement proposal. And so for every new thing, every new idea I have, it begins as riffing or vibing as you would, but it's all about developing this document called a PEP and the PEPs have numbers. So it could be PEP dash zero zero three one. Uh, maybe I'll need 10,000 at some point, but for now I feel like, you know, for, you know, numerals there is, is just fine, but that's the essence for the most part is like draft peps so That I understand full context. Let's, let's dream and build and think. And that thing has statuses. It has all the artifacts that pep lives in its own directory. You can, the agent can add more things like maybe it completed something. So it's a completion report. We complete this PEP, but don't just update the PEP document. Give me a whole separate document that is about the completion of it. And how did it work? So I can go back and learn and read later on. You know, you even have things like knowledge base. So I have it draft knowledge base articles. So once we're done with the PEP and we've done X, Y, or Z, not just the completion report inside that PEP, but more like, what do we learn that is now an institutional knowledge for you as An agent or any other agents or any of the team members I bring on later on? Even internal blogs or what I'm calling builder logs. It's still blog. It's B-L plural, but I'm calling them builder logs. So this is more of an internal free form, but just go and dream what happened here. And I'll go back and read those as my agent babysitter hat. And I learn with it how to direct it. And so this entire flow that I've sort of learned as a skill set, I've been calling agent flow. What do you think? Beyang Liu I think that fits with the way that a lot of people use AMP, actually. I think the more sophisticated, intentional users have a very intentional design process that often starts with the human generating a lot of tokens describing what they want. And the length of the complexity of the task that you want it to handle, there's a linear correlation between the complexity of the task and how much direction you want to give it. So if I'm doing a thing where I know exactly what I want, and I want the agent to get as far as it can on its own, then I will do things that are very similar to what you described. Like, hey, I want you to generate a plan. It's got to have these properties. Make sure that each step is annotated with the relevant context files that you'll want to address when you're actually implementing this plan. And maybe I'd do a couple iterations on the plan itself, if there's any course corrections, like, oh, use this technology, not that technology, or use this library, not that library. And then once that plan is generated, then you have another thread that then goes and says, okay, go and execute this plan, or do steps one through two of this plan, and then let me review It. ([Time 0:31:52](https://share.snipd.com/snip/0ef3ad0b-429d-41d4-96e1-bd422fce0f66)) - **Shipping A Weekend Spike With A Toddler** - Beyang described finishing a spike while caring for his two-year-old by using AMP asynchronously. - He let the agent run, checked results occasionally, and shipped work he couldn't have done otherwise that weekend. Transcript: Beyang Liu Yeah. It's, it's absolutely incredible. And I think one of the, the first like, what's like wow moments or like points of realization for me with Amp too. It was like, this is very early on. It was like back in like probably like April or something. It was like, I had one of these weekend experiences where I was taking care of our kid. We have like a two-year at home. So he's like running around. He's an agent of chaos. And so it's like, it's very difficult to be in the same room with him and do anything for longer than like, uh, two minutes at a time. And I was literally running back and forth between like playing with him and trying to like get the spike of this like feature up and running. And, uh, I ended up like getting to a point where I was like, Oh, this is actually good. And I, I'm going to push this like over the course of a Sunday. And it's like without the benefit of this tool, it would not have happened. Because in order to write anything trivial, you need that mental space to get into flow and to think through the things and page in all the context. And every interaction that. Whereas with the agent, if the thing that you're doing is within the capability of the agent and you're able to remain at the high level, you can absolutely do this async thing where it's Like, let me tell you what I want you to do. Let me give you some detail. Let me maybe give you some pointers for how to validate it and verify that you're doing the right thing. And then I'll fire it and then I'll go and like do something else. ([Time 0:50:34](https://share.snipd.com/snip/5e84ddbf-e0ce-41f6-8d84-ec5fcfcc5b93)) - **Keep Threads Short And Atomic** - Long threads can degrade quality and increase cost because models must reprocess all prior messages. - Start fresh threads for new atomic tasks to reduce confusion, latency, and token cost. Transcript: Beyang Liu Me some of your threads that I think one of the cool things about amps you could share your threads with other people and so like oftentimes we'll get users sharing their threads and saying Like hey you know this is how I built this or like hey you know can you help me improve and understand what I could do better? I would love to take a look at how you're using it and maybe give you best practice tips from what we've learned. I will say my intuitions are almost inverse from yours. If anything, starting fresh is great, because you know the context window is clean. The more things that you accrue in the context over time, the more possibility for confusing the model there is. If anything, it's like back when the context window limit for Claude's sonnet was around 200K, I think one of the things we noticed was there was severe... You started to notice a little bit degradation, actually around 70K. And then a much steeper degradation past 120 tokens. It was always one of those things where it was vibe-based. So we never built that directly into the product saying, oh, don't go past 120. Because sometimes there's a legitimate use case that does involve taking advantage of a lot of the previous context. So it's a matter of user taste. And plus, we're all still learning in real time, so we didn't want to be too prescriptive. But I think maybe now we're at the point where we've seen enough where we will be offering some more visual indicators around how much of the context has been used and where the sweet spot Starts to end for some of these models. Now it's like you can go up to a million, and I think the quality has gotten better beyond 70K now, but it's still to the point where all things considered, I prefer the kind of like ripoff Note, let's start a fresh thing. And it's almost like the context that you wanted to remember hopefully is captured within the files that it's modified. So it's like, you this edit to the code. Now it does this thing. Now let's start a new thread and say, now it does the thing that I just asked it to do, but now make this other modification. It's almost like, as a human, when you git commit, you're kind of saying like, okay, I'm committing this. This is one atomic change. And now let me start from a clean slate to do the next incremental thing, where I don't have to worry about all the other stuff, all the other changes that I made to do the previous thing. It's sort of like mentally clear to have the separation. ([Time 1:10:50](https://share.snipd.com/snip/5e290b73-1a43-4ac0-a1ec-5984acdd8041)) - **Watch The Context Meter** - Monitor the context-meter and avoid overfilling the model's working memory. - Start new threads or commit work so the next task has a clean context window. Transcript: Beyang Liu Yeah, no, I totally get that. And it's something that we've been actively thinking about for a while now. I think we're one of the first to ship the context window meter. So just showing you how much of the context window has been filled up, that's an important indicator. I think now I've seen it pop up in a couple of other coding agents. But that to us is one signal that's useful to look at. Because the higher that percentage goes, the more costly each additional message becomes. You take a latency hit as well. And you start to see quality degradation at some of these key fall-off points. I will say for the folks listening, my personal best practice for using agents is, again, the analogy that I draw when explaining this to people is the context window is sort of analogous To the human brain's working memory. So, you know, it's like, there's the old adage, like, you can't think of more than, like, five or seven. I forget what the exact number is, but, like, there's only a certain amount of, like, distinct concepts that you can have in your brain's, kind of like L1 cache or working state that are, Like, immediately instantaneously accessible at a given time. And the context window is essentially that, but for LLMs. So the more you try to cram in there, at some point, it's got to pick and choose which pieces of that it's actually going to pay attention to, and then it has to effectively ignore the rest. ([Time 1:18:54](https://share.snipd.com/snip/33ae6bf7-20fa-42e8-a8e6-41e4a4500105)) - **LLMs Are Powerful Pattern Matchers** - Treat modern LLMs as universal pattern matchers, not conscious entities. - Use them to fit repeatable patterns in tasks rather than expect human-like understanding. Transcript: Beyang Liu And then I go and use it and can't even figure out how to SSH into my remote machine. I don't know how to do that. I can't do that. Yes, you can. I would say philosophically, what the technology is, it's not consciousness. It's not a human replacement. It's not a god. What it is, is it's a universal pattern matcher. If you go and watch Demis' Nobel lecture, he's the head of DeepMind, received a Nobel Prize for the work that they did on that team in protein folding. It's a very short lecture. It's like 30 minutes. But he says something very insightful there where it's like the transformer architecture, what it really does is it allows you to fit any pattern that is either observable in nature Or that you can synthetically generate. So as long as you have a way to collect the data from observing nature or generate that data in an environment that closely enough approximates what you're getting after, what the transformer Model architecture does is allows you to train a model that fits that pattern. They're great curve fitters or pattern matchers. And so that is not nirvana, but it is a useful tool. ([Time 1:48:32](https://share.snipd.com/snip/085b6627-b74d-4c29-8b79-da34b030aeba)) - **Start With A Small Out-Of-Band Project** - Try agents on projects outside your wheelhouse to see quick wins. - Pick a small, practical app and iterate to learn capabilities and limits fast. Transcript: Beyang Liu And that segues into my practical advice, which is if you want to experience the wow in a way that's tangible and delightful and also practical, I think the first thing I would do is pick An app or pick a domain that's somewhat outside of your wheelhouse as a developer. Maybe you're a hardcore systems engineer, but you've never built an iPhone app. But you have a three-year kid and you want to build a game that teaches him how to spell basic words or something like that. Go and sit down. It could be with AMP. It could be with any of the other coding agents. For this particular task, I think most of them that you've heard of will do a pretty good job of building a basic app that's outside of your wheelhouse to do something that is simple to A lot of people out there, but hard for you. And if you do that, I think probably with 98% confidence, you'll have a good experience with the proper mindset. ([Time 1:51:04](https://share.snipd.com/snip/8d56354d-3718-4c19-8f16-e77d7424924e)) - **Libraries Will Shift Toward Domain Value** - As code becomes cheaper to generate, the role of libraries will shift. - Expect fewer trivial utility packages and more domain-specific, robust libraries around new hardware and APIs. Transcript: Beyang Liu But at the same time, I do think that there remains a use case for having libraries. I just think that the nature which libraries are really popular is probably going to change. I think prior to agents and AI, a lot of the most popular libraries in existence were some form of middleware or a piece of abstraction that helped make using a particular API or technology Or a piece of physical hardware more accessible. And so that was a big problem that libraries, like open source packages and libraries, solve for you. I think now that there's kind of less demand for that form of package, but still a large amount of demand for libraries that are robust, they're well-tested, that provide—you still Need some amount of abstraction, but maybe fewer layers of abstraction over an underlying capability. I don't know if that makes sense. Would say my strongly stated opinion, but maybe weakly held, given how quickly things are changing, is that we'll see far fewer libraries that are purely just like, hey, this is a neat Way to read files in Node.js or something like that. And much more libraries like, hey, there's this new piece of hardware that got developed. Or maybe there's some new biotechnology that is available now. And someone built an API for that, and it exposes all the key hooks. And because software is so cheap now, code is cheaper to generate with agents than with humans manually hand coding everything. We'll just see a lot more things expose software endpoints and we'll see a lot richer playground for people building software to be able to hit different things that do things either In cyberspace or in the physical world. ([Time 1:56:34](https://share.snipd.com/snip/c201e720-4221-44cf-a371-4b0eabe8c1bc))