Agents that remember - CourtneyR.dev's 2nd Brain

Agents that remember ![rw-book-cover](https://readwise-assets.s3.amazonaws.com/static/images/article2.74d541386bbf.png) 🔗 **Source:** [Agents that remember](https://www.youtube.com/watch?v=geUv4CjPpxI) > [!note]+ Document Notes > tags: [ai-agents, memory-systems, multi-agent-orchestration] > author: Kevin > source: Anthropic > reference: https://www.youtube.com/watch?v=geUv4CjPpxI > date created: {{date:YYYY-MM-DD}} {{time:HH:mm}} > > # 🏷️ Title: Agents that remember >  ## 🧭 Table of Contents - [📝 Summary](#-summary) - [📖 Main Content](#-main-content) - [🗒️ Notes](#️-notes) - [📚 References](#-references) - [🗣️ Paraphrase](#️-paraphrase) - [📰 Newsletter](#-newsletter) - [✅ Action Items](#-action-items) - [🧪 Hypotheses](#-hypotheses) - [📊 Methodology](#-methodology) - [❓ Questions](#-questions) - [🔮 Future Research](#-future-research) - [💡 Implications](#-implications) - [🧠 Learning Styles Assessment](#-learning-styles-assessment) - [📊 Accessibility Assessment](#-accessibility-assessment) - [🌍 DEIB Assessment](#-deib-assessment) - [✅ Logical Validity](#-logical-validity) - [🛡️ DARVO Analysis](#-darvo-analysis) - [🎯 Conclusion](#-conclusion) ## 📝 Summary - Anthropic's Kevin demonstrates new memory capabilities for Claude agents that enable persistent information storage across sessions. - Memory stores act as file system-like storage that agents can read from and write to, solving the isolation problem where agents forget previous interactions. - "Dreaming" is an asynchronous multi-agent process that organizes, fact-checks, and enriches memory stores to prevent unbounded growth. - The system uses three composable layers: sessions (isolated instances), memory stores (cross-session persistence), and dreaming (memory optimization). - Key stakeholders include developers building agent workflows, organizations managing AI systems, and users requiring continuity in agent interactions. ## 📖 Main Content Anthropic has introduced memory capabilities to their Claude Manage Agents platform to solve a fundamental limitation: agent isolation. Previously, each agent session was isolated, meaning agents couldn't remember information from previous interactions or transfer knowledge to future sessions. ### Memory Stores Memory stores are persistent, file system-like storage that attach to agent sessions as resources. Key characteristics: - **File system interface**: Agents can use bash commands, grep for searches, and standard file operations - **Flexible boundaries**: Organizations can create memory stores per user, workspace, or other logical divisions - **Access controls**: Sessions can have read-write or read-only access - **Versioning**: All memory files are versioned for change tracking When creating a session with a memory store, developers can include steering prompts to guide what information agents should prioritize remembering. ### Dreaming Process As memory stores grow, they face typical knowledge management challenges: duplication, staleness, and disorganization. Anthropic's solution is "dreaming" - an asynchronous multi-agent harness that: - **Fact-checks** existing memories against session transcripts - **Enriches** information with additional details like dates and identifiers - **Organizes** memory files and removes duplicates - **Creates indexes** for more efficient future retrieval The dreaming process is non-destructive, creating a new output memory store rather than modifying the original. It uses Claude Opus or Sonnet models and can process dozens to hundreds of sessions depending on scale needs. ### Technical Implementation The system runs on Anthropic's existing Claude Manage Agents infrastructure: - Memory stores mount as file systems in session containers - Dreaming jobs spawn sub-agents (one per input session) coordinated by an orchestrator - High cache hit rates (~95%) reduce token costs despite exhaustive processing - Sessions, memory stores, and dreams are composable layers that can be mixed and matched ## 🗒️ Notes #### ⏱️ [00:01:30](https://www.youtube.com/watch?v=geUv4CjPpxI&t=90) Problem Statement - Agents today are isolated - no information transfer between sessions - Limits usefulness in real-world workflows requiring continuity #### ⏱️ [00:03:45](https://www.youtube.com/watch?v=geUv4CjPpxI&t=225) Memory Store Concepts - File system-like persistent storage attached to sessions - Agents get read/write tools for memory manipulation - Can organize per user, workspace, or other boundaries #### ⏱️ [00:08:20](https://www.youtube.com/watch?v=geUv4CjPpxI&t=500) Live Demo - Creating Memory - Kevin demonstrates CLI commands for memory store creation - Shows console interface for viewing memory contents - Agent automatically saves information from user messages #### ⏱️ [00:12:15](https://www.youtube.com/watch?v=geUv4CjPpxI&t=735) Memory Retrieval Demo - New session with same memory store can access previous information - Agent uses grep to search for relevant keywords in memory - Successfully recalls CMA talk information from previous session #### ⏱️ [00:16:30](https://www.youtube.com/watch?v=geUv4CjPpxI&t=990) Dreaming Introduction - Addresses memory store growth and organization problems - Multi-agent harness for memory optimization - Runs asynchronously to avoid blocking active sessions #### ⏱️ [00:19:45](https://www.youtube.com/watch?v=geUv4CjPpxI&t=1185) Dreaming Process Demo - Creates dream job with Opus model and session inputs - Shows console monitoring of dream job progress - Non-destructive - creates new output memory store #### ⏱️ [00:23:10](https://www.youtube.com/watch?v=geUv4CjPpxI&t=1390) Dreaming Results - Shows diff of memory improvements - Added index file for efficient retrieval - Enriched existing memories with additional details - Better organization and structure #### ⏱️ [00:26:00](https://www.youtube.com/watch?v=geUv4CjPpxI&t=1560) Enhanced Retrieval Demo - Session using dreamed memory store shows improved responses - More comprehensive information retrieval - Better organization leads to more relevant answers #### ⏱️ [00:28:30](https://www.youtube.com/watch?v=geUv4CjPpxI&t=1710) Cost Considerations - Exhaustive by design but high cache hit rates reduce costs - Exploring batch API pricing for further discounts - Token budgeting controls available ## 📚 References - [Anthropic Claude Manage Agents Platform](https://console.anthropic.com) - [Workshop Repository](https://github.com/anthropics/anthropic-cookbook) ## 🗣️ Paraphrase Anthropic has solved a major limitation in AI agents: the inability to remember information across conversations. Their new memory system works like giving agents a persistent notebook they can read and write to across multiple sessions. When these notebooks get messy over time, an automated "dreaming" process - essentially other AI agents - cleans them up, fact-checks information, and organizes everything for better future retrieval. This creates a three-layer system where individual conversations can build on past knowledge, and that knowledge stays organized and useful over time. ## 📰 Newsletter 🧠 **AI Agents Get Memory Upgrades** Anthropic just dropped some serious agent capabilities that WordPress developers should pay attention to. Their new memory system lets AI agents remember information across sessions - imagine a chatbot that actually recalls your previous conversations and builds on them. The system works with three layers: - **Sessions**: Individual conversations (like current chatbots) - **Memory stores**: Persistent file storage agents can read/write to - **Dreaming**: Background processes that organize and improve memory over time Why care? If you're building AI features into WordPress sites - customer support bots, content assistants, or automated workflows - this persistent memory could be a game-changer for user experience. No more "I don't remember what we discussed yesterday" frustrations. The technical implementation is clever too: memory mounts as a file system, so agents can use familiar commands like grep and bash to search and organize information. WordPress developers could potentially integrate this for plugin development, content management workflows, or customer service automation. Worth exploring: [Anthropic's documentation](https://console.anthropic.com) and their workshop examples for inspiration on WordPress AI integrations. ## ✅ Action Items #### DevRel - **Demo development**: Create WordPress plugin integration examples using memory-enabled agents - **Workshop content**: Develop "AI Agents for WordPress" tutorial series incorporating memory concepts - **Community engagement**: Host webinar on persistent AI agents for WordPress developers #### WordPress - **Core AI**: Evaluate memory store patterns for future WordPress AI features - **Plugin directory**: Research existing AI plugins that could benefit from memory capabilities - **Documentation**: Create guidance for developers on responsible AI memory usage in WordPress contexts #### Community - **Ecosystem**: Explore partnerships with AI/ML tool providers for WordPress integrations - **Contributors**: Identify developers interested in AI features for potential collaboration - **Events**: Propose WordCamp talks on AI agent integration best practices ## 🧪 Hypotheses 1. **Memory persistence increases agent utility exponentially** - Agents with cross-session memory will handle complex, multi-step workflows significantly better than isolated agents 2. **File system interface adoption reduces learning curve** - Developers will more readily adopt agent memory systems that use familiar file operations rather than specialized APIs 3. **Automated memory organization prevents degradation** - Without active memory management (dreaming), agent performance degrades as memory stores grow beyond optimal size 4. **Multi-agent memory processing scales better than single-agent** - Parallel processing of memory optimization using multiple specialized agents outperforms sequential single-agent approaches ## 📊 Methodology ### Testing Memory Effectiveness - A/B testing: Sessions with vs without memory stores across identical task sequences - Metrics: Task completion rates, response accuracy, user satisfaction scores - Control variables: Task complexity, session length, information density ### File System Interface Usability - Developer surveys: Learning curve assessment, preference rankings vs API alternatives - Time-to-implementation studies: Speed of memory integration in existing workflows - Error analysis: Common mistakes when using file system operations vs specialized commands ### Memory Degradation Analysis - Longitudinal studies: Memory store performance over time with/without dreaming - Information retrieval accuracy: Precision/recall metrics as memory size increases - Cost analysis: Token usage patterns with organized vs unorganized memory stores ## ❓ Questions - How does memory store performance scale with size - is there a practical upper limit before retrieval becomes inefficient? - What security and privacy implications exist when agents persist user information across sessions? - How does the dreaming process handle conflicting information or corrections to previously stored memories? - What happens when multiple agents write to the same memory store simultaneously - are there concurrency controls? - How does memory store organization affect agent reasoning quality beyond just retrieval efficiency? ## 🔮 Future Research - **Selective memory mechanisms**: Research optimal strategies for determining what information deserves long-term storage vs ephemeral handling - **Memory sharing protocols**: Investigate secure methods for agents to share memory stores across organizations or domains - **Hierarchical memory systems**: Explore multi-tier memory (short-term, working, long-term) similar to human cognitive models - **Memory compression techniques**: Develop methods to maintain memory utility while reducing storage costs and retrieval time - **Cross-modal memory integration**: Study how to effectively store and retrieve non-text information (images, audio, structured data) in agent memory systems ## 💡 Implications ### Practice - **Workflow continuity**: Enables agent deployments in customer service, project management, and content creation where context spans multiple interactions - **Cost optimization**: Memory systems may reduce redundant context provision and improve task efficiency - **User experience**: Persistent memory could eliminate frustrating "restart from scratch" scenarios in AI interactions ### Policy - **Data retention**: Organizations need clear policies about what agent memory stores contain and how long information persists - **Privacy compliance**: Memory persistence raises GDPR, CCPA implications for user data handling across sessions - **Access controls**: Need frameworks for managing who can access, modify, or delete information in shared memory stores ### Security - **Memory poisoning**: Attackers might attempt to inject false information into memory stores to influence future agent behavior - **Cross-session information leakage**: Shared memory stores require careful isolation to prevent unintended information sharing between users - **Audit trails**: Memory versioning enables forensic analysis but also creates compliance obligations ### Education - **Developer training**: Need new curricula covering memory-enabled agent design patterns and best practices - **Ethics education**: Training on responsible AI memory usage, particularly around user privacy and consent - **System thinking**: Developers need to understand multi-session, multi-agent workflows rather than single-interaction patterns ## 🧠 Learning Styles Assessment **Reading level estimate**: Flesch-Kincaid Grade Level 12-13. Technical vocabulary and complex sentence structures require advanced comprehension. #### 👀 Visual learners - **Present**: System architecture diagram showing session → memory store → dreaming workflow - **Improve**: Add flowcharts for decision trees (when to use memory vs not), visual diff comparisons of pre/post-dreaming memory organization, screenshot walkthroughs of console interface #### 🔊 Aural learners - **Present**: Live demonstration with verbal explanation of each CLI command and console interaction - **Improve**: Audio summary of key concepts, podcast-style discussion of use cases and implications #### 🗣️ Verbal learners - **Present**: Clear definitions (memory store, dreaming, sessions), analogies to human memory and file systems - **Improve**: Glossary of technical terms, story-driven examples showing problem → solution progression #### 🧩 Kinesthetic learners - **Present**: Hands-on workshop format with step-by-step CLI commands and immediate results - **Improve**: Interactive sandbox environment, coding exercises building memory-enabled agents, troubleshooting scenarios #### 🧠 Logical learners - **Present**: Three-layer architectural model, clear cause-effect relationships between memory problems and solutions - **Improve**: Decision matrices for memory store design choices, cost-benefit analysis frameworks, performance metrics tables #### 🧑‍🤝‍🧑 Social learners - **Present**: Workshop format with Q&A, audience participation, real-world use case discussions - **Improve**: Peer code review exercises, collaborative memory store design sessions, community forum discussions #### 🧘‍♀️ Solitary learners - **Present**: Self-guided workshop repository with complete examples - **Improve**: Individual reflection prompts on use cases, self-assessment checklists for implementation readiness #### 🌿 Naturalistic learners - **Present**: Analogies to human memory formation and organization - **Improve**: Examples from biological systems, ecosystem metaphors for multi-agent memory management ## 📊 Accessibility Assessment #### Screen reader semantics - **Present**: Structured headings, descriptive link text in transcript - **Improve**: Alt text for console screenshots, table structure for technical specifications, ARIA labels for interactive demo elements #### Keyboard navigation - **Present**: CLI-based demos are inherently keyboard accessible - **Improve**: Ensure console interface supports full keyboard navigation, provide keyboard shortcuts documentation #### Contrast and readability - **Present**: Terminal/CLI output typically has high contrast - **Improve**: Verify console interface meets WCAG contrast requirements, offer high-contrast theme options #### Text resizing and reflow - **Present**: Text-based content scales appropriately - **Improve**: Test console interface at 200% zoom, ensure code examples wrap properly #### Media and captions - **Present**: Video has transcript provided - **Improve**: Add chapter markers, synchronized captions with technical term definitions #### Forms and interactions - **Present**: CLI commands have clear error messaging - **Improve**: Form validation for console interface, accessible error states and success confirmations ## 🌍 DEIB Assessment #### Cultural breadth - **Present**: Technical content focuses on universal computing concepts - **Improve**: Include use cases from diverse industries and regions, consider multilingual memory store examples #### Inclusive language - **Present**: Uses technical terminology appropriately, avoids exclusionary jargon - **Improve**: Define all technical terms on first use, provide plain-language summaries of complex concepts #### Neurodiversity support - **Present**: Clear structure with timestamps and sections - **Improve**: More frequent headings, visual breaks between concepts, consistent formatting patterns #### Representation - **Present**: Single presenter (Kevin) from Anthropic engineering team - **Improve**: Include diverse voices in use case examples, show varied applications across different domains #### Psychological safety - **Present**: Encourages audience participation, allows for questions - **Improve**: Explicitly welcome different experience levels, provide multiple ways to engage with content ## ✅ Logical Validity #### Strong reasoning - **Problem-solution structure**: Clearly identifies agent isolation problem, then systematically introduces memory stores as solution. Location: Opening section establishing the base case. - **Causal explanation**: Demonstrates how memory persistence directly enables cross-session information transfer through concrete examples. Location: Live demo comparing sessions with and without memory. - **Inductive generalization**: Uses specific CMA talk example to illustrate broader principle of agent memory utility. Location: Demo sequences showing information storage and retrieval. #### Fallacies - **Appeal to novelty**: Implies memory-enabled agents are automatically better without considering contexts where isolation might be preferable. Fix: Acknowledge trade-offs and appropriate use cases for both approaches. - **Hasty generalization**: Token cost claims (95% cache hit rate) presented without sufficient context about usage patterns. Fix: Provide more specific conditions under which these rates apply. #### Tone of voice - **Technical but accessible**: Uses specialized terminology while providing explanations. Examples: "file system-like store," "multi-agent harness." - **Demonstrative confidence**: Presents capabilities as established solutions rather than experimental features. Examples: "This is how you solve the problem," definitive statements about system performance. ## 🛡️ DARVO Analysis No DARVO patterns detected. This is a straightforward technical demonstration without accusations, defensive rhetoric, or victim-offender role reversal. The presentation maintains focus on technical capabilities and implementation details without deflecting criticism or reframing problems as external fault. ## 🎯 Conclusion Anthropic's memory-enabled agents represent a significant advancement in AI agent capabilities, moving from isolated interactions to persistent, evolving knowledge systems. The three-layer architecture of sessions, memory stores, and dreaming provides a scalable solution to information continuity challenges that have limited agent utility in complex workflows. For developers, this system offers practical benefits: agents that remember user preferences, build on previous work, and maintain context across extended interactions. The file system interface lowers the learning curve by using familiar operations rather than specialized APIs. The broader implications extend beyond individual agent capabilities to organizational knowledge management and user experience design. As these systems mature, they'll likely influence how we design AI-human collaboration patterns and think about persistent digital assistance. Next steps for readers include exploring Anthropic's workshop materials, considering memory integration points in existing workflows, and evaluating privacy and security requirements for persistent agent memory in their specific contexts. --- ## Full Transcript [00:19] Hello everyone. Thank you all for [00:21] joining us today. Um, my name is Kevin. [00:24] Uh, I'm an engineer here at Anthropic. [00:27] And today we'll be learning about how to [00:29] build agents that remember. Um so uh [00:33] today we're going to talk a little bit [00:34] about the base case with agents today uh [00:37] which is that they are isolated and this [00:39] kind of limits their usefulness in a lot [00:40] of real world workflows. Uh we'll look [00:43] at how we can actually solve this [00:44] problem uh with our new agent memory [00:46] stores feature that we've launched. Uh [00:48] this will give agents access to a live [00:51] memory store that they can read and [00:52] write to over multiple sessions. And [00:55] then we'll look at how we can improve [00:56] these memory stores over time uh using a [00:59] new feature that we call dreaming. Uh [01:01] and then finally we'll get to see how [01:03] all of this ties together with both our [01:05] CLI and also our awesome console [01:07] interface. [01:12] Um so in the previous uh workshops I [01:14] think we've learned a little bit about [01:16] how cloud manage agents has these [01:17] concepts called an agent environment and [01:20] a session. Uh in this workshop we're [01:22] going to add two additional concepts [01:23] here. The first one here is a memory [01:26] store. Uh so a memory store is a [01:28] persistent file system-l like store that [01:31] gives uh that attaches as a resource to [01:34] sessions that you create and it gives [01:36] agents access to uh it gives agents the [01:38] ability to read and write information [01:40] across sessions. [01:42] A dream uh is what is a asynchronous job [01:46] that runs in the background. uh it looks [01:48] over an input memory store and a bunch [01:50] of your previous sessions that are [01:53] represented as transcripts and then we [01:55] run a harness over them to distill new [01:58] information that maybe the original [01:59] agents missed. We do things like [02:01] factchecking. Uh we also organize and [02:04] consolidate and duplicate information so [02:06] that your memory stores don't grow [02:07] unbounded over time. [02:12] So uh also in case anyone in this room [02:15] has not already uh downloaded the [02:16] workshop repository you'll need for this [02:19] uh here is the URL. I'll give a few [02:21] seconds for folks to just grab that. [02:24] Okay. Um so let's kind of like talk a [02:27] little bit about the problem today. So [02:28] when you when you create agents on cloud [02:30] manage agents today and sessions um most [02:33] of the time you're creating one session [02:34] at a time and these are isolated right? [02:36] the agent doesn't remember information [02:37] from the past and it doesn't transfer [02:39] information to future sessions. Um so I [02:41] want to kind of like take us through [02:43] this this base case today. Uh so if I [02:45] switch over to my computer [02:50] um I'm in our workshop repository here. [02:52] Uh hopefully everyone can see this uh in [02:54] the back but I've just run the bootstrap [02:56] script that we've uh included in the [02:58] repository. Uh and what this has done is [03:00] basically created uh some seed [03:02] information for us. We have an agent uh [03:04] we have an environment and we have a few [03:06] like previous sessions that talk about [03:08] things like the keynotes uh from day one [03:11] as well as you know uh a previous [03:13] workshop. And so uh what I'm going to do [03:16] from here is actually go ahead and start [03:20] walking through this guide with you [03:21] guys. Feel free to follow along your [03:23] computers. I'll be using the CLI and [03:26] then also showing kind of the console [03:28] interface uh for each of these steps. So [03:31] first we're going to create a session [03:32] that basically tells it some some new [03:34] information. Uh so I'm going to copy [03:36] this command here. [03:39] And as you can see we are creating a [03:42] session with the agents that we've [03:43] created before uh with the environment [03:45] ID. Uh and we've given a title here of [03:47] just like write test with no memory. [03:51] Great. Uh so now that we created the [03:53] session uh I'm going to quickly switch [03:54] over to my console here. Um and you'll [03:57] see that [03:59] just move this over. You'll see that the [04:01] session shows up in console. Uh it [04:02] doesn't it has a status of idle. Nothing [04:04] running yet. Uh so then we're going to [04:06] switch back and then I'm going to send [04:08] this session some new information. Uh so [04:12] once I copy this [04:18] and what I'm doing here is sending a [04:20] first user message here with information [04:22] about the CMA talk yesterday. uh naming [04:26] a few keywords like multi- aent [04:27] orchestration multi- aent orchestration [04:29] outcomes in memory and I've also just [04:31] given it this URL you know example that [04:33] I took notes and uploaded them here. [04:38] So, if I switch back over to my console, [04:41] uh, we should see this event pop up. [04:45] And what we sort of expect the the agent [04:48] or the model will do here is it'll just [04:49] respond and say like, great, thanks for [04:51] the information. Not sure what else you [04:53] want me to do here. Uh, so we'll give it [04:55] a sec and maybe folks to catch up. [05:01] See, we get a little refresh. [05:04] Ah, sorry. Yeah. So, it looks like the [05:06] model responded here. [05:07] Just collapse this. Yeah, it's basically [05:10] saying, okay, great. Thanks for telling [05:11] me this information. Uh, so then if we [05:13] go, [05:14] >> sorry, [05:16] >> uh, this is using, I believe, sonnet, [05:18] the agent that was created at the [05:21] beginning. [05:22] Yeah. Um, [05:25] cool. So, if I go back here, uh, to my [05:28] workshop here, we're going to create [05:29] basically a second section that's kind [05:31] of the retest. So, I'll go through these [05:33] steps a little bit faster. [05:38] Uh, and then again I'll send it a [05:41] message here. That's this time I'm going [05:42] to ask it for information about the [05:45] stuff that I just told it. [05:48] And if I go back to our console here and [05:50] check out the other session that should [05:52] have been created, [05:55] we would expect the agent to basically [05:58] um say something like, "Yeah, I don't [06:00] really have access to this information. [06:01] I can help you in these various ways." [06:04] Great. So that's effectively the base [06:06] case today. Uh so if I go back to the [06:08] slides [06:10] real quick, this is a quick recap of [06:12] what we did. We told it something, asked [06:14] another session about it later. No [06:16] information is transferred between the [06:17] sessions. [06:19] So how do we solve this problem? Right? [06:21] Um just like in humans, uh we've [06:23] introduced the sort of concept of [06:24] memory. And again, a memory store here [06:27] in the cloud manage agents platform is a [06:30] file system like store. uh under the [06:32] hood you can or you can create as many [06:35] memory stores as you like here. So you [06:36] don't have to necessarily restrict a [06:38] memory store to one organization. You [06:40] could create it per user, per workspace, [06:42] etc. It's up to you to kind of define [06:44] what the boundaries of the memory store [06:46] are. Uh and then under the hood, this [06:48] memory store gets attached as a file [06:50] system to the the session container and [06:52] the model has tools to read and write to [06:54] it. Uh the actual interesting thing here [06:56] is that we've actually you mounted it as [06:58] a file system because it's such a [06:59] powerful interface for the model. uh you [07:01] can use things like bash to like uh like [07:05] explore the file system. It can use GP [07:08] to kind of search for keywords. It can [07:10] also read files and do a bunch of like [07:11] really powerful things um that make it [07:14] much more useful for the agent. [07:17] Um so I'll switch back to my computer [07:20] here and we'll walk through kind of how [07:22] to create a memory store. [07:25] Um so first things first is actually [07:27] creating it. So I'm going to copy this [07:29] uh command here. Feel free again to [07:31] follow along on your computers. Um, and [07:34] the kind of parameters that you need [07:35] here are mainly just like a name. Uh, so [07:37] I'm calling mine CWC memory. You can [07:40] give it a quick description. Uh, and [07:41] then I'll show you also in console later [07:43] that there are two additional parameters [07:45] that you can set here. But let me just [07:46] follow through with a simple example. [07:49] Uh, once I create this memory store, you [07:51] can actually see it in console under [07:54] manage agents memory stores. [07:58] and you'll see that it's active. Uh you [08:00] can actually click into it and view uh [08:04] essentially a file system viewer of [08:05] what's currently in it. Of course, [08:07] there's nothing in it right now. [08:09] Additionally, uh we offer functionality [08:12] uh we offer the ability for you to like [08:13] add manually add memories. So you can [08:15] create like a file under a specific [08:17] path, add some content, etc. [08:20] If I go back just a brief second and [08:23] talk about uh creating a new memory [08:24] store, there's actually two sort of [08:26] additional parameters that you can uh [08:30] set on the memory store uh when we [08:32] actually mount it. So if I go back to [08:36] the repository here, um the next step [08:39] once you create a memory store is to [08:40] actually use it with your sessions. So [08:43] I'm going to again copy a few of these [08:45] commands. So the first one here is [08:47] basically just giving us the shape that [08:50] we need to pass the sessions API [08:52] request. I'll paste it in our terminal [08:55] here so we can see it better. [08:57] Um so again the memory store here you [08:59] just pass in a memory store ID. Uh and [09:01] you can also give it a prompt around uh [09:04] that will steer the agent to read and [09:06] write specific information. So you might [09:07] want it to focus on maybe like a [09:10] specific um link or a specific like area [09:13] of focus on. Let's say you're making [09:15] making like a investment agent, right? [09:17] And you wanted to focus on specific [09:18] things to remember future. Uh so you can [09:20] do that with a prompt parameter. [09:22] Additionally, there is an access field [09:25] uh that defaults to read or write. Um [09:28] you can change that to read only which [09:30] will make it so that the session and the [09:32] agent will only be able to read from [09:34] that memory store. It cannot update it. [09:36] So once I run this um you'll see I have [09:39] created a new session in console. [09:44] Uh, and this time it'll have a memory [09:46] store attached. [09:49] And then if I send an event here, this [09:50] time we'll just basically repeat the [09:52] test that we did just before. So let me [09:54] grab this [10:02] again. Same text as before. We're [10:03] telling it new information. Uh, and [10:05] hopefully we'll be able to observe a [10:07] different behavior this time. [10:15] Yeah. So clicking into uh if you click [10:16] into the session details, you'll see the [10:18] the information that I just gave it. [10:21] And now the model is like first looking [10:23] at memory to see okay, was there [10:24] anything that like I need to remember [10:26] for this conversation. [10:28] Uh and now it's going to actually like [10:30] of course there's nothing in our memory [10:31] store. So now what it's going to do is [10:33] actually save the content that I told it [10:35] to that memory store directly. and it [10:38] saved it under this like sessions.mmd [10:40] file and it's great telling me that uh [10:43] what it did. [10:48] Um, so then if I go back and do that [10:50] same test that we did before, this time [10:52] I'll just copy both. [10:57] Sorry, [11:02] again where this time we're going to [11:03] create a new session with the same [11:05] memory store that we were just using. [11:09] Uh, and we will send it an event here [11:13] asking it what are the things that it [11:15] found or it learned from the CMA talk. [11:21] Once again, going back to our console [11:23] UI, we can see the recall test running. [11:27] And as we would expect, the model is now [11:30] first looking at its memory store to see [11:32] if there's any information. And again, [11:35] it's now using GP to find uh any sort of [11:38] keywords here. It's looking for CMA. And [11:40] great, it found like a lot of [11:42] information that we just told it from [11:43] the from a previous session. And now [11:45] it's able to answer my question, right? [11:48] And this is like of course a very simple [11:49] example, but this illustrates kind of [11:50] the power of of memory. And this is like [11:52] something that was kind of difficult to [11:54] do before, right? [12:01] I'll give quick pause in case anyone's [12:04] struggling. [12:08] Um, okay, great. So what what else kind [12:10] of can can you do with a memory store [12:11] here? Well, we actually offer uh [12:13] additional sort of endpoints that allow [12:15] you to manually inspect the store [12:17] itself. Uh so I'll use the CLI here, but [12:20] for instance, you can list all of the [12:21] memory files that are in the memory [12:23] store. [12:25] We can like do that. Um there's also [12:29] each uh memory a memory stores memory [12:32] files in a memory store are also [12:33] versioned. Uh so anytime you make a [12:35] change to a file, etc., there's a new [12:37] version that's created and we offer a [12:39] set of endpoints for that. [12:43] And then I can also kind of take you [12:44] through the memory store UI. [12:52] So each memory store you create again is [12:54] here. And if we go back to our like kind [12:56] of file system viewer, you can actually [12:58] see the files that it created. Uh there [13:00] will be like a directory structure here. [13:02] If Claude is creating kind of [13:03] subdirectories to organize memory files, [13:06] you can actually edit these memory files [13:08] directly if you wanted to. So for [13:09] instance, if Claude wrote something u [13:11] that was incorrect or maybe you just [13:13] wanted to add more information, you can [13:15] do that. And again, as we saw before, [13:16] you can add additional memories to a [13:18] memory store. [13:26] Uh okay, so I'm going to go back to the [13:29] slides real quick. [13:32] And as we just talked about, uh this is [13:34] how you create a memory store and then [13:36] mount it on a session that you want to [13:38] use it on. Again, it's up to you to kind [13:39] of decide which of your sessions will [13:41] use memory, which ones will not. [13:47] Uh, we also saw how you can like list [13:49] memories and see what's currently in a [13:50] memory store. [13:54] And let's move on to talk a little bit [13:56] about dreaming. Now, um, so when you [13:59] have agents that are reading and writing [14:01] to this memory store over time, uh, [14:03] we've noticed that often times it can [14:04] start just kind of dumping information [14:06] to that memory store. So it'll start [14:08] writing maybe every every task you ask [14:10] it to do maybe records information right [14:12] and over time your memory store is going [14:13] to grow right and there's no real [14:15] process uh before that would allow you [14:17] to sort of organize that memory maybe [14:20] check to see if anything was stale and [14:22] and consolidate any duplicates right and [14:24] so this is where dreaming comes in uh [14:26] dreaming is a is a batch process that [14:29] runs again asynchronously you launch it [14:32] uh using uh our API or through console [14:34] and it'll run a a new dreaming harness [14:37] that we built that is a multi- aent [14:39] setup. It will look over each of the [14:42] input sessions that you've given it. So [14:44] you specify like an input memory store [14:46] that you want it to dream over along [14:47] with like a group of transcripts that [14:49] you think might help or enrich that [14:51] memory store. It'll look through each [14:53] one, do again factchecking, enriching [14:56] with additional details, maybe dates, [14:58] specific identifiers. Uh and then it [15:00] will also organize those memory files [15:02] and see if there's any duplicates, [15:03] anything that can help uh fix so that [15:06] when you and it'll produce an output [15:08] memory store such that in the future [15:10] when you attach that output to [15:11] additional sessions, it will hopefully [15:14] increase efficiency uh efficiency of [15:16] like information retrieval. It also [15:18] hopefully increase the intelligence of [15:19] the agent. [15:23] So let's take a look at how this works. [15:25] Again, switch back to my computer and [15:27] we'll walk through it together. [15:36] Uh so what you to to get get started [15:38] with dreaming basically you'll need to [15:40] actually create the dream job [15:43] and I'll go through a little bit of the [15:45] parameters here. So the model uh here [15:47] that we're choosing is cloud opus 47. [15:49] You can choose between opus 47 or sonnet [15:51] 46 depending on kind of the level of [15:53] quality you want as well as maybe like [15:55] token costs. Uh it takes in two inputs. [15:58] So you'll need the memory store that you [16:00] want it to dream over as well as a list [16:03] of session ids. Uh so this is up to you [16:05] to decide. You can you know maybe dream [16:07] over daily and dream over maybe like 10 [16:10] sessions at a time or 20, right? It [16:11] could go you know all the way up to like [16:12] a 100. We're also looking to scale it [16:14] further uh beyond that. Uh optionally [16:17] you can also provide the dream job some [16:19] additional instructions that you might [16:20] add. So we provide it with a default [16:22] prompt that does a bunch of things. If [16:25] you wanted dreaming to for instance [16:27] specifically fix a few things like maybe [16:29] you're working in a domain that requires [16:30] like very specific details right you [16:32] might ask a dream job to really focus on [16:35] hey make sure you back fill these [16:37] details so I remember for the future [16:38] right you can also get it to you can [16:40] also steer it to maybe like organize [16:42] files a bit more like I want this [16:43] specific structure in my memory store [16:45] please do that uh so let's actually go [16:48] and run this command here [16:50] great we'll get we'll get back a dream [16:52] ID and then once Again, I'll go back to [16:55] our console here. And so dreams are [16:57] under again manage agents dreams. [17:01] And once I create the dream job, it'll [17:02] start p it'll start with pending, but [17:04] it'll start running pretty shortly [17:05] after. And you can actually see the [17:08] status of the job both in console and [17:09] through the API. I'll show both. But in [17:11] console here, you'll see the input [17:13] memory store as well as a token count. [17:16] And generally like uh dream job can take [17:18] you know depending on the size or the [17:20] number of transcripts that you give it. [17:22] It could take anywhere from you know a [17:23] couple minutes to hours at a time. Uh [17:26] and that's really the benefit of doing [17:28] it asynchronously right like this is not [17:29] something that you want to do live while [17:31] your agents are working. And we'll just [17:33] give it a a minute here to to run. As [17:35] you can see as it's running as agents or [17:38] as the harness itself is running we're [17:39] updating the token count for you so you [17:40] can track its progress over time. The [17:43] other really cool thing here is that uh [17:44] dreaming is actually built directly on [17:46] top of cloud manage agents primitives. [17:48] Uh so you can actually see that we are [17:50] creating a session for the dream job [17:53] itself and you can actually click into [17:54] it see exactly what the dream is doing. [17:56] Uh this offers a really nice amount of [17:58] like observability and so you can like [17:59] diagnose issues potentially. Um and so [18:02] you can see the prompt that we give it. [18:04] Uh there's a lot of details here that [18:05] I'll kind of that you can explore on [18:08] your own time, but the under the hood, [18:10] the the dreaming harness itself is [18:12] launching sub agents to look over all of [18:15] the transits that you've given it. And [18:18] each sub agent essentially has a system [18:20] prompt that tells it what to do. Look [18:22] over and the orchestrator is responsible [18:24] for just like making sure all the agents [18:26] are running and kicking them off as they [18:27] go. [18:29] Uh so we'll give it another minute for [18:31] to let this run. [18:56] Uh, another thing to call out here is [18:58] that while uh we don't actually touch [19:01] the input memory store at all that you [19:03] create. So this is a non-destructive [19:04] process. What we actually do is we will [19:06] clone your input memory store uh into [19:08] what's called an output memory store. [19:10] And the dream job will be writing [19:11] basically to a new memory store such [19:13] that any edits that are made are [19:15] non-destructive. And then we'll see down [19:17] the line how you can utilize this output [19:19] memory store in your future sessions. [19:24] Generally this takes about a minute or [19:26] so depending on uh how fast the job [19:28] runs. [19:45] Uh, one other thing is that you'll see [19:47] that I'm sort of checking on the job [19:48] periodically uh as it's running in [19:50] console. Uh, when you do this [19:51] programmatically, we offer an API that [19:53] allows you to just essentially query uh [19:56] for the dream job and it'll have a [19:58] status uh so you can pull for the [19:59] status. Great. So, looks like it just [20:02] completed. Um and the cool thing here is [20:04] that in console we actually show you a [20:06] diff of what it did. Uh so you can see [20:09] dreaming here it created an index file. [20:11] Uh this index file has sort of these [20:14] slugs that reference the various like [20:17] memory files that uh a future agent [20:19] might need. And the main goal of this is [20:21] really just that future agents uh it's a [20:24] lot more efficient to kind of look at an [20:25] index file and quickly grock like what [20:27] it needs to go look for instead of maybe [20:28] doing a a wider GP. [20:31] Additionally, it's actually adding like [20:33] additional information that was not [20:34] present in the in the first couple [20:36] sessions that I created. So, it's [20:38] creating this event logistics file. It [20:39] gives the whole schedule of code with [20:41] claude. Um, a bunch of names as well. [20:46] Uh, and again schedule for day two. [20:49] Uh, and you also see that it actually [20:51] kind of reformatted the memory file that [20:53] I created in a previous session. So this [20:54] time it is adding again a slug, a [20:57] description of the event, some [20:59] additional metadata and again adding [21:01] more details. Um and generally we find [21:03] that like uh more information actually [21:06] really does help future sessions. And if [21:07] you think about intuitively um while an [21:10] agent's working on a task currently, [21:12] it's kind of hard to predict down the [21:14] line what it might need, right? That's [21:15] just generally a harder prediction [21:16] problem. So it's actually good to kind [21:18] of write additional details down that a [21:20] future agent might remember. And [21:21] Dreaming can always like go back and [21:23] like remove stuff that is no longer [21:24] needed. [21:26] Uh and if I go to the output memory [21:28] store here, I'll just click on it. [21:32] Um again, you can see all the files that [21:36] I created. It's another good way to sort [21:37] of see what's going uh what dreaming [21:40] did. Uh and if you wanted like a human [21:42] in the loop kind of review process, this [21:44] is kind of where uh this is super [21:46] helpful. uh human can kind of go in and [21:47] see if the dreaming harness made any [21:50] mistakes. [21:54] Okay, great. Um just going to switch [21:56] back to the slides real quick because I [21:57] want to show a diagram. [22:01] Um [22:03] yeah, so again under the hood, this is [22:05] sort of what how dreaming works, right? [22:07] This is a multi- aent harness. uh we [22:10] have an orchestrator that is mainly [22:12] responsible for spinning up sub aents [22:13] and again we spawn one sub aent per [22:16] input session that you give it. Um and [22:19] kind of the the reasons behind this are [22:20] we actually kind of want dreaming to be [22:22] exhaustive by design. Uh if you give it [22:25] 100 trans like claude is looking over [22:28] all the information to make sure it's [22:29] not missing anything, right? Um [22:32] great. So now me let me switch back to [22:33] my computer again. [22:36] Uh and this time I'm going to actually [22:37] like walk through how we might use this [22:39] in a future session. [22:44] So uh once the dream uh is done, you can [22:47] actually go and grab the output memory [22:48] store using this. So again, we're we're [22:50] just retrieving the dream resource uh [22:52] with the dream ID that we created and [22:54] then uh querying or just grabbing the [22:57] the JSON memory store ID. [23:00] Great. So this is the memory store and [23:03] again you can look through what memories [23:06] it created. [23:13] Uh and now we'll do this sort of test [23:15] again that we did before with the two [23:17] sessions. So I will create a new session [23:19] here [23:21] and you'll notice that this is now using [23:23] the output memory store from dreaming. [23:30] Once again we'll send it an event here. [23:39] We're just asking it what sessions I [23:41] attended, what resources do I have links [23:43] for, and what follow-ups I flag. [23:51] Once again, going back to the console, [23:53] it's really a great way to, you know, [23:55] visualize what's going on here. [24:00] And you'll see that this time when it's [24:02] reading from memory, you'll see all the [24:04] stuff that Dreaming did, right? So, the [24:06] index, the event logistics, [24:11] and it's now reading kind of it's [24:13] starting to read the index first. [24:17] Okay, great. Now it knows I'm going to [24:19] go straight to the sessions file. [24:24] It's being a little exhaustive here. [24:26] Just checking for, you know, event [24:27] logistics. [24:42] Okay, great. Let's see what it came up [24:43] with. So, as you can see, I I'd have to [24:47] go back and show you the the previous [24:49] session, but this time I think there's a [24:50] lot actually a lot more information [24:51] here. So, it gave me a recap of all the [24:53] sessions I I attended. Uh, it's giving [24:55] me timestamps now about like all the [24:57] sessions that were that were planned for [24:58] day two as well as like the resource [25:00] links, right? I think this kind of [25:02] showcases again like how dreaming can [25:04] really enrich information that is [25:06] transferred between sessions. [25:11] And then optionally um at the end you [25:13] know if you're really happy with the [25:15] output memory store here you can go [25:16] ahead and actually retire the old memory [25:18] store. Um so this won't affect like your [25:20] previous sessions. It just means that [25:22] you no long it will it will keep you the [25:23] number of memory stores in your [25:24] organization down to a reasonable level. [25:31] Um great. So I'm going to kind of switch [25:33] back to the slides here [25:39] and talk a little bit how you can kind [25:40] of view sessions, memory stores, and [25:43] dreaming as three composable layers [25:45] here, right? If you think of a session [25:47] as a as an isolated instance of an agent [25:49] running, uh it's one usually typically [25:51] one conversation thread, typically [25:53] ephemeral, right? A memory store [25:56] augments that. So now you can connect [25:58] information between your sessions across [26:00] multiple sessions, right? And then [26:02] finally with dreaming uh you're now [26:04] organizing, enriching and improving your [26:08] memory stores over time so that as you [26:10] scale up the number of sessions you have [26:12] and as you scale up the information that [26:13] it's being processed through those [26:15] sessions, your memory stays at a [26:18] reasonable level. It's manageable. It [26:19] doesn't blow up. Also, it checks for [26:22] things like staleness to make sure all [26:23] the information is most is up to date, [26:25] right? I wanted to highlight some of the [26:27] maybe good questions that I got from the [26:29] audience here. So one of them was around [26:31] uh generally like sort of what is the [26:33] like token usage of this feature like? [26:35] Um so as I said before like I think by [26:37] design we actually do want it to be [26:38] exhaustive. So we do expect it to use a [26:40] lot of tokens. Um the the nice thing [26:43] here is that because most of the [26:45] processing is agentic uh most of the [26:47] tokens are actually cached. So we're [26:49] expecting like about a 95% cash rate [26:52] cash hit rate on um like most dream [26:55] sessions right and additionally we are [26:58] exploring like other ways of offering [27:00] this at lower costs to you. So uh [27:03] example of this would be like similar to [27:05] our batch API we could offer things at a [27:07] 50% discount by scheduling it at [27:09] different times. Uh other additional [27:11] like token usage controls include like [27:13] switching the model, steering the prompt [27:15] a little bit more, also providing more [27:17] like just general budgeting of tokens. [27:22] >> Um [27:26] great. Um so I think we're just about [27:29] winding down. So I'm going to quickly [27:32] kind of go over again for the folks [27:34] remaining in the room uh like what we [27:36] went over today. Uh so again we talked [27:39] about the problem that most agents face [27:41] today which is like how do you remember [27:43] information across sessions? I think [27:44] this is a pretty like wellestablished [27:47] kind of problem now. And we talked about [27:49] how memory is kind of the first step to [27:50] addressing this right you give agents [27:52] access to something where they can dump [27:54] the information read from it etc. Uh but [27:57] we also saw how this creates a problem [27:59] where memory stores are can grow [28:01] unbounded over time. They can grow [28:02] disorganized information can grow stale [28:05] etc. And then we saw how dreaming can be [28:07] used as a way to mitigate this problem. [28:09] Right? So we we take another set of [28:12] agents that their entire job is to [28:14] improve that memory store for future [28:16] use. [28:21] Um and with that I'd like to thank [28:23] everyone for for coming to today's [28:24] workshop. Um hopefully it was helpful [28:27] and hopefully by the end you'll learn [28:29] how to uh you'll know how to like [28:31] integrate maybe memory into your own use [28:32] cases.  ## Highlights - Token cost claims (95% cache hit rate) presented without sufficient context about usage patterns. Fix: Provide more specific conditions under which these rates apply. - Note: 🔍 Fallacy: Hasty Generalization #fallacy/hasty-generalization