How Shortwave Wants to Reinvent Email With AI  ## Metadata - Author: [[The New Stack Podcast]] - Full Title: How Shortwave Wants to Reinvent Email With AI - Category: #podcasts - URL: https://share.snipd.com/episode/c10d1989-320d-47c4-98aa-cb65a9a90e00 ## Document Notes ## Highlights - **Google Inbox Innovation Story** - Google Inbox was a Gmail redesign that tested new ideas but was ultimately killed despite its popularity. - This experience exemplifies how large companies struggle with innovation in mature products. Transcript: Andrew Lee I can point to is what happened with Google Inbox, if you remember Google Inbox. Yeah, describe that quickly for those who might not remember what Google Inbox was. JOHN MUELLER- Yeah, so basically, Google figured out in the mid-2010s that the existing Gmail experience they had was kind of outdated. It hadn't really thought hard about what to do when you had a lot of email coming in, how to do things well on mobile. And they built this totally new product called Google Inbox that incorporated just a ton of very interesting new product ideas. And people really liked it. It had a big following, especially among journalists, actually. That's one of the reasons we got so much press initially. A lot of journalists used it. And I thought it was a great product. A lot of people on our early team thought it was a great product. And the Inbox folks went back to the main gmail leadership like gmail put their many of their best people on this new team and they built this they put it out there it was working well and They went back to their leadership and said okay we've done it we've made this product it's awesome like let's push this out to everybody and the gmail team basically said ah too risky You know like we got all these users over here and they're all used to this other thing and do we really want to force them to migrate? And do we really want to have two products? And they ultimately made the decision, despite the fact this had tens of millions of users on it, they killed it off without really a replacement for those users. A lot of the people that were working on that product left and went to do other things. So for example, there were several people on the Firebase team that were like refugees from the team, which is before they actually shut it down, but who kind of saw the writing of the Wall there. And then the folks who were left in the Gmail team had a very clear signal of like, don't try to innovate too much, right? If you innovate too much, you might, you know, your project might get killed. And so, you know, if you look at the last, you know, what, six years since that product got killed, Gmail hasn't done a lot. They've kind of shoved the Gemini sidebar in there, but like, there's no, there's been no like radical efforts to like rethink the product. Frédéric Lardinois Yeah, yeah, yeah. If you innovate too much, you have to go and work on Firebase. ([Time 0:04:13](https://share.snipd.com/snip/4c3b37e1-2c6b-4751-af58-717fc4dcdc34)) - **Beyond Keyword Search with Embeddings** - Shortwave leverages embeddings combined with multiple databases for powerful semantic email search. - This system surpasses Gmail's keyword search by indexing conceptual information. Transcript: Andrew Lee Text of those emails into the large language model. And then it does whatever it needs to do from there. Frédéric Lardinois The tool calling, obviously, you know, you stack agents. It's all we've been thinking about over the last few months, I think, is how are these models using tools? Right. How well is that working for you? Because often I come across like really great demos. But in reality, things still fail sometimes. ([Time 0:17:03](https://share.snipd.com/snip/1fef12ba-c167-43fa-8a06-95df6ab81b6b)) - **Anthropic Models Excel at Tool Calling** - Anthropic's Claude Sonnet 3.5 enables AI agents to call multiple tools sequentially, boosting reasoning. - This advancement marks a radical improvement over earlier AI models limited to single tool calls. Transcript: Andrew Lee The big change actually happened in December of last year when we switched to Cloud Sonnet 3.5. And we discovered that it seems like Anthropic has been doing some very focused reinforcement learning specifically on agentic tool calling use cases. And last summer, this time, the OpenAI models, they could effectively call one tool, but you couldn't really get them to call a bunch of tools or a sequence of tools. And the Anthropic models starting in December were able to call a tool, up information, take that information to figure out the next tool call, call another tool and do that, you know, Sort of an endless and an endless sequence. And the result of that was a radically smarter agent. So I think, you know, if you've been a long time user, you probably noticed that like the AI, you know, kind of worked mostly up until December and then like January, like 12th or whenever We launched a new agent suddenly got really smart. ([Time 0:18:33](https://share.snipd.com/snip/b3e2f4c5-9608-430a-9469-143bc8aafe61)) - **Leverage Dynamic Prompting** - Use AI models that tolerate dynamic prompt and tool changes to scale integrations effectively. - Avoid rewriting system prompts for every new tool to prevent AI failures. Transcript: Andrew Lee Ability for you to dynamically add stuff into the prompt. So a year ago, anytime you changed models, anytime you added a tool into the prompt, you basically have to totally rewrite your system prompt to make it do anything sane. Because like any little tweak in that system prompt, the thing would fall over. And as the models have gotten better, they've gotten more robust to adding stuff in. So MCP, I think the technology that really unlocked MCP was just the ability for you dynamically adding a tool into a system without having the system fall over. The standard itself is very But the blocker of that was anytime you just dynamically add a tool, you'd have to re-art your system prompt, and that kind of broke it. But once the models got good enough that you could dynamically add them, and it would do something sane, you had this explosion with MCP. I think the limit right now is you can't add very many of these before ([Time 0:21:32](https://share.snipd.com/snip/033e1090-4c6a-4eb9-a94f-c781dbb511a7)) - **Balancing Tool Quantity vs. Reasoning** - AI agents struggle with reasoning when overloaded with too many tools simultaneously. - Smaller tool sets improve reliability but limit creative problem-solving potential. Transcript: Andrew Lee To break down. I start to see a lot of errors and stuff. So I think, yeah, one of the things is like, can these models scale not to just having, you know, a small number of integrations, but many integrations, and you got to keep in mind, like, Each integration might have many tools. So like the Asana integration is like dozens and dozens and dozens of tools. So yeah, I think we really need models that can handle like 1000 tools. And without without breaking a sweat, I think that's what that's what we need. Is it? Frédéric Lardinois I mean, one thing I keep hearing is that kind of one of the best practices these days is to give every agent one tool and have them kind of figure it out that way. Because once you add too many, maybe not one tool, but maybe up to five, six tools or something like that. Because after that, things start to break down and they don't know what tools to call anymore. Has that also been your experience? Andrew Lee I think there's pros and cons. I think you can get more reliable behavior if you keep the tool set small. But the downside of that is you can't have the LLM like reason across a broader set of tools. Like there may be creative solutions that it can come up with to solve your problem if it understands the full range of what's possible. And so I've had this discussion with with many people i think in our for the product that we're building in ([Time 0:22:40](https://share.snipd.com/snip/d5540ee4-cefa-477f-b453-d629c15c1c45)) - **Unified Agent with Multiple Tools** - Shortwave runs a single iterative conversational agent that selectively uses multiple tools as needed. - This unified agent approach supports complex workflows without a hierarchy of sub-agents. Transcript: Andrew Lee Yeah, for the main kind of conversational AI, I want to know, we have a lot of other we have autocliped those AI features, but for the main conversational AI, it is a single iterative agent With a whole bunch of tools that, and it can sort of pull in the tools it needs at different times. Yep. Got it. Got it. Frédéric Lardinois What's, you know, we've ([Time 0:24:20](https://share.snipd.com/snip/b2a3ea12-b617-472b-a922-6fd872feee65)) - **Future of Asynchronous AI Agents** - Shortwave plans to build asynchronous AI agents that automate complex email workflows proactively. - These agents will handle events like email receipt and recurring tasks without user presence. Transcript: Frédéric Lardinois You know, without giving off? Well, you can give away your roadmap, but without giving away your complete roadmap, but like high level of what, what else is on? Andrew Lee Yeah. I mean, there, I think there's a, there's a, there's a lot of obvious stuff on our plate here. If you've used the product, lots of little like, you know, rough edges and things missing on mobile and stuff like that. But I think the big focus area for us is really driven around asynchronous execution. So I think we do a pretty good job today if you're sitting in front of your computer and you're typing and asking it to do things. But a lot of the use cases people have for AI is like, I wanted to do it when I'm not in front of my computer. Right? Like the email comes in, I want to go do a bunch of processing ahead of time for me or automatically reply or whatever it is. And we have some very simple features here now. So we have this thing called AI filters that allows you to run a prompt. And based on the prompt, you can like add a label or you can archive it or you can make a bunch of like relatively simple actions. But you can't like, you know, keep a notion database up to date with every, you know, VC associate who's emailed you or, you know, take all the uploads, uploaded files and put them in Google Drive. And we want to take that to the next level and say, hey, what if your AI filters weren't just AI filters, but they were like AI agent, like asynchronous AI agents that every time you get An email, you can run the full agent and do everything the agent is capable of. And not just on email receipt, I think there's other events that we would want to handle, you know, recurring tasks or, you know, every time you add a label or things like that. So I think look out for powerful, asynchronous, you know, automatic execution of like complex agent workflows. Frédéric Lardinois Yeah, that'll be interesting to see. ([Time 0:26:25](https://share.snipd.com/snip/8c01412a-a72d-4a08-99e3-772595d7e373)) - **AI Boosts Shortwave's Dev and Biz** - Shortwave heavily uses AI tools like CloudCode and Cursor for coding and business automation. - They've also AI-assisted content creation and candidate screening, boosting productivity without replacing humans yet. Transcript: Frédéric Lardinois Is happening with the help of AI right now in the company? Andrew Lee Sons. So we internally, we've tried a lot of different products and we're still using a bunch of different products. But I think the two that we use the heaviest by far are CloudCode and Cursor. I think we've been especially impressed by CloudCode with Opus. We've had some pretty impressive cases of it, like one-shotting complex bug fixes or new features, where you give it the right prompt and you give it five minutes to think but then it Just comes back with with a working PR. I, you know, I don't think it's, it's replaced, any humans on the team yet. But I think, as a fraction of the code that's being committed, it's now possibly, you know, possibly the majority of the code written at our company is now AI. And I think that number is just going to increase and increase and increase and everybody's using it every day. So yeah, we're heavily using AI for coding. For sure. Frédéric Lardinois Hasn't replaced anyone yet, but maybe you're making fewer hires because of it at this point, maybe. Potentially. Andrew Lee We're hopefully just moving faster, right? We also, it's not just code either. There's other processes that get automated. So for example, the blog post and the newsletter around our MCP announcement were AI generated. Basically, I wrote a prompt and the prompt included an outline of what I wanted to say. And then I copy pasted a whole bunch of other docs with context. So I copy pasted like some PRD docs. I copy pasted a bunch of previous blog posts that we had written that I thought were well-written. And I copy pasted like some general information about MCP, but I had this big long prompt. It was basically like, write a blog post with this outline. And then here's a bunch of information that you probably want to know. And then here's some examples of well-written posts. And it generated a blog post that was like 95% of the way there. And I, you know, I tweaked and modified it from there, but I think that saved me several hours of work. And then the newsletter was basically like, okay, now take this blog post and like, you know, give me a newsletter version of it. And here an example of the past of like the newsletter version of the blog post that I did. And so yeah, things like our content is all, you know, AI assisted at this point. Even like our candidate screening process, there is some AI automation. We auto categorize emails that come in from candidates. I parse out the resumes and stuff looking for certain types of information because we can parse PDFs in our AI. So every business process that we have are looking for ways to streamline and automate processes everywhere we can. ([Time 0:28:35](https://share.snipd.com/snip/fedbdef1-8e9e-4c77-b63c-9e22defa1c43)) - **AI-Created Dynamic User Interfaces** - Dynamically generated UIs built by AI can tailor interfaces to specific user tasks. - This approach could redefine productivity apps by adapting layout to different workflows automatically. Transcript: Frédéric Lardinois Totally. What else gets you excited right now about AI? What other areas are you super interested in? Andrew Lee Yeah, that's a, that's a good question. I, I'm very excited about dynamically generated UI. So one of the, like, if you think about it, like, the models are at the point where they can generate code for UIs pretty well. I can go into cloud code and be like, hey, give me a widget that does blah, blah, blah. And it does a pretty good job there. But if you think about it, it doesn't just have to do that when coding. You can just live in the app. You can be like, me a widget and you can take the HTML and you can just shove that in a modal and boom, you've got a UI. And you may not want to directly eval HTML, but we've had some experiments where we just gave it a list of sort of like pre-configured React components. And we're like, hey, build a UI that solves this problem using these elements. And it's done a pretty good job. And I see a future where you can sort of get purpose-built UI to solve any problem that you have. For example, you're reviewing candidates, right? And maybe the ideal email client when you're reviewing candidates is different than the ideal email client when you're handling customer support. And so I think the future ideally would be like, you go into the AI and it'd be like, lay out my client differently. So that's really optimized for me to like read resumes. Right. And then it just like reconfigures and then suddenly it's like a resume reviewing tool. I think that's totally possible. I think within the next couple of years, it's gonna be a lot of, a lot of apps that work that way. So yeah, dynamic UI, I think is a big one. I'm, I ([Time 0:31:50](https://share.snipd.com/snip/3db118f9-1775-4336-99b8-fadfad6aab62)) - **Faster AI Expands Use Cases** - Speed and cost reductions in AI models expand practical use cases like proactive email responses. - Faster models make AI assistance valuable even for quick interactions like meeting delay notices. Transcript: Andrew Lee To large contact sizes, continuing improvements to speed, because I think there's a lot of things today that you can do, but maybe are impractical. So like RAI, for example, like one of the biggest complaints we have is it's just slow, right? It can go, it's genius. It can go off and like write an email for me, but like, you know, if I'm just running late to a meeting, it's much faster if you just type myself. But if these things can get much faster and much cheaper, I think there's a lot of use cases where it'll suddenly become very valuable. Because for example, like we can do things proactively. What else am I excited about? ([Time 0:33:35](https://share.snipd.com/snip/d55ebe5d-5474-4949-83ac-9c0ddbdd5bf8))