# A series of miracles (with Jason Cohen)  ## Episode metadata - Episode title: A series of miracles (with Jason Cohen) - Show: The Panel - Owner / Host: Justin Jackson, Brian Casel, Jordan Gal - Guests: [Jason Cohen](https://share.snipd.com/person/9be5a751-caff-43ed-98ad-cb0fa62b9281) - Episode publish date: 2026-05-08 - Episode AI description: Jason Cohen, founder of WP Engine and author of Hidden Multipliers, shares his approach to writing and startup thinking. He debates whether distribution is really a moat, explores what makes companies like Basecamp and Lemon Squeezy durable, and explains his funnel idea that the quality of who enters the top matters most. Conversational, strategic, and full of practical frameworks. - Mentioned books: [Hidden Multipliers](https://share.snipd.com/book/ff856dba-e78b-4fdf-a65b-633c3ba1033b) by [Jason Cohen](https://share.snipd.com/person/9be5a751-caff-43ed-98ad-cb0fa62b9281) - Duration: 01:19:08 - Episode URL: [Open in Snipd](https://share.snipd.com/episode/6d080592-bf48-42fe-abb7-ab4f42856c81) - Show URL: [Open in Snipd](https://share.snipd.com/show/27d8f9cc-847c-4007-aa5c-0bcd13e5e0a1) - Export date: 2026-07-08T01:11:43 ## Snips ### [Why Jason Stepped Away From WP Engine](https://share.snipd.com/snip/41bf90c3-385a-43b7-916a-d5f895475b92) 🎧 02:16 - 03:36 (01:19) <iframe src="https://share.snipd.com/embed/obsidian-player/snip/c2193377-81e3-4d1b-bd8c-3872891e93ca" width="100%" height="100" style="border: none; border-radius: 12px;" sandbox="allow-scripts allow-same-origin allow-forms allow-popups allow-clipboard-write" ></iframe> - Jason left WP Engine after 15 years to rest, spend time with family, and write rather than join another company. - He visited colleges with his daughter, traveled, and explicitly avoided starting another company for at least two years while writing a book. #### 💬 Quote > I left WP Engine about two years ago. I've been writing a book, which is now on pre-order. I'm not starting another company. Here it's two years later. I have not started another company. > — Jason Cohen Jason Cohen on leaving WP Engine and his post-exit priorities #### 📚 Transcript **Jason Cohen:** I said SmartBear. That's how long it was. I left WP Engine about two years ago. I've been writing a book, which is now on pre-order. I'm not here to push the book. That's not why we're doing this. So I'll let Justin push it at some point. Hey, I will push the book. Just pushing it. All you want is it's called Hidden Multipliers. Oh, look at that. You have pulled up and everything. Okay, so if go to hiddenmultipliers.com, there it is. There, I did the pitch. So, yes, so it's funny. We were just talking before the show. Um, I think when founders or executives, uh, leave, they always have an excuse and the typical excuses, I'm leaving to spend time with my family. Cause we can argue with that. It sounds great. Everyone goes, Oh, that's so noble. And then they leave. And then three weeks later you read so-and has been, you know, been appointed the senior vice president of sales of North America for, and you're like, wait a minute, that was not a lot of time with your family dude and so uh so i was really sensitive to that so when we were talking about it internally i was like look it's just there's nothing wrong nothing bad it's just been 15 years it's just time like i just want to rest a little i'm older my daughter's about to go to college i want to go visit colleges with her a little bit but like it's just mostly it's been a very long time. It's just simply maybe time for the next thing, whatever that is, which is not another company. I said at the time, I'm not starting another company. Here it's two years later. I have not started another company. --- ### [Writing Without Deadlines Preserves Quality](https://share.snipd.com/snip/d1650975-821e-4aa7-8286-4c85d76814e4) 🎧 06:08 - 11:55 (05:46) <iframe src="https://share.snipd.com/embed/obsidian-player/snip/91438400-e2ca-4971-a774-b67fe4cced98" width="100%" height="100" style="border: none; border-radius: 12px;" sandbox="allow-scripts allow-same-origin allow-forms allow-popups allow-clipboard-write" ></iframe> - Jason treats writing as a freedom-driven craft: no deadlines, big content buffer, and variable cadence to preserve quality. - He keeps hundreds of drafts, buffers social posts months ahead, and prioritizes craft over algorithmic consistency. #### 💬 Quote > I have a rule from the beginning that it's supposed to be fun. I have no deadlines ever for anything. My buffer is deep. I'm not joking. I'm not consistent at all. I post very randomly. > — Jason Cohen Jason Cohen on his writing process and editorial freedom #### 📚 Transcript **Jason Cohen:** Well, you know, since I've been writing for almost 20 years, as you know, and, and, and fortunately, never for money, never trying to make my, you know, and so I've been able to do whatever I feel is best for me. So I've had a rule from the beginning that it's supposed to be fun. And so for example, I have no deadlines ever for anything. And so sometimes I've had years where I publish every week like clockwork and years where I publish once or twice only the whole year. For whatever reason, I don't have the energy. I have the time. I don't want to. It's whatever. So as a result, I've been able to keep the quality high because I'm allowing things, like you said, consistency. When you said that, I thought, I'm not consistent at all. I post very randomly. But as a result, things like the craft or the style or the ideas, the consistency of whether the idea is good enough, I've been able to keep up because I let things like deadlines not drive things. And of course, for algorithms, that's bad. And so a lot of people say, you should have 10 times the number of followers on Twitter on, you know, blog, I don't know. And it's like, yeah, maybe, but I'm just not going to do that algorithm stuff. Because again, I just have a different set of constraints that I'm running under. But it allows me to have the quality, at least the best of my own ability, whatever that is. And so for anyone who likes that, then good. **Brian Casel:** I do love the idea of like lack of deadlines. I'm always really curious about what the life cycle of every... So if we see something come out on Twitter from you or an article drop from you, what was the lead up to that? How many days of development? How many drafts? How, how, how do you get to the point of like, this is ready to hit publish? Yeah. **Jason Cohen:** Nothing's real time. Like even the Twitter stuff, that's all buffered up. And so a lot of times surprised myself, like, Oh, I wrote that like nine months ago. I mean, my, my buffer is deep. I'm not joking. **Brian Casel:** And with one of the nice things, it does that way from the surface. I assume that there must be some sort of like cue, **Jason Cohen:** right? It is a massive cue. And what's nice about being so ahead of the cue like that is I can say things which in the moment, if I didn't have such a deep cue, there's things I wouldn't have said because it was so close to the moment that the person or event or thing I'm talking about, they would know or other people in their team would know. And that's the last thing I ever want to do to anybody, right? Like it's nice to say here's something I observed or a lesson. Very bad if I'm calling out someone even by accident, right? Or at least I say, right, that's just my attitude. But if there's a nine-month queue and I write that, there's no chance you could – anyone. Yeah. They could maybe think, oh, was he talking about that thing from a year ago? But that's so vague that at some point, it's just my experience. I have to relate it. Or I don't have to, but it makes sense. So it's actually great because I can say more things than I might have otherwise said without worrying about how I may be affecting other people in that particular sense. The posts, I have literally hundreds of posts and drafts, some of which are like an idea in one sentence and some are long things. I just don't – I'm just not feeling it or I'm feeling the editing or maybe the idea is not good enough yet or something. And of course, most of them will never work. And so it is, again, kind of sporadic. Sometimes I'll just have an idea and bam, I write the whole first draft when one's sitting and then all it needs is editing. And then other times – and so even so, that will be hours because i go over it and i edit it a lot i want to find every problem i want the style again i'm not just trying to put out something so like is this written well enough is this is it structured well enough that sort of thing are the examples good and of course what's what's too long and all that um so it still will take a lot of time in editing but sometimes the original is fast and sometimes the whole thing gets rewritten a bunch there's one people like a lot that's called the elephant curve it's called the elephant in the room but it's about the elephant curve which is just a name i came up with um for this particular idea that one is very long and has a million graphs and data points because i have this thesis about how growth works or how exponential growth doesn't work, you might say. And I have this thesis about, like, here's how growth works and why. And then I back it up with, like, the zillion data points from all kinds of companies, bootstrapped, venture-funded, the biggest ones ever, old ones, new ones, and, of course, my own data. That one took 10 years because I had this idea that long ago i published this i think in 2022 but i i had that idea because i i remember publishing it like i've had this idea for so long and the reason there's so much data is that i had this idea and i never got around to like making it into some sort of thesis but every time i saw another example i'm like god damn it there's it there it is again and i would snag it and throw it in the file you know so the file ended up being like i have 10 not 10 i have maybe three times as much stuff as what's in that article even right of like accumulated and then finally for no reason i can i can explain to you i was just like i had the bug and i had and i just it's like it's time to write this damn thing after all it's ready to get it out and so that's part of why it's so deep so it can be like that too i i think so that's not it's kind of a non-answer of like, what's my process? I guess the answer **Brian Casel:** is, yeah. **Jason Cohen:** Yeah. The answer is like different things just happen differently. I allow that. And I don't worry. I don't sweat that. And a real professional writer cannot act like me, you know, right? Like if you need to publish every day, if you're writing for a newsletter or even if it's for money and so, and you're getting paid $20,000 for an amazing article in The Atlantic, you still need to go like do that thing for a few months. And so like since I don't need to, the freedom of that – but what that means is it's – I can do my best work. I don't let it intrude. I cannot let it intrude on the rest of life. But that means I can be the best writer I can be, the best thinker I can be, put out the work that I'm most proud of. That doesn't mean it's all great necessarily. It's just that my best that I can do is what, that's why I keep repeating that. And that's something everyone can adopt and do. --- ### [Use AI To Spew Then Edit Every Word](https://share.snipd.com/snip/7661ba52-6cb7-4372-a2a2-5fb504331f0d) 🎧 16:18 - 16:53 (00:35) <iframe src="https://share.snipd.com/embed/obsidian-player/snip/485224f2-f21d-4b19-926c-72d8829dd569" width="100%" height="100" style="border: none; border-radius: 12px;" sandbox="allow-scripts allow-same-origin allow-forms allow-popups allow-clipboard-write" ></iframe> - Use AI to turn verbal spew or half-baked drafts into structured outlines, then edit every word yourself. - Leverage AI as an editor and outline generator to get past writer's block, but do heavy human editing to retain voice. #### 💬 Quote > Often just talking aloud, you end up with something which you can now reform. And if you don't care so much about your own writing craft, of course, you can use AI and say, all right, I have this half written thing. > — Jason Cohen Jason Cohen on using AI in the writing process #### 📚 Transcript **Jason Cohen:** And if you don't care so much about your own writing craft, of course, you can use AI and say, all right, I have this half written thing. I have this spew of ideas. Like let's, let's first decide what kind of article structure makes sense. Like, is it, is it like surprising thing, background context, here's the solution. And here's like a couple of tips, which is a lot of mine to be fair. Like, is that the right? And so you could, you can work with the AI a little bit, like what's the right outline structure. And then like, okay, we'll take all the shit I said and, like, sort of do that. And it won't be very good. But, like, maybe, as we all know, editing is easier than writing from scratch. --- ### [Moat Means Being Unstoppable Not Just Hard To Copy](https://share.snipd.com/snip/6794f1d8-dccb-47aa-bc00-a775e6c4ab29) 🎧 23:50 - 25:19 (01:29) <iframe src="https://share.snipd.com/embed/obsidian-player/snip/1bd5ada0-546c-4f5e-b6fe-341e2c8f6f3c" width="100%" height="100" style="border: none; border-radius: 12px;" sandbox="allow-scripts allow-same-origin allow-forms allow-popups allow-clipboard-write" ></iframe> - 'Moat' means competitors cannot stop you from running your business, not merely 'hard to copy' features. - Jason frames moats like Warren Buffett: persistent defensive advantages such as network effects or entrenched distribution partners. #### 💬 Quote > When I hear moat I just think – Warren Buffett... competitors cannot stop us from running the business we're running. To me, that's what the moat is. > — Jason Cohen Jason Cohen on the definition of a moat #### 📚 Transcript **Jason Cohen:** moat in this world? I agree with Jordan that the phrase, you know, you read into it what you want and therefore it doesn't mean anything. **Justin Jackson:** And **Jason Cohen:** it's also not new. I mean, to the extent that distribution could be a moat, that isn't new information. **Justin Jackson:** And **Jason Cohen:** to the extent that it's not a mo it's not a the phrase is not accurate there is a very so so in that sense it embodies the very funny thing uh i forget where this comes from it's very old but there's like a beginning of a paper or something and it says there's much in this paper that is new and much that is true but the stuff that is true is not new and the stuff that is new is not true so this reminds me of like this is distribution as a moat is kind of an embodiment of that like this is a nonsense thing it's either not true or or we it's not saying anything new so therefore this is so i think that's what jordan was saying too and and i agree **Justin Jackson:** i **Jason Cohen:** do think um so when i hear moat i just think – Warren Buffett, of course, coined this phrase. And so I think of the Warren Buffett version of it, which is – maybe you could have other ones. That's fine. That's just what I think of, which is that competitors cannot stop us from running the business we're running. To me, that's what the moat is. Now, you normally think of it as, you often think, and I've said it as, a permanent competitive advantage. **Justin Jackson:** I **Jason Cohen:** think that's a fine way to say it. But if we're going back to the original thing, it's sort of that it's not even an advantage. --- ### [Reputation And Distribution Can Become Real Moats](https://share.snipd.com/snip/e0db2e52-eb22-47ad-b81e-c863e5007813) 🎧 25:51 - 29:37 (03:45) <iframe src="https://share.snipd.com/embed/obsidian-player/snip/ee960bd5-d780-4591-8ba9-c6cc6b8fef0d" width="100%" height="100" style="border: none; border-radius: 12px;" sandbox="allow-scripts allow-same-origin allow-forms allow-popups allow-clipboard-write" ></iframe> - Reputation and positioning can create an unassailable place in market segments where others cannot dislodge you. - Examples: WP Engine's reputation as the expensive-best provider and QuickBooks embedding CPAs as distribution moats. #### 💬 Quote > We had the reputation of where the most expensive and also the best. ... Once QuickBooks got the CPAs to use QuickBooks now that's the distribution moat to getting to customers. > — Jason Cohen Jason Cohen on how reputation and indirect distribution create moats #### 📚 Transcript **Jason Cohen:** were doing? We had the reputation of where the most expensive and also the best. A lot of people don't want the most expensive and don't need the best. In fact, most people don't. And I agree. And that's why GoDaddy is a great business because for $3, you can get a lot. Like what you get for $3 is insanely good. Like the ROI of GoDaddy is insane. **Justin Jackson:** That's **Jason Cohen:** fine. So like they have that. Someone else could also be expensive and good. But that can't stop us from being expensive and good. And it can't stop our reputation for being sort of first and best in that category. **Justin Jackson:** So in that sense, is reputation the key phrase there? Like is reputation the moat? **Jason Cohen:** So again, like, I guess I go back to what Jordan says, is this, are these useful, are these useful things to say? If you have a social media channel, no one else can stop you from having a good one. And so even if they have a good one, you can probably still do really well. On AdWords, it's not true. You can have a really good AdWord, but if 10 other people are better, then you're just not shown. Or it's too expensive to have good ROI or whatever, right? So in that sense, AdWords is heavily competitive. Social media, while competitive, and even a big company with lots of money and customers, they sort of can't stop you from having a better social media channel than that other company. The big company can overwhelm you in AdWords actually. They won't necessarily. Like it's a fight. It's just that you can't – it's not like they can't – they could. Whereas with social media, there's kind of nothing the big entrenched, mature, rich company can do to stop you from having a good social media presence. So in that sense, you have this unassailable advantage. to me, that's this idea of a moat is that you this unassailable position. Now, just the social media channel, I wouldn't call that a moat. I'm just making a simpler example. So in business, might be something like, I mean, everyone says network effects. Okay, yes. Because if someone else comes up with a better tool um they just they just can't stop the network effects they can compete on features and try to steal people and maybe they will but what they can't do is is lower the value of your of your um network effects **Justin Jackson:** right **Jason Cohen:** they can't they can't do anything they can't launch a feature that that for that particular so therefore the the network effect is a moat. So in that sense, is distribution a moat? Well, if you just run AdWords, then distribution is not a moat for you because you're doing things that are not only replicable. See, that's another thing. You think of it as can someone copy you. That's not what a moat is, right? Like if you have a network effect because you're a social media app, the next social media app has a network effect of course they can copy that it's not even the point in fact part of the definition of emote probably is that even if they copy it it's still emote for you yep **Justin Jackson:** so **Jason Cohen:** in that sense like this a lot of distribution is not a moat in that other people can in fact outdo you and beat you yeah but but if your distribution is good, then it can become a moat that can't be defeated. If you have a YouTube channel with a million subscribers, literally, that does seem like this kind of like forever is a big word, but let's say long-term advantage that no one can rest from you. Everyone just going to go god damn it and no matter what they do here you have it so i think distribution could be a moat if so like another way would be once quickbooks got the cpas to use quickbooks now that's the distribution moat to getting to customers because the cpa will tell their their clients oh use this because then it plugs into my thing. That sales system that's indirect through the CPA, that is a moat because no matter what you do on the outside to sell directly to those clients, it doesn't matter if their CPA says, no, use this. --- ### [Basecamp's Founders Built The Real Moat](https://share.snipd.com/snip/414ac058-1211-4d2a-a70d-d71bfe340d8b) 🎧 33:24 - 36:13 (02:48) <iframe src="https://share.snipd.com/embed/obsidian-player/snip/84b78ffb-e73b-49c1-869e-fea785e2cceb" width="100%" height="100" style="border: none; border-radius: 12px;" sandbox="allow-scripts allow-same-origin allow-forms allow-popups allow-clipboard-write" ></iframe> - Basecamp's success leaned heavily on the founders' public personalities and contrarian thought leadership. - Jason argues the 37signals blog and founders' public voice created the tribe and reputation that made Basecamp succeed more than product uniqueness alone. #### 💬 Quote > If you never had the 37 Signals blog that went alongside Basecamp, no one would use Basecamp. It's the very public personalities of the two founders. > — Jason Cohen Jason Cohen on Basecamp's competitive advantage #### 📚 Transcript **Jason Cohen:** me, it's the very public personalities of the two founders. If you never had the 37 Signals blog that went alongside Basecamp, no one would use Basecamp. Brian, **Brian Casel:** what do you think their moat is? I think that's absolutely true, especially in the early phase, or you might even say like the first 10 years of Basecamp, and sure it still has an effect. I think if the question is about like, why does it work today in 2026 for Basecamp, I think it's their longevity, right? Like they've been around for so long that they have this customer base, they already have a well-known established brand, at least in some circles, probably many circles. And I think that flywheel is just well on its way at this point. **Justin Jackson:** Jordan? **Jordan Goff:** I think they got the boulder up the hill with their very clear contrarian thinking and great marketing of that thinking that established a spot in the market where they just have their own space and people think of them in a unique way. And then once that boulder started rolling down the hill on the product, now the product fulfills the promise that they're making on the site and it's unstoppable. It's **Jason Cohen:** beautiful. **Jordan Goff:** I'd **Jason Cohen:** also push back a little to say that it is a moat. I've never met someone that used Basecamp. know now i know i know a million people use base camp and that's not what i'm saying what i'm saying is i could list you a lot of other products that have a hundred times the number of users that's okay because that's not their goal the goal is not to be the biggest whatever right so the goal is not in fact the goal is explicitly not to become jira right so that's not a criticism. I'm just saying like, let's put it this way. Let's say the team didn't, none of this existed. What happened is they vibe-coded Basecamp to the point it is right now as a product and then launched it right now. **Justin Jackson:** Would **Jason Cohen:** they reachieve their market share in let's say five or ten years? Are you going to it enough time because of anything? Good marketing, the product's better, differentiated. No, because the product's not any of those things it's it is a little different than others like it's good it's a great product there's a lot of great products like that though you're telling me people who already use notion for that would switch away from notion and to base camp even though they could have done that for the last 10 years and didn't just on the basis of the product i don't think so So like the product isn't differentiated and it would not succeed now. And so it would just be another indie. I made a project management software. People say, yeah, you know, there are some nice things about this, but I'm not going to switch to it, right? And so it isn't better. It's good, but it's not like there's nothing special there. That's why I go back to like, it's the cult of the founders. **Justin Jackson:** Yeah. **Jason Cohen:** Right. Like, I don't know what else there is that would make it succeed. Now, with this cult of founders, could it succeed now? --- ### [Signup To Cancel Feels Like A Series Of Miracles](https://share.snipd.com/snip/b7481423-fa36-487a-9cd7-2a24390dfade) 🎧 43:05 - 45:54 (02:48) <iframe src="https://share.snipd.com/embed/obsidian-player/snip/0f6841be-a6df-4ec6-b52b-2bc1825f1dd8" width="100%" height="100" style="border: none; border-radius: 12px;" sandbox="allow-scripts allow-same-origin allow-forms allow-popups allow-clipboard-write" ></iframe> - Purchases are a series of improbable events; cancellations after purchase often feel like a cascade of tiny failures. - Jason calls signups-to-cancellations a 'series of miracles' and uses it to motivate digging into onboarding and retention. #### 💬 Quote > Most ad viewers don't click. Most of those don't click past the landing page. Most of those don't consider. Most of those don't buy. It's the rare wins that keep us going. A series of miracles. > — Jason Cohen Jason Cohen on how unlikely conversions and subsequent cancellations feel #### 📚 Transcript **Jason Cohen:** longer version of this is how I kick off one of the chapters of the book which is a chapter about cancellation because i have this i've always had this feeling and i you know try to put into words of like how impossible it is that anyone would buy the damn thing like first you have to see your ad which is already like what one out of maybe of one percent of people see the ad that you expose it to that's pretty good just order of magnitude maybe it's 0.1 i don't know and so then they clicked it well let's click through rates one percent that's good okay **Justin Jackson:** then **Jason Cohen:** they hit the home page and then what 70 bounce 90 depends on the source. Okay. Yeah. That means they spent more than like three seconds, right. And then they read it and they're like, Oh, interesting. Really? Thank God. You know, you're the first one today. Um, you know, and then they, they looked around and they, they didn't, they didn't get bored and they looked, the features are like, Ooh, really? Thank God. And then they, then they looked at the pricing page and they didn't just leave you know like oh that's reasonable or it's not too low and they think you're shitty or too high and they think you're you know out of their league and they look the tears they're like i can do this and then maybe they looked at competitors maybe they didn't but supposedly they did whatever they still decided to buy from you they got out of the credit card and it worked i mean that's already something you know there's a fall off there yeah a series of miracles a series of miracles and then they got in and then what they onboarded they typed stuff in they uploaded a photo of themselves the seven millionth time this week like they dedicated all this time out of their life to the yeah maybe they call tech support to get unfucked and and like that then they and also with if it's at work i'm usually i'm usually thinking of a b2b context everything i just said is absolutely true and b2c i i'm just i just uh am expert in b2b and not b2c so i always think that way and so anyway in a business context they probably told their peers like oh you need to check out the software. I checked this out. It's really cool. Like they've started putting their own reputation on the line. And then after all of that, they cancel. And you're like, what? That's what lightning in a bottle. And then you're like, nah, like what? Like you have to go just emotionally, like forget financially, emotionally, you have to find out what is going on because they bought your argument. They bought your thesis. They bought your promise. They have the budget. It too expensive. They chose you over a competitor. They invested all this time. And they said, no, what did you do wrong? This is totally on you now, right? Now, maybe they were confused. Okay, but I'm being extreme a little bit, but to make the point. So then, of course, I go on to talk about there is a financial aspect that's very important as well. --- ### [Optimize Top-Of-Funnel Quality First](https://share.snipd.com/snip/06895dc0-a3d3-41d4-9d7f-828d834d32f6) 🎧 50:53 - 52:37 (01:43) <iframe src="https://share.snipd.com/embed/obsidian-player/snip/10c18d0e-b35f-4138-bb41-c44da2c89355" width="100%" height="100" style="border: none; border-radius: 12px;" sandbox="allow-scripts allow-same-origin allow-forms allow-popups allow-clipboard-write" ></iframe> - Prioritize the quality of who enters the top of your funnel over optimizing downstream steps. - Aim for fewer, higher-quality clicks that match your ICP because better traffic outweighs marginal landing page AB tests. #### 💬 Quote > The quality of who enters the top of the funnel is more important than every other step combined. You'd rather have fewer clicks. If the quality is great, the rest of it doesn't matter. > — Jason Cohen Jason Cohen on funnel strategy and the importance of ICP #### 📚 Transcript **Jason Cohen:** So it's exactly what, actually, what and Jordan were getting at. Because Brian was talking about ICP, and Jordan was talking about looking through the, **Justin Jackson:** watching **Jason Cohen:** people and saying, wait a minute, there is stuff I could do here. Consider this. Let's say in your funnel, I put nothing but the perfect best customers, like your ICP only is coming to the top of the funnel on the website. Even if your copy is just okay, but it just is talking to them essentially, like you're going be you're gonna be okay people will still buy because they want it so bad right and if your sales team if there is sales uh is kind of bad it's okay because like they kind of wanted to buy anyway right prevent the sale in other words even if everything even if you're bad at everything if i only put icps in the top you'll be okay. Now the opposite. I put nothing but garbage traffic, **Justin Jackson:** click **Jason Cohen:** fraud, whatever. Now I don't care how good your funnels are and your testing, your sales team, you're not going to get a lot of customers. What that tells you is the most important thing in the entire funnel is the quality entering the top of the funnel. **Justin Jackson:** We **Jason Cohen:** just said, if the quality is great, it doesn't really matter. The rest of it doesn't matter. If the quality is bad, the rest of it doesn't matter. So of course I'm exaggerating when I say the rest doesn't matter. Of course it does, but I'm exaggerating to make the point that the first thing entering, not even a step in the funnel, the stuff entering the funnel, the quality of that is more important than every other step combined. Once you understand that, you realize what Brian said. Go get your ICP because that's what it means to have quality at the top. You'd rather have fewer clicks. --- ### [Prevent A/B Tests From Optimizing Garbage Traffic](https://share.snipd.com/snip/ad586541-a399-4189-ace1-888ee44aadb3) 🎧 54:23 - 55:46 (01:23) <iframe src="https://share.snipd.com/embed/obsidian-player/snip/f8da2bdd-e59d-483c-8573-192349f79cda" width="100%" height="100" style="border: none; border-radius: 12px;" sandbox="allow-scripts allow-same-origin allow-forms allow-popups allow-clipboard-write" ></iframe> - Beware AB tests that optimize for low-quality traffic; they can improve conversion metrics while worsening overall customer economics. - Segment high-quality sources separately and measure conversion to end outcomes, not just next-click rates. #### 💬 Quote > If 90% of the traffic is garbage, the only way to up that conversion rate is to convert the garbage. What you do is you say I'm trying to up the conversion rate. That's what the A/B test will do. Just what you don't want. > — Jason Cohen Jason Cohen warning about misleading A/B test effects when traffic quality is poor #### 📚 Transcript **Jason Cohen:** That's not how we normally think, but that would be how you took this information and apply it properly. Or another thing would be how you segment, you know, in whether it's Google ads or Facebook ads or, okay, social media, other things, like how you segment the audience is really important. And again, it's better to have fewer leads that are more poignant. So here's another thing. If you don't do this, then your A-B testing in your website also optimizes for garbage because most of the traffic to your site is garbage. **Justin Jackson:** Yes. **Jason Cohen:** And what do you do when you optimize a step in the funnel, like optimizing the pricing page conversion or optimize the homepage, even a landing page click to just go on to the next page. What you do is you say, I'm trying to up the conversion rate. Right. But if 90% of the traffic is garbage, the only way to up that conversion rate in a material sense is to convert the garbage. If **Justin Jackson:** you convert that 10% **Jason Cohen:** and you grow from 10% to 11%, that's actually what you want to do. That's a 10% improvement. That's huge. But you'll never even be able to measure that. The only thing you can measure is if the shit traffic that you don't want is nevertheless fooled to continue. Then the rate of that step goes up. So that's what the AV test will do. Just what you don't want. Again, this is assuming 90% is garbage, which again, comes back to the top of the funnel. --- ### [Measure Conversion Deeper Into The Funnel](https://share.snipd.com/snip/ee4ca4f6-1d49-4dff-a588-addd161dcfce) 🎧 56:31 - 57:14 (00:42) <iframe src="https://share.snipd.com/embed/obsidian-player/snip/3d1161e1-4a65-46f1-8fd5-f76ce9ffb778" width="100%" height="100" style="border: none; border-radius: 12px;" sandbox="allow-scripts allow-same-origin allow-forms allow-popups allow-clipboard-write" ></iframe> - Measure deeper into the funnel (to purchase or retention) whenever possible to align experiments with business outcomes. - For simple products, track end-to-end conversion so tests reflect true impact, not intermediate clicks. #### 💬 Quote > The deeper you can measure into something useful, then it does work because you can say no, I'm not converting just the next step of the funnel. I'm measuring my conversion rate to the end of the funnel. > — Jason Cohen Jason Cohen on choosing better funnel metrics #### 📚 Transcript **Jason Cohen:** That helps. Another one is if you can measure end-to more. Part of the fallacy here is measuring just one step, this landing page to this next click. That's too myopic. If you can measure all the way to the sale, which for simple products, maybe you can, for complex ones, you cannot, but let's just say the deeper you can measure into something useful, then it does work because you can say, no, I'm not converting just the next step of the funnel. I'm measuring my conversion rate to the end of the funnel. If that goes up a little bit, yes, now you've done the right thing. Hooray. So expanding the, what do you mean convert? Convert to what? The deeper you can convert to what? The more now you really are measuring the thing you wanted to measure all along. --- ### [Expect Slow Feedback And Use Multiple Metrics](https://share.snipd.com/snip/2c2d8e70-35bd-4f14-b607-d96497da4f99) 🎧 01:01:59 - 01:06:31 (04:31) <iframe src="https://share.snipd.com/embed/obsidian-player/snip/efa693f9-9471-4c1f-b4fb-bf0d69901ad2" width="100%" height="100" style="border: none; border-radius: 12px;" sandbox="allow-scripts allow-same-origin allow-forms allow-popups allow-clipboard-write" ></iframe> - Long feedback cycles create an 'uncertainty principle' in funnel metrics: end-to-end outcomes can take months, so short-term signals are noisy. - Jason recommends multiple complementary metrics and a clear vision to guide action despite slow feedback. #### 💬 Quote > There are these... analogies. One is in the stuff like data and funnels. You might say it will take you a year to find out whether a new marketing source creates people who cancel more often than other people. You have to wait. > — Jason Cohen Jason Cohen on metric timelines and the difficulty of end-to-end learning #### 📚 Transcript **Jason Cohen:** here's a, here's a, okay, let's do another physics example. There's, there's, there's the like uncertainty principle. If the more you know about, you know, position, the less you know about momentum and vice versa, or the less you can be precise about. So to me, there are these, in complex systems, there's, again, not literally, but figurative analogies of this. And one of them is in the stuff like data and funnels like you're talking about. So you might say, everyone in the company would have the end-to funnel view. And although they may be working on or they may have control over one set of mechanisms within that system, which is obviously true, they nevertheless are all optimizing across the whole system. That's what you would like. Because of course, you want maximum throughput or equality, you might say, for the whole system. And then you have people who are specialized in different areas working on it. That's what you want. But you have this uncertainty problem, which is anything that's end-to has all these characteristics that make it very hard to operate. Like it will take you a year to find out whether a new marketing source creates people who cancel more often than other people. And you can't speed that up because whether they cancel in six months, take six months. Like there's nothing you can do to accelerate the pace at which you can find out whether someone's going to cancel in six months. You have to wait. And also like, if it's a brand new channel, you probably need to give them chances to optimize a little bit. Like you can't just declare, you know, so that's why I say it might take a year and there's nothing you can do. You can be smarter and less smart. You cannot make it faster than that. Well, people can't operate on a cycle, on a feedback cycle of a year. They have to operate, they have to have something faster, but those things that are faster are probably more localized and don't tell you the full picture. And so this is the uncertainty principle of metrics and timelines, you might say, right? So to me, it's like, okay, what do we do with that fact? And timeline is only one of many things to me about metrics that you might observe. to me, it okay, that means there's not one true metric. That's a stupid idea. We just explain why that doesn't work. So you need different metrics for different things, different goals or operational or whatever. You need different ones. that together tell me all the stuff I need to know. And we cannot think that a given metric will tell us things other than what it actually tells us. None of them will be the totality of what we want. And somehow a synthesis of all this is how we have to operate, even though we don't like it. Just like, sorry, you're just never going to know where the electron is and which direction it's going ever. But that doesn't mean you can't have some measurements and do some stuff and that's what we're going to do and so still i would come back i would i would still it's still useful to me to do what you said which i think is right and come back and like all right everyone let's keep our eye on the on the prize the prize is overall optimization although we can't run our day-to based on that comma, it is the, like, this is why I like vision statements. Vision statement is something like, you would like that to be the case. You're trying to do that. You may never even reach it ever, but like, at least it tells us what we what we are trying to do if we could. And that is still useful because it gives us like that. Anything that's worked toward that is correct. Anything that's like to the side or away from it is not right. So I would use your statement as a vision statement rather than a mandate, an operational mandate, which is we're trying to make it so that everything that comes in the top of the funnel is perfect and people stay forever. And we're all trying to do that. We're trying to optimize for the whole funnel. Now, since none of us can day to day measure and do that, we will need things that as best we can tie into that vision and we'll need to check in. So I'll, because we can measure that vision over a span of months. So we will, it's just that we know that like, if make a change today, we're not going to see that number change immediately. And even in six months, we may not because it's multi-factor. So many different other things go into it. We still won't know. We just have to acknowledge that's true. But we can still measure the overall vision, which is the overall funnel thing, like you said. That we're still going to measure because that's what we're trying to do. We will have to use our wits and our yes, our intuition and where we can data. And sometimes we'll think of a slightly better metric that still doesn't solve the whole equation, but like this one's better than before. --- ### [How A Dumb Survey Improved WP Engine's Data Team](https://share.snipd.com/snip/2fc63029-13d5-4b90-b551-bb0aa32eae8c) 🎧 01:09:38 - 01:14:24 (04:45) <iframe src="https://share.snipd.com/embed/obsidian-player/snip/145c0efa-9268-49c9-88e7-628bec04feaa" width="100%" height="100" style="border: none; border-radius: 12px;" sandbox="allow-scripts allow-same-origin allow-forms allow-popups allow-clipboard-write" ></iframe> - Jason used a simple monthly survey to show WP Engine's internal data team how customers perceived their work. - The survey revealed more positive sentiment than expected, improved feedback tone, and prompted better meetings and explanations. #### 💬 Quote > They had no metrics and no conception about their customer. I said just take a step back... What would it look like if we were crushing it for our customers? We could give them a really dumb survey. The scores came up over time. > — Jason Cohen Jason Cohen describing using a basic survey to align an internal team's work with customer needs #### 📚 Transcript **Jason Cohen:** Anyway, so I took what's in there, quite literally sent the article, like just as an example with the data team, which meaning the internal corporate data team at WP Engine. And, um, they were, they were sort of rethinking how to do all their work and what tools they're using and what metrics they should use. So we used exactly this article. Um, in fact, I think the article was already published at the time. So it was literally using this And it was like, it was interesting. All the metrics on the data team were things like, are we syncing from the source to that source? Is this thing broken? How would we know that? What are these timeframes? And that's all true. Those are all good operational metrics for the team itself of like, they, is their own work and own systems doing whatever the team wanted it to do? Right? Very team centric. But to your point, like those are the easier ones because it's your stuff. What about the customer? Now, in the case of the data team, the customer is the rest of the company, especially the other data analysts in the rest of the company, but even other people too. That's their customer. And they had no metrics and no conception about their customer. So I would say things – so they would do things like, oh, so-and in this other department is mad at us because, I don't know, we didn't work on this thing that they need. But it was really urgent. We didn't have the time. So in other words, they're talking about normal product development and client work. But since it's all within the company, we can do whatever we want with it, which is a nice kind of microcosm. And so I was like, well, so we were talking about this, like, don't even know what metrics we would use to say, like, do we deliver something? Because like, we may or may not be able to do any one task in any one sprint. So like that, I don't even know what a metric would be. And I said, okay, well, just take a step back and ask, and this gets back to the vision. Just if you imagine, like, if just nevermind our tech and how we do it, but like our customers, all these people in the company, what would it look like if we were like just the best ever and just crushing it for our customers? What would that look like? This is, of course, what the vision statement is. It's describe what the future looks like if we're crushing it, right? Yeah, yeah. What we look like if we're crushing it. And're like well i don't know they get back to the numbers i'm like no no not numbers just like without numbers what what what and just like they would just i don't know like i guess if you just talk to any of them they would be happy with us and and then like yeah i agree like if they all said it's awesome working with the data team then that would be success like yeah but we don't know what's awesome. We don't like, oh, I know, I know, I know they may not even know, but you just said it. You just put a vision statement that, that you want everyone to. And so how could we start to in fact measure it very poorly, like intentionally measure it poor. Like I know it's hard. Like how would we measure this even poorly? Well, we could give them like a really dumb survey, really basic survey, not NPS, of course, because they're not referring us to a friend. Anyway, I hate NPS, but just a basic survey about how they feel with it, like a fill, like a big fill in box so they can just type shit. And, um, we could just send that every month to 20 of our so-called customers. **Brian Casel:** I love how like so many of your ideas start with like, like just start with the ideal. Let's work backwards from that. What needs to be true and **Jason Cohen:** what needs to be true. So they did. And the first thing that was interesting is it wasn't as bad as they thought, which is, which makes sense because like if you deliver data to another person and it's just what they want, often what happens is nothing. They just use it. Right. So you have no positive feedback. Occasionally you do. Usually people are like, Oh, thanks for that. Right. But a lot of times if you're doing your job well in data or security, if you do your job well, you're invisible. Then that's a good thing. Like, Ray, I didn't think about my website performance today. Right. That's good. So that, but that means you're not getting the positive feedback. So when you do get feedback is when people are upset and they're like, ah, website down, whatever. And you're like, ah, so that's all you see. So your perception is there's problems all the time because there are problems all the time. It's just that you're not seeing how that's, how that's counterbalanced by, by when there's not. So first of all, it wasn't as bad, which was a morale boost because they kind of felt like maybe people were upset with them. And it was like, oh, no, they're actually fundamentally not upset with us. They just want certain things. Also, just the fact that they were curious about whether their customers were happy or not made their customers like go, oh, oh, that's good. And all of a sudden they were more constructive in their feedback. Oh, you actually want to improve? Oh, well, you know, it's easier to be nicer in that case, more understanding. And if you think that maybe your feedback is not being heard, even if it is being heard, you think it's not. So the whole dynamics changed to be better right off the bat. And then, of course, they did look at this and they did blah, blah, blah. --- ### [Use Metrics To Inform Not Decide](https://share.snipd.com/snip/f70ae6cf-d825-42e4-a36d-544ffe1ddf93) 🎧 01:17:27 - 01:18:01 (00:33) <iframe src="https://share.snipd.com/embed/obsidian-player/snip/7ee1a64e-5b1b-4a9c-a5b6-df572795a7bd" width="100%" height="100" style="border: none; border-radius: 12px;" sandbox="allow-scripts allow-same-origin allow-forms allow-popups allow-clipboard-write" ></iframe> - Treat metrics as informative inputs, not definitive answers; avoid incentivizing single metrics that encourage gaming. - Use metrics to guide conversation and behavior, not as absolute proof or sole basis for rewards. #### 💬 Quote > Metrics should be informative, but never decisive. People bringing you the metrics should not present them as definitive. Despite these metrics, we think, and that's good. > — Jason Cohen Jason Cohen on healthy metrics usage and avoiding metric overreach #### 📚 Transcript **Jason Cohen:** Like, is the data collected even right? We're not sure. Are we interpreting it correctly? We're not sure. We're using stats. Are those the right stats? We're not sure. Is this the full picture? You know, certainly not. It is certainly not the whole picture. We know that definitely. We just don't know exactly how much. People want it to be definitive. They want it to be accurate. Yeah, And the people bringing you the metrics are also should not be bringing you those metrics as if that's the definitive they should say. And yet, despite these metrics, we think, and that's good. So metrics should be informative, but never decisive. --- Created with [Snipd](https://www.snipd.com) | Highlight & Take Notes from Podcasts # Snips updated after 2026-04-05T10:16:17 ### [Why Jason Stepped Away From WP Engine](https://share.snipd.com/snip/41bf90c3-385a-43b7-916a-d5f895475b92) 🎧 02:16 - 03:36 (01:19) <iframe src="https://share.snipd.com/embed/obsidian-player/snip/c2193377-81e3-4d1b-bd8c-3872891e93ca" width="100%" height="100" style="border: none; border-radius: 12px;" sandbox="allow-scripts allow-same-origin allow-forms allow-popups allow-clipboard-write" ></iframe> - Jason left WP Engine after 15 years to rest, spend time with family, and write rather than join another company. - He visited colleges with his daughter, traveled, and explicitly avoided starting another company for at least two years while writing a book. #### 💬 Quote > I left WP Engine about two years ago. I've been writing a book, which is now on pre-order. I'm not starting another company. Here it's two years later. I have not started another company. > — Jason Cohen Jason Cohen on leaving WP Engine and his post-exit priorities #### 📚 Transcript **Jason Cohen:** I said SmartBear. That's how long it was. I left WP Engine about two years ago. I've been writing a book, which is now on pre-order. I'm not here to push the book. That's not why we're doing this. So I'll let Justin push it at some point. Hey, I will push the book. Just pushing it. All you want is it's called Hidden Multipliers. Oh, look at that. You have pulled up and everything. Okay, so if go to hiddenmultipliers.com, there it is. There, I did the pitch. So, yes, so it's funny. We were just talking before the show. Um, I think when founders or executives, uh, leave, they always have an excuse and the typical excuses, I'm leaving to spend time with my family. Cause we can argue with that. It sounds great. Everyone goes, Oh, that's so noble. And then they leave. And then three weeks later you read so-and has been, you know, been appointed the senior vice president of sales of North America for, and you're like, wait a minute, that was not a lot of time with your family dude and so uh so i was really sensitive to that so when we were talking about it internally i was like look it's just there's nothing wrong nothing bad it's just been 15 years it's just time like i just want to rest a little i'm older my daughter's about to go to college i want to go visit colleges with her a little bit but like it's just mostly it's been a very long time. It's just simply maybe time for the next thing, whatever that is, which is not another company. I said at the time, I'm not starting another company. Here it's two years later. I have not started another company. --- ### [Writing Without Deadlines Preserves Quality](https://share.snipd.com/snip/d1650975-821e-4aa7-8286-4c85d76814e4) 🎧 06:08 - 11:55 (05:46) <iframe src="https://share.snipd.com/embed/obsidian-player/snip/91438400-e2ca-4971-a774-b67fe4cced98" width="100%" height="100" style="border: none; border-radius: 12px;" sandbox="allow-scripts allow-same-origin allow-forms allow-popups allow-clipboard-write" ></iframe> - Jason treats writing as a freedom-driven craft: no deadlines, big content buffer, and variable cadence to preserve quality. - He keeps hundreds of drafts, buffers social posts months ahead, and prioritizes craft over algorithmic consistency. #### 💬 Quote > I have a rule from the beginning that it's supposed to be fun. I have no deadlines ever for anything. My buffer is deep. I'm not joking. I'm not consistent at all. I post very randomly. > — Jason Cohen Jason Cohen on his writing process and editorial freedom #### 📚 Transcript **Jason Cohen:** Well, you know, since I've been writing for almost 20 years, as you know, and, and, and fortunately, never for money, never trying to make my, you know, and so I've been able to do whatever I feel is best for me. So I've had a rule from the beginning that it's supposed to be fun. And so for example, I have no deadlines ever for anything. And so sometimes I've had years where I publish every week like clockwork and years where I publish once or twice only the whole year. For whatever reason, I don't have the energy. I have the time. I don't want to. It's whatever. So as a result, I've been able to keep the quality high because I'm allowing things, like you said, consistency. When you said that, I thought, I'm not consistent at all. I post very randomly. But as a result, things like the craft or the style or the ideas, the consistency of whether the idea is good enough, I've been able to keep up because I let things like deadlines not drive things. And of course, for algorithms, that's bad. And so a lot of people say, you should have 10 times the number of followers on Twitter on, you know, blog, I don't know. And it's like, yeah, maybe, but I'm just not going to do that algorithm stuff. Because again, I just have a different set of constraints that I'm running under. But it allows me to have the quality, at least the best of my own ability, whatever that is. And so for anyone who likes that, then good. **Brian Casel:** I do love the idea of like lack of deadlines. I'm always really curious about what the life cycle of every... So if we see something come out on Twitter from you or an article drop from you, what was the lead up to that? How many days of development? How many drafts? How, how, how do you get to the point of like, this is ready to hit publish? Yeah. **Jason Cohen:** Nothing's real time. Like even the Twitter stuff, that's all buffered up. And so a lot of times surprised myself, like, Oh, I wrote that like nine months ago. I mean, my, my buffer is deep. I'm not joking. **Brian Casel:** And with one of the nice things, it does that way from the surface. I assume that there must be some sort of like cue, **Jason Cohen:** right? It is a massive cue. And what's nice about being so ahead of the cue like that is I can say things which in the moment, if I didn't have such a deep cue, there's things I wouldn't have said because it was so close to the moment that the person or event or thing I'm talking about, they would know or other people in their team would know. And that's the last thing I ever want to do to anybody, right? Like it's nice to say here's something I observed or a lesson. Very bad if I'm calling out someone even by accident, right? Or at least I say, right, that's just my attitude. But if there's a nine-month queue and I write that, there's no chance you could – anyone. Yeah. They could maybe think, oh, was he talking about that thing from a year ago? But that's so vague that at some point, it's just my experience. I have to relate it. Or I don't have to, but it makes sense. So it's actually great because I can say more things than I might have otherwise said without worrying about how I may be affecting other people in that particular sense. The posts, I have literally hundreds of posts and drafts, some of which are like an idea in one sentence and some are long things. I just don't – I'm just not feeling it or I'm feeling the editing or maybe the idea is not good enough yet or something. And of course, most of them will never work. And so it is, again, kind of sporadic. Sometimes I'll just have an idea and bam, I write the whole first draft when one's sitting and then all it needs is editing. And then other times – and so even so, that will be hours because i go over it and i edit it a lot i want to find every problem i want the style again i'm not just trying to put out something so like is this written well enough is this is it structured well enough that sort of thing are the examples good and of course what's what's too long and all that um so it still will take a lot of time in editing but sometimes the original is fast and sometimes the whole thing gets rewritten a bunch there's one people like a lot that's called the elephant curve it's called the elephant in the room but it's about the elephant curve which is just a name i came up with um for this particular idea that one is very long and has a million graphs and data points because i have this thesis about how growth works or how exponential growth doesn't work, you might say. And I have this thesis about, like, here's how growth works and why. And then I back it up with, like, the zillion data points from all kinds of companies, bootstrapped, venture-funded, the biggest ones ever, old ones, new ones, and, of course, my own data. That one took 10 years because I had this idea that long ago i published this i think in 2022 but i i had that idea because i i remember publishing it like i've had this idea for so long and the reason there's so much data is that i had this idea and i never got around to like making it into some sort of thesis but every time i saw another example i'm like god damn it there's it there it is again and i would snag it and throw it in the file you know so the file ended up being like i have 10 not 10 i have maybe three times as much stuff as what's in that article even right of like accumulated and then finally for no reason i can i can explain to you i was just like i had the bug and i had and i just it's like it's time to write this damn thing after all it's ready to get it out and so that's part of why it's so deep so it can be like that too i i think so that's not it's kind of a non-answer of like, what's my process? I guess the answer **Brian Casel:** is, yeah. **Jason Cohen:** Yeah. The answer is like different things just happen differently. I allow that. And I don't worry. I don't sweat that. And a real professional writer cannot act like me, you know, right? Like if you need to publish every day, if you're writing for a newsletter or even if it's for money and so, and you're getting paid $20,000 for an amazing article in The Atlantic, you still need to go like do that thing for a few months. And so like since I don't need to, the freedom of that – but what that means is it's – I can do my best work. I don't let it intrude. I cannot let it intrude on the rest of life. But that means I can be the best writer I can be, the best thinker I can be, put out the work that I'm most proud of. That doesn't mean it's all great necessarily. It's just that my best that I can do is what, that's why I keep repeating that. And that's something everyone can adopt and do. --- ### [Use AI To Spew Then Edit Every Word](https://share.snipd.com/snip/7661ba52-6cb7-4372-a2a2-5fb504331f0d) 🎧 16:18 - 16:53 (00:35) <iframe src="https://share.snipd.com/embed/obsidian-player/snip/485224f2-f21d-4b19-926c-72d8829dd569" width="100%" height="100" style="border: none; border-radius: 12px;" sandbox="allow-scripts allow-same-origin allow-forms allow-popups allow-clipboard-write" ></iframe> - Use AI to turn verbal spew or half-baked drafts into structured outlines, then edit every word yourself. - Leverage AI as an editor and outline generator to get past writer's block, but do heavy human editing to retain voice. #### 💬 Quote > Often just talking aloud, you end up with something which you can now reform. And if you don't care so much about your own writing craft, of course, you can use AI and say, all right, I have this half written thing. > — Jason Cohen Jason Cohen on using AI in the writing process #### 📚 Transcript **Jason Cohen:** And if you don't care so much about your own writing craft, of course, you can use AI and say, all right, I have this half written thing. I have this spew of ideas. Like let's, let's first decide what kind of article structure makes sense. Like, is it, is it like surprising thing, background context, here's the solution. And here's like a couple of tips, which is a lot of mine to be fair. Like, is that the right? And so you could, you can work with the AI a little bit, like what's the right outline structure. And then like, okay, we'll take all the shit I said and, like, sort of do that. And it won't be very good. But, like, maybe, as we all know, editing is easier than writing from scratch. --- ### [Moat Means Being Unstoppable Not Just Hard To Copy](https://share.snipd.com/snip/6794f1d8-dccb-47aa-bc00-a775e6c4ab29) 🎧 23:50 - 25:19 (01:29) <iframe src="https://share.snipd.com/embed/obsidian-player/snip/1bd5ada0-546c-4f5e-b6fe-341e2c8f6f3c" width="100%" height="100" style="border: none; border-radius: 12px;" sandbox="allow-scripts allow-same-origin allow-forms allow-popups allow-clipboard-write" ></iframe> - 'Moat' means competitors cannot stop you from running your business, not merely 'hard to copy' features. - Jason frames moats like Warren Buffett: persistent defensive advantages such as network effects or entrenched distribution partners. #### 💬 Quote > When I hear moat I just think – Warren Buffett... competitors cannot stop us from running the business we're running. To me, that's what the moat is. > — Jason Cohen Jason Cohen on the definition of a moat #### 📚 Transcript **Jason Cohen:** moat in this world? I agree with Jordan that the phrase, you know, you read into it what you want and therefore it doesn't mean anything. **Justin Jackson:** And **Jason Cohen:** it's also not new. I mean, to the extent that distribution could be a moat, that isn't new information. **Justin Jackson:** And **Jason Cohen:** to the extent that it's not a mo it's not a the phrase is not accurate there is a very so so in that sense it embodies the very funny thing uh i forget where this comes from it's very old but there's like a beginning of a paper or something and it says there's much in this paper that is new and much that is true but the stuff that is true is not new and the stuff that is new is not true so this reminds me of like this is distribution as a moat is kind of an embodiment of that like this is a nonsense thing it's either not true or or we it's not saying anything new so therefore this is so i think that's what jordan was saying too and and i agree **Justin Jackson:** i **Jason Cohen:** do think um so when i hear moat i just think – Warren Buffett, of course, coined this phrase. And so I think of the Warren Buffett version of it, which is – maybe you could have other ones. That's fine. That's just what I think of, which is that competitors cannot stop us from running the business we're running. To me, that's what the moat is. Now, you normally think of it as, you often think, and I've said it as, a permanent competitive advantage. **Justin Jackson:** I **Jason Cohen:** think that's a fine way to say it. But if we're going back to the original thing, it's sort of that it's not even an advantage. --- ### [Reputation And Distribution Can Become Real Moats](https://share.snipd.com/snip/e0db2e52-eb22-47ad-b81e-c863e5007813) 🎧 25:51 - 29:37 (03:45) <iframe src="https://share.snipd.com/embed/obsidian-player/snip/ee960bd5-d780-4591-8ba9-c6cc6b8fef0d" width="100%" height="100" style="border: none; border-radius: 12px;" sandbox="allow-scripts allow-same-origin allow-forms allow-popups allow-clipboard-write" ></iframe> - Reputation and positioning can create an unassailable place in market segments where others cannot dislodge you. - Examples: WP Engine's reputation as the expensive-best provider and QuickBooks embedding CPAs as distribution moats. #### 💬 Quote > We had the reputation of where the most expensive and also the best. ... Once QuickBooks got the CPAs to use QuickBooks now that's the distribution moat to getting to customers. > — Jason Cohen Jason Cohen on how reputation and indirect distribution create moats #### 📚 Transcript **Jason Cohen:** were doing? We had the reputation of where the most expensive and also the best. A lot of people don't want the most expensive and don't need the best. In fact, most people don't. And I agree. And that's why GoDaddy is a great business because for $3, you can get a lot. Like what you get for $3 is insanely good. Like the ROI of GoDaddy is insane. **Justin Jackson:** That's **Jason Cohen:** fine. So like they have that. Someone else could also be expensive and good. But that can't stop us from being expensive and good. And it can't stop our reputation for being sort of first and best in that category. **Justin Jackson:** So in that sense, is reputation the key phrase there? Like is reputation the moat? **Jason Cohen:** So again, like, I guess I go back to what Jordan says, is this, are these useful, are these useful things to say? If you have a social media channel, no one else can stop you from having a good one. And so even if they have a good one, you can probably still do really well. On AdWords, it's not true. You can have a really good AdWord, but if 10 other people are better, then you're just not shown. Or it's too expensive to have good ROI or whatever, right? So in that sense, AdWords is heavily competitive. Social media, while competitive, and even a big company with lots of money and customers, they sort of can't stop you from having a better social media channel than that other company. The big company can overwhelm you in AdWords actually. They won't necessarily. Like it's a fight. It's just that you can't – it's not like they can't – they could. Whereas with social media, there's kind of nothing the big entrenched, mature, rich company can do to stop you from having a good social media presence. So in that sense, you have this unassailable advantage. to me, that's this idea of a moat is that you this unassailable position. Now, just the social media channel, I wouldn't call that a moat. I'm just making a simpler example. So in business, might be something like, I mean, everyone says network effects. Okay, yes. Because if someone else comes up with a better tool um they just they just can't stop the network effects they can compete on features and try to steal people and maybe they will but what they can't do is is lower the value of your of your um network effects **Justin Jackson:** right **Jason Cohen:** they can't they can't do anything they can't launch a feature that that for that particular so therefore the the network effect is a moat. So in that sense, is distribution a moat? Well, if you just run AdWords, then distribution is not a moat for you because you're doing things that are not only replicable. See, that's another thing. You think of it as can someone copy you. That's not what a moat is, right? Like if you have a network effect because you're a social media app, the next social media app has a network effect of course they can copy that it's not even the point in fact part of the definition of emote probably is that even if they copy it it's still emote for you yep **Justin Jackson:** so **Jason Cohen:** in that sense like this a lot of distribution is not a moat in that other people can in fact outdo you and beat you yeah but but if your distribution is good, then it can become a moat that can't be defeated. If you have a YouTube channel with a million subscribers, literally, that does seem like this kind of like forever is a big word, but let's say long-term advantage that no one can rest from you. Everyone just going to go god damn it and no matter what they do here you have it so i think distribution could be a moat if so like another way would be once quickbooks got the cpas to use quickbooks now that's the distribution moat to getting to customers because the cpa will tell their their clients oh use this because then it plugs into my thing. That sales system that's indirect through the CPA, that is a moat because no matter what you do on the outside to sell directly to those clients, it doesn't matter if their CPA says, no, use this. --- ### [Basecamp's Founders Built The Real Moat](https://share.snipd.com/snip/414ac058-1211-4d2a-a70d-d71bfe340d8b) 🎧 33:24 - 36:13 (02:48) <iframe src="https://share.snipd.com/embed/obsidian-player/snip/84b78ffb-e73b-49c1-869e-fea785e2cceb" width="100%" height="100" style="border: none; border-radius: 12px;" sandbox="allow-scripts allow-same-origin allow-forms allow-popups allow-clipboard-write" ></iframe> - Basecamp's success leaned heavily on the founders' public personalities and contrarian thought leadership. - Jason argues the 37signals blog and founders' public voice created the tribe and reputation that made Basecamp succeed more than product uniqueness alone. #### 💬 Quote > If you never had the 37 Signals blog that went alongside Basecamp, no one would use Basecamp. It's the very public personalities of the two founders. > — Jason Cohen Jason Cohen on Basecamp's competitive advantage #### 📚 Transcript **Jason Cohen:** me, it's the very public personalities of the two founders. If you never had the 37 Signals blog that went alongside Basecamp, no one would use Basecamp. Brian, **Brian Casel:** what do you think their moat is? I think that's absolutely true, especially in the early phase, or you might even say like the first 10 years of Basecamp, and sure it still has an effect. I think if the question is about like, why does it work today in 2026 for Basecamp, I think it's their longevity, right? Like they've been around for so long that they have this customer base, they already have a well-known established brand, at least in some circles, probably many circles. And I think that flywheel is just well on its way at this point. **Justin Jackson:** Jordan? **Jordan Goff:** I think they got the boulder up the hill with their very clear contrarian thinking and great marketing of that thinking that established a spot in the market where they just have their own space and people think of them in a unique way. And then once that boulder started rolling down the hill on the product, now the product fulfills the promise that they're making on the site and it's unstoppable. It's **Jason Cohen:** beautiful. **Jordan Goff:** I'd **Jason Cohen:** also push back a little to say that it is a moat. I've never met someone that used Basecamp. know now i know i know a million people use base camp and that's not what i'm saying what i'm saying is i could list you a lot of other products that have a hundred times the number of users that's okay because that's not their goal the goal is not to be the biggest whatever right so the goal is not in fact the goal is explicitly not to become jira right so that's not a criticism. I'm just saying like, let's put it this way. Let's say the team didn't, none of this existed. What happened is they vibe-coded Basecamp to the point it is right now as a product and then launched it right now. **Justin Jackson:** Would **Jason Cohen:** they reachieve their market share in let's say five or ten years? Are you going to it enough time because of anything? Good marketing, the product's better, differentiated. No, because the product's not any of those things it's it is a little different than others like it's good it's a great product there's a lot of great products like that though you're telling me people who already use notion for that would switch away from notion and to base camp even though they could have done that for the last 10 years and didn't just on the basis of the product i don't think so So like the product isn't differentiated and it would not succeed now. And so it would just be another indie. I made a project management software. People say, yeah, you know, there are some nice things about this, but I'm not going to switch to it, right? And so it isn't better. It's good, but it's not like there's nothing special there. That's why I go back to like, it's the cult of the founders. **Justin Jackson:** Yeah. **Jason Cohen:** Right. Like, I don't know what else there is that would make it succeed. Now, with this cult of founders, could it succeed now? --- ### [Signup To Cancel Feels Like A Series Of Miracles](https://share.snipd.com/snip/b7481423-fa36-487a-9cd7-2a24390dfade) 🎧 43:05 - 45:54 (02:48) <iframe src="https://share.snipd.com/embed/obsidian-player/snip/0f6841be-a6df-4ec6-b52b-2bc1825f1dd8" width="100%" height="100" style="border: none; border-radius: 12px;" sandbox="allow-scripts allow-same-origin allow-forms allow-popups allow-clipboard-write" ></iframe> - Purchases are a series of improbable events; cancellations after purchase often feel like a cascade of tiny failures. - Jason calls signups-to-cancellations a 'series of miracles' and uses it to motivate digging into onboarding and retention. #### 💬 Quote > Most ad viewers don't click. Most of those don't click past the landing page. Most of those don't consider. Most of those don't buy. It's the rare wins that keep us going. A series of miracles. > — Jason Cohen Jason Cohen on how unlikely conversions and subsequent cancellations feel #### 📚 Transcript **Jason Cohen:** longer version of this is how I kick off one of the chapters of the book which is a chapter about cancellation because i have this i've always had this feeling and i you know try to put into words of like how impossible it is that anyone would buy the damn thing like first you have to see your ad which is already like what one out of maybe of one percent of people see the ad that you expose it to that's pretty good just order of magnitude maybe it's 0.1 i don't know and so then they clicked it well let's click through rates one percent that's good okay **Justin Jackson:** then **Jason Cohen:** they hit the home page and then what 70 bounce 90 depends on the source. Okay. Yeah. That means they spent more than like three seconds, right. And then they read it and they're like, Oh, interesting. Really? Thank God. You know, you're the first one today. Um, you know, and then they, they looked around and they, they didn't, they didn't get bored and they looked, the features are like, Ooh, really? Thank God. And then they, then they looked at the pricing page and they didn't just leave you know like oh that's reasonable or it's not too low and they think you're shitty or too high and they think you're you know out of their league and they look the tears they're like i can do this and then maybe they looked at competitors maybe they didn't but supposedly they did whatever they still decided to buy from you they got out of the credit card and it worked i mean that's already something you know there's a fall off there yeah a series of miracles a series of miracles and then they got in and then what they onboarded they typed stuff in they uploaded a photo of themselves the seven millionth time this week like they dedicated all this time out of their life to the yeah maybe they call tech support to get unfucked and and like that then they and also with if it's at work i'm usually i'm usually thinking of a b2b context everything i just said is absolutely true and b2c i i'm just i just uh am expert in b2b and not b2c so i always think that way and so anyway in a business context they probably told their peers like oh you need to check out the software. I checked this out. It's really cool. Like they've started putting their own reputation on the line. And then after all of that, they cancel. And you're like, what? That's what lightning in a bottle. And then you're like, nah, like what? Like you have to go just emotionally, like forget financially, emotionally, you have to find out what is going on because they bought your argument. They bought your thesis. They bought your promise. They have the budget. It too expensive. They chose you over a competitor. They invested all this time. And they said, no, what did you do wrong? This is totally on you now, right? Now, maybe they were confused. Okay, but I'm being extreme a little bit, but to make the point. So then, of course, I go on to talk about there is a financial aspect that's very important as well. --- ### [Optimize Top-Of-Funnel Quality First](https://share.snipd.com/snip/06895dc0-a3d3-41d4-9d7f-828d834d32f6) 🎧 50:53 - 52:37 (01:43) <iframe src="https://share.snipd.com/embed/obsidian-player/snip/10c18d0e-b35f-4138-bb41-c44da2c89355" width="100%" height="100" style="border: none; border-radius: 12px;" sandbox="allow-scripts allow-same-origin allow-forms allow-popups allow-clipboard-write" ></iframe> - Prioritize the quality of who enters the top of your funnel over optimizing downstream steps. - Aim for fewer, higher-quality clicks that match your ICP because better traffic outweighs marginal landing page AB tests. #### 💬 Quote > The quality of who enters the top of the funnel is more important than every other step combined. You'd rather have fewer clicks. If the quality is great, the rest of it doesn't matter. > — Jason Cohen Jason Cohen on funnel strategy and the importance of ICP #### 📚 Transcript **Jason Cohen:** So it's exactly what, actually, what and Jordan were getting at. Because Brian was talking about ICP, and Jordan was talking about looking through the, **Justin Jackson:** watching **Jason Cohen:** people and saying, wait a minute, there is stuff I could do here. Consider this. Let's say in your funnel, I put nothing but the perfect best customers, like your ICP only is coming to the top of the funnel on the website. Even if your copy is just okay, but it just is talking to them essentially, like you're going be you're gonna be okay people will still buy because they want it so bad right and if your sales team if there is sales uh is kind of bad it's okay because like they kind of wanted to buy anyway right prevent the sale in other words even if everything even if you're bad at everything if i only put icps in the top you'll be okay. Now the opposite. I put nothing but garbage traffic, **Justin Jackson:** click **Jason Cohen:** fraud, whatever. Now I don't care how good your funnels are and your testing, your sales team, you're not going to get a lot of customers. What that tells you is the most important thing in the entire funnel is the quality entering the top of the funnel. **Justin Jackson:** We **Jason Cohen:** just said, if the quality is great, it doesn't really matter. The rest of it doesn't matter. If the quality is bad, the rest of it doesn't matter. So of course I'm exaggerating when I say the rest doesn't matter. Of course it does, but I'm exaggerating to make the point that the first thing entering, not even a step in the funnel, the stuff entering the funnel, the quality of that is more important than every other step combined. Once you understand that, you realize what Brian said. Go get your ICP because that's what it means to have quality at the top. You'd rather have fewer clicks. --- ### [Prevent A/B Tests From Optimizing Garbage Traffic](https://share.snipd.com/snip/ad586541-a399-4189-ace1-888ee44aadb3) 🎧 54:23 - 55:46 (01:23) <iframe src="https://share.snipd.com/embed/obsidian-player/snip/f8da2bdd-e59d-483c-8573-192349f79cda" width="100%" height="100" style="border: none; border-radius: 12px;" sandbox="allow-scripts allow-same-origin allow-forms allow-popups allow-clipboard-write" ></iframe> - Beware AB tests that optimize for low-quality traffic; they can improve conversion metrics while worsening overall customer economics. - Segment high-quality sources separately and measure conversion to end outcomes, not just next-click rates. #### 💬 Quote > If 90% of the traffic is garbage, the only way to up that conversion rate is to convert the garbage. What you do is you say I'm trying to up the conversion rate. That's what the A/B test will do. Just what you don't want. > — Jason Cohen Jason Cohen warning about misleading A/B test effects when traffic quality is poor #### 📚 Transcript **Jason Cohen:** That's not how we normally think, but that would be how you took this information and apply it properly. Or another thing would be how you segment, you know, in whether it's Google ads or Facebook ads or, okay, social media, other things, like how you segment the audience is really important. And again, it's better to have fewer leads that are more poignant. So here's another thing. If you don't do this, then your A-B testing in your website also optimizes for garbage because most of the traffic to your site is garbage. **Justin Jackson:** Yes. **Jason Cohen:** And what do you do when you optimize a step in the funnel, like optimizing the pricing page conversion or optimize the homepage, even a landing page click to just go on to the next page. What you do is you say, I'm trying to up the conversion rate. Right. But if 90% of the traffic is garbage, the only way to up that conversion rate in a material sense is to convert the garbage. If **Justin Jackson:** you convert that 10% **Jason Cohen:** and you grow from 10% to 11%, that's actually what you want to do. That's a 10% improvement. That's huge. But you'll never even be able to measure that. The only thing you can measure is if the shit traffic that you don't want is nevertheless fooled to continue. Then the rate of that step goes up. So that's what the AV test will do. Just what you don't want. Again, this is assuming 90% is garbage, which again, comes back to the top of the funnel. --- ### [Measure Conversion Deeper Into The Funnel](https://share.snipd.com/snip/ee4ca4f6-1d49-4dff-a588-addd161dcfce) 🎧 56:31 - 57:14 (00:42) <iframe src="https://share.snipd.com/embed/obsidian-player/snip/3d1161e1-4a65-46f1-8fd5-f76ce9ffb778" width="100%" height="100" style="border: none; border-radius: 12px;" sandbox="allow-scripts allow-same-origin allow-forms allow-popups allow-clipboard-write" ></iframe> - Measure deeper into the funnel (to purchase or retention) whenever possible to align experiments with business outcomes. - For simple products, track end-to-end conversion so tests reflect true impact, not intermediate clicks. #### 💬 Quote > The deeper you can measure into something useful, then it does work because you can say no, I'm not converting just the next step of the funnel. I'm measuring my conversion rate to the end of the funnel. > — Jason Cohen Jason Cohen on choosing better funnel metrics #### 📚 Transcript **Jason Cohen:** That helps. Another one is if you can measure end-to more. Part of the fallacy here is measuring just one step, this landing page to this next click. That's too myopic. If you can measure all the way to the sale, which for simple products, maybe you can, for complex ones, you cannot, but let's just say the deeper you can measure into something useful, then it does work because you can say, no, I'm not converting just the next step of the funnel. I'm measuring my conversion rate to the end of the funnel. If that goes up a little bit, yes, now you've done the right thing. Hooray. So expanding the, what do you mean convert? Convert to what? The deeper you can convert to what? The more now you really are measuring the thing you wanted to measure all along. --- ### [Expect Slow Feedback And Use Multiple Metrics](https://share.snipd.com/snip/2c2d8e70-35bd-4f14-b607-d96497da4f99) 🎧 01:01:59 - 01:06:31 (04:31) <iframe src="https://share.snipd.com/embed/obsidian-player/snip/efa693f9-9471-4c1f-b4fb-bf0d69901ad2" width="100%" height="100" style="border: none; border-radius: 12px;" sandbox="allow-scripts allow-same-origin allow-forms allow-popups allow-clipboard-write" ></iframe> - Long feedback cycles create an 'uncertainty principle' in funnel metrics: end-to-end outcomes can take months, so short-term signals are noisy. - Jason recommends multiple complementary metrics and a clear vision to guide action despite slow feedback. #### 💬 Quote > There are these... analogies. One is in the stuff like data and funnels. You might say it will take you a year to find out whether a new marketing source creates people who cancel more often than other people. You have to wait. > — Jason Cohen Jason Cohen on metric timelines and the difficulty of end-to-end learning #### 📚 Transcript **Jason Cohen:** here's a, here's a, okay, let's do another physics example. There's, there's, there's the like uncertainty principle. If the more you know about, you know, position, the less you know about momentum and vice versa, or the less you can be precise about. So to me, there are these, in complex systems, there's, again, not literally, but figurative analogies of this. And one of them is in the stuff like data and funnels like you're talking about. So you might say, everyone in the company would have the end-to funnel view. And although they may be working on or they may have control over one set of mechanisms within that system, which is obviously true, they nevertheless are all optimizing across the whole system. That's what you would like. Because of course, you want maximum throughput or equality, you might say, for the whole system. And then you have people who are specialized in different areas working on it. That's what you want. But you have this uncertainty problem, which is anything that's end-to has all these characteristics that make it very hard to operate. Like it will take you a year to find out whether a new marketing source creates people who cancel more often than other people. And you can't speed that up because whether they cancel in six months, take six months. Like there's nothing you can do to accelerate the pace at which you can find out whether someone's going to cancel in six months. You have to wait. And also like, if it's a brand new channel, you probably need to give them chances to optimize a little bit. Like you can't just declare, you know, so that's why I say it might take a year and there's nothing you can do. You can be smarter and less smart. You cannot make it faster than that. Well, people can't operate on a cycle, on a feedback cycle of a year. They have to operate, they have to have something faster, but those things that are faster are probably more localized and don't tell you the full picture. And so this is the uncertainty principle of metrics and timelines, you might say, right? So to me, it's like, okay, what do we do with that fact? And timeline is only one of many things to me about metrics that you might observe. to me, it okay, that means there's not one true metric. That's a stupid idea. We just explain why that doesn't work. So you need different metrics for different things, different goals or operational or whatever. You need different ones. that together tell me all the stuff I need to know. And we cannot think that a given metric will tell us things other than what it actually tells us. None of them will be the totality of what we want. And somehow a synthesis of all this is how we have to operate, even though we don't like it. Just like, sorry, you're just never going to know where the electron is and which direction it's going ever. But that doesn't mean you can't have some measurements and do some stuff and that's what we're going to do and so still i would come back i would i would still it's still useful to me to do what you said which i think is right and come back and like all right everyone let's keep our eye on the on the prize the prize is overall optimization although we can't run our day-to based on that comma, it is the, like, this is why I like vision statements. Vision statement is something like, you would like that to be the case. You're trying to do that. You may never even reach it ever, but like, at least it tells us what we what we are trying to do if we could. And that is still useful because it gives us like that. Anything that's worked toward that is correct. Anything that's like to the side or away from it is not right. So I would use your statement as a vision statement rather than a mandate, an operational mandate, which is we're trying to make it so that everything that comes in the top of the funnel is perfect and people stay forever. And we're all trying to do that. We're trying to optimize for the whole funnel. Now, since none of us can day to day measure and do that, we will need things that as best we can tie into that vision and we'll need to check in. So I'll, because we can measure that vision over a span of months. So we will, it's just that we know that like, if make a change today, we're not going to see that number change immediately. And even in six months, we may not because it's multi-factor. So many different other things go into it. We still won't know. We just have to acknowledge that's true. But we can still measure the overall vision, which is the overall funnel thing, like you said. That we're still going to measure because that's what we're trying to do. We will have to use our wits and our yes, our intuition and where we can data. And sometimes we'll think of a slightly better metric that still doesn't solve the whole equation, but like this one's better than before. --- ### [How A Dumb Survey Improved WP Engine's Data Team](https://share.snipd.com/snip/2fc63029-13d5-4b90-b551-bb0aa32eae8c) 🎧 01:09:38 - 01:14:24 (04:45) <iframe src="https://share.snipd.com/embed/obsidian-player/snip/145c0efa-9268-49c9-88e7-628bec04feaa" width="100%" height="100" style="border: none; border-radius: 12px;" sandbox="allow-scripts allow-same-origin allow-forms allow-popups allow-clipboard-write" ></iframe> - Jason used a simple monthly survey to show WP Engine's internal data team how customers perceived their work. - The survey revealed more positive sentiment than expected, improved feedback tone, and prompted better meetings and explanations. #### 💬 Quote > They had no metrics and no conception about their customer. I said just take a step back... What would it look like if we were crushing it for our customers? We could give them a really dumb survey. The scores came up over time. > — Jason Cohen Jason Cohen describing using a basic survey to align an internal team's work with customer needs #### 📚 Transcript **Jason Cohen:** Anyway, so I took what's in there, quite literally sent the article, like just as an example with the data team, which meaning the internal corporate data team at WP Engine. And, um, they were, they were sort of rethinking how to do all their work and what tools they're using and what metrics they should use. So we used exactly this article. Um, in fact, I think the article was already published at the time. So it was literally using this And it was like, it was interesting. All the metrics on the data team were things like, are we syncing from the source to that source? Is this thing broken? How would we know that? What are these timeframes? And that's all true. Those are all good operational metrics for the team itself of like, they, is their own work and own systems doing whatever the team wanted it to do? Right? Very team centric. But to your point, like those are the easier ones because it's your stuff. What about the customer? Now, in the case of the data team, the customer is the rest of the company, especially the other data analysts in the rest of the company, but even other people too. That's their customer. And they had no metrics and no conception about their customer. So I would say things – so they would do things like, oh, so-and in this other department is mad at us because, I don't know, we didn't work on this thing that they need. But it was really urgent. We didn't have the time. So in other words, they're talking about normal product development and client work. But since it's all within the company, we can do whatever we want with it, which is a nice kind of microcosm. And so I was like, well, so we were talking about this, like, don't even know what metrics we would use to say, like, do we deliver something? Because like, we may or may not be able to do any one task in any one sprint. So like that, I don't even know what a metric would be. And I said, okay, well, just take a step back and ask, and this gets back to the vision. Just if you imagine, like, if just nevermind our tech and how we do it, but like our customers, all these people in the company, what would it look like if we were like just the best ever and just crushing it for our customers? What would that look like? This is, of course, what the vision statement is. It's describe what the future looks like if we're crushing it, right? Yeah, yeah. What we look like if we're crushing it. And're like well i don't know they get back to the numbers i'm like no no not numbers just like without numbers what what what and just like they would just i don't know like i guess if you just talk to any of them they would be happy with us and and then like yeah i agree like if they all said it's awesome working with the data team then that would be success like yeah but we don't know what's awesome. We don't like, oh, I know, I know, I know they may not even know, but you just said it. You just put a vision statement that, that you want everyone to. And so how could we start to in fact measure it very poorly, like intentionally measure it poor. Like I know it's hard. Like how would we measure this even poorly? Well, we could give them like a really dumb survey, really basic survey, not NPS, of course, because they're not referring us to a friend. Anyway, I hate NPS, but just a basic survey about how they feel with it, like a fill, like a big fill in box so they can just type shit. And, um, we could just send that every month to 20 of our so-called customers. **Brian Casel:** I love how like so many of your ideas start with like, like just start with the ideal. Let's work backwards from that. What needs to be true and **Jason Cohen:** what needs to be true. So they did. And the first thing that was interesting is it wasn't as bad as they thought, which is, which makes sense because like if you deliver data to another person and it's just what they want, often what happens is nothing. They just use it. Right. So you have no positive feedback. Occasionally you do. Usually people are like, Oh, thanks for that. Right. But a lot of times if you're doing your job well in data or security, if you do your job well, you're invisible. Then that's a good thing. Like, Ray, I didn't think about my website performance today. Right. That's good. So that, but that means you're not getting the positive feedback. So when you do get feedback is when people are upset and they're like, ah, website down, whatever. And you're like, ah, so that's all you see. So your perception is there's problems all the time because there are problems all the time. It's just that you're not seeing how that's, how that's counterbalanced by, by when there's not. So first of all, it wasn't as bad, which was a morale boost because they kind of felt like maybe people were upset with them. And it was like, oh, no, they're actually fundamentally not upset with us. They just want certain things. Also, just the fact that they were curious about whether their customers were happy or not made their customers like go, oh, oh, that's good. And all of a sudden they were more constructive in their feedback. Oh, you actually want to improve? Oh, well, you know, it's easier to be nicer in that case, more understanding. And if you think that maybe your feedback is not being heard, even if it is being heard, you think it's not. So the whole dynamics changed to be better right off the bat. And then, of course, they did look at this and they did blah, blah, blah. --- ### [Use Metrics To Inform Not Decide](https://share.snipd.com/snip/f70ae6cf-d825-42e4-a36d-544ffe1ddf93) 🎧 01:17:27 - 01:18:01 (00:33) <iframe src="https://share.snipd.com/embed/obsidian-player/snip/7ee1a64e-5b1b-4a9c-a5b6-df572795a7bd" width="100%" height="100" style="border: none; border-radius: 12px;" sandbox="allow-scripts allow-same-origin allow-forms allow-popups allow-clipboard-write" ></iframe> - Treat metrics as informative inputs, not definitive answers; avoid incentivizing single metrics that encourage gaming. - Use metrics to guide conversation and behavior, not as absolute proof or sole basis for rewards. #### 💬 Quote > Metrics should be informative, but never decisive. People bringing you the metrics should not present them as definitive. Despite these metrics, we think, and that's good. > — Jason Cohen Jason Cohen on healthy metrics usage and avoiding metric overreach #### 📚 Transcript **Jason Cohen:** Like, is the data collected even right? We're not sure. Are we interpreting it correctly? We're not sure. We're using stats. Are those the right stats? We're not sure. Is this the full picture? You know, certainly not. It is certainly not the whole picture. We know that definitely. We just don't know exactly how much. People want it to be definitive. They want it to be accurate. Yeah, And the people bringing you the metrics are also should not be bringing you those metrics as if that's the definitive they should say. And yet, despite these metrics, we think, and that's good. So metrics should be informative, but never decisive. ---