Why AI Coding Claims Don't Add Up  ## Metadata - Author: [[The Changelog: Software Development, Open Source]] - Full Title: Why AI Coding Claims Don't Add Up - Category: #podcasts - URL: https://share.snipd.com/episode/7b235afe-c522-491d-8f3b-807318117bcd ## Document Notes ## Highlights - **AI Coding Gains Are Overstated** - Mike Judge found no statistically significant productivity gains from AI coding in his six-week self-test. - He concluded AI tools don't produce the claimed 2x speedups and may not meaningfully increase output. Transcript: Jerod Santo Why AI coding claims don't add up. Mike Judge, no, not that Mike Judge, has been coding for 25 plus years. He was an early adopter of AI coding and a fan until a couple months ago. Now he's mad, furious even. Quote, I read the meter study and suddenly got serious doubts. In that study, the authors discovered that developers were unreliable narrators of their own productivity. They thought AI was making them 20% faster, but it was actually making them 19% slower. This shocked me because I had just told someone the week before that I thought AI was only making me about 25% faster, and I was bummed it wasn't a higher number. I was only off by 5% from the developer's own incorrect estimates. End quote. That study left him unsettled, so he put himself to the test. For six weeks, he tested his own productivity with and without AI. What he found was really disappointing. Quote, I discovered that the data isn't statistically significant at any meaningful level. That I would need to record new data points for another four months just to prove if AI was speeding me up or slowing me down at all. I can say definitively that I'm not seeing any massive increase in speed, as in 2x, using AI coding tools, end quote. ([Time 0:00:44](https://share.snipd.com/snip/ac394d84-e0a9-43e9-8968-46a82fdf254e)) - **Perception Versus Reality Of AI Productivity** - Developers often misremember how much AI helps them, believing in speedups that studies don't confirm. - The lack of an indie software boom questions widespread claims that AI dramatically boosts developer output. Transcript: Jerod Santo In that study, the authors discovered that developers were unreliable narrators of their own productivity. They thought AI was making them 20% faster, but it was actually making them 19% slower. This shocked me because I had just told someone the week before that I thought AI was only making me about 25% faster, and I was bummed it wasn't a higher number. I was only off by 5% from the developer's own incorrect estimates. End quote. That study left him unsettled, so he put himself to the test. For six weeks, he tested his own productivity with and without AI. What he found was really disappointing. Quote, I discovered that the data isn't statistically significant at any meaningful level. That I would need to record new data points for another four months just to prove if AI was speeding me up or slowing me down at all. I can say definitively that I'm not seeing any massive increase in speed, as in 2x, using AI coding tools, end quote. That got Mike thinking, is he the only one or are we all delusional? To answer that, he asked a simple question. Quote, if so many developers are so extraordinarily productive using these tools, where is the flood of shovelware? We should be seeing apps of all shapes and sizes, video games, new websites, mobile apps, software as a service apps. We should be drowning in choice. We should be in the middle of an indie software revolution. ([Time 0:01:04](https://share.snipd.com/snip/cf3e39ca-9bed-47dc-aa35-ea827b48fa2f)) - **Use Cactoid For Low-Friction RSVPs** - Try Cactoid to create events quickly without forcing account sign-ups or waits. - Use the unique URL RSVP flow for simple, low-friction event coordination. Transcript: Jerod Santo Click through for receipts. Cactoid wants to be the ultimate RSVP platform. I've been prompting the open source community kill meat up for years. It looks like the folks behind Cactoid are taking a crack at it. Quote, like the cactus, great events bloom under any condition when managed with care. Cactoid helps you streamline RSVPs, simplify coordination, and keep every detail efficient. So your gatherings are resilient, vibrant, and unforgettable. End quote. They've started off simply, which is great. You can create an event in seconds, get a unique URL to share, and people can use that link to RSVP to the event. No accounts, no waiting. Is that too simple? Perhaps yes, especially if it gets popular, which always brings abuse. But for now, it's refreshing. Go ahead, create an event and share it with friends. ([Time 0:02:39](https://share.snipd.com/snip/f025eae6-007a-48e6-b059-0e616270c7ff)) - **How RSS Beat Microsoft’s ICE** - Ryan Farley recounts how ICE, a Microsoft-led spec, lost to simple open RSS. - He highlights that simplicity and openness let RSS survive despite bigger corporate-backed rivals. Transcript: Jerod Santo Here's Ryan Farley writing for ButtonDown. Quote, not many people talk about how or why RSS won the content syndication war because few people are aware that a war ever took place. Everyone was so fixated on the drama over RSS's competing standard, Atom vs. RSS 2.0, that they barely registered the rise and fall of the Information and Content Exchange, ICE, specification, which had been created, funded, and eventually abandoned by Microsoft, Adobe, CNET, and other household names. End quote. I had never even heard of the ICE specification, which predated RSS by almost exactly a year and was philosophically almost a complete opposite approach. Thankfully, simple and open beat out complex and closed. Eventually, Google killed RSS by shuttering Google Reader, but that's not the whole story either. Quote, all RSS had to do to weather ice, Twitter, AI, and whatever comes next was keep things simple and let users build their own feeds, filters, lists, and aggregators. Like email, it probably won't make anyone a billion dollars or reshape entire industries, but it will always be wholly yours. ([Time 0:03:36](https://share.snipd.com/snip/25db9043-492e-4374-b1d5-2d52edb23b90)) - **Switch To Podman To Avoid Docker Daemon Risks** - Consider replacing Docker with Podman if you worry about a long-running daemon and security. - Podman offers daemon-free operation, better system de-integration, and smoother Kubernetes alignment. Transcript: Jerod Santo Ditching Docker for Podman. Dominic Cizamansky is old enough to remember when Vagrant looked like a promised land where every development environment would look the same. Then came Docker. Quote, Docker wasn't just a tool. It fundamentally changed how we thought about application development and deployment. Having a repeatable, separated environment from your local system was refreshing and looked like a superpower, end quote. After many years of Docker use, Dominic's thoughts on Docker began to change, quote. Along the way, the quiet Docker daemon running in the background felt less like a comfortable constant and more like a ticking bomb, end quote. The long-running daemon, Dominic believes, is Docker's security downfall, so he went looking for alternatives, enter Podman. Quote, beyond the obvious daemon advantages, Podman brings some genuinely clever features that make day-to container work more pleasant, such as system de-integration that doesn't Suck, Kubernetes alignment that's not just marketing, and the Unix philosophy done right, end quote. If that's enough to get your attention, I have good news. ([Time 0:05:50](https://share.snipd.com/snip/fdd6dda4-e281-42d7-913f-3bdef01765e6)) - **Tempo Is A Payments-Focused Blockchain** - Stripe built Tempo as a payments-first Layer 1 blockchain focused on stablecoins and high-volume transfers. - Tempo aims for EVM compatibility but targets payment throughput rather than general-purpose decentralization. Transcript: Jerod Santo Developed in partnership with leading fintechs and Fortune 500s, Tempo is a new payments-focused blockchain that supports all major stablecoins. But why? Quote, stablecoins enable instant, borderless, programmable transactions, but current blockchain infrastructure isn't designed for them. Existing systems are either fully general or trading-focused. Tempo is a blockchain designed and built for real world payments. Tempo will be EVM compatible, but it isn't meant to displace other general purpose blockchains like Ethereum, Solana, etc. Because it's entirely focused on high volume payment use cases. That being said, high volume payment use cases that route around the banks and credit card companies stranglehold on transfer fees are, so far at least, the killer app of general purpose Blockchains. We'll have to wait and see how this plays out. Tempo has been announced, but it's not yet public. There is an invite only testnet at the moment and it will be validated by a diverse group of independent entities at launch. ([Time 0:07:05](https://share.snipd.com/snip/b9b8a79d-fce3-46c9-8648-09cfc52b04c3))