The "Confident Idiot" Problem  ## Metadata - Author: [[Changelog Master Feed]] - Full Title: The "Confident Idiot" Problem - Category: #podcasts - URL: https://share.snipd.com/episode/7c8d495f-69ce-476a-8d2c-f74bd6e1043a ## Document Notes ## Highlights - **LLM Checks Create Dangerous Circularity** - Treat LLMs as software with failure modes, notacles of pure judgment oracles. - Relying on one model to judge another creates circular sycophancy and won't fix hallucinations. Transcript: Jerod Santo The confident idiot problem, or why AI needs hard rules, not vibe checks. If you've been following the how do we actually use AI in production conversation stream, you've probably heard people propose a strategy where one LLM checks another LLM's results. But will that work? Quote, we are told to ask GPT-40 to grade GPT-3 We are told to fix the vibes, but this creates a dangerous circular dependency. If the underlying models suffer from sycophancy, which is agreeing with the user, or hallucination, a judge model often hallucinates a passing grade. We are trying to fix probability with more probability. That is a losing game. End quote. One possible way of dealing with these confident idiots we've introduced into our software stacks the last few years is to stop treating agents like magic boxes and start treating Them like software. Hence, the Steer SDK was created. Quote, Steer is an open source Python library that intercepts agent failures such as hallucinations, bad JSON, PII leaks, et cetera, and allows you to inject fixes via a local dashboard Without changing your code. ([Time 0:00:34](https://share.snipd.com/snip/99913c8d-dca4-4269-bf3b-0ad9ebef02c0)) - **Anthropic Joins Forces With Bun Team** - Anthropic hired the Bun team rather than just forking Bun's open source runtime. - Jerod suggests the Bun team's expertise was the real asset, not the code alone. Transcript: Jerod Santo Bunn is joining Anthropic. The company behind Bunn, which is the open source runtime for CloudCode, is joining Anthropic. We discussed the big acquisition slash aqua hire on last week's Friends episode, but at the time, I hadn't quite considered this move and how contrary it is to Anthropik's party line That AI agents are replacing software engineers. From Anthropik's announcement, quote, we've been a close partner of Bun for many months. Our collaboration has been central to the rapid execution of the CloudCode team, and it directly drove the recent launch of CloudCode's native installer. We know the Bun team is building from the same vantage point that we do at Anthropic, with a focus on rethinking the developer experience and building innovative, useful products. End quote. Bun is open source. Why not just fork it and have a Cloud Code-powered engineer make all the necessary changes and upgrades to the runtime that Anthropic needs. Perhaps because there's no getting there from here. At least not yet. Jared Sumner and the Bun team's expertise is what's so valuable. ([Time 0:01:55](https://share.snipd.com/snip/89269aab-217d-4783-ade8-142817ad8744)) - **Claude Fails Recreating Space Jam Site** - Jonah Glover tried to recreate the 1996 Space Jam site with Claude and it repeatedly failed. - Jerod relates the failures to common hallucination and anchoring problems when using current front-end AI tools. Transcript: Jerod Santo Claude can't recreate classic Space Jam site. Jonah Glover tried to recreate everyone's favorite 1996 website by giving Claude Code, which is running Opus 4.1, a screenshot of the site and all the associated assets. It failed repeatedly in all the ways I would expect from my own front-end and design attempts with the tool. Jonah's finding, which is quite relatable, quote, once Claude's version existed, every grid overlay, every comparison step, every precise adjustment was anchored to his layout, Not the real one. At the end of all of this, I'm left with the irritating fact that, like many engineers, he's wrong and he thinks he's right. What this teaches me is that Claude is actually kind of a liar, or at least Claude is confused. However, for the drama, I'll assume Claude is a liar, end quote. I've been giving Claude Code a lot of props lately, but I've also been giving it a lot of tasks that it can't quite accomplish. This process starts off as fun and interesting, but each time it ends in failure, I am perplexed by all the possible failure paths. Was it me and my prompting? Was it the agent? Was it the model? Or perhaps I'm asking for things that aren't easily accomplished with today's tech. I can be quite demanding. This all makes me yearn for the days when the only one to blame for my failures was me. ([Time 0:03:00](https://share.snipd.com/snip/c0f56aee-d9ab-47dc-909b-c8445207389d)) - **Chrome Restores JPEG XL Support** - Chromium reversed its previous decision and re-enabled JPEG XL support in Blink. - Jerod predicts wide adoption given Chrome's market position. Transcript: Jerod Santo Google unkills JPEG XL. Quote, in a dramatic turn of events, the Chromium team has reversed its obsolete tag and has decided to support the format in Blink, which is the engine behind Chrome, Chromium, and Edge. Given Chrome's position in the browser market share, I predict the format will become a de facto standard for images in the near future. ([Time 0:05:25](https://share.snipd.com/snip/a6324348-8a7d-4176-9b32-9de5188027b0)) - **Drive Linux Adoption With Game-First Distros** - Focus on gaming-first distros to grow Linux desktop adoption. - Jerod highlights Bazite's tweaks like Steam preinstalled, HDR/VRR, and scheduler improvements to improve gaming on Linux. Transcript: Jerod Santo If the mythical year of the Linux desktop is ever to materialize, it will first be preceded by a sea change in gaming options for the venerable open source OS. The gaming sea change appears to be in full swing with Steam on Linux hitting an all-time high of over 3% usage last month. Enter Bazite, a Fedora-based Linux distro that's hyper-focused on making gaming awesome. Quote, Bazite is designed for Linux newcomers and enthusiasts alike with Steam pre-installed, HDR and VRR support, improved CPU schedulers for responsive gameplay, and numerous Community-developed tools and tweaks to streamline your gaming and streaming experience. End quote. The project began back in 2023, but it appears to be maturing and aiming at sustainability by setting up ways to donate with its latest update. Quote, as Bazite matures, we begin to tackle more ambitious projects such as proper secure boot, support for more handheld devices, and conference attendance, which means more costs For us. And we would gladly appreciate the help in covering them. ([Time 0:06:04](https://share.snipd.com/snip/589b9d99-b7bc-4c52-ba82-ce4ed7e95010))