1002: The Real Pricing of LLMs  🔗 **Source:** [1002: The Real Pricing of LLMs](https://share.snipd.com/episode/47ae25ae-a9b8-47e7-b9c4-f300b06924fd) ## Highlights - **The Free Lunch For LLMs Is Ending** - LLM usage is becoming dramatically more expensive as providers move from flat subscriptions to token-based billing. - Wes and Scott note GitHub multipliers make some models effectively 27× costlier, causing heavy subsidized usage to end. Transcript: Wes Bos So if you didn't see this, you probably got an email about it. But GitHub sent out an email saying, like, we're moving to usage-based billing rather than these, like, all-you subscriptions. And I think we're starting to see this not just with GitHub Copilot, but we're seeing it with a lot of people is they didn't, nobody realized how much we are going to be using these models, Like initially it was like you just type you have tab completion or you type in the box and you get an answer back and then you copy paste and apply it to thing and now people are just like Agent maxing running like 30 at a time you have all these like sub things going you're using it to simply people are just using it to simply like ask it to get commit, you know, like unreal Usage that people are using. Yeah. And unfortunately, change the background, right? Scott Tolinski Yeah. Yeah. Wes Bos Change the background. But unfortunately, this stuff is is very, very expensive. And for the last, I don't know, year, two years, whatever, we've been we've been getting heavily subsidized ai models for us and i think they're sort of looking at how much they're losing On these is not just github but like anthropic had they were testing removing cloud code from the 20 plan they're really tightening down how much you get on a 200 plan and and that's not Because they're being greedy it's simply because they're probably bleeding money. They're probably bleeding thousands and thousands of dollars on single people every single day. And the VCs are sort of picking up the bill for a lot of the stuff right now. But eventually things are are going to have to be paid for. And I know that like, I don't know, what do you think the computer? Do you think that the stuff will get cheaper eventually? You know, like computers always get better and cheaper? Yeah. Scott Tolinski Yeah. I mean, you'd think so, but it is interesting to see, you know, you know, what is it? The it's not necessarily you could call it a rug pull of sorts, but like I think any Anybody who's been paying attention could have seen this coming, given the progressive movement of All of these companies moving in this direction. And I remember people talking last year saying, you know, the $200 a month quad coat max plan and codex plans that exist like that as well. Like those aren't going to be here forever this is like i i saw people say spend your tokens now because they're going to cost a lot more in the future and i i mean i think that's really what We're seeing i don't know if i mean i'm yes obviously uh computing will improve uh we'll be able to use these things for cheaper but better models are going to continue to come out we saw Most recently with the latest Claude Opus 4.7 because of how they're tokenizing it's like using a ton more per general prompts and because of that it's not just that the bottle is more Expensive or whatever it's like the actual amount of tokens you're using is more so therefore you're going to be spending you're good yeah anybody who's worked in this stuff will see Like the transition into api-based billing and that api-based billing that usage-based billing yeah it gets expensive quickly. Yeah. Holy cow. Wes Bos Sometimes I run the numbers on the tokens that I've used in a single day and I'm like, that would have been like 300 bucks in a day. Which is crazy for what? Yeah. It's nuts. So GitHub released their what's called their multipliers, which is basically like if you're using these different models, here is how much faster you're going to be using it, right? And like cloud opus for six, the multiplier was three, the new one is 27. So they're saying like, this is essentially like multiple times more expensive than it actually was. ([Time 0:01:24](https://share.snipd.com/snip/d317b7f5-34f3-4308-afbb-4c6c6aa44d1e)) - **Small Specialized Models Will Replace One-Size-Fits-All** - Expect a shift toward many smaller, targeted models rather than one giant model for all tasks. - Scott predicts specialized models will handle narrow tasks (commit messages, tests) to reduce token costs and speed up workflows. Transcript: Scott Tolinski Here's what I think is gonna happen. I think we're all going to get better at using smaller models in more targeted workflows. One model does one thing very specifically. It costs way less. It needs to do way less. And we're going to move on from opening up one app and just being like can you please do this and commit this and then run my checks and do this like it because you don't need opus 4.7 to run Your test suite you don't need it to run get commit or create a commit message you can use much smaller models than that and i think we're going to start to see systems pop up that allow and Are based around the idea. ([Time 0:08:07](https://share.snipd.com/snip/63d9cecf-5e36-4f55-8923-9912ffcec7a8)) - **Recruiter Repo Nearly Exfiltrated Env Vars** - Adib Hanna nearly got hacked when a recruiter had him clone a repo that exfiltrated process.env to a remote server. - Wes describes the repo used a pre-commit/backdoor with an Axios call and base64 payload that logged environment variables. Transcript: Wes Bos Adib Hana, he posted this crazy story about how he almost got hacked by like a recruiter. So basically he says, I had an interview with a crypto recruiter. We talked about 40 for 40 minutes and they asked me to look at some code. Their first instruction was to clone the repo. I didn't. They seemed surprised. So I told them I and I waited a moment to check whether it was safe or not. I ran a quick analysis with Claude. Turns out the code had a back door that would have copied my environmental variables and sent them to the remote server. The recruiter was speechless and ended the call pretty quickly. Be careful. And then he goes on to like post the actual code of like what happened and simply by like cloning this thing. And I think it was like a pre-commit, but basically it was an Axios call that spread process.env, which I didn't know you could spread process.env. I thought you had to know the actual key for it. Like I know you can't use object.keys on process.env. Maybe you can. But yeah, it's crazy. There's a little base64 encoded string, which turns out to be like a Vercel hosted IP checker. Like it looks like it was just simply a call to check what your IP was to get the country and to come back. But really, it was just an app that was just like logging absolutely everything. ([Time 0:13:03](https://share.snipd.com/snip/88636d5c-f7ee-4898-b062-402dbc07e225)) - **Prioritize Results Over Chasing AI Hype** - Focus on the results you want, then learn only the tools that deliver those results instead of chasing every new AI feature. - Wes recommends using AI to solve problems you're excited about and warns against building tools-for-tools that fade fast. Transcript: Scott Tolinski I totally unplugged from coding. My only connection to the web dev world was listening to your podcast. Thank you. Now I'm getting back to work and it feels like the landscape has completely shifted. The whole AI agent space exploded while I was gone. Everyone seems to be talking about open code and building these ultra complex agent setups. But not just that. It feels like the whole ecosystem has moved forward a lot. From the first time in my life, I genuinely feel left behind. How do I get back to that cutting edge state quickly? What is your advice and the best way to catch up on these new workflows to figure out what's actually worth learning thanks and greetings from down on down this is like a whole thing i actually Just gave a keynote on this today about staying up to date with stuff in this world because as we as we march on and every single person that you know is creating an AI agent orchestrator Like everybody is like, oh, I'm going to have this really novel idea of creating an agent orchestrator, you know, something that's never been thought of. Like browsers are still shipping stuff like just yesterday. Firefox shipped the ability to use light dark function without. Oh, hell yeah. Because light dark was just colors and now you can use it with images too. Man, that feels like exciting to me. And it's so easy to lose track of all of that, progress that's happening in native APIs, because we're all built. We're all like exploring this whole new world of AI, and it's exciting. And I think at the end of the day, you know, you've got to like think about what it is that you're actually care about working on. Do you care about working on this AI stuff or do you care about building great experiences for the web? And you need to focus your time and energy on the tools and techniques that will get you to those results, rather than worrying about the tech, worry about the results. And I think that's something that people really lose in this stuff. And then they lose motivation, because it's so overwhelming, there's this new harness, or this new model, or this new thing. And it's like, what are we actually building? Are you building anything? Like sometimes I spend a whole week working on something that is just a tool as a tool for a tool rather than actually building anything useful. And like that, that sucks to feel that way. Like, wait a second, what am I actually building here? Right. What am I actually needing to focus on? So I would I would say focus on the results first and then work backwards from there. Wes Bos Yeah, I think I think that's really important is focus on using this new tech that we have to get you better results. A lot of people are simply just focusing on the harness and whatever that we actually use to write code. And the frustrating part about that is in two months, a lot of that is unnecessary because it's just been rolled into the models are just are good at it. One thing I often think about is that so many times there's like, oh, we should have a feature for X, Y, and Z, or there should be a button for this. Like I even went into the cursor chat and I was like, hey, like it would be nice to have a button for X, Y, and Z. I forget what it was. And their answer was just ask it to do that. And I was like, oh, damn it. Yeah, you're right. Like they don't even need to build that feature because you can simply just ask it to do that. So it's kind of frustrating that it moves so fast and what you're excited about one day is totally gone. And I agree with Scott where you should focus on how do you use this tech to build something that you are excited about or something that you previously couldn't do, something that has Interested you in a while. Like the other day, I found this little Budweiser cup at a thrift store, and it has like a little Bluetooth thing. It connects to your phone, and it lights up when your team scores. And it's like 12 years old or whatever. So I was like, I bet I can hack this. And I couldn't figure it out because it wasn't standard Bluetooth stuff. I've done quite a bit of web Bluetooth stuff, but I just couldn't crack it and couldn't figure out what commands to send to it. So I was like, why don't we try like decompiling the Android app? And then so I found it, I gave it to Opus, and it decompiled it. And I was like, that was that was really fun. Like that was I don't think I would have been able to do that without without, or at least in this amount of time. So just keep pushing into the different areas that you're excited about. ([Time 0:18:32](https://share.snipd.com/snip/f4cbec4c-5630-4854-b4f1-137e362c2b98)) - **Lean On Native Browser Primitives First** - Prefer native browser primitives (details/summary, customizable select, CSS anchor) when they cover your needs to avoid unnecessary JS. - Scott and Wes show a details/summary accordion is 4 lines versus a heavy Base UI component with many wrappers. Transcript: Wes Bos He says, I'm interested in your take on using React UI components versus native browser solutions. A specific example, base UI's accordion component versus HTML details summary. I'll throw in another example. It's just like dropdowns, customizable dropdowns. We now have that in customizable select in the browser, right? It's a select. You can make it look like however you want. I understand that the base UI version is fully battle-tested and covers all possible accessibility concerns, but to me it feels like adding accordion. Battle-tested. It's true, though. Sometimes there's edges with accordions. To me, it feels like I'm adding unnecessary JS overhead. I like leaning on native browser behavior when possible, and I generally assume native browser behavior would have no accessibility holes, but this may not be the case. And he goes on to sort of give one example. I think the native browser stuff is currently being slept on because like Scott said earlier, everybody is so hyped on all this AI stuff. People are not realizing that we have CSS Anchor and customizable select. And like we have, there's like, we've talked about this on the podcast every week. There's like 30 new things in the last year or so that absolutely nobody is using. You know, like I saw Apple.com using CSS Anchor and ScrollSnap for one of those things. I was like, this might be the first time I've seen like a big website actually using a lot of this new native stuff. And I think it's made worse by the LLMs not giving you those things by default. I have to explicitly ask and know I want to use CSS Anchor. I want to use customizable select. I want to use the details box. Don't just make your own version of it. You have to explicitly go and ask for all of those things. I think it will get much better once these types of things are sort of, they make their way into the component libraries. I don't think we're going to get rid of component libraries. Probably we won't have a need for some of the components here or there. But generally, I think what will happen is we'll get rid of a whole bunch of extra JavaScript. And then the shad CN implementation of a lot of these things is simply just going to be some HMO with some classes and maybe a little bit of JavaScript here and there to fill in different Interactions that you want. ([Time 0:25:11](https://share.snipd.com/snip/8fcfe8fb-3b30-40e0-8788-fcc03fe0b7c1)) - **CSS Linting Is Evolving To Tame AI Slop** - New CSS tooling is emerging to lint AI-generated slop and enforce sensible defaults. - They highlight Project Wallace and csskit.rs (Keith) as promising tools to analyze and lint CSS at scale. Transcript: Scott Tolinski Hey, webmaster of Project Wallace here. I love it, webmaster. Thanks for the shout out on episode 998. Just wanted to let you guys know that we are working on a big style lint plugin pack for exactly the reasons you guys mentioned on the podcast. Determ sensible defaults and more agent ready tools with wallace related tools now my question scott clint seems nice clint was my little uh css linting project i threw together have You looked into css kit.rs yet it aims to be a sort of oxc but for CSS. The maintainer is Keith. Keith, I'm so sorry. Is it Circle or Kirkle? C-I Sorry, Keith. He works on the Firefox browser. It uses the same CSS parser as Lightning CSS. He also wants to incorporate linting into CSS kit. Looks like you are both in the same boat. Anyways, wanted to let you know. I have not heard about this, Bart, and I really sincerely appreciate this. I ultimately don't want to be the one who makes this. So if, oh, this is, this is a cute little website. So if Keith is working on this, it looks like there is linting. CSS kit lint styles.css. Very similar based on Lightning CSS as well. Analyze for deep. Yes. Shout out to Bart. Shout out to Project Wallace, one of the coolest CSS projects out there. For those people who don't know, Project Wallace does a great job of analyzing your CSS in all kinds of ways, both as a CLI, but also as a cool little website. So, yeah. ([Time 0:32:47](https://share.snipd.com/snip/33d2c684-bfc3-486f-9ab0-435460fb3e15)) - **Use Slow Or Weak Models To Evaluate Interview Skills** - In interviews, give candidates a fast, lower-quality model and a real task so you can observe their problem-solving rather than copy-pasting. - Scott suggests using models like Composer 1 to force hands-on iteration and see their feedback loop. Transcript: Wes Bos There's a really good post by someone that said, use Composer 1 or 1.5 or use a model that is fast, but not very good. That like you you want to gauge how somebody thinks and tackles and approaches working on a code base with these tools right because like quite honestly whether you like it or not that's That's a skill in itself but you also don't want them just to simply just like copy paste the requirement paste it in the box and then sit there for while it generates right So with a faster, Lower power model, whether it's like Composer or Kimmy or something like that, you can get an idea of like people's feedback loops going back and forth. It doesn't write all of the code for you. So you'll have to kind of jump in there and do a lot of that type of stuff yourself. I think that that's such a good use case. Here's the model that you're allowed to use. Here's the thing that I want you to build. Um, now go at it. And I kind of want to see you working on this type of thing because I don't think you can anymore. ([Time 0:36:34](https://share.snipd.com/snip/68576144-b22e-4e3d-a6e6-d1257874de5c)) - **Pin Node And Package Manager Per Project With VitePlus** - Use tools that manage Node and package manager per-project rather than global nvm. - Wes recommends VitePlus to pin Node version and package manager per project and mentions corepack and 'n'/'fnm' as alternatives. Transcript: Wes Bos Queasy. Scott Tolinski I know it's a bad Australian. Okay. Do you have any best practice suggestions for how to manage your version dependent JavaScript development environments? In particular, I mean Node, NPM, PNPM, Bun, or any other tooling that comes with different versions. I thought we, the community, had settled on NVM, Node Version Manager, but that manages node together with npm while it looks like pnpm should be preferred these days all of my research So far has just left me puzzled with options and a little concrete solutions should i just globally yolo install node and pnpm and then fix problems as i might run into them that's kind Of what i do that's kind of, that's kind of what I do, Niels. I just fix them as they, they fall in front of me here. I deeply hope the answer is not dev containers piece. Yes. Oh man. I deeply hope that too, because all of that stuff just seems like a lot. So for me, what do I do? I, uh, before I, I, you know, it's pretty much PNPM for everything with me. And that's just how I manage things. NVM always just has like almost always given me grief at points where I'm like, I don't know what version is correct. And it's telling me it's not a cross, you know, just like little things where I get frustrated. Wes Bos I get this one error with NPM that it's like something regressed in this version of NPM and you have to run this like shmod command and that'll fix it. And like, it just comes back all the time. And it drives me nuts. Scott Tolinski Like what's happening here? Yeah, I know. So for me, but more recently, I have been just using Vite Plus for everything. But again, I haven't really gotten into Node version management. It's so funny because so many of my projects, I'm just like, whatever, use the latest version, use the latest LTS. And I don't have to worry about it because even though I have a lot of projects, I get to be in control of that. And I don't end up having that many Node version issues using older versions of Node. So I am not the right person. Wes, what do you do? Yes. Wes Bos So node has something built in for the package manager aspect of it. It's called core pack. And essentially in your package JSON, you define if you ever NPM something or PNPM install something, you'll notice that PNPM adds a little package manager property to your package.json. And CorePak is this idea that when you have a project, CorePak will figure out or knows that that project is using NPM, PNPM, or Yarn. I think those are the three that it supports. Scott Tolinski I remember there's a lot of controversy when that first. Wes Bos Yeah. Whether that should go in there. Yeah. So those are the big three, right? Core pack is is kind of a solution to that but quite honestly i don't use it but i do like it when projects ship the support for core pack because it's just like oh it uses yarn Figure it out. Install the version of yarn that you need and figure out the weird command that you need to run. And I don't want to have to, like anytime there's a project that's yarn, it's like just yarn start. I'm like, I'm dead. I don't, it's not going to work. But if you use core pack, it kind of just like does that for you, which is really nice. So there's that, but I'll quite honestly, I don't use core pack all that often. What I use is I use one called N, which is you just go to to npm.im forward slash N, you can find it. FNM is like the new Rust-based one that seems to everybody's using. We looked at that in the Stata.js episode. It seems like everyone's using that. But honestly, the VitePlus is where you want to be because VitePlus manages both your environment, so which version of Node do you want to use, and it manages your package manager, which Is great because you simply just need to type vpstart or vpdev, vprun, whatever, and it will actually run it for you. Or vpinstall, you don't have to think oh this project uses npm to install and like that's another weird thing is if you npm install on like a on a pmp package it like hangs forever you don't Have to think about any of that it simply just translates the command to whatever package manager that is using so unfortunately the answer to all of these different tools is just like One tool that like sits on top and and puppeteers all the things and then it also will change your node version as well but it will also change the node version just for a specific project Which is so good if you have like i often have like my own course thing which needs a specific version of node but then i want to try a lot of the cutting edge Node features. So I have to switch in between them as a global Node version. And that's annoying because it takes forever. It has to copy, paste, pseudo. It has to do all those things. So I've been absolutely loving VitePlus for this type of stuff. ([Time 0:41:41](https://share.snipd.com/snip/55dacd26-f6d3-47be-9ced-22dbb29d3bd2)) - **Practical Desk Lighting And Cleaning Hacks** - Wes and Scott share desk/lighting hygiene picks: Wes uses an Amaran Halo light for consistent mounts, Scott uses Zeiss lens cleaner for screens and glasses. - Scott prefers standards-based lights to attach softboxes; both stress cleaning gear regularly. Transcript: Wes Bos I got a sick pick for you, Scott. This is a little Amaran light. I know you have a couple of these as well, and that's why I initially got my first one. But I have the like big Amaran 150C, which is just like the sun monster. Scott Tolinski Yeah. Wes Bos And it does different colors and everything like that. And I needed another one that was just like much smaller just for like kind of like a little bit of side light right here. Let me see if I can actually turn it on. There we go. You can hardly see it in there. I need to repoint it a little bit. But this is the MRN Halo 100X. And if you just need like a slim, it's a couple inches thick light that still takes standard attachments. I think they're, they're called like a Bowden mount or something like that. That's what you need because so many people, when they're like getting like, Oh, I wish my webcam was a little bit better. And you go on Amazon and you buy some like newer thing or whatever. Those are, are okay lights, but they're not like, they don't take standard attachments. They don't have standard inputs. They don't have standard hooks. And then when it comes time to like, oh, I want to add like a little soft box to this, you know, like maybe a tiny little soft box, or I want to diffuse it a little bit, then there's like no solution For whatever it is that light that you bought. So buying, again, I'm a big standards guy, buying stuff that is standard spaces was really important to me. So I got this little Ameren Halo light, and it does just warm and cool. I don't think it does colors. And it has like a nice little Bluetooth app. That's the one thing I don't like about it is the Bluetooth on my desktop app. I have to restart the app all the time to connect to it. But I really like it because you can change it warm, cool. Scott Tolinski You can match the color on your camera, all that type of stuff I like that, I've never heard of that they make great lights and I do love my massive one right here and we've been doing more Lighting setups I don't know if you've seen this but in Courtney's podcast Wes, we now have like a multi-cam setup and so I've been like having to really get into lighting and multi-cam Setups now because we started doing video for that. So that's been a whole thing. I got like back lights and side lamps. And yeah, it's a whole thing. I'm going to pick a neat and neat and a cheap little hack here that I have for keeping my computer screen clean, but also my glasses. Because I'm I did. You see me. I probably fix my hair a whole bunch. I touch my face all the time. I touch my glasses. I'm just kind of a fidgety guy like that. And my glasses get all dirty, but also my screen does because I'm always touching it. So I bought this Zeiss lens cleaner. It's a glasses lens cleaner. It's really, really, it's like non-alcohol based it's water proprietary detergents and preservatives so who knows what's in this thing but uh for 12 bucks you get two of these giant Bottles that you will absolutely never run out of never and like i i've been using this thing for so long and it's like not even you know, three quarters of the way or like it's still three Quarters of the way full. And I use it to clean my computer screen all the time. I use it to clean my keyboard, wipe it down, trackpad, whatever. And I'm not going to be the person that says to just use it on your screen because who knows if that could remove. But I've been using it on my screen now for like two years, and I've never had any issues with the screen, whatever the screen texture or whatever getting wiped out. Wes Bos The nano coating. Scott Tolinski So either way, screen cleaner. Clean your dang screen. Your screens are dirty. And I found this stuff works really well. ([Time 0:47:01](https://share.snipd.com/snip/9129e32b-3437-444c-a87a-0a45a9ce4470))