Prime Is (Mostly) Right About AI - CourtneyR.dev's 2nd Brain

Prime Is (Mostly) Right About AI ![rw-book-cover](https://img.youtube.com/vi/VDPMXSAxiWk/maxresdefault.jpg) 🔗 **Source:** [Prime Is (Mostly) Right About AI](https://www.youtube.com/watch?v=VDPMXSAxiWk) > [!note]+ Document Notes > tags: #ai-economics #compute-constraints #developer-tools #pricing-models #ai-infrastructure > author: Theo - t3.gg > source: YouTube > reference: https://www.youtube.com/watch?v=VDPMXSAxiWk > date created: 2025-01-09 12:00 > > # 🏷️ Title: Prime is (mostly) right about AI >  ## 🧰 Action Inbox - [H1] Track enterprise AI compute costs — company spending exceeds salary costs #ai-economics - [H2] Analyze personal AI usage patterns — measure cost per task completion #developer-tools - [H3] Monitor GPU availability trends — capacity constraints driving pricing #compute-constraints ## 🧭 Table of Contents - [[#📝 Summary]] - [[#📖 Main Content]] - [[#🗒️ Notes]] - [[#📚 References]] - [[#🗣️ Paraphrase]] - [[#📰 Newsletter]] - [[#✅ Action Items]] - [[#🧪 Hypotheses]] - [[#📊 Methodology]] - [[#❓ Questions]] - [[#🔮 Future Research]] - [[#💡 Implications]] - [[#🧠 Learning Styles Assessment]] - [[#📊 Accessibility Assessment]] - [[#🌍 DEIB Assessment]] - [[#✅ Logical Validity]] - [[#🛡️ DARVO Analysis]] - [[#🎯 Conclusion]] ### 🔗 References - [Primagen's Original Video](https://www.youtube.com/watch?v=VDPMXSAxiWk) - [Artificial Analysis Intelligence Index](https://artificialanalysis.ai) ## 📝 Summary - AI economics are shifting due to compute constraints, not profitability issues - Companies are restricting usage to preserve enterprise compute capacity - Pricing changes reflect GPU scarcity rather than revenue optimization - Cost per intelligence unit is actually decreasing despite higher token prices - Enterprise customers pay significantly more than consumer subscription rates ## 📖 Main Content Theo provides a nuanced response to Primagen's analysis of AI economic challenges, arguing that the core issue isn't subsidization ending but compute scarcity driving changes. While agreeing that AI economics are shifting, Theo contends the problem isn't revenue optimization but GPU availability constraints. The key distinction lies in understanding pre-training versus post-training costs. Pre-training (like GPT-4.5) requires massive upfront investment, while post-training iterations (4.6, 4.7) cost significantly less through reinforcement learning techniques. This explains why some model releases appear less profitable - they're iterative improvements using cheaper post-training methods. Companies like Anthropic, Microsoft, and Google face compute bottlenecks that force difficult choices about allocation between subsidized consumer plans and profitable enterprise contracts. The pricing changes and signup restrictions represent attempts to preserve capacity for high-value customers rather than squeeze additional revenue from individuals. ## 🗒️ Notes #### ⏱️ [00:00](https://youtu.be/VDPMXSAxiWk?t=0) — [H1] AI economics nuance beyond subsidization - Key idea: Economics shifting due to compute constraints, not just revenue optimization - Evidence: "The problem isn't that there isn't enough money to spend... They don't have enough Nvidia graphics cards" #### ⏱️ [03:45](https://youtu.be/VDPMXSAxiWk?t=225) — [H2] Cursor pricing model changes as early indicator - Key idea: First company forced to abandon message-based pricing due to cost variability - Evidence: "Some users would do $10 of inference with their 200 messages. Others would do thousands of dollars" #### ⏱️ [07:12](https://youtu.be/VDPMXSAxiWk?t=432) — [H3] Anthropic's time-based usage experiments - Key idea: Peak hour restrictions reveal compute capacity management strategies - Evidence: "Between 5:00 a.m. and 11:00 a.m. PT... you'll move through your 5-hour session limits faster" #### ⏱️ [12:30](https://youtu.be/VDPMXSAxiWk?t=750) — [H2] Enterprise revenue vs subscription revenue priorities - Key idea: Companies optimize for enterprise customers, not individual subscribers - Evidence: "They don't care that much about people going from the $20 plan to the $100 plan" #### ⏱️ [18:45](https://youtu.be/VDPMXSAxiWk?t=1125) — [H1] Pre-training vs post-training cost differences - Key idea: Model development costs vary dramatically between fundamental and iterative improvements - Evidence: "GPT-4.5 probably cost hundreds of millions... GPT-4.6 probably won't be that big of a cost" #### ⏱️ [25:20](https://youtu.be/VDPMXSAxiWk?t=1520) — [H3] GitHub Copilot pricing reflects compute scarcity - Key idea: 7.5x multiplier for GPT-4.5 based on capacity, not API costs - Evidence: "One message... has been going for almost an hour and a half... upwards of $100 of compute" #### ⏱️ [32:15](https://youtu.be/VDPMXSAxiWk?t=1935) — [H2] Microsoft compute constraints forcing changes - Key idea: Signup pauses indicate capacity issues, not revenue optimization - Evidence: "You don't pause signups because you want to make more money. You pause signups because you don't have capacity" #### ⏱️ [38:40](https://youtu.be/VDPMXSAxiWk?t=2320) — [H3] Google's aggressive subsidization and quick reversal - Key idea: Google subsidized harder initially but had to restrict faster - Evidence: "Google was the first company to start doing the restrictions... people building plugins started getting banned" #### ⏱️ [45:30](https://youtu.be/VDPMXSAxiWk?t=2730) — [H2] Enterprise vs consumer pricing disparities - Key idea: Business customers pay dramatically more for same compute - Evidence: "I pay 200 bucks a month for it. They pay 2,000 bucks for it" #### ⏱️ [52:10](https://youtu.be/VDPMXSAxiWk?t=3130) — [H1] Cost per intelligence decreasing despite higher token prices - Key idea: Efficiency improvements make tasks cheaper even with expensive models - Evidence: "GPT-4.5 medium cost $1,200 where GPT-4.4x high costs $2,800" #### ⏱️ [58:25](https://youtu.be/VDPMXSAxiWk?t=3505) — [H3] Token efficiency improvements in newer models - Key idea: Newer models solve problems with fewer tokens, reducing actual costs - Evidence: "GPT-4.5 low... only 7 million tokens to score higher than DeepSeek V4" #### ⏱️ [65:40](https://youtu.be/VDPMXSAxiWk?t=3940) — [H2] Real-world compute usage examples - Key idea: Individual usage can exceed enterprise allocation costs - Evidence: "Engineers spending more on compute... than they are paid in salaries" ## 📚 References - [Anthropic Claude Pricing](https://www.anthropic.com/pricing) - [GitHub Copilot Plans](https://github.com/features/copilot) - [Artificial Analysis Benchmarks](https://artificialanalysis.ai) ## 🗣️ Paraphrase - AI companies are restricting user access not to increase profits but to preserve limited GPU capacity for enterprise customers who generate actual revenue - The shift from message-based to usage-based pricing reflects the reality that some AI tasks cost pennies while others cost hundreds of dollars to compute - While frontier AI models become more expensive per token, they're actually becoming more cost-effective per task through efficiency improvements and better reasoning ## 📰 Newsletter The AI economics landscape is shifting, but not in the way many developers think. Recent pricing changes from Anthropic, Microsoft, and Google aren't about squeezing more money from users - they're about managing severe GPU shortages. **Key insight**: Companies are compute-constrained, not revenue-desperate. When GitHub Copilot paused signups, it wasn't to boost profits but to preserve capacity for enterprise customers paying significantly more. **Practical application**: Understanding these dynamics helps developers make better tool choices. Consumer subscriptions remain heavily subsidized compared to enterprise API costs, making personal plans excellent value for individual developers. **For the WordPress community**: This compute scarcity could affect AI-powered development tools and hosting services that rely on machine learning features. Plan for potential usage restrictions and consider diversifying AI tool dependencies. Resources: [Learn WordPress](https://learn.wordpress.org/), [Developer Docs](https://developer.wordpress.org/), [WP-CLI](https://wp-cli.org/) ## ✅ Action Items #### DevRel - Education: Create workshops explaining AI cost structures and compute economics - Developer experience: Document cost-effective AI integration patterns - Community management: Address concerns about AI tool pricing changes #### OSPO - Strategic Alignment: Monitor AI infrastructure costs and capacity constraints - Developer Support: Provide guidance on enterprise vs consumer AI tool usage - Community Engagement: Track open source AI model development and hosting costs #### WordPress - Core AI: Evaluate compute requirements for AI features in WordPress core - Performance: Benchmark AI-powered features against compute budgets - Hosting: Assess impact of AI features on hosting provider costs ## 🧪 Hypotheses - Compute constraints will drive consolidation among AI providers as smaller companies can't secure GPU capacity - Enterprise AI costs will stabilize as companies develop better usage governance - Consumer AI subscriptions will become more restrictive but remain heavily subsidized for marketing purposes - Open source AI models will gain adoption as companies seek cost predictability ### 📊 Methodology - Track enterprise AI spending across different company sizes and usage patterns - Monitor GPU availability and pricing trends from major cloud providers - Compare cost per task completion across different AI models and pricing tiers ### ❓ Questions - How will compute scarcity affect AI model development timelines and innovation? - What governance frameworks will emerge for enterprise AI usage management? - How will open source AI models compete with proprietary alternatives on cost efficiency? ### 🔮 Future Research - Analysis of AI compute capacity expansion plans from major providers - Study of enterprise AI ROI measurement and budget allocation strategies - Investigation of alternative compute architectures for AI inference ### 💡 Implications - Practice: Developers should diversify AI tool dependencies and understand usage patterns - Policy: Organizations need AI governance policies that account for compute costs - Security: Compute constraints may affect AI-powered security tool availability - Education: Technical education should include AI economics and resource management - Funding: AI infrastructure investments will require careful capacity planning ### 🧠 Learning Styles Assessment Reading level estimate: 12th Grade #### 👀 Visual learners - Present: Cost comparison charts, usage graphs, pricing tier diagrams - Improve: Infrastructure diagrams showing compute allocation, capacity visualization #### 🔊 Aural learners - Present: Video format with detailed explanations and examples - Improve: Podcast summary, audio-only version for commuting #### 🗣️ Verbal learners - Present: Clear definitions of technical terms, analogies to traditional pricing - Improve: Glossary of AI economics terms, comparison frameworks #### 🧩 Kinesthetic learners - Present: Real examples of usage patterns and cost calculations - Improve: Interactive cost calculators, hands-on pricing experiments #### 🧠 Logical learners - Present: Data-driven arguments, clear cause-effect relationships - Improve: Decision trees for tool selection, cost-benefit analysis frameworks #### 🧑‍🤝‍🧑 Social learners - Present: Response video format engaging with community discussion - Improve: Discussion forums, peer comparison of usage patterns #### 🧘‍♀️ Solitary learners - Present: Detailed analysis suitable for individual study - Improve: Self-assessment tools, personal usage tracking templates #### 🌿 Naturalistic learners - Present: Ecosystem analogies for AI provider relationships - Improve: Resource allocation metaphors, sustainability considerations ### 📊 Accessibility Assessment #### Screen reader semantics - Present: Clear heading structure, descriptive video content - Improve: Transcript with technical term definitions, audio descriptions #### Keyboard navigation - Present: Standard video player controls - Improve: Chapter navigation, transcript synchronization #### Contrast and readability - Present: Good contrast in video player interface - Improve: High-contrast transcript option, customizable text sizing #### Text resizing and reflow - Present: Responsive video embedding - Improve: Reflow-friendly transcript layout #### Media and captions - Present: Auto-generated captions available - Improve: Human-reviewed captions, downloadable transcript #### Forms and interactions - Present: Standard video controls - Improve: Accessible chapter selection, speed controls ### 🌍 DEIB Assessment #### Cultural breadth - Present: US-focused examples and companies - Improve: Global AI provider landscape, international regulatory considerations #### Inclusive language - Present: Technical language with explanations - Improve: Plain language summaries, cultural context for economic concepts #### Neurodiversity support - Present: Detailed explanations with examples - Improve: Content warnings for rapid technical changes, summary sections #### Representation - Present: Tech industry perspective - Improve: Diverse stakeholder viewpoints, small business considerations #### Psychological safety - Present: Respectful critique of other creators - Improve: Acknowledgment of different valid perspectives, learning opportunities ### ✅ Logical Validity #### Strong reasoning - Deductive strength: Clear premises about compute constraints leading to pricing changes - Inductive strength: Multiple examples supporting pattern of capacity-driven decisions - Causal reasoning: Well-established links between GPU scarcity and usage restrictions - Analogical strength: Effective comparison to traditional resource pricing models #### Fallacies - Hasty generalization: Some broad claims about Google's strategy based on limited examples - Appeal to authority: Relies heavily on insider knowledge without fully sourcing claims #### Tone of voice - Overall tone: Analytical and educational with respectful disagreement, some frustration with misconceptions ### 🛡️ DARVO Analysis - Deny: No concerning denials present - Attack: Constructive criticism without personal attacks - Reverse victim/offender: Not applicable - Emotional cues and rhetoric: Passionate but factual presentation, appropriate technical frustration ## 🎯 Conclusion The AI economics shift represents compute scarcity rather than the end of subsidization. Companies are optimizing for enterprise revenue while managing limited GPU capacity, not trying to extract maximum profit from individual users. Understanding these dynamics helps developers make informed decisions about AI tool adoption and usage patterns. The cost per unit of intelligence continues declining even as headline prices increase, suggesting sustainable long-term economics for AI-assisted development. ## 🧱 Refactor Appendix ### 📚 Literature Split Pad Purpose: Stage CEE units that become Literature notes. Split level: H3 ### [H1] AI pricing reflects compute constraints not profit optimization > [!quote] Prime is (mostly) right about AI by Theo - t3.gg via YouTube. Published 2025-01-09. Accessed 2025-01-09. > "The problem isn't that there isn't enough money to spend... They don't have enough Nvidia graphics cards in their server farms" Economic analysis of AI industry changes through compute availability lens rather than traditional business models. [view highlight](https://youtu.be/VDPMXSAxiWk?t=1935) #xref #articleseed ID:rw_ai_economics ### [H2] Pre-training costs dwarf post-training model iterations > [!quote] Prime is (mostly) right about AI > "GPT-4.5 probably cost hundreds of millions, if not billions of dollars. GPT-4.6 probably won't be that big of a cost" Understanding model development economics requires distinguishing between fundamental advances and iterative improvements. [view highlight](https://youtu.be/VDPMXSAxiWk?t=1125) #xref ID:rw_model_training ### [H3] Cost per intelligence decreasing despite token price increases > [!quote] Prime is (mostly) right about AI > "GPT-4.5 medium cost $1,200 where GPT-4.4x high costs $2,800" Efficiency improvements make AI more cost-effective even as frontier models become more expensive. [view highlight](https://youtu.be/VDPMXSAxiWk --- ## Full Transcript [00:00] So, I'm sure many of you have noticed I [00:01] kind of like AI for development work. I [00:03] think it's pretty solid overall. This is [00:05] a big change from how I used to feel [00:07] about it. And over time, I find myself [00:08] reaching for AI in my development work [00:10] more and more. But not every developer [00:12] feels the same way. One in particular is [00:14] an old friend of mine, Primagen, and he [00:15] just put up this video about how the AI [00:17] economy is about to change. This isn't [00:19] going to be a usual reaction video cuz I [00:20] actually already watched it. But I want [00:22] to break down some of his takes cuz [00:24] there's so much truth to them that I [00:26] feel like that the nuance of it being [00:27] missed. But on the other hand, there are [00:30] some things that I I feel like aren't [00:31] well understood around the economics of [00:34] all of this. In particular, once [00:35] Microsoft and Google get mentioned, I [00:37] feel like the the details get a bit [00:39] lost. I want to try to find a middle [00:40] ground that makes sense here, cuz Prime [00:42] already is kind of in the middle between [00:44] the haters and the full AI psychosis [00:46] folks. I also like to think I'm [00:48] somewhere in the middle there. I know [00:49] people think I'm much more on the [00:50] psychosis side, and I can see why. It's [00:52] a great video, and I want to do my best [00:54] to to find the nuance within it. So, [00:56] we're going to dive right into this [00:57] video and talk all about Prime's takes [00:59] right after a quick word from today's [01:00] sponsor. You've probably already heard [01:01] me talk about today's sponsor. It's [01:03] Blacksmith. Wait, they ship MacRunners. [01:06] Okay, I am acting here, but I was that [01:08] excited, if not more so, when I heard [01:10] about this because one of the slowest [01:12] parts of the T3 code build process right [01:14] now is the Macr runners. In order to [01:16] build a Mac app, you need to use a Mac. [01:18] And since we were building our app for [01:19] Mac, we kind of got stuck on GitHub's [01:21] Macr runners, which are really, really [01:22] bad. Our whole build process used to [01:24] take 16 minutes. And a lot of this was [01:26] time spent on the x64 Intel builds for [01:29] the Mac version, which would take eight [01:31] plus minutes. They now take 3 minutes [01:33] and 34 seconds. And the ARM 64 builds, [01:35] you know, the Apple silicon ones, would [01:37] take 6 minutes and 11 seconds on GitHub. [01:39] And now they take under 4 minutes, 332. [01:42] This made us go from annoyed about [01:44] releases to kind of excited to do them. [01:45] And if you're curious why we now ship a [01:47] nightly with T3 code, it's because of [01:49] how much cheaper and faster our build [01:51] process has become. I'm going to be so [01:52] real with you guys. If you're building a [01:54] Mac app on GitHub and you're not using [01:56] Blacksmith yet, you're wasting so much [01:58] time. And if you're not building Mac [02:00] apps, but you're still using GitHub [02:01] actions, you're still wasting time [02:02] because Blacksmith is 2x faster in less [02:04] than half the price. Stop wasting time [02:06] and money and build faster at [02:07] soyv.link/blacksmith. [02:09] Let's see what Prime has to say. The [02:11] cracks are starting to show. The [02:13] foundations looking a little bit shaky. [02:15] I'm talking talking about the AI [02:17] economy, the token economy. Uh, the [02:19] first one comes courtesy of course of [02:21] Anthropic, everybody's favorite, the [02:23] good guys. Am I right? I love their [02:25] lawyers. What ended up happening just a [02:28] couple days ago is Anthropic did [02:29] something called a painted door test. At [02:31] least that's the term I've always heard. [02:33] I think I've seen other people use fake [02:35] door test where you actually show [02:36] alternative pricing on your page to see, [02:39] hey, how much more money could we make [02:42] if we charged people more? How many more [02:44] people would leave the pricing page? So, [02:46] let me give you a hypothetical. So, he's [02:47] talking about here is a change that [02:48] happened a few weeks ago where Enthropic [02:50] wanted to figure out if they could move [02:51] Claude code out of the $20 tier so that [02:54] you had to be on the hundred or $200 [02:56] tier to get Cloud Code usage. The reason [02:58] they wanted to make this change is [03:00] because Claude Code is expensive to run [03:02] and they wanted to try and save compute. [03:05] There are pieces he's missing here, [03:08] though. The first thing he misses is [03:10] that this is not the first time the [03:12] economy started to break down here. I [03:14] would argue the first time actually goes [03:16] all the way back to July of last year. [03:19] Cursor got hit hard in this case. I'm [03:22] bringing up cursor because previously [03:24] they build by a number of messages. So [03:26] you would pay an amount of money and [03:28] then get a certain number of messages a [03:30] month. That sucks because some messages [03:32] cost a few cents to run and other ones [03:34] cost many dollars to run. Fun fact, [03:37] GitHub at this point in time still [03:39] prices based on number of messages. They [03:41] are also coming around and changing this [03:43] at the start of next month. I've [03:44] actually had a terminal running trying [03:46] to solve a cryptography challenge and it [03:49] has been going for almost an hour and a [03:50] half on one message. You get 1,500 [03:54] messages a month on the $40 plan. [03:57] Entirely unrealistic. And we'll come [03:58] back to this, don't worry. But this was, [04:00] I would argue, where the start of the [04:02] end was. When Cursor, who doesn't have [04:05] the ability to subsidize because they [04:06] have to pay the labs for the inference, [04:08] like they have to pay OpenAI and [04:10] Anthropic cash for each request, they [04:13] couldn't eat the subsidization because [04:14] some users would do $10 of inference [04:16] with their 200 messages. Others would do [04:18] thousands of dollars of inference with [04:20] their 200 messages. And Cursor couldn't [04:22] eat that. So they made changes where [04:24] your pricing is now more reflective of [04:26] what it cost to run your requests, not [04:29] how many requests you did. So this was [04:31] the first start of the end here. The [04:34] next really big one was when Enthropic [04:36] started changing when you could use the [04:38] inference. This might not seem like it [04:40] fits cuz it seems like more inference, [04:42] but hear me out because this is a really [04:44] big change when Enthropic made it in [04:46] March of this year. A small thank you to [04:48] everyone using Claude. We're doubling [04:50] usage outside of peak hours for the next [04:53] 2 weeks. This was the first experiment [04:55] that Enthropic ran in order to figure [04:58] out how to price things based on the [05:00] time of day. They have a fixed number of [05:02] GPUs available, a fixed amount of [05:04] compute that can run requests. So when [05:06] there's a bunch of users during the [05:08] workday that are trying to get a ton of [05:10] usage out, that means that people who [05:13] are running things 24/7, it's more [05:15] expensive for them during that window [05:17] effectively. Because if you have 100 [05:19] GPUs and during the workday 95 are taken [05:23] up, the last five are going to be seen [05:25] very differently than off peak hours [05:27] when you have 50 available. So, [05:29] Enthropic was trying to steer the power [05:31] users towards off hours because they [05:34] want that compute available during the [05:36] workday. This did not go as well as they [05:38] hoped. Just a week and a half later, [05:41] Thoric announced publicly that in order [05:43] to manage the growing demand for Claude, [05:45] they would adjust their 5-hour session [05:47] limits for free pro and max plans during [05:49] peak hours. Your weekly limits remain [05:52] unchanged. So, between 5:00 a.m. and [05:54] 11:00 a.m. PT or 1 to 7 GMT, you'll move [05:56] through your 5-hour session limits [05:58] faster than before. This is the key [06:00] piece I wanted to highlight. The [06:02] previous thing was them seeing if they [06:04] could push us to using off peak hours [06:06] more. It wasn't enough. So they had to [06:08] do the opposite and reduce the amount of [06:10] usage we get during specific hours. To [06:12] go back to cursor for a second because [06:14] again cursor can't subsidize the same [06:16] way the other labs can. They don't own [06:18] this nearly infinite amount of compute [06:20] and GPUs. Cursor is the customer of [06:22] anthropic and OpenAI the same way we [06:24] are. So they're much more sensitive to [06:25] these things. That's why they had to [06:27] stop subsidizing early. And this is also [06:29] why they do a lot of internal auditing [06:31] figuring out how much compute you get [06:33] per dollar on these other platforms. So [06:35] internally they measure how much you can [06:37] get out of a $200 cloud code sub. [06:39] Previously you could do up to $2,000 in [06:41] compute. Now they're seeing even crazier [06:43] numbers where you can get up to $5,000 [06:45] of inference in that $200 a month plan [06:48] because Anthropic is subsidizing so [06:50] aggressively. So I just wanted to give a [06:52] bit more history because Prime started [06:54] with a recent drama which was them [06:56] trying to move Claude Code out of the [06:58] $20 plan. I see that less as trying to [07:01] push people up to the $100 plan and more [07:04] as empropic trying to get all of these [07:06] like low tier users to stop using claude [07:09] code. Their goal is to claw back [07:11] compute, not just to increase profit [07:13] here. And I really want to highlight [07:14] that because Anthropic's growth is crazy [07:17] in enterprise, which is where they make [07:19] most of their revenue. They don't care [07:21] that much about people going from the [07:22] $20 plan to the $100 plan. what they [07:24] care about is much more specifically [07:26] that they are able to get their compute [07:28] available so that they can use it for [07:30] all the other enterprise customers. So [07:32] just want to make sure I jump on that [07:33] because the enterprise revenue is what [07:35] Anthropic cares about. The subscription [07:37] revenue is mostly just a marketing thing [07:39] and the reason they do this crazy [07:41] subsidization isn't because they think [07:43] they can make the money back or they're [07:45] just trying to crush and win. It's [07:46] mostly a marketing play. The issue is [07:48] that they're trading compute for [07:49] marketing effectively where they own the [07:51] GPUs. giving it away for these prices [07:53] helps them be seen better and be seen [07:56] more often. But now that they're low on [07:58] compute, it's harder for them to get [08:00] away with that. But this is much more [08:01] about the compute than just spiking [08:03] revenue. Skipping a bit because I want [08:05] to get to the next core point, which is [08:07] how the costs actually work out as a [08:09] user and as a business for a company [08:11] that is doing inference. Well, they're [08:13] making money off of inference. Like [08:15] every time they do like a request call, [08:18] of course they make money. And it's [08:19] like, yeah, I guess if you measured how [08:21] much a shoe cost by only paying the [08:24] employee to sell the shoe, then you're [08:26] like, yeah, you're making money off of [08:27] every shoe. They're making money off of [08:28] every request. Yeah, that makes sense, [08:31] but you're not actually considering the [08:33] real cost. Remember, people that used [08:35] Opus 45 are now using Opus 46. Nobody's [08:38] using Opus 47. A kind of tarted model. [08:40] No, no one really likes that one. But [08:43] that means all the cost that went into [08:45] 45 if they didn't recoup that in [08:49] inference cost 45 just cost them money. [08:51] And this is obviously what's happening. [08:53] Open AI got a 100 talk about the open AI [08:56] stuff in a sec. I want to talk about the [08:58] examples here with anthropic because for [09:00] the most part he is correct. If you're [09:02] looking at the cost in simple numbers [09:04] like you spent $15 in API cost that cost [09:09] Anthropic $1.50 in electricity. They did [09:12] make money on that, but there's a couple [09:14] layers where that breaks down. The first [09:15] is the subscription model where some [09:17] users are using $5,000 of inference for [09:19] 200 bucks. That's a hard enough [09:21] subsidization that they're probably [09:22] losing money on it even just from the [09:24] electricity costs. Not counting the cost [09:26] for the GPUs, not counting the cost to [09:27] train or the employees or any of the [09:29] other things just counting electricity. [09:31] There are users who are costing [09:32] anthropic money purely on electricity [09:34] due to the sheer level of subsidization [09:36] they are doing. Estimates for how much [09:38] the compute costs relative to the API [09:40] requests are as low as 15 to 20%. So the [09:44] raw cost that Anthropic or OpenAI eats [09:47] is likely 15 to 20% of what you spend on [09:50] the API requests. But again, when you're [09:52] subsidizing 20x plus, that washes out [09:55] and these companies are spending money [09:57] effectively to give you that level of [09:59] subsidization. And if those numbers for [10:00] the cost and electricity sound crazy, I [10:02] have a 5090 doing some fun training [10:05] tasks on the side. And it alone raised [10:07] my electric bill by about $1,000. Like [10:10] my electric bill is around $1,800 a [10:12] month across my AC, my computers, and [10:14] everything here in San Francisco. It's [10:16] insane. Electricity is expensive, and [10:18] it's cheaper other places, but it's like [10:19] half the price maybe. Running a high-end [10:21] GPU at a full,000 watts 24/7 in even the [10:24] like cheapest places in the US, you're [10:26] talking$500 plus dollars of electricity [10:28] a month. Like it's crazy how expensive [10:31] that gets. So there's other parts here I [10:33] want to break down. Specifically the [10:35] amortization of the price over time [10:37] where you have to make the amount you [10:39] spent on the model. There are two pieces [10:41] here I want to talk about. I'm going to [10:42] break this down to pre-training and [10:43] post-training because what Prime is [10:45] referring to here is largely the case in [10:47] the pre-training part. There's a really [10:49] good Daario interview where he talks [10:51] about this. There's kind of like two [10:53] different ways you could describe what's [10:55] happening in the model business right [10:56] now. So let's say in 2023 you train a [10:59] model that costs $und00 million um and [11:02] then you deploy it in 2024 um and it [11:06] makes $200 million of revenue. [11:09] Meanwhile, because of the scaling laws [11:10] in 2024, you also train a model that [11:13] costs a billion um and then in 2025 you [11:16] get $2 billion of revenue from that um [11:19] $1 billion and mean and you spend $10 [11:21] billion to uh train the model. So if you [11:24] if you look in a conventional way at the [11:26] profit and loss of the company, um you [11:28] know, you've you've lost $100 million [11:31] the first year, you've lost $800 million [11:33] the second year, and you've lost $8 [11:34] billion in the in the third year. So it [11:36] looks like it's getting worse and worse. [11:38] Um if you consider each model to be a [11:41] company, um you know, the the model that [11:44] was trained in in in in 2023, um was was [11:48] profitable. You you paid a hundred [11:50] million and then it made 200 million of [11:51] revenue. there's some um you know cost [11:53] to inference with the model. Um but you [11:55] know let's just assume in this [11:57] cartoonish cartoon example that even if [11:59] you add those two up you're you know [12:02] you're kind of in a good state. So if [12:04] every model was a company the the model [12:06] is actually you know in in this in this [12:09] example is actually profitable. So this [12:12] is roughly what prime was talking about [12:14] there. If you look at each model and how [12:16] much it costs and then look at how much [12:18] money it makes, it comes out as [12:20] profitable. Usually, if hypothetically [12:23] speaking, Sonnet 35 cost 100 mil and [12:25] they made back 200 mil in revenue, that [12:27] model was profitable. Even if they spent [12:29] 400 mil to train the next one, the [12:31] company might have lost money, but the [12:33] model itself, the money in to the money [12:35] out worked out profitably, but they have [12:38] to keep scaling up the cost of making [12:40] the new model, which only works if their [12:41] revenue scales alongside it. The thing [12:44] prime said that I don't agree with [12:45] because it doesn't fit this model [12:46] correctly is that Opus 47 and Opus 46 [12:50] even to an extent lost money because [12:52] they weren't as big of a jump as Opus 45 [12:54] was and 45 and to an extent 46 are the [12:56] ones people are using. It's possible [12:58] that they're losing money on these model [13:00] drops that get replaced quicker and they [13:02] move on from faster or that don't get as [13:04] much adoption. That is not the case. The [13:06] reason why that's not the case is the [13:08] different types of training that are [13:10] done. I'm sure you guys have noticed by [13:12] now that sometimes a new model drop just [13:14] feels like the last one but slightly [13:16] smarter, slightly faster, slightly more [13:19] capable in specific domains. And then [13:20] other times it feels entirely different. [13:23] Like going from GPD 52 to 53 codeex was [13:26] like an oh, it talks to me more but it's [13:28] roughly as smart. And then 53 codeex to [13:31] 54 was like oh yeah, it overreaches a [13:33] bit but they felt similar. 55 feels [13:36] entirely different to the point where it [13:38] probably should have been given a [13:39] different name. Either something [13:40] separate from the GPT line or GPT6 [13:43] honestly because the model is so [13:45] fundamentally different. The reason it's [13:47] so different is because of the [13:49] pre-training. There's two main types of [13:52] training with models. You have the [13:53] pre-training and the post-training. The [13:55] pre-training is how you take all of the [13:57] information, all the data you've [13:58] collected and bake it into this pile of [14:01] parameters that is a model. then you [14:04] want it to behave certain ways. So the [14:06] pre-training is how you get the [14:08] knowledge in effectively and the [14:10] post-training is how you get the [14:11] behavior out of it. This is a gross [14:13] gross simplification of all of the stuff [14:15] here. If you're into it, I highly [14:17] recommend looking into it more. It's [14:18] really cool stuff. I'm just trying to [14:19] get the point across here that [14:21] pre-training is the big expensive thing [14:23] that costs hundreds of billions if not [14:25] billions of dollars of compute to try [14:27] and take these terabytes of data and [14:30] compress them into this particular shape [14:33] that allows you to generate text [14:35] programmatically off it as an LLM. The [14:38] post-training is how you refine the way [14:40] it behaves. And postraining has been way [14:43] more rewarding than people expected, [14:44] especially RLVR and RHF, which are [14:46] reinforcement learning strategies. Turns [14:48] out those are really good at making the [14:50] model good at things like long tool call [14:52] runs, long agent runs, doing code, and [14:55] all those types of tasks. You can RL [14:57] models into getting way better at that. [14:59] We know this because models like [15:00] Composer 2 by Cursor aren't actually a [15:03] new model. They are fine-tuned versions [15:06] of older models like Kimmy K25. Kimmy [15:08] K25 from the moonshot guys over in [15:11] China's at open weight model. And since [15:13] the weights are out there, you can RL it [15:15] and you can fine-tune it externally [15:17] yourself. This isn't the thing that like [15:19] anyone can do if the model weights are [15:21] open. It is also generally speaking [15:24] often much less expensive than the [15:26] pre-training is. So Opus 45 was almost [15:29] certainly new pre-training. It was a [15:31] fundamental change in how Opus works and [15:34] behaves. That's why it got three times [15:35] cheaper than previous Opus models [15:37] because they made it smaller than it was [15:39] before probably and ended up making it [15:41] way smarter than they expected to. And [15:43] it is likely from that point on that [15:45] focus 4 6 and 7 were post training which [15:48] was less expensive. You can usually see [15:50] this just by the cadence of release [15:52] cycle. But there are exceptions to this [15:54] too. Like some of my research friends in [15:55] chat are pointing out that gro 4 spent [15:57] more time in compute on RL they did on [15:59] pre-training. Even something like [16:01] composer 2 from cursor allegedly spent [16:04] 4x more on the compute for post-raining [16:08] than Kimmy did on the pre-training in [16:10] the first place. So, there are [16:12] exceptions here, but generally speaking, [16:14] especially with the big labs, once they [16:16] do a new pre-training, the future [16:17] iterations aren't as expensive. They'll [16:20] often have five or six of them [16:21] internally, and they take the one they [16:23] like the most and put that one out. The [16:25] individual updates to the model once [16:27] they have that pre-training are not [16:28] these billion dollar investments like [16:30] the new models were before. So, 55 [16:33] probably cost hundreds of millions, if [16:35] not billions of dollars. 56 probably [16:37] won't be that big of a cost in [16:39] comparison. Opus 45 probably cost near a [16:42] billion dollars. Opus 46 probably a lot [16:45] less. Just wanted to make that point [16:46] because it's important to understand. [16:48] Let's see what Prime has to say about [16:50] OpenAI here. Open AI got 122 or $120 [16:55] billion investment and that's enough [16:57] money to run for 18 to 24 months. That [17:01] is like 5 to7 billion every single month [17:05] in the hole for the next 18 to 24 [17:08] months. They had to do this test because [17:10] they have to know how much money they [17:11] can make because if they don't make some [17:13] sort of change, they're going to [17:14] continue to lose billions of dollars. [17:16] Now, if this just happened by itself, [17:18] I'd say, "Hey, Claude just needs to make [17:20] more money. They need to start becoming [17:22] more competitive cuz they're competing [17:23] against OpenAI." And then I'd be like, [17:25] "Okay, that's that's just that." But [17:27] this isn't the only case of this [17:28] happening because just a couple days [17:30] ago, guess who else decided to make a [17:32] bit of a price change? Microsoft. Oh, [17:34] beautiful Microsoft. I love Satia. I [17:37] love co-pilot. Isn't co-pilot is this? [17:39] Is this the greatest? You're probably [17:40] confused when I say co-pilot because [17:42] you're probably thinking of like one of [17:43] their 50 services. I I'm talking about [17:45] the GitHub, the GitHub co-pilot in this [17:47] case. See, the GitHub co-pilot, you used [17:49] to pay some sort of amount of money and [17:51] then you could perform actions on [17:52] behalf. You had some sort of amount of [17:54] actions you could execute. Well, what's [17:56] the problem with that? Not every model [17:58] cost the same. That's not the problem [17:59] with this. Yes, different models cost [18:01] different amounts, but you can make [18:03] certain models two or three times as [18:05] many messages per message. The problem, [18:08] and this is actually the point that [18:10] triggered me enough to make my response [18:11] video, is not that some models are [18:13] expensive and some are cheap. It's that [18:15] pricing the number of messages makes no [18:17] sense at all. Imagine if Walmart priced [18:20] by the amount of items you had in your [18:22] cart. Or if they priced by how many [18:24] people walked in, they charge you when [18:26] you walk in, you go take whatever you [18:27] want and leave. Some people go in and [18:29] take two pieces of candy. Some people go [18:31] in and grab five TVs. If you're pricing [18:33] by the number of items, that doesn't [18:35] work. And the worst part is that Copilot [18:38] already is doing this. If you go into [18:40] Copilot and you pick a different model, [18:42] it'll tell you what the multiplier rate [18:44] is for different models. For example, [18:45] here 54 is 1x, but 55 is 7.5x. That's [18:50] the number of messages it uses. So on [18:53] the $40 plan, which is what I'm on with [18:55] Copilot, I believe I'm being comped [18:56] because of open source. So, thank you to [18:58] GitHub for that. They give you 1,500 [19:00] messages a month. The problem is that [19:02] some messages cost pennies to respond [19:04] to. Whether it is Opus or 54 or [19:07] whatever, doesn't matter. What matters [19:09] is what is it doing and how long does it [19:11] take? And remember earlier I said I was [19:13] running this request. This request is [19:16] nearing 2 hours. I sent one message and [19:19] it has been going for 2 hours. This one [19:22] message is going to use upwards of $100 [19:25] of compute by itself. This makes no [19:28] sense. You cannot charge based on the [19:30] number of items in somebody's cart. You [19:32] charge based on how expensive the things [19:34] are. The problem with Copilot and the [19:36] reason they're making this change now [19:37] isn't because the subsidization is over. [19:40] It's because Microsoft is just a slower [19:42] moving company and it took them longer [19:44] to fix this. That is all that matters [19:46] here. And the 7.5x part actually [19:49] emphasizes the point I want to make, [19:51] which is that these numbers and these [19:53] costs are not based on what the API [19:55] prices are or what the labs are [19:57] charging. These prices are based on how [20:00] much compute they have available to run [20:02] this model and how expensive it is for [20:04] them to give that compute up to the [20:06] subsidized co-pilot users. Historically, [20:08] C-pilot users were light enough users [20:11] that they could just eat the cost. If [20:13] some of those $40 a month users cost [20:15] them 800 bucks, whatever, it's [20:17] Microsoft. They make money. They can [20:18] afford that. The problem is there's [20:20] enough of them now. And it's not just [20:22] money that they are spending. It's [20:23] compute they are losing. And they are [20:25] compute constraint. They are fighting to [20:27] reserve as much compute as they can on [20:29] Azure for all of the other deals for all [20:30] the other companies they're doing. They [20:32] need that compute. It's not just like [20:34] you're costing them too much money when [20:36] you use co-pilot. is that they need the [20:37] GPUs that Copilot is using available. [20:40] The bigger deal about their announcement [20:42] isn't that they change the pricing. The [20:44] bigger deal is that they paused signups [20:46] entirely. You don't pause signups [20:49] because you want to make more money. You [20:51] pause signups because you don't have [20:53] capacity. And this is the really big [20:55] point I want to drive home. The problem [20:57] isn't that there isn't enough money to [20:59] spend. The problem isn't that they are [21:01] spending too much on these users of [21:02] these $20 plans and they want to get you [21:04] to spend more. This is not the [21:05] traditional up to the right problem that [21:08] it is being made out to be. This isn't [21:09] these companies trying to squeeze more [21:11] pennies out of you as the user. They [21:13] don't care about you. All they care [21:15] about is their enterprise customers that [21:17] they make actual money off of. And they [21:19] don't have enough Nvidia graphics cards [21:21] in their [ __ ] server farms to afford [21:23] those customers and to sell them what [21:25] makes them actual money. Do you [21:27] understand how crazy it is that GitHub [21:29] and Microsoft had to pause signups? That [21:32] is not revenue clawing. That is not a [21:34] desperate attempt to make more money. [21:35] That is trying to reserve compute so you [21:38] stop losing enterprise customers. That [21:40] is the real story here and I want to [21:42] make sure that is emphasized. You're [21:43] thinking about composer too fast from [21:45] cursor. That thing costs like nothing. [21:48] When you're thinking about Opus 47, that [21:50] costs a lot more. We're talking like 20 [21:53] times more the cost. So obviously [21:56] co-pilot they had to make some sort of [21:58] reduction and that's what they did. you [22:00] no longer just get some amount of [22:02] executes. [22:03] >> I hate to skip over this part, but it's [22:04] just entirely wrong. Like I again, the [22:07] core points Prime's trying to make here [22:08] are right. He's just not in the weeds [22:10] enough to know these details. And the [22:12] usage base change isn't they're trying [22:14] to claw money. It's they're trying to [22:16] make sure that users of a $200 a month [22:18] thing or a $40 a month thing or $20 a [22:20] month thing aren't using so much compute [22:22] that they don't have any available for [22:24] their customers doing enterprise [ __ ] [22:26] As my research friends in chat are [22:27] pointing out, capacity is going to be a [22:29] huge problem. I am very scared. Chips [22:32] can't be made fast enough. That is the [22:34] problem here. The next section is where [22:35] Prime starts comparing Microsoft to the [22:37] other companies in the space and he [22:38] brings up one that's very interesting [22:40] that is also very different. See the [22:42] difference between Microsoft and [22:43] Anthropic is Microsoft makes money. I [22:46] know crazy concept for anthropic, but [22:48] Microsoft makes a lot of money. whether [22:50] you like it or not, they're one of the [22:52] biggest companies in the world. And [22:54] therefore, [22:56] they can actually kind of take like a [22:57] bit of a nose dive for a while [23:00] accumulating users. But even at this [23:02] point, they're going, "Hey, we can't we [23:05] can't do this. We got to make more. This [23:07] this is this is silly." This is the same [23:09] mistake as earlier. I also would argue [23:12] this is a misunderstanding of the [23:14] anthropic change before, too. They [23:16] didn't change the potential availability [23:18] of claude code on the $20 a month plan [23:21] to push people up. What they would do is [23:23] heavily restrict how much cloud code you [23:25] get on the $20 a month plan so you're [23:26] incentivized to upgrade. They don't care [23:28] about those subscriptions. All they want [23:30] is more enterprise customers. That is [23:33] what the chain should be seen as. The [23:34] removal of cloud code on the $20 a month [23:36] cloud plans isn't them pricing you up. [23:39] It is them trying to slow down the fire [23:43] hose of compute being wasted as more and [23:46] more users are signing up every day. [23:47] It's the exact same thing as the C-pilot [23:49] pause. Think of those the same way. The [23:51] $20 month change for Claude Code and the [23:54] pausing of signups for Copilot are the [23:56] same. This isn't because Microsoft was [23:58] burning too much money and they're [23:59] clawing it back and trying to make more [24:01] now. This is because they both have the [24:03] same problem. They're both out of [24:04] compute because neither Microsoft nor [24:06] Anthropic are Nvidia. They don't have [24:08] access to unlimited compute. Even Nvidia [24:11] doesn't really. They can just [24:12] manufacture it and they're limited by [24:14] the companies they rely on to actually [24:16] fab the silicon and [ __ ] There is not [24:18] enough compute in the world right now. [24:20] That is the problem. We are working with [24:21] a limited resource and the limited [24:23] resource isn't money. And if we keep [24:24] trying to make it about money and not [24:26] the amount of compute available, you're [24:28] going to start going down conspiracy [24:30] theory paths that just aren't true. [24:31] These behaviors do represent the end of [24:34] something, but this isn't the end of the [24:35] subsidy economy. This is the start of [24:38] compute restrictions really affecting [24:39] the way companies think. That is the key [24:42] difference here. Kic just linked me this [24:43] tweet from two years ago from Sam. It's [24:46] actually almost three years now. We are [24:48] pausing new chat GBT plus signups for a [24:50] bit. Brown frowny face. The surge in [24:52] usage postdev day has exceeded our [24:54] capacity and we want to make sure [24:55] everyone has a great experience. You [24:56] still sign up to be notified within the [24:58] app when subs reopen. Again, OpenAI was [25:01] compute limited so they bought literally [25:02] all of the compute they possibly could. [25:04] It started coming in last year and this [25:06] year and now they don't have these [25:08] problems. Not because they have more [25:10] money than the other companies or [25:11] they're more willing to subsidize than [25:12] the other companies. They just have more [25:14] compute. This all comes down to compute [25:16] in the end. All of this, even Opus 47 [25:18] being dumber is arguably because of [25:20] compute. What's going on here? This [25:22] doesn't even make sense according to [25:23] Microsoft. Now, the real winner honestly [25:25] from all of this is Google. Classic [25:28] Google. They are pouring like a hundred [25:30] plus billion dollars a year into AI and [25:33] they can just do that. And guess what? [25:35] After they pour a hundred billion [25:36] dollars, $200 billion into AI, they [25:40] still make money. That's wild. Like they [25:43] can do so much money for the next year [25:45] after year and they don't have to worry [25:47] about will investors still find me [25:49] attractive. That's probably why you're [25:51] not getting the same level of hype [25:53] coming from Google that you get from [25:55] these companies. I just think this whole [25:57] part is just entirely wrong. That's not [25:59] how it's working for Google. Google is [26:01] providing more free compute than anyone [26:04] right now by far. By [ __ ] far. I'll [26:07] give an example. I'm going to Google [26:09] search for the primogen and it just [26:11] shows me his links. But watch what [26:13] happens if I ask who is the primogen. [26:16] Now we get an AI overview. This is free [26:19] compute. Watch what happens when I do it [26:20] on a signed out account. Oh, it still [26:23] did it. It still did it. These AI [26:26] overviews are free compute. How did he [26:30] get popular? Now I am doing literal free [26:33] compute on google.com. Signed out incog [26:37] browser doing this for free. The people [26:39] saying, "Oh, isn't it cashed? Why am I [26:41] able to do follow-ups? This is a full [26:44] built-in chat interface as part of [26:46] Google. They do a shitload of free [26:48] compute." And I have been told by so [26:50] many people that they're using [26:51] anti-gravity because of how aggressive [26:53] the subsidizations were. In fact, the [26:55] Google models are so bad that they [26:57] included Opus 45 in your Google [26:59] subscription that you could use in [27:00] anti-gravity. And I knew a number of [27:02] people that were using anti-gravity just [27:04] for the subsidized opus usage within it. [27:06] And that ended up exploding so hard that [27:08] Google was the first company to start [27:09] doing the restrictions for where you can [27:11] use the inference where like people who [27:13] were even just building plugins to track [27:15] their anti-gravity usage started getting [27:17] banned. People who were doing plugins to [27:19] link up anti-gravity with open code and [27:20] [ __ ] all started getting banned as well. [27:22] They've been aggressive with that [27:24] because they were too generous initially [27:26] and have had to walk it back since. The [27:28] reason you don't think of Google as [27:29] subsidizing is because they did it so [27:31] aggressively initially they had to clot [27:33] back first. And Google models are so [27:35] [ __ ] trash that nobody talks about [27:37] it. So this is just an entire like blind [27:39] spot for many people in the developer [27:41] world. Google is doing more free compute [27:42] than anybody else. They were subsidizing [27:44] harder than anybody else. They had to [27:45] clot back faster than anybody else and [27:47] more aggressively than anybody else. [27:49] Google's arguably the most extreme [27:51] example of all of this and they make [27:53] their own compute because they make [27:54] their TPUs and they are really behind on [27:56] that to the point where there are rumors [27:58] that they use CPUs both for training and [28:01] for inference because that's what they [28:03] have around. Google is as computed if [28:05] not more so than the other companies and [28:07] has acted accordingly. I don't blame [28:09] anyone for not being in the weeds on [28:10] this [ __ ] because you got to pay a lot [28:12] of attention to notice these things [28:13] happening but this is just a big miss. [28:16] Google is as good of an example as you [28:18] can get of the things I'm talking about [28:19] here. The subsidized compute era isn't [28:23] over, but it is closing because the [28:25] amount of compute available is less and [28:27] less. The real story isn't that your [28:29] subscriptions are being limited or the [28:30] $20 a month tiers are getting more [28:32] restricted. The real story is that RAM [28:33] prices went up. That is what this really [28:35] should be focused on. The cost problem [28:37] here isn't one related to what you're [28:40] getting for 20 bucks. It's one related [28:41] to how expensive the compute has gotten. [28:44] Think about that for a second. Google's [28:46] also competing on the frontier. Google's [28:48] also attempting to win the market. [28:50] Google's also trying to convince [28:51] everybody that their AI is the best AI, [28:53] that they're going to be able to [28:54] shepherd it and take it into the future [28:55] despite the fact of inventing the T and [28:57] GPT and somehow fumbling the bag and not [29:00] being the first one to market. Nobody [29:02] knows the answer to that one. And AI is [29:04] really expensive. Uber just got done [29:07] claiming that within 4 months they spent [29:09] their entire year's budget on AI. Gee, I [29:12] can't believe this is happening. How [29:14] could you tell every employee to [29:16] maximally use AI? By the way, we're [29:18] judging you on AI usage. Oh my gosh, [29:20] you're using too much AI. How'd that [29:22] happen? What the hell? You're not [29:24] supposed to be using a year budget in 4 [29:26] months. What are you even doing? What [29:28] are people doing with all those tokens? [29:30] And here is where the point I've been [29:32] making the whole time comes in. Uber is [29:34] not using the $200 a month subscriptions [29:36] on Cloud Code. They're certainly not [29:38] using the $20 a month ones. Uber is [29:40] using inference through the API. So they [29:43] are paying directly to Anthropic and [29:45] OpenAI. The API costs are what are being [29:47] talked about here. They are paying full [29:49] rate. My usage of these tools is using [29:51] the personal subscriptions which are way [29:54] more generous. I am probably doing [29:55] similar amounts of inference to the [29:57] average employee at Uber. The difference [29:59] is I pay 200 bucks a month for it. They [30:02] pay 2,000 bucks for it. That's the key [30:04] part here. I have heard crazy numbers [30:06] from other companies that I'm hopeful I [30:08] can share in the near future from inside [30:10] sources in many different places. I know [30:12] companies where there are engineers [30:13] spending more on compute and spending [30:15] more on inference than they are paid in [30:17] salaries like pretty often because the [30:20] companies have to be under different [30:22] terms because a lot of these [30:24] subscriptions explicitly say you can't [30:26] use these when we should be negotiating [30:28] a enterprise contract. By the way, that [30:31] one message I sent earlier is still [30:33] going. We're now over 2 hours of this [30:36] message going. And if you're wondering [30:37] why you can't use the sub for your [30:38] enterprise job, I just asked outright. [30:40] Here you go. The consumer terms exclude [30:42] commercial offerings by design. They [30:44] open with a notice clarifying the [30:45] commercial terms govern API keys. The [30:47] console and other offerings that [30:48] reference them explicitly cover that [30:49] does not include Claudia or Cloud Pro [30:50] for individuals. Cloud Code's own legal [30:52] compliance page confirms the split. [30:54] Commercial terms apply to teams. [30:55] Consumer terms apply to free. The [30:57] ordinary individual usage assumption. [30:59] The Cloud Code legal page states that [31:00] advertised usage limits for Pro and Max [31:02] plans assume ordinary individual usage [31:04] of the cloud code in the agency. This is [31:06] a soft cap on what counts as legitimate [31:07] use. There's also a whole section about [31:09] business domains and non-commercial [31:10] evaluation. Sign up with an email owned [31:12] by your employer. Your account may be [31:13] linked to your org's enterprise yada [31:15] yada yada. Where Enthropic permits [31:16] limited evaluation. The evaluation use [31:19] is restricted to personal non-commercial [31:20] use only. They also prevent you from [31:23] using it to make competing products [31:25] which basically everyone is at this [31:26] point like everyone is. And the biggest [31:29] deal is that when you're on the [31:30] subscription tiers, your data can be [31:32] used for training even after opt out. [31:35] Flagged content and feedback can still [31:37] be used. There is no ZDR. There is no [31:40] proper isolation of your data. And most [31:42] businesses will never let you get away [31:44] with that. BAAS only extend to cloud [31:46] code if the customer has executed a BAA [31:48] and has zero data retention enabled. [31:49] Both of which require an enterprise [31:51] contract. Max plans cannot be HIPPA [31:53] covered. They want you on enterprise [31:55] plans. If you're doing enterprise work, [31:56] just be realistic here. So again, if [31:58] you're at an enterprise, you're not [32:00] paying the 200 bucks a month. You're [32:01] paying for your tokens and your tokens [32:02] are much more expensive. Just want to [32:04] make sure you understand that cuz a lot [32:05] of people don't. [32:06] >> I don't even know. Like I don't think [32:07] we're going to be like, "Well, back it [32:08] up everybody. We're not going to use AI [32:10] anymore. You're going to have to go back [32:11] to hand coding everything because AI is [32:13] just not economically viable." No, no, [32:15] no, no, no, no, no, no, no, no. They [32:17] will find a way. Sure, it may be in a [32:20] couple years, but you know what? They're [32:22] going to find a way to make this thing [32:24] viable, but for now, we're starting to [32:26] see the cracks. Things just can't be as [32:28] free as they once were, and the amount [32:30] of usage you're going to be getting is [32:31] clearly and obviously going down. I [32:33] don't want to be the person that's like, [32:34] "Oh, Mr. anti AI for all reasons. Like [32:37] that's why I make this video to show you [32:38] how ridiculous their marketing is. This [32:40] is why they do such hyped out marketing. [32:43] This is why Daario is constantly telling [32:45] you, "Hey, you're out of a job here very [32:46] very soon. You you you Yeah, we're going [32:48] to take your job." I feel super bad [32:50] about it. Oh, I I I want to talk a bit [32:53] about the point he just made before [32:55] about the like amount of compute we get [32:58] going down. I really don't see that. If [33:00] you measure it purely by dollars, sure. [33:03] If you measure it by tokens to the best [33:05] frontier models maybe, but the cost of [33:08] intelligence is consistently going down. [33:10] A good example of this is the artificial [33:12] analysis intelligence index. GBT55 is a [33:15] more expensive model than prior. They 2x [33:18] the token costs, but they also made it [33:20] more efficient. And if you look at the [33:22] highest end X high versions of the [33:25] highest end models, 55 is smarter than [33:28] 54. 55 got 60 points and 54 got 57. It [33:31] is smarter. Where this gets much more [33:33] interesting is when you turn on 5.5 [33:36] medium. When you put 5.5 medium in the [33:39] graphs, you'll notice 5.5 medium [33:42] performed roughly as well as 5.4x high [33:44] did. Same score there. But what's much [33:46] more interesting is the actual cost to [33:48] run. This isn't the cost per token. [33:50] People get too crazy about those [33:53] numbers. To do the grocery store example [33:55] again, if somebody was buying grapes and [33:56] you wanted to charge them, you could [33:58] charge them by the number of grapes or [33:59] by the weight of the grapes. And we are [34:02] looking at the cost per grape [34:04] effectively when we look at the token [34:06] counts, but the actual cost is not just [34:09] how expensive it is per token. It's how [34:11] many tokens were needed to solve your [34:13] problem. And that shows very interesting [34:15] information here where GPT 54 and 55x [34:19] high. Despite, remember 55 is 2x more [34:22] expensive per token, it used fewer [34:25] enough tokens that it only cost like 20% [34:28] or so more from 2,850 bucks to 3350 [34:31] bucks for the exact same data drop run [34:36] everything. And Sonnet, which is cheaper [34:38] than GPT55 per token, ended up being [34:41] even more expensive because it used way [34:43] more tokens. And obviously, Opus is [34:45] basically double the price at $5,300. [34:47] But here's where things get cool, where [34:50] we don't have to worry about cost going [34:51] up a ton. If you were happy with how [34:52] smart 54 was, you're in for a treat [34:55] because 55 medium is just as smart and [34:57] costs less than half as much. 55 medium [35:00] cost $1,200 where 54x high costs $2,800. [35:05] That is what's exciting here. While the [35:07] frontier continues to get more expensive [35:09] as we're pouring more compute into it, [35:11] at any given level of intelligence, [35:13] prices are dropping fast. 54 happened a [35:16] few months ago, like two months ago, not [35:18] even. And 55, a month and a half or so [35:21] later, took that level intelligence, cut [35:23] the price in half, and then offered more [35:25] intelligence for 20% more money. And if [35:28] you're willing to go a little bit lower [35:29] and you want to try out 55 low, which is [35:31] honestly quickly becoming my default, it [35:33] only cost $500 to run this. That is a [35:37] sixth the price of 54. And the [35:39] intelligence level isn't that far off. [35:41] Like, yeah, it's dumber, but it's still [35:43] 51 points. That still puts it up there [35:45] with like Claude Sonnet 46 which was way [35:48] more expensive. And again, if we compare [35:49] the number of tokens, things are very [35:52] entertaining because Sonnet 46 max did [35:55] 200 million tokens to run this bench. [35:57] Opus was more efficient at 110 million, [36:00] but those tokens are very expensive. 54x [36:02] high did 120 mil tokens, so slightly [36:04] more than Opus. When we look down here, [36:06] 55 was down to 75 million. Massive token [36:09] reduction. 55 medium was only 22 million [36:12] and 55 low was the lowest amount of [36:15] tokens used of any of the things we have [36:17] selected at 7 million tokens to score [36:19] higher than deepseeek 4. The models on [36:22] the furthest left and right in this [36:23] chart got the same score. The one that [36:25] used the most tokens, Deepseek V4, and [36:28] the one that used the least tokens 55 [36:30] low got the same score. That's what's so [36:32] cool. The labs that are putting their [36:34] effort in the right places are making [36:36] things more efficient. There is effort [36:38] to make these things cheaper. You might [36:40] not see this because you look at how [36:41] expensive the subscriptions are, how [36:43] many messages you get, or how expensive [36:45] the tokens are, but when you look at how [36:47] expensive real work is to complete, the [36:50] cost is going down rapidly as long as [36:53] you're not using anthropic models. So [36:54] again, just to emphasize how silly and [36:57] [ __ ] all the numbers here are and [36:59] that like the way things are priced to [37:00] you is not based on the amount of [37:02] electricity that it costs, but it's [37:04] based on the availability of compute. [37:06] We've now established that 55, despite [37:08] being 2x more expensive by tokens, ends [37:11] up being a lot cheaper per run, [37:13] especially if you use it on medium or [37:15] low. It ends up being way cheaper than [37:17] 54. To go back to our favorite example [37:19] from earlier, Microsoft, why is it that [37:21] 55, which worst case is 2x more [37:24] expensive, but generally speaking is [37:25] actually quite a bit cheaper than 54, [37:27] why is it a 7.5x [37:29] messages per request when 54 is 1x? If [37:32] this was about the cost to run, these [37:35] numbers make literally no sense at all. [37:37] The reason these numbers are here in [37:39] this way and that Opus is 15x is because [37:42] this actually comes down to how much [37:44] provisioning is available. How many GPUs [37:46] are running 54? How many are running 55? [37:49] How much demand is there for those [37:50] different clusters? How much [37:51] availability is there in those clusters? [37:53] 55 costs more opportunity for Microsoft [37:56] to serve because they're trying to sell [37:58] the 55 inference to companies and [38:00] enterprises right now. These numbers are [38:02] not meant to actually make the more or [38:04] less money. They are meant to get you as [38:06] an individual paying 40 bucks a month [38:08] out of the way of random Fortune 500 [38:11] companies that are paying way more per [38:13] token. That's all this is. And if you [38:15] think of this any differently than that, [38:17] you are falling for conspiracy theory [38:19] [ __ ] That isn't actually how any of [38:20] it works. And by the way, we're now way [38:23] over 2 hours and this one request is [38:25] still running and my co-pilot usage is [38:27] still at 0.4%. So if you think that this [38:30] change that GitHub Copilot is making is [38:33] them being evil and greedy, you just [38:35] don't understand how these costs work [38:37] out. You just don't. And that's fine, [38:39] but don't complain about things you [38:40] don't understand. Go understand it. So I [38:42] wanted to jump on that because it's just [38:43] a a very very common misconception that [38:46] I've seen around a lot. I just feel so [38:49] bad about me taking all of your money. [38:51] I'm so sorry. It's so dangerous. But [38:55] this is the reason why they're doing it. [38:57] They need to raise money. This is why [38:59] Google doesn't do it. They don't need to [39:01] raise the same kind of capital that uh [39:03] Daario and Sam need to raise. [39:05] >> Yeah. Again, I just this is wrong. [39:07] Google was subsidizing really hard. [39:09] Google and Microsoft's problem isn't [39:10] they have more money so they could [39:11] subsidize longer and Google chose not [39:13] to. Microsoft chose to. It's just that [39:16] enterprise these things take a longer [39:18] time to do and do right. Microsoft [39:21] doesn't want to take hits to their [39:22] reputation. So they extended co-pilot's [39:25] generosity for longer than they should [39:27] have. Google doesn't really know what [39:28] reputation means, so they made changes [39:30] way quicker that made their already bad [39:32] product basically unusable. That's the [39:34] other piece here is that co-pilot being [39:36] part of VS Code makes it solid. [39:38] Generally speaking, the C-pilot [39:40] developer ecosystem is okay. It's not my [39:42] favorite thing. There are other [39:44] harnesses I like much more, but the [39:45] Copilot CLI is totally fine. This is not [39:47] the case for Google solutions. Google's [39:50] CLI is not great, and Google's IDE is [39:52] absolute garbage. So, Google didn't have [39:55] much to lose there. But I feel like all [39:56] of these nuances were missed because [39:58] there's so much nuance here. There is so [40:01] many things you have to be tapped in on [40:02] to understand what's going on. And [40:04] again, I do not fault Prime for not [40:06] understanding these things and not [40:08] communicating these things because his [40:09] core point still stands. Things are [40:11] changing and we're getting used to [40:13] things working one way. It is getting [40:14] different. But I also think they're [40:16] getting better in a lot of ways, too. [40:18] The fact that you can pay a hundred [40:20] bucks a month and have enough compute to [40:22] build like three full startups as an [40:24] enduser is unbelievable. The fact that [40:27] these models are getting so much more [40:28] efficient and smarter is incredible. My [40:31] previous video where I said that we were [40:32] hitting the ceiling for what models were [40:34] capable of is so stupid in hindsight and [40:36] retrospect that it's actually kind of [40:38] funny. So, I have made much dumber flubs [40:40] and much less informed videos and [40:42] opinions than anything I'm talking about [40:44] here, but I ended up going way deeper in [40:46] the weeds and learning all of these [40:48] things. So, that's why I wanted to jump [40:49] on this. I still think you should go [40:51] watch the original video. It's linked in [40:52] the description. While I overall agree [40:54] with Prime's analysis here that the [40:57] economics of all of this are shifting [40:58] and changing, I think it's a lot less [41:00] about how much money they can squeeze [41:02] out of you as an end user and it's much [41:04] more about the amount of compute they [41:06] have available. That's all I have to say [41:07] on this one. I hope this breakdown was [41:08] helpful and until next time, save your [41:10] GPUs.  📄 **Full text:** ![[Full Document Contents/Articles/Prime Is (Mostly) Right About AI.md|Prime Is (Mostly) Right About AI by YouTube]] ## Highlights - figuring out how much compute you get per dollar on these other platforms. So internally they measure how much you can get out of a $200 cloud code sub. Previously you could do up to $2,000 in compute. Now they're seeing even crazier numbers where you can get up to $5,000 of inference in that $200 a month plan because Anthropic is subsidizing so aggressively. So I just wanted to give a bit more history because Prime started with a recent drama which was them trying to move Claude Code out of the $20 plan. I see that less as trying to ([View Highlight](https://read.readwise.io/read/01kqw3ekvyh0x676s60zr75vkk)) - Tags: 👻 ai highlighted - much compute they have available to run this model and how expensive it is for them to give that compute up to the subsidized co-pilot users. Historically, C-pilot users were light enough users that they could just eat the cost. If some of those $40 a month users cost them 800 bucks, whatever, it's Microsoft. They make money. They can afford that. The problem is there's enough of them now. And it's not just money that they are spending. It's compute they are losing. And they are compute constraint. They are fighting to reserve as much compute as they can on Azure for all of the other deals for all ([View Highlight](https://read.readwise.io/read/01kqw3ekzsejjhc1bpkastqhgh)) - Tags: 👻 ai highlighted - the other companies they're doing. They need that compute. It's not just like you're costing them too much money when you use co-pilot. is that they need the GPUs that Copilot is using available. The bigger deal about their announcement isn't that they change the pricing. The bigger deal is that they paused signups entirely. You don't pause signups because you want to make more money. You pause signups because you don't have capacity. And this is the really big point I want to drive home. The problem isn't that there isn't enough money to spend. The problem isn't that they are ([View Highlight](https://read.readwise.io/read/01kqw3em55cxzs9ptwdsx17dk8)) - Tags: 👻 ai highlighted - no longer just get some amount of executes. >> I hate to skip over this part, but it's just entirely wrong. Like I again, the core points Prime's trying to make here are right. He's just not in the weeds enough to know these details. And the usage base change isn't they're trying to claw money. It's they're trying to make sure that users of a $200 a month thing or a $40 a month thing or $20 a month thing aren't using so much compute that they don't have any available for their customers doing enterprise [ __ ] As my research friends in chat are pointing out, capacity is going to be a huge problem. I am very scared. Chips ([View Highlight](https://read.readwise.io/read/01kqw3emac30ax6kstnen7qeys)) - Tags: 👻 ai highlighted - is more enterprise customers. That is what the chain should be seen as. The removal of cloud code on the $20 a month cloud plans isn't them pricing you up. It is them trying to slow down the fire hose of compute being wasted as more and more users are signing up every day. It's the exact same thing as the C-pilot pause. Think of those the same way. The $20 month change for Claude Code and the pausing of signups for Copilot are the same. This isn't because Microsoft was burning too much money and they're clawing it back and trying to make more ([View Highlight](https://read.readwise.io/read/01kqw3emg3r3c06da92t9bd110)) - Tags: 👻 ai highlighted - compute limited so they bought literally all of the compute they possibly could. It started coming in last year and this year and now they don't have these problems. Not because they have more money than the other companies or they're more willing to subsidize than the other companies. They just have more compute. This all comes down to compute in the end. All of this, even Opus 47 being dumber is arguably because of compute. What's going on here? This doesn't even make sense according to Microsoft. Now, the real winner honestly from all of this is Google. Classic Google. They are pouring like a hundred ([View Highlight](https://read.readwise.io/read/01kqw3emma4cqaak5906g77x7x)) - Tags: 👻 ai highlighted - for inference because that's what they have around. Google is as computed if not more so than the other companies and has acted accordingly. I don't blame anyone for not being in the weeds on this [ __ ] because you got to pay a lot of attention to notice these things happening but this is just a big miss. Google is as good of an example as you can get of the things I'm talking about here. The subsidized compute era isn't over, but it is closing because the amount of compute available is less and less. The real story isn't that your subscriptions are being limited or the ([View Highlight](https://read.readwise.io/read/01kqw3emrhv8tq7g44kdn09eq7)) - Tags: 👻 ai highlighted - $20 a month tiers are getting more restricted. The real story is that RAM prices went up. That is what this really should be focused on. The cost problem here isn't one related to what you're getting for 20 bucks. It's one related to how expensive the compute has gotten. Think about that for a second. Google's also competing on the frontier. Google's also attempting to win the market. Google's also trying to convince everybody that their AI is the best AI, that they're going to be able to shepherd it and take it into the future despite the fact of inventing the T and GPT and somehow fumbling the bag and not ([View Highlight](https://read.readwise.io/read/01kqw3emvgqw6s94rqme6zxcpz)) - Tags: 👻 ai highlighted