Anthropic Just…wait What - CourtneyR.dev's 2nd Brain

Anthropic Just…wait What ![rw-book-cover](https://img.youtube.com/vi/3pkz-Ie_k_c/maxresdefault.jpg) 🔗 **Source:** [Anthropic Just…wait What](https://www.youtube.com/watch?v=3pkz-Ie_k_c) > [!note]+ Document Notes > tags: #anthropic #xai #spacex #cursor #claude #compute-shortage #ai-infrastructure > author: Theo - t3․gg > source: YouTube > reference: https://www.youtube.com/watch?v=3pkz-Ie_k_c > date created: 2025-01-14 21:22 > > # 🏷️ Title: Anthropic just…wait what >  ## 🧰 Action Inbox - [H1] Track Anthropic's compute scaling challenges — 80x growth in Q1 #anthropic - [H2] Monitor XAI-Cursor $60B acquisition timeline — data partnership implications #xai - [H3] Assess impact of OpenAI's AWS deployment — competitive dynamics shift #openai ## 🧭 Table of Contents - [[#📝 Summary]] - [[#📖 Main Content]] - [[#🗒️ Notes]] - [[#📚 References]] - [[#🗣️ Paraphrase]] - [[#📰 Newsletter]] - [[#✅ Action Items]] - [[#🧪 Hypotheses]] - [[#📊 Methodology]] - [[#❓ Questions]] - [[#🔮 Future Research]] - [[#💡 Implications]] - [[#🧠 Learning Styles Assessment]] - [[#📊 Accessibility Assessment]] - [[#🌍 DEIB Assessment]] - [[#✅ Logical Validity]] - [[#🛡️ DARVO Analysis]] - [[#🎯 Conclusion]] ### 🔗 References - [Anthropic Claude](https://claude.ai) - [XAI Grok](https://grok.x.ai) - [Cursor AI](https://cursor.sh) ## 📝 Summary - What happened: Anthropic partnered with SpaceX/XAI for compute access despite historical animosity between the companies - Why it matters: Reveals the severity of AI compute shortages and how they're driving unprecedented industry partnerships - Key stakeholders: Anthropic (needs compute), XAI (needs data/research), Cursor (caught in the middle), OpenAI (the target everyone fears) - Risks and tradeoffs: Anthropic risks reputation damage working with Elon, but gains 300MW of desperately needed compute capacity - Bottom line: The AI compute crisis is so severe that companies are partnering with enemies to survive ## 📖 Main Content This video reveals the dramatic partnership between Anthropic and SpaceX/XAI, despite Elon Musk previously calling Anthropic "misanthropic" and banning XAI from using Claude models. The partnership stems from Anthropic's severe compute shortage - they experienced 80x growth in Q1 2024 instead of their planned 10x annual growth. The three requirements for AI success are: research talent, quality training data, and compute infrastructure. OpenAI has all three, while other companies have gaps. XAI has massive compute (Colossus 1 with 280,000 H100s) but lacks quality coding data. Anthropic has world-class research and some data but desperately needs compute. The XAI-Cursor deal ($10B for partnership, $60B acquisition option) is primarily about data acquisition. Cursor has the most valuable coding interaction data from developers using multiple AI models across different scenarios. This back-and-forth coding data is crucial for training models that can handle complex, multi-step programming tasks. Anthropic's compute is split across incompatible architectures: Amazon's Trainium chips, Google TPUs, and NVIDIA GPUs. Researchers prefer NVIDIA/CUDA, creating operational complexity. The partnership gives Anthropic access to 300MW of NVIDIA capacity (nearly all of Colossus 1), while XAI moves training to the larger Colossus 2 facility. ## 🗒️ Notes #### ⏱️ [00:00](https://youtu.be/3pkz-Ie_k_c?t=0) — [H1] Theo predicted compute was the pricing bottleneck - Key idea: Anthropic's high pricing was due to compute constraints, not profit maximization - Evidence: "It's not pricing power yet, but capacity" #### ⏱️ [02:13](https://youtu.be/3pkz-Ie_k_c?t=133) — [H2] Historical animosity between Anthropic and XAI - Key idea: These companies have significant bad blood but partnered anyway - Evidence: "Anthropic banned XAI from using anthropic models" in January #### ⏱️ [03:47](https://youtu.be/3pkz-Ie_k_c?t=227) — [H1] Elon's public criticism of Anthropic - Key idea: Musk repeatedly called Anthropic "misanthropic" over bias concerns - Evidence: "Your AI hates whites and Asians, especially Chinese heterosexuals and men" #### ⏱️ [04:58](https://youtu.be/3pkz-Ie_k_c?t=298) — [H1] Elon changed his stance after meeting Anthropic team - Key idea: Direct interaction changed Musk's perception of Anthropic's intentions - Evidence: "Everyone I met was highly competent and cared a great deal about doing the right thing" #### ⏱️ [10:15](https://youtu.be/3pkz-Ie_k_c?t=615) — [H1] Anthropic's 80x growth caught them unprepared - Key idea: They planned for 10x annual growth but saw 80x in Q1 alone - Evidence: "In the first quarter of this year, we saw, if you were to annualize it, 80x growth per year" #### ⏱️ [12:45](https://youtu.be/3pkz-Ie_k_c?t=765) — [H2] OpenAI planned better for compute needs - Key idea: OpenAI bought maximum compute while Anthropic was conservative - Evidence: "OpenAI bought as much compute as they possibly could" #### ⏱️ [14:32](https://youtu.be/3pkz-Ie_k_c?t=872) — [H2] Anthropic splits compute across incompatible architectures - Key idea: Running on Trainium, TPUs, and NVIDIA creates operational complexity - Evidence: "Three entirely different types of hardware they have to run on" #### ⏱️ [18:47](https://youtu.be/3pkz-Ie_k_c?t=1127) — [H1] Compute utilization economics drive subscription limits - Key idea: Unused servers lose money every minute, driving usage restrictions - Evidence: "Every minute your compute isn't running, you are losing money" #### ⏱️ [22:15](https://youtu.be/3pkz-Ie_k_c?t=1335) — [H2] Claude rate limits massively increased - Key idea: More compute availability enabled 4-5x increases in API limits - Evidence: "Tier three went from 800k into 5 mil in more than 4xing" #### ⏱️ [27:33](https://youtu.be/3pkz-Ie_k_c?t=1653) — [H1] XAI-Cursor deal about data acquisition - Key idea: The $60B acquisition is primarily to secure coding interaction data - Evidence: "$10 billion for our work together" plus acquisition rights #### ⏱️ [32:18](https://youtu.be/3pkz-Ie_k_c?t=1938) — [H1] Three requirements for AI success framework - Key idea: Need research talent, quality data, and compute infrastructure - Evidence: "You need research, you need data, and you need compute" #### ⏱️ [35:42](https://youtu.be/3pkz-Ie_k_c?t=2142) — [H2] Cursor has the most valuable coding data - Key idea: Multi-model interaction histories are crucial for training - Evidence: "Cursor has runs for everything" while other labs only have their own model data #### ⏱️ [45:15](https://youtu.be/3pkz-Ie_k_c?t=2715) — [H1] Anthropic gets 300MW of Colossus 1 capacity - Key idea: XAI gave away almost all of Colossus 1 because Grok has few users - Evidence: "300 out of the 425 megawatt and 220 out of the 280 GPUs" #### ⏱️ [47:28](https://youtu.be/3pkz-Ie_k_c?t=2848) — [H1] Compute crisis drives enemy partnerships - Key idea: Desperate need for resources makes companies ignore animosity - Evidence: "The willingness of all of these people to set aside their morals to get compute" ## 📚 References - [Anthropic Claude API Documentation](https://docs.anthropic.com/) - [XAI Colossus Supercomputer Details](https://x.ai/) - [Cursor AI Editor](https://cursor.sh/) ## 🗣️ Paraphrase - Anthropic desperately needed compute after experiencing 80x growth instead of planned 10x, forcing them to partner with XAI despite mutual animosity - XAI has massive unused compute capacity (280k H100s) but lacks the quality coding data that Cursor possesses from developer interactions - The three pillars of AI success - research, data, and compute - reveal why these unlikely partnerships formed, with each company trying to fill critical gaps ## 📰 Newsletter **Compute Crisis Drives Unlikely AI Partnerships** The AI industry just witnessed something unprecedented: Anthropic partnering with SpaceX/XAI despite years of public feuding. This partnership reveals how severe the compute shortage has become. **What happened:** Anthropic saw 80x growth in Q1 2024 instead of their planned 10x annual growth, leaving them compute-starved. Meanwhile, XAI's massive Colossus 1 facility (280k H100s) sits mostly unused because Grok has few users. **The bigger picture:** Success in AI requires three things - research talent, quality data, and compute infrastructure. OpenAI has all three, while competitors have gaps they're desperately trying to fill through partnerships. **For developers:** Claude's API rate limits just increased 4-5x, and the peak-hour restrictions are gone. If you've been hitting limits, now's the time to scale up your AI-powered applications. Resources: [Anthropic API Docs](https://docs.anthropic.com/), [Claude Pricing](https://www.anthropic.com/pricing), [AI Compute Economics](https://epochai.org/) ## ✅ Action Items #### DevRel - Education: Create workshops on AI compute economics and scaling challenges - Developer marketing: Content series on choosing AI providers based on infrastructure reliability - Community management: Monitor developer sentiment around API limit changes and pricing #### OSPO - Strategic Alignment: Track compute partnerships affecting open source AI model availability - Community Engagement: Evaluate impact of proprietary compute deals on open source alternatives - License Compliance: Review terms for AI models hosted across different cloud infrastructures #### Community - Contributors: Document best practices for handling API rate limits in AI applications - Ecosystem: Monitor how compute shortages affect smaller AI companies and open source projects #### WordPress - Core AI: Assess impact of increased Claude limits on WordPress AI integrations - Performance: Evaluate trade-offs between different AI providers for WordPress features ## 🧪 Hypotheses - Companies with insufficient compute planning will be forced into disadvantageous partnerships or acquisitions within 12 months - The value of high-quality interaction data (like Cursor's) will exceed $10B annually as models become commoditized - OpenAI's "all three pillars" advantage will diminish as competitors fill gaps through partnerships - Compute capacity will become the primary differentiator between AI companies, not model architecture ### 📊 Methodology - Track compute capacity announcements vs. actual usage patterns across major AI labs - Analyze API pricing and limit changes as indicators of supply/demand imbalances - Monitor partnership announcements for patterns indicating resource gaps ### ❓ Questions - How will Anthropic maintain research independence while depending on XAI's compute infrastructure? - Can the XAI-Cursor data partnership create a viable challenger to OpenAI's integrated approach? - What happens when Colossus 2 comes online - does the partnership dynamic shift again? ### 🔮 Future Research - Investigate the true cost of operating AI models across different hardware architectures - Analyze the strategic value of developer interaction data for AI training - Study how compute shortages are reshaping AI industry consolidation patterns ### 💡 Implications - Practice: Developers should diversify across AI providers to avoid single points of failure during compute shortages - Policy: Regulators may need to address compute concentration and its effects on AI competition - Security: Dependency on competitor infrastructure creates new attack surfaces and data leakage risks - Education: Understanding compute economics becomes crucial for AI strategy decisions - Funding: Massive compute investments (billions) are now table stakes for frontier AI companies ### 🧠 Learning Styles Assessment Reading level estimate: 12th grade #### 👀 Visual learners - Present: Charts showing compute capacity comparisons, partnership diagrams - Improve: Infrastructure architecture diagrams, data flow visualizations #### 🔊 Aural learners - Present: Video format with clear narration and emphasis - Improve: Podcast version, audio summaries of key technical concepts #### 🗣️ Verbal learners - Present: Clear definitions of technical terms, analogies for compute economics - Improve: Glossary of AI infrastructure terms, more business analogies #### 🧩 Kinesthetic learners - Present: Step-by-step breakdowns of partnership implications - Improve: Interactive cost calculators for AI compute, hands-on API limit testing #### 🧠 Logical learners - Present: Clear cause-and-effect reasoning for partnerships, numerical data - Improve: Decision trees for AI provider selection, ROI calculations #### 🧑‍🤝‍🧑 Social learners - Present: Discussion of industry relationships and dynamics - Improve: Community forums for discussing AI infrastructure strategies #### 🧘‍♀️ Solitary learners - Present: Comprehensive analysis suitable for independent study - Improve: Self-assessment tools for AI compute needs, reflection prompts #### 🌿 Naturalistic learners - Present: Ecosystem analogies for AI industry relationships - Improve: Sustainability implications of massive compute infrastructure ### 📊 Accessibility Assessment #### Screen reader semantics - Present: Clear heading structure, descriptive video content - Improve: Detailed alt text for charts and diagrams mentioned #### Keyboard navigation - Present: Standard video controls accessible via keyboard - Improve: Timestamp navigation, transcript search functionality #### Contrast and readability - Present: High contrast video player, clear audio - Improve: Visual elements in charts need higher contrast ratios #### Text resizing and reflow - Present: Responsive video player - Improve: Scalable transcript text, adjustable playback speed #### Media and captions - Present: Audio content with clear narration - Improve: Auto-generated captions, downloadable transcript #### Forms and interactions - Present: Standard video controls - Improve: Accessible speed controls, chapter navigation ### 🌍 DEIB Assessment #### Cultural breadth - Present: Global perspective on AI industry dynamics - Improve: Include impacts on AI development in different regions #### Inclusive language - Present: Technical jargon explained in context - Improve: Avoid gendered assumptions about tech industry roles #### Neurodiversity support - Present: Clear section breaks, consistent terminology - Improve: More frequent summarization, visual organization #### Representation - Present: Focus on companies and technical concepts - Improve: Acknowledge diverse teams behind these companies #### Psychological safety - Present: Professional tone despite discussing conflicts - Improve: Content warnings for competitive/adversarial framing ### ✅ Logical Validity #### Strong reasoning - Deductive strength: Clear logical chain from compute shortage to partnership necessity - Inductive strength: Pattern recognition across multiple company behaviors - Causal reasoning: Well-established cause-effect relationships between growth and resource needs - Analogical strength: Effective comparison of AI requirements to traditional business needs #### Fallacies - Ad hominem: Some personal attacks on company leadership detract from technical analysis - Hasty generalization: Broad claims about company cultures based on limited public information #### Tone of voice - Overall tone: Analytical and technical with occasional inflammatory language about company relationships ### 🛡️ DARVO Analysis - Deny: No significant denial patterns detected - Attack: Some attacks on Anthropic's business practices and Google's execution - Reverse victim/offender: Frames companies as both victims --- ## Full Transcript [00:00] Looks like Theo was spot on in his [00:01] latest video. Compute was the [00:02] bottleneck. Pretty sure you called this [00:04] the other day. So Theo was right. It's [00:06] not pricing power yet, but capacity. [00:08] Let's hope it lasts at Theo. Compute [00:11] really was the reason for egregious [00:12] pricing. It's a good day to be right. [00:14] Seriously though, I just don't get how [00:15] anyone didn't understand this before. It [00:17] was pretty obvious Anthropic's biggest [00:19] problem is that they didn't have enough [00:20] compute to keep up with the demand that [00:22] they were seeing. And that demand has [00:23] been crazy. Absolutely unprecedented. [00:25] We'll talk a bit about that, but first [00:27] we need to talk about today's news. the [00:29] new partnership between Enthropic and [00:30] SpaceXi/ [00:32] whatever Elon calls the company now. [00:34] It's kind of crazy when you know the [00:36] history of these companies, but this is [00:37] real. You can even see it in this uh oh [00:40] no, this is a post from January when [00:42] Anthropic banned XAI from using [00:46] anthropic models. Huh. Definitely not [00:48] related to today's announcement with [00:50] higher usage limits for Claude and a [00:52] compute deal with SpaceX. Yeah, this was [00:54] not an expected thing. There has been a [00:57] lot of historical beef between these two [00:59] parties and seeing them come together in [01:00] this way shows just how bad the compute [01:03] crisis is. Anthropic does not like [01:06] SpaceX. Anthropic does not like XAI. [01:10] Anthropic does need compute. Thankfully, [01:13] Elon's a huge fan of Anthropic and is [01:15] more than happy to do this partnership. [01:16] Oh, your AI hates whites and Asians, [01:19] especially Chinese heterosexuals and [01:21] men. This is misanthropic and evil. Fix [01:23] it. Frankly, I don't think there's [01:25] anything you can do to escape the [01:26] inevitable irony of anthropic ending up [01:28] being misanthropic. This is a joke he [01:30] has been making a lot a lot. Just search [01:35] Twitter for Elon posts that say [01:37] misanthropic and there are dozens. But [01:40] apparently that changed too. By way of [01:42] background, for those who care, I spent [01:44] a lot of time last week with senior [01:45] members of the anthropic team to [01:47] understand what they do to ensure Claude [01:49] is good for humanity and was impressed. [01:51] Everyone I met was highly competent and [01:53] cared a great deal about doing the right [01:55] thing. No one set off my evil detector. [01:57] So long as they engage in critical [01:59] self-examination, Claude will probably [02:00] be good. After that, I was okay with [02:02] leasing Colossus 1 to Anthropic as [02:05] SpaceX had already moved training to [02:07] Colossus 2. This one sentence has so [02:11] much information in it that I am very [02:13] excited to break down. But unlike Elon [02:15] and Anthropic, I don't have billions of [02:18] dollars of GPUs. I only got a couple [02:20] 5090s and if I want to get any more, I'm [02:22] going to have to take a quick break for [02:23] today's sponsor. Stop me if you've heard [02:25] this one before. You've been working on [02:26] some code with an agent and you feel [02:28] like you got to comb through every [02:29] single line it spits out because it just [02:31] keeps getting this simple thing wrong. [02:33] Maybe you even have a PR bot that's [02:35] going to review the code once it gets [02:37] shipped up onto GitHub. But what good is [02:38] that going to do when it's running in [02:39] your agent? You just sit there watching [02:41] so you can tell it what it did wrong to [02:42] fix itself. Wouldn't it be nice if it [02:44] knew what it was doing wrong when it [02:46] made those mistakes so that you didn't [02:48] have to be there monitoring it the whole [02:49] time? Well, today's sponsor is Code [02:51] Rabbit, and you might think of them as [02:52] the code review tool that happens on [02:54] GitHub, and it is really good at that. I [02:56] even found myself pushing up PRs for [02:58] code that wasn't ready yet, just because [03:00] I wanted the feedback that I got from [03:02] Code Rabbit. Well, now they have a [03:03] really, really cool additional feature [03:05] that makes it so much better for those [03:07] agentic loops, the CLI. You might be [03:09] thinking, why do I need a CLI to review [03:11] code? I just use the GitHub CLI to push [03:14] the code. Well, this is actually what's [03:16] cool about it. Sure, you can run the [03:17] command to review some code before you [03:19] push it up to GitHub, but you can also [03:21] let your agent run it. You even tell [03:23] your agent that it should run the CLI [03:25] for a review before telling you that [03:27] it's done. And it will just work. It'll [03:29] run the CLI and it'll take advantage of [03:31] their crazy context that knows [03:32] everything about not just the one [03:34] project you're working on, but the other [03:35] repos that your organization has in [03:37] order to find how one thing in one place [03:39] can break something else somewhere else. [03:41] It really does feel like the enterprise [03:42] ready AI code reviewer that isn't just [03:45] again on GitHub. It is everywhere else [03:47] in your stack, too. By the way, you can [03:49] use it with Slack now. It can even make [03:50] PRs. It's pretty dang cool. I'll talk [03:52] more about this very soon, I promise. [03:54] It's free to get started and free for [03:55] open source. What are you waiting for? [03:56] Get better reviews at sidv.link/code [03:59] rabbit. This is a wild partnership and [04:02] we have a lot of things to talk about [04:04] within it. The obvious one is how does [04:07] this affect us specifically cloud code [04:10] users? There's a question of what's [04:12] Anthropic's compute problem? Like why is [04:15] this a thing that they're willing to do? [04:16] Is it really that bad that they're going [04:18] to partner with a person that they hate [04:19] and hates them? Like that just seems [04:21] insane, right? But most importantly, WTF [04:24] is going on at XAI. I honestly think [04:27] this is the most interesting part of all [04:29] of this. The financials and direction of [04:32] XAI is fascinating. Reminder, they just [04:36] put out a really weird bid for Cursor. [04:39] So, I'm going to do a lot of speculation [04:41] on what's going on at XAI and Elon. I [04:43] have historically said that I still [04:45] think XAI has a real chance of catching [04:47] up and being a frontier lab, but the [04:49] reason why has historically been [04:51] compute. And believe me, we will be [04:53] talking about that. So, make sure you [04:54] watch to the end on this one because [04:56] this sentence here is probably the most [04:58] important piece of all of this [05:00] information. Going to change the order a [05:01] little bit and we're going to start with [05:02] what Anthropics compute problem is. A [05:05] lot of people seem to think the changes [05:07] the Anthropic's been making recently are [05:09] in order to make more money off their [05:11] end users. While making money is [05:13] absolutely part of their plan and what [05:15] they're trying to do, it is not the [05:18] reason that these changes are happening. [05:20] If you saw the attempts to remove Claude [05:22] Code from the pro plan and thought, "Oh, [05:24] they're trying to charge us more money [05:25] and get a bunch of people to jump to the [05:27] $100 tier." You just don't get what's [05:28] going on. They don't want to sell more [05:31] of any of those tiers. Their goal was [05:33] explicitly to try and free up compute [05:35] because they just did not buy enough [05:37] GPUs. All of these announcements [05:38] happened today at the Code with Claude [05:41] conference that for some reason I didn't [05:43] get invited to. I don't know. Invite [05:45] must have been lost in the mail. [05:46] Everyone knows I'm a huge fan of Claude [05:47] and Cloud Code. So, I surprised I didn't [05:49] get invited. Anthropic failed to predict [05:52] how much compute they were going to need [05:54] this year. And that failure has made [05:56] things very hard for them for serving [05:58] their customers. Normally they'd kind of [06:01] dance around these details, but I was [06:03] surprised with how candid Dario was in [06:05] this conference and I wanted to just [06:07] share what he had to say. [06:09] >> PR's inflects due to due to the work [06:12] that that Claude is doing and we've seen [06:14] it externally because actually this is [06:16] the first year we've grown faster than [06:18] the exponential. So, you know, we tried [06:21] to plan very well for a world of 10x [06:24] growth per year. Um, in the first [06:27] quarter of this year, we saw, [06:30] >> if you were to annualize it, 80x growth [06:33] per year in in in in in in in in [06:37] revenue and usage. Also, ADX more ins [06:41] than necessary, but regardless, [06:44] >> ADX growth is kind of insane. No one can [06:48] plan for that properly. I can sympathize [06:51] with them for that. But it is also worth [06:54] noting OpenAI did kind of plan for this. [06:57] OpenAI bought as much compute as they [07:00] possibly could. Daario was admittedly [07:03] quite conservative with his compute [07:04] purchases and allocations. He wanted to [07:07] wait and not over buy and overcommit to [07:11] buying compute and then be in a rough [07:13] spot because they had more compute than [07:15] they knew what to do with. It kind of [07:16] seems like right now the biggest winners [07:18] are the people who jumped on the [07:19] realization that you need a lot of [07:20] compute. So they bought as much as [07:22] possible except for one company because [07:25] compute is only useful if there's people [07:28] around to use it and when you have 80x [07:31] growth in all these enterprises who are [07:33] trying to use your inference then you [07:36] need the compute but if you don't [07:38] matters less in fact it actually ends up [07:41] being quite expensive. Remember that [07:44] fact because we'll be going back there [07:45] in a bit but I want to talk more about [07:47] what this means for us as users first. [07:50] Anthropic didn't have the compute they [07:52] needed and they also have to split their [07:54] compute across multiple different [07:55] providers that are all entirely [07:57] different from each other. For example, [07:59] they're doing 5 gawatt of compute with [08:01] Amazon which is going to be on tranium [08:04] which is one architecture that notably [08:06] isn't Nvidia and isn't CUDA. They also [08:08] have a deal with Google and Broadcom. [08:11] This deal with Google and Broadcom is [08:13] Google's TPUs which notably also are not [08:15] Nvidia. They separately have a [08:17] partnership with Microsoft and Nvidia [08:19] that includes $30 billion of Azure [08:20] capacity and they're helping invest in [08:23] American AI info with Fluid Stack. But [08:25] these three here are three entirely [08:27] different types of hardware they have to [08:30] run on. They even say here that they [08:31] train and run Claude on a range of AI [08:33] hardware. Tranium, Google TPUs, and [08:35] Nvidia GPUs. It is my personal belief [08:38] that the researchers do not want to use [08:40] the first two here. Ironically enough, [08:43] the training isn't happening on Tranium [08:46] because the researchers all want to work [08:47] in CUDA. I have yet to meet an AI [08:49] researcher that doesn't prefer to just [08:51] use CUDA for everything. So, my [08:53] hypothesis for a while has been that [08:55] they're trying to move as much comput as [08:56] possible for inference over to Tranium [08:58] and Google TPUs to free up their NVIDIA [09:01] allocation so the researchers can use [09:03] it. But even then, they didn't have [09:05] enough and they needed more. Do you know [09:07] who does have a lot of NVIDIA compute? [09:10] SpaceX. We'll go back to that soon [09:12] because the cool thing for all of us is [09:15] that they are increasing the usage [09:16] limits now because they are finally not [09:18] as compute constrained. Remember, I've [09:20] been saying this many times. The way [09:22] that these subscriptions work isn't that [09:25] they have a thing that they're trying to [09:27] sell for an amount of money and they're [09:28] trying to just get you up to that tier. [09:29] It's actually the opposite. They're [09:31] making these subscription tiers to get [09:33] people into the platform, but the cost [09:36] is not so simple to measure. is not like [09:40] selling computers directly where if you [09:42] make a thousand laptops and you sell 800 [09:44] of them, you're losing money because you [09:46] didn't sell those 200 laptops, but then [09:48] you can just keep them around and slowly [09:49] sell them until you get your profit in [09:51] the end. Every minute your compute isn't [09:53] running, you are losing money. So, if [09:55] you have thousands of servers, let's say [09:57] you have 10,000 servers that can serve [09:59] users, your peak hours, you're hitting [10:02] those 10,000, but in your less peak [10:03] hours, you're only using 6,000 of them. [10:06] you have 4,000 compute units, however [10:08] you want to measure it, that are just [10:10] sitting there doing jack [ __ ] That's [10:11] money you're potentially losing. But if [10:13] you have more requests coming in than [10:15] you have the compute to serve, you're [10:17] screwed. And a lot of the changes we've [10:18] been experiencing as cloud code users on [10:20] those traditional subscription tiers [10:22] isn't that they are out of computers to [10:24] sell us, is that there is no compute [10:26] available at certain times. That's why [10:28] they did the thing where they had the [10:29] peak hour reduction where during work [10:31] hours we got less inference than we [10:34] would on off hours because again they're [10:36] just trying to use the compute they have [10:37] available to them. Now they have way [10:39] more compute available. So the peak hour [10:41] limit reduction is gone. They also [10:43] doubled the cloud code 5h hour rate [10:45] limits. Sounds awesome and in a lot of [10:47] ways it is. But there is a catch. [10:50] There's two types of usage when you use [10:52] cloud code in cloud subs. You have your [10:55] 5h hour limit. So that's your current [10:57] session. when you send a message, it [10:59] starts and then you can only do so much [11:01] inference within a 5 hour window. [11:03] Separately, there's a weekly limit that [11:05] resets every seven days that is on top [11:08] of your 5h hour limit. So, if you are [11:10] hitting the 5 hour limit a lot cuz [11:12] you'll like code really heavily for a [11:14] couple hours and then not touch code for [11:16] a few days, this will help you a lot. [11:18] But if you're using this multiple times [11:20] a day over multiple windows every day [11:22] and you were hitting the weekly limits, [11:24] you're no better off than you were [11:26] before. And if you're using all these [11:27] fancy new parallel agents and like auto [11:30] review tools that they're starting to [11:31] introduce that keep these things running [11:33] way longer, you're not going to benefit [11:35] a lot from these changes. If you're [11:37] using things in bursts and you hit the 5 [11:39] hour limit all the time, awesome, you're [11:41] in a good spot. But the most exciting [11:42] thing for me is the bump to the API rate [11:45] limits. This is for the enterprise [11:48] customers. We got screwed hard with this [11:50] whenever a new model came out and they [11:52] would only allocate us so much tokens [11:55] per minute when the new models drop and [11:58] then T3 chat would have problems because [12:00] users would be trying to use Opus and [12:02] some of these limits were egregious. [12:04] Like 160,000 output tokens per minute on [12:07] tier 3 might sound like a lot, but I've [12:10] done individual runs that can hit the [12:12] millions. Obviously, those took more [12:14] than a couple minutes, but if you have [12:16] three people doing output at the same [12:19] time, hitting these limits was not very [12:21] hard, but they have massively increased [12:24] them, especially on the input side. On [12:26] tier one, the input token per minute [12:28] limit was 30,000, which was nothing. It [12:31] was like you could get cut off on one [12:33] request and often were. And the 8,000 [12:36] output was also just laughable. This [12:38] means you could never have more than two [12:40] requests running at once, probably not [12:41] more than one. and the input token limit [12:43] you could hit as soon as you make a [12:44] request. So that being bumped is huge. [12:47] Tier two went from 450k into 2 mil in. [12:49] Tier three went from 800k into 5 mil in [12:53] more than 4xing which is crazy. And then [12:55] tier 4 went from 2 mil to 10 mil 5xing [12:59] the input tokens. Very nice for those of [13:01] us who are trying to host things through [13:03] anthropic. I have a lot of fun [13:05] conspiracies I could dive into about [13:06] this part too. Opus models can be hosted [13:09] in multiple different places. As [13:11] mentioned before, they work on Tranium, [13:13] they work on Google TPUs and they also [13:16] can be hosted in Azure as well as of [13:18] course Google and Amazon servers. So [13:20] this is one of the reasons that [13:21] Anthropics done so well in enterprise is [13:23] because Anthropic positioned themselves [13:24] to be in every single cloud. So if your [13:26] company's a Google cloud company, if you [13:28] guys are an Azure company or most likely [13:30] you're an AWS company, you could use [13:32] cloud models within your existing setup [13:35] which was huge for them. It's a big part [13:36] of why Anthropic did so well and why [13:38] OpenAI was struggling because not only [13:40] was OpenAI limited to just serving on [13:42] Azure and their own inference, Azure was [13:45] not hosting their inference properly. I [13:47] after 14 months of bitching, I got them [13:49] to fix it finally, but we'll talk about [13:50] that in a little bit. The thing I want [13:52] to showcase here that I think is really [13:54] interesting is around how these [13:56] different deployments are going. I spend [13:59] way too much time in these charts. This [14:00] is the output speed over time. This [14:03] measures how fast the model runs in the [14:05] different providers. And you'll notice [14:07] here, Amazon is by far the fastest, [14:09] regularly over 80 TPS, sometimes hitting [14:11] as high as 100, when Anthropic's own [14:13] official hosting is usually in the 50s. [14:15] Google is a little lower, but still [14:17] solid in the like 50 to 70 range and [14:20] meaningfully better than Anthropic and [14:22] Azure. But if we turn off Amazon and [14:25] Google, you might notice something here. [14:27] These numbers are really close. Like [14:31] absurdly close. I've been hearing some [14:34] conspiracies that I'm starting to [14:36] believe that Azure isn't actually [14:39] hosting anthropic models right now. [14:41] Rather, they are piping through requests [14:43] to the official anthropic APIs. It could [14:46] be that there's just another NVIDIA [14:48] cluster and that they're hosting them [14:49] the same way and that the inference is [14:50] roughly the same speed. But I don't [14:52] necessarily believe that at this point [14:54] because I've seen two companies hosting [14:55] Nvidia and not getting any even close to [14:58] similar performance OpenAI. This [15:01] mirroring near exact mirroring suggests [15:05] to me that these models are being piped [15:09] through Anthropic on Azure right now. I [15:11] do see this potentially changing in the [15:13] future, but it does seem very clearly [15:16] like these are mirroring each other such [15:18] that it would surprise me if they were [15:20] coming from different data centers. I [15:21] could be entirely wrong on this, but I [15:23] wanted to bring it up cuz I find it [15:24] really interesting, especially because [15:25] both Amazon and Google have meaningfully [15:28] better performance and also meaningfully [15:29] better latency characteristics over [15:31] time. Just thought that was an [15:32] interesting thing worth calling out. I [15:34] have heard others say that Amazon and [15:36] Google are also piping through to [15:38] Anthropic and that is obviously wrong [15:40] for a billion [ __ ] reasons. But now [15:42] you can see in this chart, yeah, [15:43] obviously. So if you think that all of [15:45] them are just piping through, no. If you [15:47] think Azure might be piping through, [15:49] potentially. So we've covered the [15:51] compute problem. We've covered how this [15:52] affects us. Time to talk about the [15:55] exciting part here. What the [ __ ] [15:57] going on at XAI? Just a few weeks ago, [16:00] this post kind of rocked my world as an [16:03] early investor in Cursor. So again, [16:05] account for bias here. Space XAI and [16:08] Cursor AI are now working closely [16:09] together to create the world's best [16:11] coding and knowledge work AI. The [16:13] combination of Curser's leading product [16:15] and distribution to expert software [16:17] engineers with SpaceX's million H100 [16:19] equivalent Colossus training [16:20] supercomputers will allow us to build [16:22] the world's most useful models. Cursor's [16:24] also given SpaceX the right to acquire [16:25] Cursor later this year for 60 billion or [16:28] pay 10 billion for our work together. [16:30] There are three things you need to be [16:32] successful in the AI space. You need [16:35] research, specifically researchers, the [16:38] scientists that are capable of making [16:39] all of these cool things happen. You [16:41] need data, both data you collect from [16:43] the rest of the world, as well as data [16:44] that you can manufacture and processes [16:46] to create that data in order to train [16:48] the models with good data. And of course [16:51] to do all of this you need compute. [16:53] Getting all of these things is [16:55] difficult. And the reason OpenAI was so [16:58] successful early on is they found novel [17:00] solutions on the research side. They [17:03] scraped and found a ton of good data. [17:05] And the compute in particular was a weak [17:08] point initially because they didn't have [17:09] the money. That's part of why there's [17:10] all of the beef going on right now with [17:12] OpenAI and Elon. The reason that they [17:14] had to go for profit is they had no [17:16] other way to acquire the compute they [17:17] needed when they started believing [17:18] scaling loss. So, we know XAI has [17:22] compute. That's been established. We'll [17:24] talk about the amount in a bit, but I [17:25] want to talk about the data side first. [17:27] You might think that X has a ton of data [17:30] because they have all of the data [17:31] historically throughout all of Twitter. [17:33] That data exists, and yes, they have it, [17:36] but personally, having spent a lot of [17:38] time on Twitter, I don't know if I would [17:39] want anything that is very aware of [17:41] Twitter on my team, much less in my [17:43] models. It is very easy for data that [17:45] seems good to cause problems. This is [17:48] one of my favorite anecdotes. This info [17:50] is shared by a researcher at Reductto AI [17:53] about the results they had before and [17:55] after cleaning up their data. They had a [17:57] bunch of loss spikes that just kind of [17:59] vanished once they cleaned out all of [18:01] the data. And the funny example somebody [18:03] shared after that I thought was [18:05] absolutely hilarious was this one about [18:08] GPT3 training having lost spikes because [18:10] they scraped from a subreddit of [18:11] microwave noises. The training batch was [18:14] literally just text like m over and over [18:17] again. This gets much funnier because [18:19] not every company has realized how [18:21] important it is to clean their data. For [18:23] example, if you ask a Gemini model to [18:25] make microwave noises, there's a good [18:27] chance you'll hit this bad data and it [18:28] will just loop infinitely until it runs [18:30] out of output tokens. I've experienced a [18:33] lot of weird things like this with [18:34] Gemini. I wasn't able to reproduce this [18:35] right now, but yeah, it is what it is. [18:38] Good, reliable data is harder to get [18:41] than you would think. And that data is [18:44] not something that XAI has. They have [18:46] Twitter, but Twitter is not the source [18:48] you want for doing the complex problems [18:50] that we all want our AI models to be [18:52] capable of. And since XAI got in a [18:55] little late, they couldn't scrape the [18:56] web for all the data because that's not [18:58] really viable anymore. And besides that, [19:00] most of the major labs are moving away [19:02] from data they scraped and towards data [19:04] they are creating, buying, [19:06] manufacturing, labeling, doing all these [19:08] things for themselves. There's even a [19:09] whole bespoke subindustry of companies [19:12] that are generating human data, labeling [19:14] it, and then selling it to the labs. [19:15] There are companies who have engineers [19:17] building full products, never with any [19:19] intent to ship the product just to ship [19:22] their history, building the product so [19:24] the models can train against that. We've [19:26] all had an experience like this with AI [19:29] chat, right? Where we use something in [19:31] cursor or another similar tool. We ask [19:33] it to start building a thing, it does a [19:35] part, we ask it to do the next thing, it [19:36] does it, we ask it to do the next thing, [19:37] etc. Like, I'm not crazy. We've all done [19:39] this back in the day using agents to [19:41] code. Once the chat appeared in our [19:43] editors, we all did stuff like this. [19:45] Now, we have these runs. We probably [19:48] have millions of these across all the [19:50] developers using all these different [19:51] tools, whether it's cursor or cloud code [19:53] or whatever else. If you're building in [19:55] this way where you have a thread where [19:57] you ask it to do a thing, it does the [19:59] first part, you ask it to do the next [20:00] part, does that, and you do that over [20:01] and over, maybe you make corrections, [20:03] too. Maybe you say here instead, uh, you [20:06] actually missed something from part two. [20:10] And the model responds, oh yeah, my bad. [20:14] I'll fix that now by yada yada. We've [20:17] all had this type of experience writing [20:20] code with models. You can take this [20:22] history and feed it back in during [20:24] training, but the behavior is not going [20:26] to be any better. You're just [20:27] reinforcing the behavior on this [20:29] existing history. Do you know how we [20:31] could make this better? Let's do one [20:33] small change. First, I'm going to break [20:35] out this start by building part one as [20:37] its own separate message. So, we have I [20:40] want to build this feature. Imagine you [20:42] describe the feature and then it says [20:43] start by building part one. So, you [20:45] separate those. And now we're going to [20:46] make a couple more changes to this [20:48] history. We're going to do next cuz [20:51] remember we're trying to make the model [20:52] behave the way that this history shows. [20:55] We want a really good history to use for [20:57] reinforcement learning so that we can [20:59] make the model behave how we want it to. [21:01] So, ready for this really complex trick? [21:03] We're going to take all of these user [21:05] messages after the initial request and [21:07] we're going to move them to the left. [21:09] Oh, huh, that looks familiar. That looks [21:12] like the oneshots that we all do now [21:14] instead. If you take good histories [21:17] where people built the thing they wanted [21:18] to and it worked, take all of the [21:21] follow-up user messages, shift them to [21:23] the left, train on it. We generate the [21:26] data the labs need by having the back [21:28] and forth with the model. We gave the [21:31] data on what it was doing wrong to the [21:34] labs and now they can use that to make [21:36] the model smarter by taking our [21:38] follow-ups, moving them over and now the [21:41] model can just be trained on this. And [21:42] we've all seen this if you read the [21:44] reasoning traces and the actual things [21:46] the model does where it breaks the thing [21:48] down into steps. It does the first step. [21:50] It notices something wrong. It fixes it [21:51] all without user interaction. All of [21:53] that is just the things we did before [21:56] being used for RL. And now we talk about [21:58] why this matters here. The data Twitter [22:02] has is the back and forths we have with [22:05] each other on Twitter as people. That's [22:08] not useful for these cases. Do you know [22:11] what source of data is incredibly useful [22:13] for this? I'll give you a hint. I don't [22:16] open it a lot anymore. Cursor has the [22:19] greatest corpus of this data imaginable [22:22] because every generation of model that [22:24] we used in cursor got all of our back [22:27] and forths in the chat. It has all of [22:29] this information. It has all of this [22:30] data. It has everything it needs to be [22:33] trained better. And that's why composer [22:35] 2 is so much better than Kimmy at coding [22:38] stuff. even though it's based on Kimmy. [22:39] It's because when you take a model that [22:41] is pretty smart and you run it through [22:43] these loops for [ __ ] ever, it gets [22:45] way more reliable at these long-term [22:47] tasks. So, we've established XAI has the [22:51] compute, but they don't have the data. [22:53] And they may or may not have the [22:54] researchers. A lot of the people they [22:55] had left, so it's hard to say for [22:57] certain at this point, but they very [22:59] well could. Hard to know for sure. They [23:01] only had the compute for sure, though. [23:03] What about Curser? Cursor doesn't have [23:05] like traditional frontier lab research. [23:08] Like they're not pre-training models [23:09] themselves right now. Could see that [23:11] changing. They do have the data. I would [23:13] argue Cursor has the best data of any [23:15] lab because the other labs like [23:17] Anthropic only have data for cloud runs. [23:20] Labs like OpenAI only have data for GPT [23:23] runs. Cursor has runs for everything. So [23:26] Curser wants to do their own [23:27] pre-training. They're investing more and [23:29] more heavily into research. They have a [23:31] shitload of data. a truly absurd level [23:34] of data, but they never had the compute. [23:36] And that's why this announcement was [23:37] very interesting and in my opinion made [23:39] a lot of sense. SpaceX and XAI do not [23:43] have the data they need to train a model [23:44] to code. They could scrape GitHub, but [23:46] scraping GitHub just shows you good code [23:48] that you would throw in pre-training. It [23:50] doesn't show you how the model should [23:52] behave to write that code. Gemini models [23:54] have the same problem here. By the way, [23:56] if you were wondering why they [23:57] subsidized using Opus so hard inside of [23:59] anti-gravity, it was this. They wanted [24:01] to get the same data. The data for how [24:03] the models behave in a thread is so [24:06] valuable and SpaceX does not have that [24:08] for code. Cursor has all of that data [24:10] for code. I would argue that the $10 [24:13] billion fee here is just for the data. [24:17] SpaceX could choose to not buy Cursor at [24:19] the end of this year. And if they choose [24:21] to not, they are effectively paying $10 [24:23] billion for the data that they are [24:26] getting out of Cursor. But if the [24:28] research at Curser and the continual [24:30] like development of the product at [24:31] Cursor stays valuable to SpaceX, they [24:34] will acquire Cursor for 60 billion. [24:36] Another thing of note is that these [24:38] numbers are so close that I can't fathom [24:40] why SpaceX would let them go. The reason [24:42] that these terms were inked is because [24:45] there is two cases. Either cursor is [24:48] only valuable because of the data they [24:49] have in that space is willing to pay 10 [24:51] bill for that or cursor is valuable [24:53] beyond that. in which case SpaceX is [24:55] willing to pay 60 bill for the whole [24:57] company for their value outside of the [24:58] data. So there's either 10 bill to take [25:00] their data or 60 bill to take [25:02] everything. That's the way to think of [25:03] this deal. But remember this from [25:05] earlier XAI staff being banned from [25:07] using anthropic models internally on [25:09] cursor because anthropic doesn't want [25:12] XAI to have the data for runs inside of [25:15] cursor using anthropic models. Because [25:17] if XAI can use Enthropic through cursor, [25:20] they can start building their own corpus [25:21] of this data and then use that to try [25:23] and make XAI better. This is an actual [25:25] way they could have potentially made [25:27] Grock better at code simply by [25:29] generating the data using cursor on [25:31] anthropic models. And anthropic is so [25:33] sensitive to their competitors having [25:35] this data that they will do whatever the [25:37] [ __ ] they have to to prevent those [25:39] competitors from getting it, including [25:41] meaningful harm to their long-term [25:43] long-standing relationship with Cursor. [25:45] They've always been very close. them [25:47] going to cursor telling them to ban XAI [25:49] is an unprecedented move especially at [25:51] that point in time. I think this was the [25:53] genesis moment that resulted in the [25:56] SpaceX cursor announcement. This was [25:59] Elon letting Anthropic know, hey guys, [26:02] you're not going to keep us from getting [26:04] that data. We are going to get all of [26:06] those chat histories that people have [26:08] using anthropic models inside of a [26:09] better harness and we're going to make [26:11] models that compete with yours by doing [26:12] it. [ __ ] you, Anthropic. But this is a [26:15] process that's going to take a long time [26:16] and XAI has a shitload of compute around [26:19] right now. It's one of the other reasons [26:21] that cursor was excited to work with [26:23] them. I am sure when this deal went [26:25] through, Anthropic was [ __ ] pissed. [26:28] Anthropic was probably ready to just [26:31] commit violent acts as a result of this [26:34] because they always do whatever they can [26:36] to prevent the data that is generated [26:38] with their models from getting to other [26:40] competitors. They have done so many [26:42] dirty things in the past. Whether it is [26:43] the horrible article they wrote about [26:45] the Chinese distillation attacks or [26:47] their banning of SpaceX or their banning [26:49] of Windsurf back in the day when they [26:50] thought OpenAI might buy them. All of [26:52] these things were really, really dirty [26:55] plays that Anthropic did because they [26:57] were willing to play dirty as long as [26:58] they got what they wanted, which is the [27:00] data not getting to their competitors. [27:02] Elon said, "Hold my beer." This almost [27:04] certainly started some heated [27:06] discussions between both parties, but [27:08] that's where we go back to our three [27:11] things you need to succeed chart. XAI [27:14] has compute. They just acquired the [27:15] data. There are a few good researchers [27:17] away from being very competitive. [27:19] Anthropic has the researchers. That's [27:21] what they have. That is where they are [27:22] strongest by far. I will never ever talk [27:24] [ __ ] on research at Anthropic beyond [27:27] their hatred of devs because the [27:28] research teams at Anthropic are [27:30] unbelievably talented and skilled. some [27:31] of the best in the [ __ ] world as good [27:33] if not better than open AIS. Anthropic [27:36] has worldleading research. Despite the [27:38] [ __ ] show that product is at Anthropic, [27:40] tools like claude code are absolutely [27:42] getting the more data they can use. And [27:44] I am nice certain that the improvements [27:46] they've seen on Opus 46 and 47 are just [27:48] taking the runs from 45 modifying them a [27:51] bit and then using that to make the new [27:53] models better via RL. But the thing [27:55] anthropics lacking is the third one, [27:58] compute. XAI didn't have research or [28:01] data, but they did have compute. [28:03] Anthropic has research and data, no [28:06] compute. Cursor kind of has research, [28:09] not the same level, so I'll give them an [28:10] X. They absolutely have the data. They [28:12] do not have the compute. OpenAI is the [28:14] only company with all three. Does [28:17] everything suddenly make sense? XAI had [28:21] holes to fill. They need to patch their [28:22] research hole and their data hole. [28:24] Enthropic had holes to patch, too. They [28:26] need to patch their compute hole. cursor [28:28] needs to figure out how they survive [28:29] this next generation. Open AAI is what [28:32] they're all fearing. And the fact that [28:34] Anthropic is willing to link up with one [28:36] of their old enemies, XAI, shows just [28:38] how desperate they are to prevent OpenAI [28:40] from coming in and slaughtering. There [28:42] are two things that are making OpenAI [28:43] much scarier to Anthropic right now. The [28:45] first is that Codeex is exploding. It's [28:48] getting significantly better to use [28:49] OpenAI models for code tasks, which [28:51] Anthropic has historically dominated. [28:53] But the other big thing is that OpenAI [28:54] now runs on AWS. Not just yet, but very [28:58] soon OpenAI will have AWS support. They [29:00] didn't before. Anthropic had a huge lead [29:03] because if you were on AWS and [29:04] everyone's on AWS, you could use [29:06] Anthropic models in bedrock and you [29:08] couldn't use OpenAI ones. Now that that [29:10] has changed, Anthropic's biggest wedge [29:12] is dead and the code capabilities wedge [29:15] is getting closed fast. So Endar is [29:18] making a pretty crazy bet here that [29:19] their current desperation from compute [29:21] being temporarily resolved by XAI is [29:24] worth the risk in the optics of this [29:26] partnership. But I still haven't talked [29:27] about that last sentence. This is again [29:30] from Elon. I was okay leasing Colossus 1 [29:32] to Anthropic as SpaceX AI has already [29:35] moved training to Colossus 2. You need [29:37] compute for two things. You need compute [29:40] for training, but you also need compute [29:42] for serving requests from users. When [29:44] users are using the models from your [29:46] company, you need GPUs or TPUs or [29:48] something to serve those requests. [29:50] Colossus one is a pretty sizable [29:53] deployment. According to epoch, Colossus [29:56] 1 was around 442 megawatt of power, [29:59] around 280,000 H100s. Truly insane [30:03] amount of GPUs. Colossus 2 is still [30:06] being produced. It's not ready yet. It [30:08] has 1.5 gawatt, so 1500 megawatts of [30:12] power and 1.4 4 million H100 [30:14] equivalents. That is a 3x in power from [30:18] 425 megaWW to,500 megawatt. And it is [30:21] also a massive increase in the number of [30:24] GPUs. It is a 5x to 1.5 mil of them [30:26] roughly. But there's a very specific [30:29] phrasing in here that I think is worth [30:31] noting. SpaceX has already moved [30:34] training to Colossus 2, not inference, [30:37] training. What that implies if you were [30:40] to drop the anthropic part and just [30:42] focus on SpaceX moving training to [30:43] Colossus 2. What that would imply is [30:46] that they're running their inference for [30:47] Grock on Colossus 1. So that's 442 [30:50] megaww allocated for Grock. I don't [30:53] think Grock needs that much inference [30:55] because I don't know a lot of people [30:56] using Grock outside of adding it on [30:58] Twitter. It's just not a good option [31:01] right now. They very briefly had a [31:03] benchmark lead when Gro 4 came out, but [31:05] they lost it fast and have never been [31:07] usable enough to be the default for [31:10] anybody. So they have 442 megawatt of [31:12] compute. They're obviously giving some [31:14] of that over to cursor. I'm assuming [31:16] more of that's going to be Colossus 2 [31:17] for training. So of the 442 megawatt on [31:20] Colossus 1, how much do you think they [31:22] gave to Enthropic? Because remember, [31:25] they have to keep some around for their [31:26] own inference. Let's hear some guesses, [31:29] chat. What percentage or what number? [31:31] You're thinking they gave away 20%, [31:32] maybe 40%, reasonable guesses. 250 [31:36] megawws, 60%, 70%. Let's look at the [31:39] official number, cuz they were actually, [31:40] for some reason, kind enough to share [31:42] this. This gives us access to more than [31:45] 300 megawatt of new capacity, over [31:47] 220,000 Nvidia GPUs within the month. [31:51] That is pretty much all of Colossus 1. [31:53] That's 300 out of the 425 megawatt and [31:56] 220 out of the 280 GPUs. That means that [31:59] XAI only needs about 100 megawws or so [32:04] for Grock inference right now, which is [32:06] very amusing. They effectively had this [32:08] giant data center with a bunch of H100s [32:10] that was meant to serve Grock that was [32:13] not getting used because nobody needs [32:15] Grock and Elon had already invested so [32:16] heavily in the next generation data [32:18] center, Colossus 2, which cost $44 [32:20] billion to build. By the way, I just [32:23] thought it was amusing that this data [32:24] center is effectively unused, not [32:26] because they have Colossus 2, simply [32:28] because they don't do any inference [32:31] because nobody's using Grock. They did [32:33] specify the agreement means they will be [32:35] able to use all of the compute capacity [32:37] at Colossus 1, but they said in the [32:39] article that it was all of the compute [32:41] within the month, which specifically [32:44] means that they are going to move [32:45] whatever little inference they're still [32:47] doing for Grock off into Colossus 2, [32:50] which allegedly will be fully online by [32:52] May 13th. Hard to know for sure. These [32:53] deadlines always get shifted around. I [32:55] just found it amusing that they're doing [32:56] so little inference that this barely [32:58] matters to them. If you take anything [32:59] from this video, it should probably be [33:01] this chart. This lays out why these [33:04] things are happening. The reason XAI [33:06] wants to buy cursor is to plug a data [33:07] gap. The reason Anthropic wants to work [33:09] with XAI is to plug a compute gap. The [33:11] reason OpenAI is ignoring all of this is [33:13] because they planned ahead slightly [33:14] better. And the reason Google is not in [33:16] this chart is because despite having all [33:18] of these things kind of, they can't stop [33:19] [ __ ] themselves long enough to make [33:21] something useful. And in a future where [33:22] Enthropic spins up way more compute over [33:26] the next few years here apparently a [33:28] gigawatt of tranium on Amazon by the end [33:30] of this year and up to five overall over [33:32] time a similar agreement with Google and [33:34] Broadcom for bedrock $30 billion on [33:37] Azure and way more investment in [33:39] American AI infra all of this suggests [33:42] that they are going to finally get ahead [33:44] on compute but as we've established many [33:46] times now compute is a thing that takes [33:48] a long lead to do you can't just say I [33:51] want this compute now and get it, you [33:53] can only get it from people who bought [33:54] it 2 years ago or you can make a buy [33:57] order for compute two plus years from [33:59] now. So Elon buying early and having no [34:01] users to serve means he has extra [34:03] compute to sell. Anthropic buying too [34:05] little and having 80x the users they [34:07] expected means they have no compute [34:08] available. Match made in heaven. As long [34:11] as you ignore Elon's genuine hatred for [34:14] anthropic that he is willing to ignore [34:16] almost entirely for enough billions of [34:18] dollars. I think this is all absurd. The [34:20] willingness of all of these people to [34:22] set aside their morals to get compute [34:25] shows just how desperate they all are to [34:28] make their investments worthwhile. And [34:30] let's be real, a significant portion of [34:32] this is influenced by the fact that Elon [34:35] hates OpenAI so deeply that he's willing [34:38] to work with Anthropic just to try and [34:40] plug these gaps to make anybody as [34:43] successful as OpenAI is. And when you [34:45] remember that these two companies detest [34:48] OpenAI, it makes all the sense in the [34:50] world they could ignore their hatred of [34:51] each other in order to compete with Open [34:53] AI more directly. The enemy of my enemy [34:56] has compute and I'm going to take [34:57] advantage of it. I got nothing else to [34:59] say here. Hope this is a fun deep dive [35:01] on the absolute chaos between all of [35:02] these labs. I think it's really fun once [35:04] you're in the weeds seeing what money [35:06] and what allocations are going where. It [35:08] makes a lot of sense when you track the [35:09] GPUs. Hope this was fun. Until next [35:11] time, peace nerds.  📄 **Full text:** ![[Full Document Contents/Articles/Anthropic Just…wait What.md|Anthropic Just…wait What by YouTube]] ## Highlights - we have a lot of things to talk about within it. The obvious one is how does this affect us specifically cloud code users? There's a question of what's Anthropic's compute problem? Like why is this a thing that they're willing to do? Is it really that bad that they're going to partner with a person that they hate and hates them? Like that just seems insane, right? But most importantly, WTF is going on at XAI. I honestly think this is the most interesting part of all of this. The financials and direction of ([View Highlight](https://read.readwise.io/read/01kr1eehgqmn7ykwhf4fc4cnb1)) - Tags: 👻 ai highlighted - time. Just thought that was an interesting thing worth calling out. I have heard others say that Amazon and Google are also piping through to Anthropic and that is obviously wrong for a billion [ __ ] reasons. But now you can see in this chart, yeah, obviously. So if you think that all of them are just piping through, no. If you think Azure might be piping through, potentially. So we've covered the compute problem. We've covered how this affects us. Time to talk about the exciting part here. What the [ __ ] going on at XAI? Just a few weeks ago, ([View Highlight](https://read.readwise.io/read/01kr1eehnfjeqn5he6w62b908g)) - Tags: 👻 ai highlighted - solutions on the research side. They scraped and found a ton of good data. And the compute in particular was a weak point initially because they didn't have the money. That's part of why there's all of the beef going on right now with OpenAI and Elon. The reason that they had to go for profit is they had no other way to acquire the compute they needed when they started believing scaling loss. So, we know XAI has compute. That's been established. We'll talk about the amount in a bit, but I want to talk about the data side first. You might think that X has a ton of data ([View Highlight](https://read.readwise.io/read/01kr1eehrkwvc4dv1c7k4jjq0z)) - Tags: 👻 ai highlighted - data not getting to their competitors. Elon said, "Hold my beer." This almost certainly started some heated discussions between both parties, but that's where we go back to our three things you need to succeed chart. XAI has compute. They just acquired the data. There are a few good researchers away from being very competitive. Anthropic has the researchers. That's what they have. That is where they are strongest by far. I will never ever talk [ __ ] on research at Anthropic beyond their hatred of devs because the research teams at Anthropic are ([View Highlight](https://read.readwise.io/read/01kr1eehxeaq4smnhtxe2jaa6v)) - Tags: 👻 ai highlighted - data, but they did have compute. Anthropic has research and data, no compute. Cursor kind of has research, not the same level, so I'll give them an X. They absolutely have the data. They do not have the compute. OpenAI is the only company with all three. Does everything suddenly make sense? XAI had holes to fill. They need to patch their research hole and their data hole. Enthropic had holes to patch, too. They need to patch their compute hole. cursor needs to figure out how they survive this next generation. Open AAI is what ([View Highlight](https://read.readwise.io/read/01kr1eej2hvws6n90ephhx44xj)) - Tags: 👻 ai highlighted - this chart. This lays out why these things are happening. The reason XAI wants to buy cursor is to plug a data gap. The reason Anthropic wants to work with XAI is to plug a compute gap. The reason OpenAI is ignoring all of this is because they planned ahead slightly better. And the reason Google is not in this chart is because despite having all of these things kind of, they can't stop [ __ ] themselves long enough to make something useful. And in a future where Enthropic spins up way more compute over the next few years here apparently a gigawatt of tranium on Amazon by the end ([View Highlight](https://read.readwise.io/read/01kr1eejdecx172xbjkdht2fbw)) - Tags: 👻 ai highlighted - let's be real, a significant portion of this is influenced by the fact that Elon hates OpenAI so deeply that he's willing to work with Anthropic just to try and plug these gaps to make anybody as successful as OpenAI is. And when you remember that these two companies detest OpenAI, it makes all the sense in the world they could ignore their hatred of each other in order to compete with Open AI more directly. The enemy of my enemy has compute and I'm going to take advantage of it. I got nothing else to say here. Hope this is a fun deep dive ([View Highlight](https://read.readwise.io/read/01kr1eejqb2bk448f83zewfvrw)) - Tags: 👻 ai highlighted