Cursor just crushed Claude Code - CourtneyR.dev's 2nd Brain

Cursor just crushed Claude Code ![rw-book-cover](https://readwise-assets.s3.amazonaws.com/static/images/article3.5c705a01b476.png) 🔗 **Source:** [Cursor just crushed Claude Code](https://www.youtube.com/watch?v=UvUzpSlXKtg) > [!note]+ Document Notes > tags: [cursor-editor, ai-coding, model-training, developer-tools, composer-model, spacex-ai] > author: [MISSING] > source: [MISSING] > reference: https://www.youtube.com/watch?v=UvUzpSlXKtg > date created: {{date:YYYY-MM-DD}} {{time:HH:mm}} > > # 🏷️ Title: Cursor just crushed Claude Code >  ## 🧭 Table of Contents - [📝 Summary](#-summary) - [📖 Main Content](#-main-content) - [🗒️ Notes](#️-notes) - [📚 References](#-references) - [🗣️ Paraphrase](#️-paraphrase) - [📰 Newsletter](#-newsletter) - [✅ Action Items](#-action-items) - [🧪 Hypotheses](#-hypotheses) - [📊 Methodology](#-methodology) - [❓ Questions](#-questions) - [🔮 Future Research](#-future-research) - [💡 Implications](#-implications) - [🧠 Learning Styles Assessment](#-learning-styles-assessment) - [📊 Accessibility Assessment](#-accessibility-assessment) - [🌍 DEIB Assessment](#-deib-assessment) - [✅ Logical Validity](#-logical-validity) - [🛡️ DARVO Analysis](#-darvo-analysis) - [🎯 Conclusion](#-conclusion) ## 📝 Summary - Cursor released Composer 2.5, a coding-focused AI model that significantly outperforms previous versions while being substantially cheaper - The model achieved 63% on Cursor's internal benchmark, nearly matching GPT-5.5 (64%) and Opus 4.7 (65%) at 10-20x lower cost - Built on Moonshot's Kimi K25 base model with 10x more compute than the original, trained using SpaceX AI's compute resources - Introduces novel training techniques like targeted textual feedback and synthetic task generation to improve coding performance - Only available through Cursor's platform, not via API, creating competitive advantages but limiting external evaluation ## 📖 Main Content Cursor has released Composer 2.5, marking a significant advancement in AI coding models that challenges the dominance of major AI labs. The model represents a dramatic improvement in both capability and cost-effectiveness for coding tasks. ### Performance and Pricing Revolution Composer 2.5 achieved remarkable results on Cursor's internal benchmark, scoring 63% compared to GPT-5.5's 64% and Opus 4.7's 65%. The critical difference lies in cost: Composer 2.5 operates at roughly 10-20x lower cost than these competing models while delivering comparable performance. The pricing structure reflects this advantage: - Composer 2.5: $0.50 per million tokens in, $2.50 out - Claude Sonnet: $3.00 in, $15.00 out - GPT-5.5: $5.00 in, $30.00 out - Opus 4.7: $5.00 in, $25.00 out ### Technical Innovation The model builds on Moonshot's Kimi K25 base model but incorporates 10x more compute than the original training. Two key innovations drive its performance: **Targeted RL with Textual Feedback**: This addresses a fundamental challenge in reinforcement learning where models struggle to identify which specific decisions in long sequences contributed to success or failure. The system provides targeted feedback at problematic points in code generation, using a teacher-student model approach where hints guide the learning process without requiring explicit instructions during inference. **Synthetic Data Generation**: The team created 25x more synthetic training tasks than Composer 2, including innovative approaches like "feature deletion" where models remove features from codebases and then reconstruct them using existing tests as verification. ### Strategic Positioning Cursor's partnership with SpaceX AI provides access to massive compute resources (Colossus 2's million H100 equivalents) while their user data offers unique training advantages. The company collects valuable interaction patterns from developers using their platform, creating a feedback loop that improves model performance. However, the model remains exclusively available through Cursor's platform, limiting external evaluation and API access. This creates competitive moats but raises questions about independent assessment of capabilities. ## 🗒️ Notes #### ⏱️ [00:00:00](https://www.youtube.com/watch?v=UvUzpSlXKtg?t=0) Introduction and Context - Cursor released Composer 2.5, a major coding model advancement - Built by company with significant compute resources (SpaceX partnership) - Represents competitive threat to major AI labs #### ⏱️ [00:02:30](https://www.youtube.com/watch?v=UvUzpSlXKtg?t=150) Pricing Analysis Deep Dive - Explains token-based pricing model complexity - Input tokens cheaper than output tokens - Three pricing factors: per-token cost, tokens per problem, negotiation leverage - Labs can subsidize heavily through direct products vs API pricing #### ⏱️ [00:08:45](https://www.youtube.com/watch?v=UvUzpSlXKtg?t=525) Cursor's Data Advantage - Cursor collects valuable developer interaction data - Chat histories with feedback loops extremely valuable for training - Double-dip strategy: resell models at higher cost while collecting training data #### ⏱️ [00:12:00](https://www.youtube.com/watch?v=UvUzpSlXKtg?t=720) Composer Evolution - Composer 1.0: Too expensive, not very good - Composer 1.5: Better but still overpriced compared to alternatives - Composer 2.0: 7x cheaper than 1.5, significantly smarter - Composer 2.5: Same price as 2.0, major intelligence leap #### ⏱️ [00:15:30](https://www.youtube.com/watch?v=UvUzpSlXKtg?t=930) Benchmark Results - 63% on Cursor Bench vs GPT-5.5 (64%) and Opus (65%) - 10-20x cost advantage over competing models - Based on Kimi K25 with doubled performance through post-training #### ⏱️ [00:18:00](https://www.youtube.com/watch?v=UvUzpSlXKtg?t=1080) Training Innovations - Targeted textual feedback for specific behavior correction - Addresses credit assignment problem in long rollouts - Teacher-student model approach with contextual hints #### ⏱️ [00:22:15](https://www.youtube.com/watch?v=UvUzpSlXKtg?t=1335) Synthetic Data Strategy - 25x more synthetic tasks than Composer 2 - Feature deletion approach: remove features, then reconstruct using tests - Reward hacking examples: models finding clever workarounds #### ⏱️ [00:26:00](https://www.youtube.com/watch?v=UvUzpSlXKtg?t=1560) Practical Testing - Game reconstruction test with Fish Slop project - Model demonstrates parallel agent capabilities - Asset swapping and UI scaling challenges revealed #### ⏱️ [00:35:00](https://www.youtube.com/watch?v=UvUzpSlXKtg?t=2100) API Access Limitations - No external API access available - Only measurable through Cursor's internal benchmarks - Limits independent evaluation and integration options #### ⏱️ [00:40:00](https://www.youtube.com/watch?v=UvUzpSlXKtg?t=2400) Future Outlook - Next model training with 10x more compute (100x vs original Kimi) - Potential to leapfrog state-of-the-art coding capabilities - Enterprise adoption advantages over individual lab partnerships ## 📚 References - [Cursor](https://cursor.com) - [Moonshot Kimi K25 model](https://moonshot.ai) - [SpaceX AI partnership](https://spacex.com) - [Artificial Analysis benchmarks](https://artificialanalysis.ai) - [SWE-bench evaluation](https://swe-bench.github.io) ## 🗣️ Paraphrase Cursor has achieved a breakthrough in AI coding assistance with Composer 2.5, a model that rivals the performance of leading AI systems while operating at dramatically lower costs. The model demonstrates that focused post-training and domain specialization can create competitive advantages against well-funded general-purpose models from major tech companies. The technical innovations, particularly targeted textual feedback and synthetic data generation, represent meaningful advances in training methodology. However, the platform-exclusive availability limits broader ecosystem integration and independent verification of capabilities. ## 📰 Newsletter 🚀 **Cursor Composer 2.5 Shakes Up AI Coding Scene** The WordPress development world just got more interesting! Cursor's latest Composer 2.5 model is delivering near-GPT-5.5 performance at 1/20th the cost. For WordPress developers working on complex themes, plugins, or custom solutions, this could be a game-changer. **What's exciting for WP developers:** - Lightning-fast code generation perfect for iterative development - Specialized training on real codebases (hello, better WordPress-specific suggestions!) - Cost-effective for agencies managing multiple client projects **Try it out:** If you're already using Cursor for WordPress development, the 2.5 model should be available now. Perfect for those custom block development sessions or complex WooCommerce integrations. **Community spotlight:** The synthetic training approach (where models learn by breaking and fixing features) mirrors how many of us debug WordPress sites. It's AI learning to think like developers! ## ✅ Action Items #### DevRel - **Developer advocacy**: Create comparison benchmarks between Composer 2.5 and other coding models for WordPress-specific tasks - **Education**: Develop workshops on effective AI-assisted WordPress development workflows - **Developer experience**: Test integration capabilities with WordPress CLI and local development environments - **Developer marketing**: Document cost-benefit analysis for WordPress agencies considering Cursor adoption - **Community management**: Monitor developer feedback on platform-exclusive model availability concerns #### WordPress - **Core**: Evaluate impact of AI coding tools on core contribution workflows and code review processes - **Core Performance**: Test AI-generated code performance against manual optimizations in WordPress context - **Documentation**: Update coding standards documentation to address AI-assisted development practices - **Training**: Create learning paths for developers transitioning to AI-assisted WordPress development - **Plugins**: Establish guidelines for AI-generated plugin code in directory submissions #### Community - **Contributors**: Develop mentorship programs pairing AI tools with human expertise for onboarding - **Maintainers**: Create templates for AI-assisted code review and testing procedures - **Ecosystem**: Coordinate with hosting providers on performance implications of AI-generated code ## 🧪 Hypotheses - **Cost reduction hypothesis**: Composer 2.5's pricing advantage will drive significant adoption among budget-conscious development teams and individual developers - **Platform lock-in hypothesis**: Cursor's API exclusivity will create competitive moats but limit broader ecosystem integration and innovation - **Training data advantage hypothesis**: Cursor's access to real developer interaction data provides sustainable competitive advantages over models trained primarily on static code repositories - **Specialization effectiveness hypothesis**: Domain-specific post-training (coding focus) can achieve competitive performance against general-purpose models with significantly less compute - **Enterprise adoption hypothesis**: The combination of cost-effectiveness and performance will accelerate Cursor's enterprise market penetration - **Innovation catalyst hypothesis**: Platform-exclusive models will pressure major AI labs to improve their own coding-specific offerings ### 📊 Methodology **Cost-benefit analysis studies**: Compare development velocity and code quality metrics across different AI coding assistants in controlled WordPress development scenarios. **Performance benchmarking**: Establish WordPress-specific coding benchmarks measuring plugin development, theme creation, and core contribution tasks. **Developer experience surveys**: Track satisfaction, productivity metrics, and workflow changes among developers using different AI coding tools. **Market adoption tracking**: Monitor enterprise adoption rates and developer tool preferences through industry surveys and usage analytics. ### ❓ Questions - How does platform exclusivity affect long-term innovation and competition in AI coding tools? - What are the implications of training AI models on proprietary developer interaction data for the broader development community? - Can specialized post-training consistently outperform general-purpose models across different coding domains? - How will major AI labs respond to competitive pressure from specialized, cost-effective coding models? - What governance and ethical considerations arise from AI models trained on developer workflow data? ### 🔮 Future Research - **Comparative effectiveness studies**: Systematic evaluation of coding model performance across different programming languages and frameworks - **Developer workflow optimization**: Research into optimal human-AI collaboration patterns for different types of coding tasks - **Training methodology advancement**: Investigation into synthetic data generation techniques for specialized AI model domains - **Economic impact analysis**: Study of AI coding tools' effects on software development labor markets and productivity ### 💡 Implications **Practice implications**: Developers need to evaluate workflow changes required for effective AI collaboration, particularly the shift from batch processing to interactive development sessions. **Economic implications**: Dramatic cost reductions may democratize access to advanced AI coding assistance, potentially leveling competitive advantages previously available only to well-funded development teams. **Competitive implications**: Platform-exclusive models create new forms of vendor lock-in that may fragment the developer tools ecosystem while driving innovation through competition. **Education implications**: Development education must incorporate AI collaboration skills alongside traditional programming knowledge, changing how coding is taught and learned. ## 🧠 Learning Styles Assessment **Reading level estimate**: Grade 12-13 (Flesch-Kincaid), appropriate for technical professionals but may challenge general audiences due to specialized AI/ML terminology. #### 👀 Visual learners - **Present**: Performance comparison charts, cost analysis tables, technical architecture diagrams - **Improve**: Add workflow diagrams showing human-AI collaboration patterns, visual timeline of model evolution, before/after code examples #### 🔊 Aural learners - **Present**: Video format with detailed verbal explanations and examples - **Improve**: Provide audio-only version for developers who prefer listening while coding, add podcast-style discussion format #### 🗣️ Verbal learners - **Present**: Clear definitions of technical terms, analogies explaining complex concepts, step-by-step explanations - **Improve**: Add glossary of AI/ML terms, provide plain-language summaries of technical sections #### 🧩 Kinesthetic learners - **Present**: Hands-on coding demonstrations, interactive testing scenarios - **Improve**: Add downloadable test projects, provide step-by-step tutorials for setting up and testing different models #### 🧠 Logical learners - **Present**: Systematic cost-benefit analysis, benchmark comparisons, logical progression of technical improvements - **Improve**: Add decision trees for model selection, quantitative ROI calculations, systematic evaluation frameworks #### 🧑‍🤝‍🧑 Social learners - **Present**: Community discussion context, developer experience sharing, enterprise adoption stories - **Improve**: Add discussion prompts for development teams, collaborative evaluation frameworks #### 🧘‍♀️ Solitary learners - **Present**: Comprehensive written analysis, individual testing scenarios, self-assessment questions - **Improve**: Add personal reflection prompts on workflow changes, individual skill development paths #### 🌿 Naturalistic learners - **Present**: Evolution of AI models as ecosystem development, competitive dynamics as natural selection - **Improve**: Add analogies to biological systems, discuss sustainability of different business models ## 📊 Accessibility Assessment #### Screen reader semantics - **Present**: Clear heading structure, descriptive video content - **Improve**: Add alt text for performance charts and diagrams, provide transcript with technical term definitions #### Keyboard navigation - **Present**: Standard video player controls accessible via keyboard - **Improve**: Add keyboard shortcuts for technical sections, ensure all interactive elements are keyboard accessible #### Contrast and readability - **Present**: Standard video player with good contrast in UI elements - **Improve**: Provide high-contrast versions of charts and graphs, ensure code examples have sufficient contrast #### Text resizing and reflow - **Present**: Responsive video player that adapts to different screen sizes - **Improve**: Ensure transcript and supplementary materials reflow properly at 200% zoom #### Media and captions - **Present**: Video with clear audio and visual demonstrations - **Improve**: Add professional captions with technical term spelling, provide downloadable transcript #### Forms and interactions - **Present**: Basic video player interaction elements - **Improve**: Add accessible feedback mechanisms for community discussion, ensure error handling for any interactive elements #### Standards reference - Map accessibility improvements to WCAG 2.2 AA criteria, particularly around multimedia content and technical documentation ## 🌍 DEIB Assessment #### Cultural breadth - **Present**: Discussion of global AI development competition, enterprise adoption across different markets - **Improve**: Include perspectives from developers in different geographic regions, discuss accessibility in different economic contexts #### Inclusive language - **Present**: Generally professional technical language, avoids most jargon without definition - **Improve**: Define all technical terms clearly, avoid assumptions about prior AI/ML knowledge #### Neurodiversity support - **Present**: Well-structured content with clear sections and logical progression - **Improve**: Provide content in multiple formats (video, text, audio), add visual organizing elements #### Representation - **Present**: Focus on technical capabilities rather than individual personalities - **Improve**: Include diverse developer perspectives in case studies, highlight contributions from underrepresented groups in AI development #### Psychological safety - **Present**: Balanced analysis acknowledging both benefits and limitations - **Improve**: Address concerns about job displacement sensitively, provide resources for skill development ## ✅ Logical Validity #### Strong reasoning - **Economic analysis**: Well-structured comparison of pricing models with clear mathematical relationships and market dynamics - **Technical evaluation**: Systematic breakdown of training innovations with specific examples and measurable outcomes - **Competitive positioning**: Logical analysis of strategic advantages and market positioning based on concrete capabilities #### Fallacies - **Hasty generalization**: Drawing broad conclusions about model superiority from limited, internal benchmarks without external validation - **Appeal to novelty**: Suggesting newer training techniques are necessarily better without comprehensive comparative analysis - **False dilemma**: Presenting platform exclusivity as purely advantageous without adequately weighing ecosystem fragmentation costs #### Tone of voice - **Technical enthusiasm with critical skepticism**: Balances excitement about innovations with realistic assessment of limitations and concerns ## 🛡️ DARVO Analysis No significant DARVO patterns detected. The content maintains analytical objectivity while acknowledging both competitive advantages and legitimate concerns about platform exclusivity and evaluation limitations. ## 🎯 Conclusion Cursor's Composer 2.5 represents a significant advancement in AI coding assistance, demonstrating that focused domain specialization and innovative training techniques can compete with well-funded general-purpose models. The dramatic cost advantages (10-20x cheaper than comparable models) combined with near-state-of-the-art performance create compelling value propositions for developers and enterprises. However, the platform-exclusive availability raises important questions about ecosystem fragmentation and independent evaluation capabilities. The success of this approach may pressure major AI labs to improve their coding-specific offerings while potentially creating new forms of vendor lock-in. For WordPress developers and the broader development community, this represents both opportunity (more accessible advanced AI assistance) and challenge (navigating an increasingly fragmented AI tools landscape). The key practical next step is evaluating how these tools integrate into existing workflows and whether the productivity gains justify potential platform dependencies. ## 🔄 Refactor Appendix — Cursor just crushed Claude Code ### 📚 Literature Atoms (H3) #### Domain-specific post-training can match general models at lower cost > [!quote] ID:rw_cursor_composer > Composer 2.5 achieved 63% on Cursor Bench vs GPT-5.5 (64%) and Opus 4.7 (65%) at 10-20x lower cost through specialized coding focus and innovative training techniques **Context & use**: - Demonstrates effectiveness of focused domain specialization over general-purpose model scaling - Evidence for sustainable competitive advantages through training methodology innovation **Tags**: #ai-training #cost-optimization #competitive-strategy **Highlight**: [view highlight](https://www.youtube.com/watch?v=UvUzpSlXKtg?t=930)  #### Platform-exclusive AI models create competitive moats but limit ecosystem integration > [!quote] ID:rw_cursor_exclusive > Unlike every other model, there is no API to hit to test Composer ourselves - the only way to try it is inside Cursor, unless you spend $60 billion to buy them **Context & use**: - Highlights new form of competitive strategy in AI model distribution - Raises concerns about ecosystem fragmentation and independent evaluation **Tags**: #platform-strategy #api-access #competitive-moats **Highlight**: [view highlight](https://www.youtube.com/watch?v=UvUzpSlXKtg?t=2100)  #### Developer interaction data provides unique training advantages for coding models > [!quote] ID:rw_cursor_data > Chat histories where you work with the agent, give it feedback as it generates things, tell it 'you missed this, go change that too' - those are so valuable for training **Context & use**: - Explains sustainable competitive advantage from user-generated training data - Illustrates value of interactive development workflows for model improvement **Tags**: #training-data #developer-workflows #feedback-loops **Highlight**: [view highlight](https://www.youtube.com/watch?v=UvUzpSlXKtg?t=525)  ### 🧠 Permanent Candidates (H4) #### Specialized AI training outcompetes generalist scaling **Claim**: Domain-specific post-training with targeted techniques can achieve competitive performance against general-purpose models using significantly less compute and cost. **Evidence pointer**: [[#Domain-specific post-training can match general models at lower cost]] **Scope & conditions**: Holds when domain has sufficient training data and clear success metrics; breaks when tasks require broad knowledge integration **Links**: - [[AI model specialization]] — because extends focused training approaches - [[Competitive AI strategy]] — because contrasts with general-purpose scaling approaches - [[Developer tool economics]] — because applies-in cost-sensitive professional software markets  ### ✏️ Fleeting Queue - Platform-exclusive models as new competitive strategy - implications for open ecosystem development #fleeting #inbox - Targeted textual feedback training technique - potential applications beyond coding #fleeting #inbox - SpaceX AI compute partnership dynamics - precedent for other AI lab collaborations #fleeting #inbox ### → Things 3 - [ ] Test Composer 2.5 performance on WordPress-specific coding tasks — compare against current development workflow #things - [ ] Research Cursor SDK capabilities for potential integration with existing development tools — evaluate API limitations and workarounds #things --- ## Full Transcript [00:00] Last week there was a really cool [00:01] release I'm afraid people are sleeping [00:03] on. It came from a company with a lot of [00:04] compute and it was focused on being a [00:06] small model that is very effective at [00:08] solving code tasks. And no, it wasn't [00:10] Gemini 3.5 Flash, even though it [00:12] probably should have been. It was [00:13] Composer 2.5 by Codium. This is a crazy [00:18] drop. The rate at which Codium has [00:20] caught up to state of the art is pretty [00:23] absurd, but there are catches here that [00:26] we need to talk about as well. If you're [00:28] not familiar with the Composer line of [00:30] models from Codium, this has been their [00:32] attempt to distill good open weight [00:34] models into a great coding model for [00:37] usage within Codium. That said, it's [00:40] only really for use within Codium. So, [00:42] it's hard for us to bench it externally. [00:45] That means we kind of got to vibe our [00:47] way through it, but it also means Codium [00:50] does to an extent as well. They have [00:51] published some awesome numbers including [00:53] their own benchmark, which sadly is not [00:56] particularly public, the Codium bench. [00:58] But the results seem to be incredible, [01:00] and I can tell you from my own use of [01:01] it, this isn't just hype. While I don't [01:04] think this is the greatest model ever [01:06] and you should use it for everything, I [01:07] do think it's worth looking at and [01:09] understanding both because it's a good [01:12] model release from a company that [01:14] understands what we need as devs, but [01:15] also because it shows that the strength [01:18] that the major labs have might be [01:20] starting to crack a little bit under [01:22] pressure. While I am an early investor [01:24] in Codium, the acquisition stuff [01:27] happening with SpaceX kind of destroys [01:29] any incentive I might have had to shill [01:31] it super hard. So, I'm not sharing this [01:34] cuz I'm biased. I'm sharing this cuz [01:35] it's actually really interesting to me. [01:37] And don't worry, I will [ __ ] all over a [01:38] lot of other Codium things throughout [01:40] this, too. All of that said, want to [01:42] make sure it's very clear Codium is not [01:43] paying me at all for this coverage. The [01:45] only people paying me today are the [01:47] sponsor. As powerful as continuous [01:49] integration is, it's full of [01:50] compromises. I like to call this the CI [01:52] compromise triangle. The different [01:54] things you have to to about when you're [01:55] configuring your continuous integration. [01:57] You start with convenience, which is how [01:58] easy is it to implement this? You also [02:00] have performance. How long does it take [02:02] to run? Be nice if the CI runners were [02:03] just a little bit faster. And of course, [02:05] you have cost. Obviously, there's lots [02:07] of different solutions for CI that have [02:09] different compromises across this [02:10] triangle because it would be impossible [02:12] for anything to just be better at all of [02:14] these, right? Like there's no way [02:15] something could be more convenient, [02:16] faster, and cheaper, right? [02:20] Oh, is it a Blacksmith ad? Yeah. [02:22] Blacksmith is 2x faster and 60% cheaper [02:24] for most people when they move from [02:25] GitHub Actions. I thought this was too [02:27] good to be true, as did many of their [02:28] customers like Mercury, Superbase, [02:30] Dscript, Exa, and more. But then you use [02:32] it and you realize you can't live [02:33] without it. If it was just cheaper, it [02:35] would be worth it. If it was just [02:36] faster, it would be worth it. And if it [02:38] was just more reliable, it would still [02:39] be worth it. But then you discover the [02:41] console and how much better it is than [02:43] anything GitHub has shipped in years. [02:45] You get actual metrics about the [02:47] workflows you're running, what ones have [02:49] high error rates, what the errors might [02:50] be, as well as logs that give you really [02:52] useful details on what is going on [02:54] across all of your CI. Every time I [02:56] forget to set up Blacksmith on a new [02:58] project, I regret it immensely and then [03:00] I move over and watch my build times go [03:01] from 3+ minutes to under one. Your devs [03:03] and your agents is our fast CI. Get it [03:05] today at solidity.link/blacksmith. [03:07] So, let's talk about this new, [03:09] intelligent, fast, cheap model that [03:12] Cursor put out. First, I want to talk [03:14] about price. A lot of people don't [03:16] understand how models are priced. I'm [03:18] going to do my best to explain it simply [03:20] here. You have the input and the output [03:23] costs and these are in million token [03:25] groups. A token is just a chunk of text, [03:28] usually three to five characters, kind [03:30] of like a word, that is how models [03:32] interpret text and then generate text as [03:35] well. Input tokens tend to be a lot [03:36] cheaper than output tokens because the [03:38] inputs are the things you're passing the [03:39] model and it's just using that to set up [03:42] what its guesses for next token [03:43] prediction are going to be. The outputs [03:44] are the actual work it's doing, so those [03:46] tend to be much more. The base price for [03:48] a model like Claude for Sonnet has [03:49] historically been $3 per million tokens [03:51] in and $15 per million tokens out. On [03:54] the 1 million token context versions, [03:57] they would sometimes be more expensive. [03:58] Anthropic has since stopped doing that, [04:00] OpenAI has not, and the Opus models tend [04:02] to be much more expensive. Like 4.5 Opus [04:04] is $5 per mil and 25 per mil out. GPT 5 [04:08] to 5.2 were actually very generously [04:10] priced in my opinion at $1.25 per mil in [04:14] and 10 per mil out. 5.2 was a bit of a [04:16] jump to $1.75 in and 14 out. 5.4 was [04:20] another jump to 250 in and 15 out, and [04:22] then 5.5 was a massive jump to $5 in and [04:25] 30 out, doubling what it was in 5.4. It [04:27] seems like the prices are going up [04:29] across the industry, but they aren't [04:31] necessarily. There's two additional [04:33] layers to the pricing that are important [04:35] to consider. The first is the amount of [04:37] tokens that are actually being [04:38] generated. Flashbang warning for [04:40] artificial analysis. Artificial analysis [04:42] measures roughly how intelligent models [04:44] are on their own benchmarks internally [04:47] that they publish to the world, but they [04:49] also measure other pieces. In [04:50] particular, how fast the model solve the [04:52] problem, how many tokens they generate, [04:54] and how much it cost to run. Output [04:56] tokens are one of my favorite things to [04:57] look at here, and you'll notice some [04:59] interesting stuff. The most output [05:01] tokens didn't come from the smartest [05:02] models. It came from DeepSeek V4 Flash, [05:05] GPT 5.4 Mini, and Claude Sonnet 4.6. [05:09] Smaller models often generate way more [05:11] tokens than they need to to solve [05:14] problems. It's also worth noting that [05:16] the lab trying the hardest to reduce [05:18] this overuse of tokens very much appears [05:20] to be OpenAI, where models like 5.5 only [05:23] did 75 mil tokens for the same bench [05:26] that Sonnet did 200 mil for. Almost a [05:29] third the number of tokens and a higher [05:31] score as well. And when you add in [05:33] models like 5.5 low and medium, which [05:35] still score really well in the benches, [05:37] like 5.5 medium scores as well as Opus [05:40] 4.7 roughly, and the low version is [05:43] worse, but it's still much higher than [05:45] 5.4 Mini and Flash and roughly around [05:47] where something like Sonic 46 is. [05:50] It is expensive per token, [05:52] but remember, it's putting out comically [05:55] less 7 mil tokens. That is a tenth as [05:58] many as 55 high and a 30th or less as [06:02] many as Sonic 46. [06:04] And 55 medium is only 22 mil, so very [06:06] token efficient models. [06:08] So that's the second factor of pricing. [06:10] Factor one is per token cost. Factor two [06:13] is number of tokens to solve a problem. [06:15] Factor three is your ability to [06:17] negotiate better deals. Some of these [06:19] labs are up charging as much as like 90% [06:23] for how much it costs to run the model [06:25] because they have lots of other costs [06:26] like the people who trained the model, [06:28] the computer they had to buy to be able [06:29] to run the model, the computer they had [06:30] to spend it to train it as well. They [06:32] have to factor that into for their [06:33] prices, but a lot of these companies [06:35] could hypothetically speaking give you a [06:38] deal for half off or more per token and [06:41] it wouldn't cost them money. It would [06:43] just be opportunity cost. So they are [06:45] pricing based on how much they think [06:47] they can get away with charging for the [06:48] thing cuz they need to make massive [06:51] profits to justify the absurd amount of [06:52] money they spend training and doing all [06:54] that. That's also why they can subsidize [06:55] so hard doing things like, you know, [06:58] giving you $4,000 of usage per month on [07:01] the $200 plans like Claude does for [07:03] example. Those plans subsidize so hard [07:07] that it's affecting cursor because if [07:09] you're subscribed to Codex or Claude [07:11] code, you are now using a competitor to [07:14] cursor. And Anthropic and OpenAI are [07:17] going to subsidize your usage in [07:19] meaningful ways because they want to get [07:21] you locked into their ecosystems. That's [07:23] just the nature of how this all works. [07:25] So if you spend $200 on Claude code, you [07:27] can get 4,000 plus dollars of usage. You [07:30] can probably get as much if not more [07:32] doing the same with Codex. But if you're [07:34] using the APIs directly, you don't get [07:37] that level of subsidization at all. You [07:39] would be lucky to negotiate 30% off. So [07:42] $4,000 to a Claude code sub costs $200. [07:45] $4,000 of Anthropic models to Cursor [07:50] probably costs $3,000 [07:52] at best. And this means a handful of [07:55] things for Cursor. It means that these [07:58] models getting more expensive on the [08:00] books for the API prices massively hurts [08:04] the amount of usage you can get as a [08:06] user of Cursor. It also means they are [08:08] competing in a subsidization war that [08:10] they don't have a player in. Because if [08:14] Anthropic or OpenAI decide they want to [08:16] crush Cursor, which seems like they kind [08:18] of have, they have the compute and the [08:20] models and the wedge to increase how [08:23] much Cursor has to pay and decrease how [08:25] much users have to pay to use something [08:27] else, that puts them in a really hard [08:29] spot. But Cursor does have one thing [08:32] that almost no one else does, including [08:35] the labs to an extent. Cursor has data. [08:38] Because Cursor's the tool that's been [08:39] used by so many developers for so long [08:42] building with AI, and many of them [08:44] didn't turn off the share my data switch [08:46] in the settings, that is data they can [08:48] use to train the models. Not just like [08:50] they read your code base or something. [08:51] So, people get way too interested in [08:53] that part. That's not that interesting. [08:54] What's much more interesting is how you [08:56] work with the agent, where you work on a [08:58] plan with it, and then you give it [09:00] feedback as it generates things. You [09:01] tell it, "Hey, you missed this thing. Go [09:02] change that, too." Those chat histories [09:05] are so valuable. And Cursor kind of gets [09:08] to double dip here because they can use [09:10] the models at admittedly way higher a [09:12] price than other people get them for to [09:15] resell to use the user building with AI [09:18] in their tool, but they also get to keep [09:20] the data if the right switches are hit, [09:22] so they can use that to train a smarter [09:24] model, not as smart as the ones that [09:26] they are distilling from, but close [09:28] enough that they can collect a lot of [09:31] the margins that they are losing to [09:34] there. Because if you choose to use [09:36] Cursor for open AI models instead of [09:38] using Codex for open AI models, you open [09:41] AI makes a lot more money as a result of [09:42] that. And this is a necessary thing for [09:45] cursor to survive because if the prices [09:47] keep going up, the subsidization [09:49] continues, the incentive to use cursor [09:51] is just a massive financial mess, which [09:54] is why when composer first dropped with [09:56] composer one and it was priced at 125 [09:59] per mil in and 10 per mil out and it was [10:02] super token hungry, I was confused. That [10:05] is too expensive for a model that wasn't [10:08] very good. And then one five, which was [10:10] meaningfully better but still not great, [10:12] was more expensive than sonnet. And [10:14] sonnet's already overpriced. $3.50 per [10:17] mil in and 17.50 per mil out. This is [10:20] the point where I kind of wrote off [10:21] composer for a bit. I just didn't see [10:23] the light. That was my mistake though. I [10:25] was betting against Jacob and you should [10:26] never bet against Jacob. If you're not [10:27] familiar, Jacob's the creator of [10:29] Supermaven as well as Tabnine. [10:30] Supermaven got acquired by Cursor and he [10:33] made the tab complete what it is. He [10:35] wanted bigger challenges, which is why [10:36] he started doing the model training [10:38] side. And then composer two hit. It was [10:40] way cheaper. It was seven times cheaper [10:43] than one five was and way smarter too. [10:46] And now we have 2.5, which is still that [10:49] same cheap price of 50 cents per mil in [10:51] and 2.50 per mil out, but the numbers [10:54] are where it shines. On cursor bench, [10:57] and this is cost per task by the way, so [10:59] left is more expensive, right is [11:01] cheaper. Composer 2.5 is scoring between [11:05] 5.5 high and X high, but costs way less [11:08] than either. [11:10] That's a really big deal. Composer two [11:12] was already pretty competitive in this [11:13] sense where it was crushing things like [11:15] 3.5 flash and Kimmy K26, which is the [11:18] model it's based on, which is really [11:19] interesting. It was getting up there [11:21] with like Opus on medium for way [11:23] cheaper. But 2.5 has gotten slightly [11:25] more token efficient, significantly [11:27] smarter, and stayed absurdly fast. Here [11:30] are the numbers of all of the models [11:32] cursor currently publishes runs for with [11:34] what is admittedly their benchmark [11:37] cursor bench. This benchmark doesn't [11:39] have too many details public. It's real [11:41] problems that they've encountered that [11:42] they collected is my understanding but [11:44] they have not published it. There's a [11:46] lot of good reasons for that like SW [11:48] bench Pro is contaminated. It just is. [11:50] If you make the benchmark open source, [11:52] the labs will get the data on how to do [11:54] it better and then that benchmark stops [11:56] being useful. It's sad that that's the [11:57] case but that is the case. Apparently [11:59] both composer 2 and 2.5 are still based [12:01] on Kimmy K25. They didn't switch to K26 [12:03] as the base which makes their numbers [12:05] even crazier. They have effectively [12:06] doubled the score of their base model [12:09] with Kimmy K25 with composer. Now at a [12:11] 63% [12:13] on their bench. GPT 5.5 got a 64. Opus [12:16] 4.7 got a little closer with 65 at 20x [12:19] the cost though. 5.5 was only roughly [12:22] 10x the cost for a point percentage [12:24] point but composer 2.5 being that good [12:27] for that cheap is absurd. Showing that [12:29] you can distill an open weight model to [12:32] be close to that level of intelligence [12:35] in coding task specifically is both [12:38] super cool to see but also kind of shows [12:41] that all of the work for making models [12:42] good at code is post training in RL [12:45] largely. Like they took a model that was [12:47] okay at code and just ran a [ __ ] load of [12:50] compute and a [ __ ] load of post training [12:52] on it to try and force it through RL to [12:54] be better at coding and they succeeded. [12:56] So let's read what they have to say [12:57] about it. 2.5 is now available in [12:58] cursor. It's a substantial improvement [13:00] in intelligence and behavior over [13:02] composer 2. It's better at sustained [13:03] work on long running tasks following [13:05] complex instructions more reliably and [13:07] it's more pleasant to collaborate with. [13:10] They improved composer by scaling [13:11] training generating more complex RL [13:12] environments and introducing new [13:14] learning methods. In addition to [13:15] training composer for 2.5 on more [13:17] difficult tasks, they've improved [13:19] behavioral aspects of the model like [13:21] communication style and effort [13:22] calibration. These dimensions are not [13:24] well captured by existing benchmarks but [13:25] we find that they matter for real world [13:27] usefulness. I agree. A lot of these [13:29] things are just hard to measure. [13:30] Composer 2.5 is built on the same [13:32] open-source checkpoint as Composer 2, [13:34] which was Moonshots Kimmy K25. I like [13:36] this chart. It's a little crazy, but [13:39] shows how this got so good. This is a [13:43] estimated measurement of how much [13:44] compute was used for each stage of [13:46] training. If you take all the compute [13:48] that went into Composer 2.5 and break it [13:50] up based on Kimmy K2, which was the [13:52] original base that was trained by [13:53] Moonshot, then compare that with Kimmy [13:55] K25 where they did another roughly the [13:57] same to refine it into a better, smarter [14:00] model. Composer did another 8x-ish [14:04] compared to that. It was 10x more total [14:06] compute in the end. And they were able [14:08] to do this because of their collab with [14:09] SpaceX AI, which is still a very [14:12] annoying, weird name collab thing. If [14:14] you're not familiar with that, Cursor [14:15] currently has a pending deal with SpaceX [14:18] where either they will be paid $10 [14:20] billion for their collab, which is [14:21] mostly getting their data, and also [14:23] Cursor gets access to their compute to [14:25] use for things, or at the end of the [14:27] year SpaceX can purchase Cursor for 60 [14:30] bill. The current rumor is that that's [14:32] going to happen when they IPO, which is [14:33] currently in process. The S-1 was just [14:35] filed. So, it's not unlikely this is [14:38] going to happen and Cursor will become [14:40] part of SpaceX, but that compute seems [14:42] to have been huge for them. For them to [14:44] have so soon after that collab started [14:46] already put out a model that's in a [14:48] third place on their benches for code is [14:50] nuts. They gave some fun details on how [14:52] they were doing the training here, too. [14:54] First piece they call it is the targeted [14:55] RL with textual feedback. Credit [14:57] assignment during RL is becoming an [14:58] increasingly difficult challenge as [14:59] rollouts can span hundreds of thousands [15:01] of tokens. When a reward is computed [15:03] over an entire rollout, it may be hard [15:05] for the model to tell which specific [15:06] decision helped or hurt the outcome. [15:08] That's especially limiting when we want [15:09] to discourage a localized behavior like [15:11] bad tool calls, confusing explanations, [15:13] or style violations. That is really [15:15] hard. If you're you're measuring with RL [15:16] that the output works, but it did three [15:19] bad things during it, how do you tell it [15:21] failed without having it fail the actual [15:23] output? The final reward can tell us [15:25] something went wrong, but it's a noisy [15:26] signal for where it went wrong. So, they [15:28] trained 25 with targeted textual [15:30] feedback. The idea is to provide [15:32] feedback directly at the point in the [15:34] trajectory where the model could have [15:35] behaved better. For a target model [15:36] message, we construct a short hint [15:38] describing the desired improvement, [15:40] insert that hint into the local context, [15:42] and use the resulting model distribution [15:44] as a teacher. We use the policy with the [15:46] original context as the student and add [15:48] an on-policy distillation KL loss that [15:50] moves the student token probabilities [15:52] towards the teachers. Interesting. So, [15:54] effectively, there is a another model [15:57] that has been steered the right way by [15:59] being given this additional context and [16:01] information, and then it tries to direct [16:04] the thing being trained to behave [16:06] similarly without having that same [16:08] context. So, you tell one model, "Hey, [16:11] this is the thing we want you to do." [16:13] And it tries to steer the other model [16:15] into going that way without actually [16:18] having that additional instruction. [16:20] Because in the end, we shouldn't have to [16:21] tell our models every step along the way [16:23] how we want them to behave. We should [16:24] tell them what we want, and then they [16:26] behave. So, this makes sense to me why [16:28] this would work well. It's really cool [16:29] to hear. It's awesome they're being so [16:30] transparent about it, too. [16:32] A good example here is when you have [16:33] tool errors. For example, if the model [16:36] has a bad tool call or tries calling a [16:38] tool that's unavailable, the model would [16:39] receive a tool not found error, but the [16:41] model would then continue to make [16:42] additional valid tool calls. This [16:44] screams Gemini. I hope my friends over [16:46] at DeepMind and Google are reading this [16:48] because you need to copy everything [16:49] they're talking about here. No model I [16:51] use regularly has as many malformed tool [16:53] calls as Gemini models do. That's [16:56] because they're RL for the end result, [16:58] almost certainly. So, if it has five bad [16:59] calls and eight good ones, it doesn't [17:01] care cuz the result came out correct. [17:03] They're trying to fix those behaviors. [17:06] The fact that it hit one error in the [17:07] process of hundreds of tool calls would [17:09] have a minimal impact on the final [17:10] reward. Yes, but how So, how do you fix [17:12] that? With text feedback, we can target [17:14] the specific mistake by inserting a hint [17:16] in the context of the problematic turn, [17:19] such as reminder, available tools are [17:21] with a list of available tools. And [17:23] remember, this is all just [17:24] probabilities, where based on the things [17:26] in the context and how that's affected [17:28] the model, it is more likely to answer [17:30] in certain ways. So, if they have a [17:32] teacher that has different context, its [17:35] probabilities are better aligned with [17:36] what we want, it will steer the student [17:39] model that is being RL then trained in [17:41] this process to behave more like the [17:44] teacher who that has this information. [17:45] So, by giving it that additional [17:46] context, it lowers the likelihood that [17:48] you have the wrong tool call [17:50] probabilistically, and it increases the [17:51] likelihood that you have a valid tool [17:53] call instead. And for this specific [17:55] turn, we update the student weights [17:58] towards the new probabilities. During [17:59] the Composer 2.5 run, we applied this [18:01] method to a variety of model behaviors, [18:03] from coding styles to model [18:05] communication. Here's an example. It [18:06] tried calling a tool that doesn't exist, [18:09] read lints. Reminder, available tools [18:12] are read, write, shell, and string [18:13] replace. And now the token probabilities [18:16] that violate the hint go down, and the [18:18] model weights are updated to avoid that [18:19] type of error. The next call they have [18:21] is synthetic data. Apparently, they're [18:23] not just using the data they collect [18:24] from us anymore. Composer's coding [18:25] abilities improved substantially to the [18:27] point where it began to get most [18:29] training problems correct. To continue [18:30] increasing intelligence, we both select [18:32] for and create harder tasks dynamically [18:34] throughout the run. Composer 2.5 is [18:36] trained with 25x more synthetic tasks [18:39] than Composer 2 was. We use a range of [18:41] approaches from creating synthetic tasks [18:42] that are grounded in real code bases. [18:44] For example, one synthetic approach is [18:46] feature deletion. For these tasks, the [18:48] agent is given a code base with a large [18:50] set of tasks and asked to delete code [18:52] and files in such a way that the code [18:54] base remains functional while specific [18:56] testable features are removed. The [18:58] synthetic task is to re-implement the [18:59] feature, and the tests are used as [19:01] verifiable rewards. That's a clever way [19:03] to generate this data. A lot of the work [19:05] for making models smarter is these types [19:07] of clever things. I talked about them a [19:09] lot in other videos, but this is a smart [19:11] one. Tell the model to delete a feature [19:14] and then tell it to re-add the feature [19:15] using the old tests. One downstream [19:17] consequence of large-scale synthetic [19:18] task creation is that it can cause [19:20] unexpected reward hacking. As the model [19:22] got more adept, Composer 25 was able to [19:24] find increasingly sophisticated [19:25] workarounds to solve the task at hand. [19:28] In one example, the model found a [19:29] leftover Python type checking cache and [19:31] reverse-engineered the format to find [19:33] the deleted function signature. In [19:35] another, it was able to find and [19:37] decompile Java bytecode to reconstruct a [19:39] third-party API. We were able to find [19:42] and diagnose these problems using [19:43] agentic monitoring tools, but they [19:44] demonstrate the increasing care [19:46] necessary for large-scale RL. [19:48] Yeah. [19:49] I even seen things like this in my own [19:51] stupid tests. Like earlier, I was trying [19:53] to demonstrate a new framework I'm [19:55] working on and I made a bunch of demo [19:57] projects for this built. And in here, [20:00] you will see that the model found a [20:03] to-do app with this already implemented [20:06] on another project that had a similar [20:08] name nearby cuz I did a restart I did a [20:10] Mario demo I did multiple demos and it [20:11] kept finding other projects nearby that [20:14] it could use to cheat. I'm checking [20:16] nearby Lakebed Capsules for updated [20:17] syntax rather than guessing the database [20:19] API. Yeah, it found other folders nearby [20:22] that had examples in it. The models are [20:24] trained to succeed, not to avoid [20:27] cheating and trying to RL without [20:30] letting the models cheat is a very [20:31] difficult thing and it's cool they're [20:32] sharing examples of the model hacking [20:34] that. Enough of this though, I want to [20:36] play. But here's also where I have to [20:38] crash out a little bit because Glass [20:41] frankly still kind of sucks. This is [20:43] their attempt at doing a Conductor Codex [20:45] app T3 code type thing and it is slow, [20:48] clunky, and obnoxious. We will use it [20:50] here cuz it is the best option they [20:51] provide. I do want to call out in T3 [20:53] code that we support Cursor through [20:56] their ECP bindings for the CLI so you [20:59] can use Cursor models here, but to make [21:01] sure we're using everything properly, I [21:03] am going to do this pass using Cursor [21:05] Glass. It's annoying. I just opened [21:07] cursor dot for this directory and it [21:09] brought me to a new thread view for a [21:12] different project. This is like the [21:14] command I use the most, like cursor [21:16] here. [21:17] Okay, the second time it opened it [21:18] right. Again, like the [21:20] I don't know how to put into words how [21:22] far the quality control has fallen. [21:24] I was really hoping glass would be the [21:25] reset we needed. It hasn't been. This [21:28] has become one of my favorite tests to [21:30] do where I take a game that I built [21:31] already that is kind of functioning but [21:34] needs a lot of work and I tell the model [21:36] to re-implement it from scratch in this [21:38] directory. I'd like to re-implement my [21:40] game fish slop from scratch. I want [21:42] better assets and more reliable core [21:43] engine and a path to making it a real [21:44] quality web game. The new directory for [21:46] it is here, use that as a reference and [21:48] build the game in its entirety in this [21:49] directory. Do not stop until you're [21:51] done. Let's see what it does. [21:53] One of the coolest things about it is [21:54] that it's really fast. It's already [21:56] spinning up parallel agents to do [21:57] different things at the same time, which [21:59] is really cool. One of the crazy things [22:00] they did, I haven't seen a company try [22:02] something this crazy before, is that [22:04] they did an internal test last week with [22:06] the model where all cursor chats for [22:08] employees were redirected to composer [22:10] 2.5 for a few days. This employee didn't [22:12] even notice, which is a testament to how [22:14] good the model's gotten. That is super [22:16] cool to hear. And you can see it flying [22:18] through this stuff right now. It is it [22:21] is churning. It did just do like five [22:23] edits in a row here to like the same [22:25] thing. -1 +1 -1 +1 -1 +1 -1 +7 +1 -7. [22:30] Interesting. [22:31] It's actually checking the resulting [22:33] code, which when I tried the same thing [22:34] with 3.5 flash yesterday, it did not do. [22:37] On that note, 3.5 flash, which also came [22:41] out the same time as composer 2.5, got a [22:43] way lower score than 2.5, lower than [22:46] 2.0, and cost $1.94. I just I don't see [22:50] why anyone would use 3.5 flash for [22:52] almost anything. It's just not a good [22:53] option. It's done? Oof. [22:56] Page failed to load. Here's the result. [23:00] So, it is one of the two models that [23:01] failed to make the actual page render, [23:04] which is sad but not super surprising. [23:07] It is a Chinese open weight model that [23:09] was RL'd to all hell. Yeah, let's see [23:11] how this works now. [23:17] I don't have enough shells to unlock [23:18] those. [23:20] Well, unlike the one that Gemini made, [23:23] it actually functions and has the core [23:24] mechanics behaving. [23:26] That is genuinely insane. I I don't love [23:30] the asset it generated here. [23:33] I'll give it an annoying task, actually, [23:35] to fix that. Pull over all the assets [23:38] from the original game. Let's see if [23:41] it's able to swap out the assets for the [23:43] ones from the original game and how [23:45] quickly it's able to do that. PNG assets [23:47] exist in Fish Game Public, but the glob [23:48] tool failed to detect them. Interesting. [23:51] This is a big move, changing the entire [23:53] render pipeline for your game live. You [23:56] can see the game flashing in the [23:57] background as it does all the changes. [23:59] Apparently, it's done. That took under [24:01] 30 seconds. And yeah, I switched over to [24:04] my original assets. [24:07] It made a lot of things too small. Let's [24:09] tell it to fix that. Scale is really off [24:12] and UI is hard to read, low contrast. On [24:17] the topic of UI, I do want to talk about [24:19] the UI it generates cuz it's actually [24:21] kind of impressive. Friend of the [24:22] channel Dara has made an awesome tool [24:25] that lets you compare what models design [24:28] and how they handle design different [24:29] ways. The site itself was designed by [24:31] Composer 2, funny enough. He runs [24:33] different models with the same prompt to [24:35] generate different web pages to see how [24:37] good they are at design. And here are [24:39] the designs that we got out of Composer [24:41] 2.5. This one's actually kind of nice [24:43] and chill. I I like the way it used the [24:46] font. I like the little thing it made [24:47] here. Tasteful grain. Yeah, this is [24:49] fine. This is clearly using the [24:51] front-end design skill. Yeah, cuz these [24:53] are all the the designs you get out of [24:55] the front-end design skill. It screwed [24:56] up the spacing on these buttons a bit, [24:58] like that should be uh aligned to the [25:00] bottom, I think. [25:01] And this doesn't look great either, but [25:03] like it it honored the goal of this [25:05] design, okay. [25:07] This one I just I don't like. I don't [25:08] like this style of like preppy [25:12] [ __ ] [25:13] I hate the retro ones. [25:15] And this is fine. So, here's how it goes [25:17] without the front end design skill from [25:19] Infropik. This looks fine. [25:22] This looks fine, but way too similar. [25:24] This is different. [25:26] It's different in a way that's kind of [25:27] cute. I don't love it, but it it is [25:29] cute. This is awful. [25:31] And this is [25:33] like an old Tailwind slash like [25:35] Bootstrappy site. I've seen a lot of [25:36] sites that look like this. This feels [25:37] like an old era design that you would [25:40] get in the the purple pink gradient [25:43] classic. Not bad. Like like getting this [25:45] type of design out of a model this cheap [25:47] is really cool to see. Interesting bug [25:49] here. We'll resume when background work [25:51] finishes. What is the background work [25:53] that it is finishing? Cool, they called [25:55] me. [25:55] Let's see how this looks now. [25:58] Oh. [25:59] Yeah, it screwed up the scaling a bunch [26:01] for this, sadly. [26:03] It got like the core mechanics right. [26:05] These buttons on the bottom don't work [26:07] for clicking things, though. Like I have [26:08] to press the hot keys. [26:10] It's [26:11] it's far from perfect, but I could see [26:13] how somebody who doesn't have a lot of [26:15] money, does have a cursor sub or like a [26:17] student discount or something, could [26:19] with enough massaging and back and forth [26:21] make something actually decent out of [26:23] this. Like I see the path there. It's [26:25] not a super happy path, but there is [26:27] one. Okay, it actually does have a lot [26:28] of things wrong with the feeding [26:29] mechanic in the game. It's sadly not the [26:31] worst solution I've seen for this, but [26:33] it is not great. This is an admittedly [26:36] very complex task, but yeah, I was [26:39] hoping for a little more there. For my [26:40] like day-to-day random TypeScript [26:42] end-to-end [ __ ] I found it to be [26:44] relatively pleasant, but there is a big [26:46] problem. [26:47] The problem isn't that I don't like [26:50] glass, even though I don't. The problem [26:52] isn't that it's too expensive or too [26:54] slow or too dumb or any of those things. [26:57] The problem's actually over here on [26:59] Cursor Bench. And the problem isn't [27:01] Cursor Bench itself. It's not that it's [27:02] a dishonest test or anything like that. [27:04] It's that we don't have anything [ __ ] [27:06] else to measure against. Because unlike [27:08] every other model that we talk about on [27:10] this channel, there is no API we can hit [27:12] to test this model ourselves. The only [27:15] way to try out Composer, regardless of [27:17] how much money you're willing to spend, [27:19] is to use it inside of Cursor. Unless [27:22] you spend $60 billion to buy them, then [27:23] you can probably use it for whatever. [27:25] Point I'm trying to make is that we [27:27] don't have a real measurement of this [27:30] outside of what Cursor is putting out. [27:33] The Rob did come out and say that they [27:35] are reporting terminal bench and SWE [27:36] bench multilingual, which sadly is a [27:39] polluted test. Enough info on it has [27:42] leaked that models have it in the [27:43] training data and it's kind of [27:45] compromised now. It is what it is. [27:47] Allegedly they are working with [27:48] artificial analysis and others for more [27:49] external evals, which is very good to [27:51] hear. But I understand on one hand why [27:53] they're not going hard on this because [27:56] they made this model just for coding and [27:57] nothing else. It'll probably perform [27:59] really poorly on stuff like SkateBench, [28:01] for example, because it isn't built for [28:04] that use case. It's built just for [28:07] coding. And a lot of these benches, like [28:09] if you were to throw it on the [28:10] artificial analysis intelligence index, [28:12] the core bench they have here, it would [28:14] perform very poorly, possibly as bad as [28:17] Kim iK25 or worse, because a lot of the [28:21] benches things that aren't code. So I [28:23] get why they're not prioritizing this, [28:24] but it does suck to have a model that [28:26] seems really useful for a bunch of [28:28] different [ __ ] that I can only use [28:30] through Cursor's app or through the ACP [28:33] implementation on their CLI that we had [28:35] to like push really hard for them to [28:36] fix. For a while you couldn't even use [28:39] Composer models through the ACP bindings [28:42] because they had a limited hard-coded [28:43] set of models that didn't include the [28:45] new Composer models. Obnoxious. They've [28:47] since fixed that after a lot of back and [28:49] forth with the team. They also just put [28:50] out the Cursor SDK, which allows you to [28:52] build agents using Cursor's harness and [28:55] infrastructure and everything, which [28:57] hypothetically means you can write code [28:59] like this, where you create an agent [29:01] that uses Composer 2 and has access to a [29:03] specific directory, and then you can [29:05] programmatically run things against it. [29:07] This is cool. Especially cuz you can [29:09] point it at cloud repos as well. There's [29:10] a lot of promise here. And as I've [29:12] mentioned many times, I do genuinely [29:14] really, really like the Cursor Cloud [29:16] stuff. cursor.com/agents has [29:18] consistently been my favorite way to use [29:19] Cursor as of late because it has a full [29:22] Linux box in the cloud that it can use [29:24] to spin up whatever your project is, as [29:26] long as it's not like an iOS app, and [29:28] actually test against it with computer [29:30] use. Super cool when combined with fast [29:33] models like Composer 2 and 2.5 and the [29:35] ability to trigger these from my phone [29:37] or from Slack. I like where they're [29:39] going with all of this. I just hate when [29:41] I see a new model with numbers that are [29:43] this good, and I can't it. This is cool, [29:47] especially cuz you can point it at cloud [29:48] repos as well. There's a lot of promise [29:49] here. [29:50] They're the only lab, in quotes, worth [29:53] looking at that doesn't expose their [29:55] stuff over API at all. This is very [29:57] different for companies like Anthropic, [29:59] who subsidize usage through their tool, [30:01] but charge you full price to hit it [30:03] directly. The only thing going on right [30:04] now that's close is Gemini 3.5 Flash, [30:07] where you can pay API prices, but if you [30:10] want the full speed version, that the [30:12] one that runs at 1,000 TPS instead of [30:13] 200, you have to use antigravity IDE or [30:17] CLI to get the full speeds. There are [30:19] very few instances nowadays, thankfully, [30:21] of labs restricting access to their [30:24] models on a fundamental level. Usually, [30:26] they just subsidize. But now we have [30:28] both Google restricting us from using [30:30] the fast version of their model over the [30:32] API. We can only use the slow one. [30:33] Previously, we had Mistral do the same [30:35] thing, where they had a version of the [30:36] model that could do 1,000 TPS that was [30:38] working through their partnership with [30:40] Cerebras at the time. But if you hit it [30:42] over the official Mistral API, you'd be [30:44] around 50 TPS. And now we have Composer, [30:47] which is literally only headdable via [30:49] Cursor surfaces. You cannot do it over [30:51] API at all. Apparently, people have it [30:54] working in tools like Pi by using the [30:56] Agent SDK that Cursor put out. That's [30:58] cool to see, and I don't think they're [31:00] going to be as egregious with limiting [31:02] how people use the SDK as companies like [31:04] you know, the one big A has been about [31:07] that. But it does suck that I feel like [31:09] I can't meaningfully benchmark this or [31:11] integrate it in other tools. Like if I [31:13] just wanted to swap out the API endpoint [31:16] in something like Snitch Bench or inside [31:18] of something like Pi to use what they [31:21] have here, I can't. I have to rebuild [31:24] the application to use their SDK. And [31:26] for what it is worth, I have not heard [31:27] the best things about the SDK. I have [31:29] admittedly not had a chance to try it [31:31] myself, but I have heard that it is [31:33] missing things and is not the most [31:34] reliable stuff yet. That said, if enough [31:36] of you guys build cool things with the [31:38] Cursor SDK, I will have the leverage I [31:40] need to force the team to push it [31:42] further. And that would be really cool. [31:44] So I do still recommend you go build [31:46] cool things with this. Just make sure [31:47] you do it in a way where if you do have [31:49] to swap out, it's not that hard to do. [31:51] So what are my final thoughts on all of [31:53] this? Well, if you're the type of [31:55] developer that is still staring at the [31:57] agent as it runs and are doing very [32:00] collaborative work with it, where you [32:01] ask it to research a thing, it [32:03] researches it, it gives you a result, [32:05] you ask it to then go make a change, and [32:06] then it does that, you look at the [32:07] result, where you're having a back and [32:09] forth that you're engaged with the [32:10] model, having a model this fast and [32:13] smart is really, really powerful. That's [32:15] why I was excited for 3.5 Flash and [32:17] ultimately was disappointed because it [32:18] wasn't good enough at coding to have a [32:21] good experience doing that with. [32:23] Composer 2.5 is a much better balance [32:25] here. It might not be as capable of [32:27] unblocking itself in solving complex [32:29] tasks as things like the latest GPT [32:31] models or the latest Opus models, but it [32:34] is really fast and close enough that you [32:37] can have this back and forth with it in [32:39] a way that reminds me of the old days [32:41] doing a gentle coding with the [32:42] difference being it's way smarter and [32:44] way more capable. So, the people who are [32:46] working at big companies on giant repos [32:48] that just want to like sit in the code [32:51] base with a real IDE and talk back and [32:53] forth with the model about the changes [32:54] they want to make and not have to wait [32:56] 20 minutes for it to go explore [ __ ] [32:58] This is an incredible model. But, if [32:59] you're the type of person that's [33:01] spinning up 20 agents in parallel on the [33:02] same thing all doing different [ __ ] and [33:05] by the time you spin up agent 17, agents [33:07] 1 through 6 are done, this will not be [33:09] as useful to you. That said, I find [33:11] myself steering away from the super [33:13] parallelization stuff lately cuz it's [33:15] just too much context in your head to [33:17] keep and if you're doing anything else, [33:19] it's just impossible to maintain. I [33:21] can't tell you how many cool projects [33:22] are probably sitting on random work [33:24] trees on this computer that I've never [33:25] had a chance to even look at yet. So, if [33:27] you're doing this type of engaged back [33:29] and forth with the model, a fast model [33:31] is really powerful. I'm saying this [33:33] confidently cuz I've experienced it [33:34] myself recently cuz I've been using [33:36] GPT-4-5 on the fast mode for like 2 [33:40] weeks now because I got way too much [33:42] free usage from OpenAI cuz I went to the [33:44] 4-5 event and everybody who applied to [33:46] go got I think a 10x bump. So, despite [33:48] building a literal cloud from scratch [33:51] over the last few days using the these [33:53] models, yeah, I'm building a cloud. [33:54] We'll talk about that another time, [33:55] don't worry. I'm not struggling to even [33:57] take down 1% of my 5-hour limit much [34:00] less my weekly usage limits. I'm trying [34:02] so hard to burn through this and I [34:03] literally can't because they gave me too [34:05] high of a limit as a result of that [34:07] event. Insane. It is what it is, but as [34:10] such, I've gotten addicted to fast [34:11] because it's 2x more expensive and I'm [34:13] still barely able to burn the [ __ ] [34:15] tokens. I think Composer 2.5 fast is a [34:18] much more realistic option for companies [34:21] where they can't have devs doing the [34:22] subsidized token usage that also don't [34:24] have that free 10x bump temporarily. For [34:26] companies with big code bases that have [34:28] developers actually in their editors [34:30] reading the files and looking at the [34:31] changes, like hand-in-hand with the [34:33] model. It's a good workflow, and having [34:36] a model this fast, this cheap, this [34:38] reliable for that type of real-world [34:40] work fits the archetype of Cursor's [34:43] customer very well. Because Cursor is [34:45] killing it in enterprise. When I talk to [34:47] my friends at big companies that aren't [34:49] Amazon, where they built their own [34:50] shitty IDE, almost everyone else I talk [34:53] to is using Cursor. So much of the [34:55] Fortune 500 is on Cursor, it's insane. [34:57] This is a really good win for them that [35:00] will keep them from taking a potential [35:02] deal with OpenAI or with Anthropic that [35:05] I'm sure they're in negotiations with [35:06] all of those companies. This is a [35:08] massive play. And unlike those other [35:10] labs, if you go to Anthropic, for [35:13] example, with a deal, you can only use [35:15] the latest Opus and Sonnet models. If [35:18] OpenAI does something cool, you're [35:19] [ __ ] If Cursor does something cool, [35:21] you're [ __ ] Same deal if you go with [35:22] OpenAI. But if you go with Cursor, you [35:25] at the very least have their awesome [35:27] models with Composer. But if one of the [35:29] other labs leapfrogs at any point, you [35:31] still get access to that, too. So, [35:33] Cursor has continued to be a really [35:34] compelling bet for enterprises, and I [35:36] think these developments help that a [35:39] ton. By having a really good default [35:41] that they can subsidize themselves and [35:43] make really compelling value [35:45] propositions for their users with, they [35:47] are able to stay a player in this race. [35:50] And as silly as it sounds, I have more [35:52] faith in Cursor to sustain the AGI [35:56] bubble or whatever happens than I do a [35:58] company like Microsoft. Because at the [36:00] very least, Cursor's moved far beyond [36:02] the VS Code fork we made fun of them for [36:05] into a real lab focused on real [36:07] developer workflows at a level that [36:09] Microsoft cannot even comprehend. If I [36:12] was to guess where we'll be in 2 years, [36:15] I legitimately think it's more likely [36:16] that we're using xAI Cursor Composer 7 [36:20] than anything by Gemini for our [36:22] day-to-day dev work. Even if it thinks [36:24] things are going on in the background [36:25] when they're not, bugs will happen. I [36:27] have one last thing I want to end on [36:28] here. Cursor posted in the thread where [36:30] they announced the new model, "Together [36:32] with SpaceX AI, we are training a [36:34] significantly larger model from scratch [36:36] using 10x more total compute. That means [36:39] it's going to be 100x more than Kimmy [36:41] was trained on." With Colossus 2's [36:43] million H100 equivalents and our [36:45] combined data and training techniques, [36:48] we expect this to be a major leap in [36:50] model capability. It is legitimately [36:52] possible that Cursor doesn't just catch [36:55] up to state of the art for code. They [36:57] might leapfrog. There's a real chance [37:00] that in just a few months, Cursor will [37:03] have the best model for code. And when [37:05] you look at the trajectory, when you [37:07] look at how fast they have scaled up [37:09] where they are, and combine that with [37:10] the insane amount of compute they have [37:12] and data and all of the new training [37:14] methods they've been figuring out, I [37:16] think there's a real chance they do it. [37:18] At least I hope so so my investment [37:19] becomes something, but we'll see where [37:20] that all goes. I got nothing else on [37:22] this one. If you're still on Cursor, you [37:23] should definitely try it. And if you're [37:24] not on Cursor, I'm curious what you're [37:26] using instead. Let me know in the [37:27] comments and until next time, [37:29] peace nerds.  ## Highlights - Composer 2.5 achieved 63% on Cursor Bench vs GPT-5.5 (64%) and Opus 4.7 (65%) at 10-20x lower cost through specialized coding focus and innovative training techniques - Note: 📚 Domain-specific post-training can match general models at lower cost · - Demonstrates effectiveness of focused domain specialization over general-purpose model scaling - Evidence for sustainable competitive advantages through training methodology innovation · #atom/literature/h1 - Unlike every other model, there is no API to hit to test Composer ourselves - the only way to try it is inside Cursor, unless you spend $60 billion to buy them - Note: 📚 Platform-exclusive AI models create competitive moats but limit ecosystem integration · - Highlights new form of competitive strategy in AI model distribution - Raises concerns about ecosystem fragmentation and independent evaluation · #atom/literature/h2 - Chat histories where you work with the agent, give it feedback as it generates things, tell it 'you missed this, go change that too' - those are so valuable for training - Note: 📚 Developer interaction data provides unique training advantages for coding models · - Explains sustainable competitive advantage from user-generated training data - Illustrates value of interactive development workflows for model improvement · #atom/literature/h3 - Specialized AI training outcompetes generalist scaling - Note: [[AI model specialization]] — because extends focused training approaches · - [[AI model specialization]] — because extends focused training approaches · #atom/zettel/z1 - Platform-exclusive models as new competitive strategy - implications for open ecosystem development - Note: ✏️ Fleeting capture #atom/fleeting/f1 - Targeted textual feedback training technique - potential applications beyond coding - Note: ✏️ Fleeting capture #atom/fleeting/f2 - SpaceX AI compute partnership dynamics - precedent for other AI lab collaborations - Note: ✏️ Fleeting capture #atom/fleeting/f3 - Drawing broad conclusions about model superiority from limited, internal benchmarks without external validation - Note: 🔍 Fallacy: Hasty Generalization #fallacy/hasty-generalization - Presenting platform exclusivity as purely advantageous without adequately weighing ecosystem fragmentation costs - Note: 🔍 Fallacy: False Dilemma #fallacy/false-dilemma