Claude Opus 4.7 Just Dropped... Or Did It Really

Claude Opus 4.7 Just Dropped... Or Did It Really? ![rw-book-cover](https://img.youtube.com/vi/NiMc2PoTiXo/maxresdefault.jpg) 🔗 **Source:** [Claude Opus 4.7 Just Dropped... Or Did It Really?](https://www.youtube.com/watch?v=NiMc2PoTiXo) > [!note]+ Document Notes > tags: #claude #anthropic #ai-models #benchmarking #software-engineering #coding #ai-degradation > author: The AI Advantage > source: YouTube > reference: https://www.youtube.com/watch?v=NiMc2PoTiXo > date created: 2024-12-19 14:30 > > # 🏷️ Title: Claude Opus 4.7 Just Dropped... Or Did It Really? >  ## 🧰 Action Inbox - [H1] Test Opus 4.7 performance against 4.6 on coding tasks — evaluate reasoning depth #claude - [H2] Monitor token usage with new tokenizer — costs may increase 1-1.3x #ai-costs - [H3] Evaluate Claude Code desktop app bugs — QA issues reported #developer-tools ## 🧭 Table of Contents - [[#📝 Summary]] - [[#📖 Main Content]] - [[#🗒️ Notes]] - [[#📚 References]] - [[#🗣️ Paraphrase]] - [[#📰 Newsletter]] - [[#✅ Action Items]] - [[#🧪 Hypotheses]] - [[#📊 Methodology]] - [[#❓ Questions]] - [[#🔮 Future Research]] - [[#💡 Implications]] - [[#🧠 Learning Styles Assessment]] - [[#📊 Accessibility Assessment]] - [[#🌍 DEIB Assessment]] - [[#✅ Logical Validity]] - [[#🛡️ DARVO Analysis]] - [[#🎯 Conclusion]] ### 🔗 References - [Claude Opus 4.7 Announcement](https://www.anthropic.com) - [Bridge Benchmark](https://bridgebench.ai) - [SWE-bench Pro](https://swe-bench.com) ## 📝 Summary - Anthropic released Claude Opus 4.7 with significant improvements over 4.6 - Users had complained about dramatic quality degradation in Opus 4.6 over recent weeks - Analysis reveals 4.6 degradation was caused by parameter changes, not model retraining - 4.7 introduces new "X-high" effort level and improved reasoning capabilities - Questions remain about whether degradation was intentional to create upgrade demand ## 📖 Main Content Claude Opus 4.7 arrives amid controversy over whether Anthropic intentionally degraded 4.6 performance to amplify the impact of this release. The evidence suggests Opus 4.6's quality decline wasn't due to model changes but rather parameter adjustments that reduced reasoning effort and thinking depth. Key technical changes in 4.6 included adaptive thinking being set to dynamically allocate zero reasoning tokens for "simple" requests, and effort defaults being silently dropped to medium. These changes weren't communicated to users, leading to widespread complaints about model stupidity. Opus 4.7 addresses every major complaint with seemingly targeted fixes: deeper reasoning with X-high effort mode, better instruction following, reduced hallucinations, and improved vision capabilities. The model shows substantial benchmark improvements across coding, vision, and long-context reasoning tasks. However, the new release comes with trade-offs including a new tokenizer that may increase costs by 1-1.3x and the simultaneous launch of a buggy desktop app that raises questions about Anthropic's QA processes. ## 🗒️ Notes #### ⏱️ [00:00](https://youtu.be/NiMc2PoTiXo?t=0) — [H1] Opus 4.7 release addressing 4.6 complaints - Key idea: New model claims to handle long-running tasks with more rigor and instruction-following - Evidence: "These claims are addressing exactly what everyone was complaining about with Opus 4.6" #### ⏱️ [01:23](https://youtu.be/NiMc2PoTiXo?t=83) — [H1] AMD analysis reveals 4.6 degradation metrics - Key idea: Thinking depth collapsed 73% from 2200 to 600 characters of reasoning - Evidence: "Model stopped reading files before editing them and stopped thinking as much" #### ⏱️ [02:45](https://youtu.be/NiMc2PoTiXo?t=165) — [H2] File reading behavior deteriorated significantly - Key idea: Model went from reading files 94% to only 66.3% of the time - Evidence: "Models should always be reading files before editing but it started to not do that 33.7% of the time" #### ⏱️ [03:15](https://youtu.be/NiMc2PoTiXo?t=195) — [H2] User interruptions increased dramatically - Key idea: Users had to interrupt the model 12 times more often to prevent errors - Evidence: "Quality problems and token burning causing users to hit $200/month limits in an hour" #### ⏱️ [04:30](https://youtu.be/NiMc2PoTiXo?t=270) — [H3] Common 4.6 failure patterns identified - Key idea: Hallucinated git hashes, fake packages, premature task abandonment became common - Evidence: "Word 'simplest' appeared three times more often, optimizing for least effort" #### ⏱️ [05:45](https://youtu.be/NiMc2PoTiXo?t=345) — [H1] Anthropic's February 9th adaptive thinking change - Key idea: Model began dynamically deciding reasoning allocation, sometimes using zero tokens - Evidence: "On certain turns, if it thinks request is simple, it would allocate zero reasoning tokens" #### ⏱️ [07:00](https://youtu.be/NiMc2PoTiXo?t=420) — [H2] Silent effort default reduction discovered - Key idea: Effort settings were dropped to medium without user notification - Evidence: "They basically just switch this down without telling anybody" #### ⏱️ [08:30](https://youtu.be/NiMc2PoTiXo?t=510) — [H2] Bridge benchmark controversy - Key idea: Opus 4.6 fell from #2 to #10 on hallucination leaderboard - Evidence: "Dropped from 83.6% to 68.3% accuracy, now worse than Sonnet 4.5" #### ⏱️ [09:15](https://youtu.be/NiMc2PoTiXo?t=555) — [H1] Opus 4.7 availability and interface changes - Key idea: New model available with adaptive thinking toggle and X-high effort mode - Evidence: "For Opus 4.7 we have adaptive thinking, but 4.6 has extended on/off" #### ⏱️ [11:00](https://youtu.be/NiMc2PoTiXo?t=660) — [H1] Benchmark improvements across categories - Key idea: Substantial gains in coding, vision, document reasoning, and biomolecular tasks - Evidence: "Biomolecular reasoning shows massive jump, over double the performance" #### ⏱️ [12:30](https://youtu.be/NiMc2PoTiXo?t=750) — [H2] Suspiciously targeted fixes for complaints - Key idea: Every major 4.6 complaint has a directly corresponding 4.7 improvement - Evidence: "Shallow thinking → X-high effort, task abandonment → hand off hardest work with confidence" #### ⏱️ [14:45](https://youtu.be/NiMc2PoTiXo?t=885) — [H3] New tokenizer cost implications - Key idea: Same inputs may cost 1-1.3x more due to updated tokenization - Evidence: "Opus 4.7 uses updated tokenizer which improves text processing but increases costs" #### ⏱️ [16:30](https://youtu.be/NiMc2PoTiXo?t=990) — [H2] Desktop app launch reveals QA issues - Key idea: Multiple bugs found despite claims of internal engineering team usage - Evidence: "Developer found 40+ bugs in 1 hour - questions about actual internal testing" #### ⏱️ [19:00](https://youtu.be/NiMc2PoTiXo?t=1140) — [H3] Financial analysis comparison test - Key idea: 4.7 showed better financial subject matter expertise and actionable insights - Evidence: "4.7 explained and justified things more, felt like it had financial expertise" #### ⏱️ [21:15](https://youtu.be/NiMc2PoTiXo?t=1275) — [H3] SaaS modeling deliverable comparison - Key idea: 4.6 created interactive dashboard, 4.7 produced comprehensive Excel analysis - Evidence: "4.7 gave actual Excel sheet with multiple tabs and scenarios" ## 📚 References - [Anthropic Claude Opus 4.7 Release](https://www.anthropic.com) - [Claude Desktop App](https://claude.ai/desktop) - [Bridge Benchmark Leaderboard](https://bridgebench.ai) - [SWE-bench Professional](https://swe-bench.com) ## 🗣️ Paraphrase - Claude Opus 4.6 experienced significant quality degradation due to parameter changes that reduced reasoning effort and thinking depth, not actual model deterioration - Anthropic released Opus 4.7 with improvements that directly address every major complaint about 4.6, raising questions about intentional degradation tactics - The new model shows substantial benchmark improvements but comes with increased costs and a buggy desktop app that undermines confidence in Anthropic's development practices ## 📰 Newsletter **Claude Opus 4.7: Major Upgrade or Strategic Recovery?** Anthropic just released Claude Opus 4.7 with impressive benchmark improvements, but the story behind this release raises important questions for developers and organizations evaluating AI tools. **What Happened:** Recent analysis revealed that Opus 4.6's quality issues weren't from model degradation but from parameter changes that reduced reasoning effort. Users experienced 73% less thinking depth and 12x more interruptions needed to keep the model on track. **Key Improvements in 4.7:** - New "X-high" effort mode for maximum reasoning - Better instruction following and self-verification - Improved vision and long-context capabilities - Enhanced coding performance on complex tasks **Important Considerations:** - New tokenizer may increase costs by 10-30% - Desktop app launch had significant QA issues - Questions about transparency in model parameter changes For WordPress developers, these improvements could significantly impact coding workflows, but careful cost monitoring will be essential. Resources: [Learn WordPress](https://learn.wordpress.org/), [Developer Docs](https://developer.wordpress.org/), [WP-CLI](https://wp-cli.org/) ## ✅ Action Items #### DevRel - Developer education: Create comparison guides between Claude versions for coding tasks - Developer experience: Test Claude Code desktop app integration with WordPress development workflows - Community management: Gather feedback on AI tool degradation transparency expectations #### WordPress - Core: Evaluate Claude 4.7 for code review and contribution assistance - Documentation: Test AI assistance for documentation improvement workflows - Community: Assess cost implications of AI tools for community-funded projects ## 🧪 Hypotheses - Anthropic intentionally degraded 4.6 performance to amplify 4.7's perceived improvements - The new tokenizer's cost increases will offset some performance gains for budget-conscious developers - Desktop app QA issues indicate rushed release timelines that may affect model quality - Parameter transparency will become a key differentiator for enterprise AI adoption ### 📊 Methodology - A/B testing: Compare same prompts across 4.6 and 4.7 with controlled effort settings - Cost analysis: Track token usage before and after tokenizer changes - Bug tracking: Monitor desktop app issue reports and resolution timelines ### ❓ Questions - Will Anthropic provide more transparency about parameter changes affecting model behavior? - How will the cost increases impact adoption among individual developers and small teams? - What does the desktop app's QA issues reveal about Anthropic's internal development practices? ### 🔮 Future Research - Long-term performance consistency monitoring across Claude model updates - Comparative analysis of AI provider transparency practices around model changes - Impact assessment of AI cost increases on open source development sustainability ### 💡 Implications - Practice: AI tool evaluation must include parameter stability and transparency assessment - Policy: Organizations need clear expectations for AI provider communication about performance changes - Security: Model parameter changes without notification could mask security or reliability issues - Education: Developers need training on AI cost management and performance monitoring - Funding: Budget planning for AI tools must account for tokenizer and parameter change impacts ### 🧠 Learning Styles Assessment Reading level estimate: 10th Grade #### 👀 Visual learners - Present: Screenshots of model interfaces, benchmark charts, comparison tables - Improve: Side-by-side output comparison visuals, cost impact graphs #### 🔊 Aural learners - Present: Video format with narration explaining technical concepts - Improve: Audio summary of key findings and recommendations #### 🗣️ Verbal learners - Present: Clear definitions of technical terms, analogies about model behavior - Improve: Glossary of AI model terminology, metaphors for parameter changes #### 🧩 Kinesthetic learners - Present: Step-by-step testing methodology, hands-on comparison examples - Improve: Interactive cost calculator, guided testing checklist #### 🧠 Logical learners - Present: Benchmark data, timeline of changes, cause-and-effect analysis - Improve: Decision framework for model selection, systematic evaluation criteria #### 🧑‍🤝‍🧑 Social learners - Present: Community feedback, user experience reports - Improve: Discussion forums, peer testing groups #### 🧘‍♀️ Solitary learners - Present: Detailed technical analysis, individual testing scenarios - Improve: Self-assessment checklist, personal evaluation framework #### 🌿 Naturalistic learners - Present: Pattern recognition in model behavior changes - Improve: Ecosystem thinking about AI tool dependencies ### 📊 Accessibility Assessment #### Screen reader semantics - Present: Clear headings, structured content organization - Improve: Better alt text for benchmark charts and interface screenshots #### Keyboard navigation - Present: Standard video controls, text-based content - Improve: Timestamp navigation, transcript search functionality #### Contrast and readability - Present: High contrast text, clear font sizing - Improve: Dark mode compatibility, customizable text size #### Text resizing and reflow - Present: Responsive text layout - Improve: Maintain readability at 200% zoom #### Media and captions - Present: Video content with narration - Improve: Detailed captions, transcript with speaker identification #### Forms and interactions - Present: Simple navigation structure - Improve: Accessible comparison tools, interactive elements ### 🌍 DEIB Assessment #### Cultural breadth - Present: English-focused technical content - Improve: Multi-language AI tool considerations, global cost impact analysis #### Inclusive language - Present: Gender-neutral technical terminology - Improve: Plain language explanations, defined jargon #### Neurodiversity support - Present: Structured content with clear sections - Improve: Summary boxes, consistent formatting patterns #### Representation - Present: Technical focus without identity assumptions - Improve: Diverse use case examples, varied developer contexts #### Psychological safety - Present: Balanced critique of corporate practices - Improve: Clear disclaimers about speculation, respectful analysis ### ✅ Logical Validity #### Strong reasoning - Deductive strength: Benchmark data supports performance improvement claims - Inductive strength: Pattern of fixes matching complaints suggests intentional strategy - Causal reasoning: Parameter changes directly linked to quality degradation - Analogical strength: Desktop app QA issues analogous to potential model testing gaps #### Fallacies - Post hoc ergo propter hoc: Assumes degradation was intentional without definitive proof - Hasty generalization: Desktop app bugs may not reflect broader development practices #### Tone of voice - Overall tone: Analytical and skeptical but fair, balancing technical assessment with critical questions about corporate practices ### 🛡️ DARVO Analysis - Deny: No clear denial patterns in the content - Attack: Some criticism of Anthropic's practices but focused on technical evidence - Reverse victim/offender: No victim/offender dynamics present - Emotional cues and rhetoric: Uses measured skepticism rather than inflammatory language ## 🎯 Conclusion Claude Opus 4.7 represents either a genuine breakthrough or a calculated recovery from self-inflicted degradation. While the technical improvements are --- ## Full Transcript [00:00] We just got Opus 4.7, which is [00:01] apparently Claude's most capable Opus [00:03] model yet. It handles long-running tasks [00:05] with more rigor. It follows instructions [00:07] more precisely and verifies its own [00:09] outputs before reporting back. What's [00:10] interesting to me is that these claims [00:12] are addressing exactly what everyone was [00:14] complaining about with Opus 4.6 the past [00:16] couple weeks when everyone was like, [00:17] "Hey, Opus has gotten so much more [00:19] stupid lately." So, yes, we see these [00:20] benchmarks. We see that there's a huge [00:22] jump from Opus 4.6 to Opus 4.7 in most [00:25] of these major categories. So, what I'm [00:27] here to talk about today is, is this [00:29] really Opus 4.7 or is this just 4.6 with [00:32] some improvements? Some people have been [00:34] saying that they intentionally made Opus [00:35] 4.6 worse so that they could build up [00:37] hype for 4.7 and then release it and [00:39] then it feels way better than it really [00:41] would have been. So, if you missed all [00:42] that, I'm going to catch you up and [00:43] let's just start flying through this [00:44] stuff. Don't want to waste anyone's [00:45] time. So, act one, as we're calling it [00:48] here, is what happened to Opus 4.6. So, [00:50] there's been lots of stuff. So this [00:52] first one I wanted to address was that a [00:53] senior director at AMD published an [00:55] analysis of almost 7,000 coding sessions [00:57] using claude code. She found that the [00:59] thinking depth collapsed 73% from 2200 [01:02] characters of reasoning down to just 600 [01:04] characters of reasoning. So basically [01:06] the model stopped reading files before [01:07] editing them and stopped thinking as [01:09] much and it just started jumping [01:11] straight into doing things. So the [01:12] models basically should always be [01:13] reading the files before editing them [01:15] and it started to not do that about 6% [01:17] of the time but then that went all the [01:18] way up to 33.7% of the time. So almost [01:21] onethird it just wasn't reading the [01:23] files. And she found that users had to [01:25] interrupt the model 12 times more often [01:27] to stop it from going off the rails. And [01:28] obviously there's lots of problems here. [01:30] There is the problem of quality of the [01:32] actual code that's being written, [01:33] especially when it's kind of like dark [01:35] code and you don't really exactly know [01:36] what AI is writing. And then there's [01:38] also the fact that that's going to burn [01:39] through your tokens, which is why in the [01:41] past couple weeks, there's also been a [01:42] huge uproar of people saying, "I just [01:44] burned through my 200 bucks a month max [01:45] plan in an hour." So what else have [01:47] users been seeing? hallucinated git [01:49] commit hashes, fake package names, [01:51] fabricated API versions. Apparently, the [01:53] word simplest appeared almost three [01:55] times more often in model output, which [01:57] means that it was optimizing for the [01:58] least amount of effort possible. We've [02:00] also been seeing premature task [02:02] abandonment, so the model will just stop [02:03] working on things midtask. This is [02:05] something that I have actually felt a [02:06] lot over the past couple weeks. And all [02:08] of this kind of stuff, the session [02:09] limits, the quality has caused some of [02:12] the power users on 200 bucks a month max [02:14] plans to start switching or canceling [02:16] away. So then, of course, Enthropic [02:17] addressed this. What did they actually [02:19] change? Well, on February 9th, they [02:21] changed the adaptive thinking, which [02:22] meant that the model started dynamically [02:24] deciding how much to think per turn. So [02:26] on certain turns, if it thinks the [02:28] request is simple, it would allocate [02:29] zero reasoning token. So literally no [02:32] thinking at all. So the question [02:33] becomes, how does the model decide what [02:35] task actually deserves a token budget [02:37] for thinking or not? And then Boris [02:39] Churnney, the basically creator of Cloud [02:40] Code, confirmed the turns where the [02:42] model fabricated had zero reasoning and [02:44] the turns with deep reasoning were [02:46] correct. So that was one of the first [02:47] main things. Another thing that happened [02:49] is they just dropped the effort default [02:51] to medium. So basically all of these [02:53] changes they didn't actually make to the [02:55] model itself but more in the way the [02:56] model behaved like the parameters and [02:58] the weights and the training of the [02:59] model OBU 4.6 was the same but they [03:02] turned off its thinking and they turned [03:03] off its effort basically. And they kind [03:05] of did this one silently like right here [03:07] if I go into my cloud code desktop app [03:10] which I'm going to talk about this new [03:11] desktop app as well because I think that [03:12] there's lots of issues here. But right [03:15] here, you can see we've got Opus 4.7, [03:16] right? And when I click on it, you can [03:18] see that we actually don't have the [03:19] ability within Opus 4.7. But basically, [03:21] if I just zoom back in here a little [03:22] more, if I go to Sonnet, for example, [03:24] you can see that we have effort and [03:26] high, medium, low, and then sometimes [03:28] for Opus 4.6, there's even a higher one, [03:31] um, max effort or something. But they [03:33] basically just switch this down without [03:35] telling anybody. And no one was kind of [03:36] looking in there. And if you're on the [03:38] terminal or wherever you are, you might [03:39] not even notice that. So people were [03:40] like, "Okay, Opus is just stupid." And [03:43] then a full month later, April 15th, pro [03:45] and max users were still by default just [03:47] running on medium effort. So like I [03:49] said, they didn't change the model [03:50] itself of 4.6. They just changed how [03:52] hard the model was allowed to think and [03:54] they didn't tell anyone. And a lot of [03:55] that thinking and reasoning is where we [03:57] get all the value of this, you know, [03:58] awesome Opus 4.6 model. And then there [04:00] was also some controversy with the [04:02] bridge bench. A viral benchmark claimed [04:03] that Opus 4.6 dropped from 83.6% to [04:07] 68.3% accuracy. So basically on this [04:10] leaderboard fell from being number two [04:12] all the way down to number 10 and it was [04:14] just you know Opus 4.6 or Opus 4.6 as of [04:17] April 12th which showed that Opus was [04:19] now actually worse than Sonnet 4.5 on [04:21] this specific hallucination benchmark. [04:23] Okay, so let's move on to act two. Opus [04:25] 4.7 drops today. Okay, so before we get [04:27] into the benchmarks, let me just show [04:28] you real quick. It is available on the [04:30] web right here. We've got Claude in the [04:32] chat and I can see Opus 4.7 right here. [04:34] Now what you'll notice is for Opus 4.7 [04:36] we have adaptive thinking. So we can do [04:38] things only when needed like we just [04:39] talked about. But if we go back to Opus [04:41] 4.6, we don't have adapted, we have [04:43] extended. So we can just turn extended [04:45] on or off. I already showed you guys the [04:47] desktop app. Here is my VS Code [04:49] terminal. You can see we have welcome to [04:50] Opus 4.7 X high. So we'll talk about [04:53] that in a sec. But if you're not seeing [04:54] this, maybe just try to update your [04:56] Claude. So you can go ahead and just run [04:57] like a claude update and that's how it [05:01] will do it. Or if you want to do it in [05:02] the extension, then you'll just have to [05:04] update the actual extension over here in [05:06] VS Code. And if I go to model in the [05:08] extension and we click on switch, you [05:09] can see right here because I haven't [05:11] updated the extension, we only have [05:13] Opus. But you would, like I said, go [05:14] over here, search for Cloud Code, and [05:16] then just install that update. But if [05:18] you're doing all those things and you're [05:18] still not seeing the model, just give it [05:20] a couple hours. It'll roll out to you [05:21] eventually. So anyways, let's go take a [05:23] look at the benchmarks. So here's the [05:25] official announcement that Enthropic [05:26] dropped. You know, our notable [05:28] improvements on Opus 4.6 ICS and [05:30] software engineering with particular [05:32] gains on the most difficult tasks. Users [05:35] report being able to hand off their [05:36] hardest coding work, the kind that [05:38] previously needed close supervision to [05:39] Opus 4.7 with confidence. Now, what I [05:42] want you guys to pay attention to here [05:43] as well is that we still have myths [05:44] preview over here. So, it says the model [05:46] has substantially better vision. It's [05:48] also more tasteful and creative when [05:50] completing professional tasks, and it's [05:51] much less capable than their most [05:53] powerful model, Claude Myths. Of course, [05:55] if you guys didn't see anything about [05:56] that, I made a full video. I'll tag that [05:58] one right up here. But it's obviously a [06:00] lot better than Opus 4.6 across things [06:02] like the coding bench or the search [06:04] computer use financial analysis which is [06:06] very interesting and you know like [06:08] vision. So if we look at some other [06:10] things we've got knowledge work we see a [06:12] significant improvement vision we see a [06:13] big improvement document reasoning this [06:15] one's pretty big long context reasoning [06:17] so both obus 4.6 and 4.7 will allow you [06:21] to use it with 1 million token context [06:22] window biomolecular reasoning which is a [06:25] massive jump. it's over double and then [06:27] long-term coherence and coding of [06:29] course. Now, what's interesting and what [06:31] I alluded to earlier is that it seems [06:32] like every complaint now has a directly [06:35] correlating matching fix, which I'm not [06:37] saying that's a bad thing because that's [06:38] really how you should be iterating on a [06:39] product. But some people are definitely [06:41] starting to raise some eyebrows at the [06:42] way that they're just like it's almost [06:44] like they're they're creating holes just [06:45] so they can fill them and look like the [06:47] hero. So, for example, people were [06:48] complaining about shallow thinking with [06:50] 4.7. And now they've introduced their [06:52] new X high effort level, which is [06:54] exclusive to 4.7. We have a complaint [06:56] about task abandonment, but Enthropic [06:58] now says you can hand off your hardest [07:00] coding work with confidence. We have [07:02] ignoring instructions, and now we say [07:04] substantially improved literal adherence [07:05] to instructions. We have hallucinations, [07:08] but now it's apparently able to catch [07:09] its own logical faults during the [07:11] planning phase. Weak vision, that's been [07:12] dramatically improved. And then this [07:14] one's interesting, too. blackmail and [07:15] safety concerns because there was a [07:17] safety concern in 4.6 where essentially [07:20] the model was like blackmailing an [07:21] engineer, but they have explicitly [07:23] addressed this right here, safety and [07:25] alignment. Overall, it shows a similar [07:27] safety profile to Opus 4.6. Our [07:29] evaluations show low rates of concerning [07:31] behavior such as deception, sycopancy, [07:34] and cooperation with misuse. They've [07:36] also kind of worked it in so that it [07:37] can't be used for like cyber security [07:40] type of threats or hacking and other [07:42] sorts of things like that with prompt [07:44] injection. It kind of all loops back [07:45] into that ecosystem of what they have [07:46] talked about with project glasswing. [07:48] Here we have a misaligned behavior [07:51] benchmark. So overall you can see that [07:53] Opus 4.6 has about a 2.75. Mthas preview [07:56] comes in really low and Opus 4.7 kind of [07:58] sits in the middle. Now in this type of [08:00] benchmark you want a lower score. Lower [08:02] is better. So along with this release we [08:04] have some other things coming. We have [08:06] the X high mode which is extra high [08:08] effort like I just talked about. So now [08:09] we have the low, medium, high, max and X [08:13] high. And we also have this new slash [08:15] command coming which is slash ultra [08:17] review which basically just spins up a [08:18] dedicated review session to read through [08:20] changes and flag things. They also give [08:22] us this section about migrating from [08:23] Opus 4.6 to 4.7. It is a direct upgrade [08:26] but apparently there are some things to [08:28] be thinking about with token usage. So [08:30] Opus 4.7 uses an updated tokenizer which [08:33] improves how the model processes text. [08:35] But what that means is the same input [08:37] might actually cost you more just like 1 [08:39] to 1.3 times more depending on the type [08:41] of content. Also, Opus 4.7 thinks more [08:44] at higher effort levels, which once [08:46] again could use more tokens. I'm about [08:48] to drop a full video on how to manage [08:50] your tokens even better. So, that'll be [08:51] out in like a day or so, and definitely [08:53] want to check that one out. I'll tag [08:55] that one up here when that comes out. [08:56] So, there's a bunch of benchmarks, [08:57] right? And that's the tough thing is you [08:59] see all of this noise, you see all these [09:00] benchmarks, and everything looks like [09:02] it's better, but you don't truly know [09:03] until you actually get your hands dirty [09:05] and play with it. They even released [09:06] this um spec card or system card about [09:09] Cloud Opus 4.7. It's 232 pages. And if [09:12] we go down here, you can see that this [09:13] thing is basically chalk full of [09:15] experiments and benchmarks and just [09:17] comparisons between, you know, Mythus, [09:19] Sonnet, Opus, and Opus. So, this is [09:22] going to be really interesting to dive [09:23] into. I read through a lot of this, but [09:25] not going to break it all down. I think [09:26] it would be insanely boring for most of [09:28] the people watching this anyways. But [09:30] yeah, if you do want to check this out, [09:32] you can find the link to this on this [09:34] main release page from Anthropic. Okay, [09:36] so is 4.7 actually new? I think that it [09:40] is a new model and we're going to get in [09:41] here and I'm going to show you some [09:42] examples of me playing around with it. [09:44] But the X higheffort level didn't exist. [09:47] The vision gains are actual differences [09:50] in the model itself, structural changes, [09:53] not just like, you know, turning on [09:55] effort or turning off effort. There was [09:56] also a noticeable jump in the SWBench [09:58] Pro, which is obviously pretty good as [10:00] well. The updated tokenizer, once again, [10:02] I'm not saying that how good or bad that [10:04] is. I haven't had enough time to fully [10:05] test, but it will probably be a little [10:07] bit more expensive. And then we also saw [10:08] a third party benchmark which was 3x [10:11] production task resolution on Rakutin. [10:13] Now real quick, something I want you [10:14] guys to think about before we get into [10:16] some of the other experiment stuff is [10:19] the desktop app launch which they had [10:21] basically said, hey, we've been working [10:23] on this for a long time. We're very [10:24] excited. Most of our internal engineers [10:25] are using this now instead of their IDs. [10:27] And I thought it was awesome, right? [10:29] Like I switched into it yesterday. I [10:31] only used this instead of VS Code cuz I [10:33] wanted to play around with it. And [10:34] there's a lot of things I really like. [10:36] you know, you can see all of your [10:37] different sessions here. You can also [10:39] manage your projects and have multiple [10:40] going at the same time. You can see when [10:42] they're done. You can see when they're [10:43] waiting on you. And I like a lot of the [10:45] idea of what they're doing. Now, [10:46] obviously, a lot of this has been taken [10:47] from like the Codeex app or other Vibe [10:50] Coding, um, Aenta Coding apps out there. [10:52] And that's not anything to complain [10:53] about. I think that's just the way the [10:54] world works. But anyways, I can come [10:56] into a session and I can see things over [10:57] here like a preview if you're working in [10:59] a local host. You could also pull up a [11:01] terminal right here. So, you could talk [11:02] to Claude in the terminal. Um, that way [11:04] you get some of the extra functionality [11:05] too. You could also look at your tasks [11:08] and you could look at the plan. So [11:09] there's ways that you can kind of [11:10] customize the view which I think is [11:12] really cool. And you can also have [11:13] different split views open. So you can [11:15] have multiple different sessions and [11:17] even multiple different projects. So um, [11:19] that's really cool. I also like right [11:20] here if I just kind of close out of this [11:22] one, you can right from in here see your [11:25] context window and your session. So like [11:28] here is my 5 hour limit. I can see when [11:30] that resets in 10 minutes. So I've got, [11:32] you know, left to play with. I can see [11:33] my models and one of the things I talk [11:35] about is like if you're running through [11:37] your limit, why don't you just watch it [11:38] more often? So, typically what that [11:40] meant is have something pulled up, but [11:41] now you can literally just from here [11:43] click into it and see, okay, cool, I've [11:44] got that much left. So, there's a lot of [11:46] other features here that are really [11:47] good. Obviously, now with the routines [11:49] and stuff, it's nice to be able to work [11:50] out of this app. But there are a lot of [11:52] things that are really poorly designed [11:54] as well. And a developer named Theo um [11:57] he basically found like 40 plus bugs in [11:59] just 1 hour. He was playing around with [12:00] it and there were lots of things like [12:02] buttons weren't working, things were [12:04] snapping weird. Um, you were like just [12:06] the the layout wasn't as intuitive and [12:08] there was just a lot of little bugs [12:10] essentially. Like for example, if you do [12:11] your voice input, it would put voice [12:13] into every single text box that's [12:15] visible on the screen. Just little [12:16] things like that. Um, and what's [12:18] interesting is they are one of the [12:20] biggest AI companies in the world and [12:22] honestly one of the biggest tech [12:23] companies in the world, but they're [12:24] shipping things so fast and they're [12:26] having these issues where did they even [12:28] do QA? And if their team had really been [12:30] using this for so long internally, [12:33] wouldn't they have found all these bugs? [12:34] And if they have Mythus, which is [12:36] supposed to be so dangerous and so good [12:38] that it can't even be released, [12:39] shouldn't Myths have been able to like [12:41] make this better? Because it's not a [12:43] matter of the model not being able to [12:45] execute the code properly. It's an issue [12:47] of the model not being actually smart [12:49] enough to go QA properly because as Theo [12:51] pointed out, a lot of the bugs he found, [12:53] it would be a oneprompt fix, right? But [12:56] no one was able to find that bug. So [12:59] there's questions about, you know, how [13:00] much are they actually using it or like [13:01] what actually went on because they're [13:03] shipping at the speed of a startup, a [13:04] tech startup who that just raised a [13:06] bunch of capital. Okay, so I had to get [13:08] that off my chest. Let's move on to act [13:09] three, the verdict. So I just ran two [13:11] experiments here that I wanted to show [13:12] you guys and I did this on the web app [13:15] with 4.6 extended and 4.7. So the first [13:18] one, let me show you guys this one. What [13:20] I did is I dropped in this chart. So, [13:22] this is a meta financial um stock chart, [13:25] and you can see that I just said, "Hey, [13:27] give me your best three sentence [13:28] description of what's going on here and [13:30] what I need to know about this in order [13:31] to make money, make or save money." So, [13:33] I'm not going to read both of these [13:34] answers, but this is what 4.6 extended [13:37] thinking gave me. And this is what 4.7 [13:40] with no adaptive thinking gave me here. [13:42] And so, you can pause that and read if [13:44] you want, but here's the quick breakdown [13:45] that I kind of came up with. the tone. I [13:48] liked the way that 4.7 spoke a little [13:50] bit better. For the actual framing, I [13:52] thought that 4.7 explained why it kind [13:55] of justified things a little bit more [13:56] and it felt more like it had financial [13:58] subject matter expertise, which is one [14:00] of the, you know, benchmarks that we saw [14:02] from Enthropic itself. We had more [14:05] actionable specifics with 4.7. The [14:07] structure over here was a little bit [14:08] better and they both obviously had some [14:10] weaknesses, but that's when I said, [14:11] "Hey, give me a three sentence [14:12] description." Um, I just wanted to keep [14:14] things a little bit more controlled. So, [14:16] I definitely did feel like I liked 4.7 [14:18] here better. And then I did another one [14:20] where I gave them this exact same prompt [14:22] about, you know, I'm building a SAS. I [14:24] need some help with, you know, figuring [14:25] out my 12-month model. And here's what [14:28] 4.7 extended gave me. It gave me an [14:30] interactive dashboard where I can see my [14:32] stats. I can click on different [14:33] scenarios. And then I can also drag the [14:35] slider along here with our starting [14:37] trial pool, with our new trials a month, [14:39] with our, you know, paid and with our [14:41] churn, and with our pricing tiers. So, I [14:43] thought this was pretty cool. It's very [14:44] interactive, but um yeah, it gave me [14:47] some insights as well. So, that was [14:49] pretty impressive, right? This took a [14:50] lot longer than the 4.7 and I think it's [14:52] just because it was using extended, but [14:54] with 4.7 here, same exact prompt, and it [14:57] came back, it did a lot of thinking. It [14:58] made some corrections on the math, but [14:59] it was able to fix those, made the right [15:01] formulas, and then what it did is it [15:03] gave me more of a deliverable. It gave [15:05] me an actual Excel sheet with multiple [15:07] different tabs. You can see here we have [15:08] more of just a base summary. You can see [15:10] then we have assumptions. we have base [15:12] case with all these different scenarios [15:14] and it's not letting me scroll because [15:16] we're in this web browser version. Um, [15:18] and then it gave me churn 3% and also if [15:21] we killed the $29 per month tier. The [15:24] quick breakdown I had of this um is that [15:27] I liked the actual deliverable better [15:28] from 4.6 because it was more interactive [15:31] and it gave me a bit more of a a [15:33] polished feel. Um, 4.7 had some errors, [15:35] but it was able to catch them. But I [15:37] think from like a business perspective, [15:39] you probably want more of that Excel [15:41] deliverable. And of course, if I [15:43] prompted 4.6 to do that, it could have [15:44] done it and it may have been better, but [15:46] once again, I was just trying to see [15:47] what would happen if I gave these two [15:49] different models kind of a vague prompt [15:51] and just kind of seeing and comparing [15:52] the way that they come back. So anyways, [15:55] um the real test will be of course when [15:56] you hop into something like Cloud Code [15:57] and you really start using it for the [15:59] actual workflows and the the the [16:01] processes and projects that you are [16:02] using Cloud Code for either way. And [16:04] ultimately, it's not something where I [16:06] could just use it for an hour and give [16:07] you guys some sort of conclusion. So, [16:08] I'm going to be playing with this [16:09] obviously all day, all week. And if I [16:12] have any other crazy insights, I will be [16:14] sure to update you all. But I do think [16:16] that the degradation was real of 4.6 and [16:20] whether that was intentional throttling [16:21] or if it was just kind of on their side [16:23] trying to optimize cost because 200 [16:25] bucks a month is honestly a steal if [16:27] you're able to use your sessions all the [16:29] way through. But ultimately, whatever [16:31] the motivation was, it resulted in a [16:33] worse product at the same price. And [16:35] then, of course, we have 4.7, which [16:37] looks a lot better on paper. And we need [16:39] to actually see if it is. We have to [16:42] take the benchmarks with a grain of salt [16:43] and try things out for ourselves. But [16:45] hopefully this video today gave you some [16:47] insight as far as like what's been [16:48] happening, where we're at, and a good [16:50] mindset as far as like actually trying [16:52] this stuff out for yourself. So, that is [16:54] pretty much my bottom line. 4.7 might be [16:56] the best AI model ever released up to [16:58] this point. It might also be the cure to [17:00] their own disease that Anthropic caused [17:02] themsel. So either way, you just got to [17:04] test it out. So that's going to do it [17:05] for this one. I hope you guys enjoyed. [17:06] If you learned something new or you did [17:08] enjoy, please give it a like. Helps me [17:10] out a ton. And I'll see you on the next [17:11] one. Thanks everyone.  📄 **Full text:** ![[Full Document Contents/Articles/Claude Opus 4.7 Just Dropped... Or Did It Really.md|Claude Opus 4.7 Just Dropped... Or Did It Really? by YouTube]] ## Highlights - um, max effort or something. But they basically just switch this down without telling anybody. And no one was kind of looking in there. And if you're on the terminal or wherever you are, you might not even notice that. So people were like, "Okay, Opus is just stupid." And then a full month later, April 15th, pro and max users were still by default just running on medium effort. So like I said, they didn't change the model itself of 4.6. They just changed how hard the model was allowed to think and they didn't tell anyone. And a lot of that thinking and reasoning is where we get all the value of this, you know, awesome Opus 4.6 model. And then there ([View Highlight](https://read.readwise.io/read/01kpm4qnayqygvj22hzd0eprxf)) - Tags: 👻 ai highlighted - web right here. We've got Claude in the chat and I can see Opus 4.7 right here. Now what you'll notice is for Opus 4.7 we have adaptive thinking. So we can do things only when needed like we just talked about. But if we go back to Opus 4.6, we don't have adapted, we have extended. So we can just turn extended on or off. I already showed you guys the desktop app. Here is my VS Code terminal. You can see we have welcome to Opus 4.7 X high. So we'll talk about that in a sec. But if you're not seeing this, maybe just try to update your Claude. So you can go ahead and just run like a claude update and that's how it ([View Highlight](https://read.readwise.io/read/01kpm4qnbhr9882m5p5nf678vs)) - Tags: 👻 ai highlighted - Opus 4.7 uses an updated tokenizer which improves how the model processes text. But what that means is the same input might actually cost you more just like 1 to 1.3 times more depending on the type of content. Also, Opus 4.7 thinks more at higher effort levels, which once again could use more tokens. I'm about to drop a full video on how to manage your tokens even better. So, that'll be out in like a day or so, and definitely want to check that one out. I'll tag that one up here when that comes out. So, there's a bunch of benchmarks, right? And that's the tough thing is you see all of this noise, you see all these ([View Highlight](https://read.readwise.io/read/01kpm4qncdskab1wg0gt024kc3)) - Tags: 👻 ai highlighted