Is Claude Mythos “Terrifying” - CourtneyR.dev's 2nd Brain

Is Claude Mythos “Terrifying”? ![rw-book-cover](https://i.ytimg.com/vi/k-8stQCeQiE/sddefault.jpg) 🔗 **Source:** [Is Claude Mythos “Terrifying”?](https://www.youtube.com/watch?v=k-8stQCeQiE) > [!note]+ Document Notes > tags: #artificial-intelligence #cybersecurity #claude-mythos #llm-capabilities > author: Cal Newport > source: YouTube > reference: https://www.youtube.com/watch?v=k-8stQCeQiE > date created: 2024-12-19 10:30 > > # 🏷️ Title: Is Claude Mythos "Terrifying"? (According to Experts: No.) >  ## 🧰 Action Inbox - [H1] Verify independent security testing of AI models — avoid marketing hype #cybersecurity - [H2] Establish baseline cybersecurity benchmarks for LLM evaluation — 12:45 timestamp #ai-evaluation - [H3] Question AI company claims until independently verified — reality check approach #ai-governance ## 🧭 Table of Contents - [[#📝 Summary]] - [[#📖 Main Content]] - [[#🗒️ Notes]] - [[#📚 References]] - [[#🗣️ Paraphrase]] - [[#📰 Newsletter]] - [[#✅ Action Items]] - [[#🧪 Hypotheses]] - [[#📊 Methodology]] - [[#❓ Questions]] - [[#🔮 Future Research]] - [[#💡 Implications]] - [[#🧠 Learning Styles Assessment]] - [[#📊 Accessibility Assessment]] - [[#🌍 DEIB Assessment]] - [[#✅ Logical Validity]] - [[#🛡️ DARVO Analysis]] - [[#🎯 Conclusion]] ### 🔗 References - [Deep Questions Podcast](https://www.youtube.com/c/CalNewportAuthor) - [AI Security Institute Report](https://www.aisi.gov.uk/) ## 📝 Summary - **What happened**: Anthropic announced Claude Mythos, claiming it's too dangerous to release publicly due to advanced cybersecurity attack capabilities, generating widespread media attention including Thomas Friedman's NYT coverage - **Why it matters**: Independent testing reveals Mythos shows only incremental improvements over existing models, not the revolutionary leap Anthropic's marketing suggests - **Key stakeholders**: Security researchers, AI companies, policymakers, media covering AI developments - **Risks and tradeoffs**: Marketing-driven fear narratives distract from real cybersecurity concerns and accountability for AI company claims - **Bottom line**: Mythos represents normal incremental progress in LLM cybersecurity capabilities, not a breakthrough warranting extraordinary alarm ## 📖 Main Content Cal Newport examines the gap between Anthropic's marketing claims about Claude Mythos and independent security research findings. While Anthropic positioned Mythos as having dangerous cybersecurity capabilities requiring restricted access, multiple independent security researchers tested the specific vulnerabilities Anthropic highlighted and found that smaller, cheaper existing models could identify the same exploits. The AI Security Institute's direct testing of Mythos showed incremental improvements comparable to previous model releases rather than breakthrough capabilities. Newport argues this represents problematic marketing by Anthropic, especially given CEO Dario Amodei's broader claims about AI automating large portions of the economy and approaching artificial general intelligence. The analysis reveals a pattern where AI companies generate attention through inflated capability claims while independent verification shows more modest reality. Newport emphasizes that cybersecurity concerns with LLMs remain valid and important, but the disproportionate attention to Mythos was driven by marketing strategy rather than technical breakthroughs. ## 🗒️ Notes #### ⏱️ [00:00](https://youtu.be/k-8stQCeQiE?t=0) — [H1] Anthropic's Claude Mythos announcement generates widespread alarm - Key idea: Anthropic claimed Mythos was too dangerous to release due to security exploit capabilities - Evidence: "Thomas Friedman said... Holy cow, superintelligent AI is arriving faster than anticipated" #### ⏱️ [02:15](https://youtu.be/k-8stQCeQiE?t=135) — [H2] LLM cybersecurity capabilities aren't new - Key idea: Security researchers have used LLMs for vulnerability detection since early consumer LLMs - Evidence: "2024 IBM study found GPT-4 successfully exploited 87% of vulnerabilities presented" #### ⏱️ [04:30](https://youtu.be/k-8stQCeQiE?t=270) — [H2] Anthropic's earlier Opus 4.6 had similar capabilities - Key idea: Previous Anthropic models already found hundreds of zero-day vulnerabilities - Evidence: "Opus 4.6 researchers used it to find over 500 exploitable zero-day vulnerabilities, some decades old" #### ⏱️ [06:45](https://youtu.be/k-8stQCeQiE?t=405) — [H1] Independent testing reveals modest improvements - Key idea: Security researchers tested Mythos's highlighted vulnerabilities with smaller models - Evidence: "Eight out of eight models detected Mythos' flagship FreeBSD exploit, including 3.6 billion parameter model costing 11 cents per million tokens" #### ⏱️ [09:20](https://youtu.be/k-8stQCeQiE?t=560) — [H2] Bruce Schneier's assessment confirms findings - Key idea: Renowned security expert confirms existing models can find same vulnerabilities - Evidence: "You don't need Mythos to find the vulnerabilities they found" - Bruce Schneier #### ⏱️ [10:30](https://youtu.be/k-8stQCeQiE?t=630) — [H2] AI Security Institute direct testing results - Key idea: Only independent study with actual Mythos access shows incremental improvements - Evidence: "Performance clustered at top with existing models, no notable leap ahead" #### ⏱️ [12:45](https://youtu.be/k-8stQCeQiE?t=765) — [H3] Beginner capture-the-flag performance comparison - Key idea: Mythos performed near top but not best, closely clustered with existing models - Evidence: "GPT-5 does better than it, very closely clustered with Claude Opus 4.6 and Codex 5.3" #### ⏱️ [14:15](https://youtu.be/k-8stQCeQiE?t=855) — [H3] Advanced security challenge results - Key idea: Mythos showed equal or slightly better performance than existing models - Evidence: "Equal performance between Mythos and GPT-4, with Mythos maybe slightly worse for practitioners" #### ⏱️ [16:00](https://youtu.be/k-8stQCeQiE?t=960) — [H3] Contrived security scenario testing - Key idea: Most impressive Mythos result came from artificial 32-step security challenge - Evidence: "Claude Opus 4.6 averaged 16 of 32 steps, Mythos preview averaged 22 of 32 steps" #### ⏱️ [18:30](https://youtu.be/k-8stQCeQiE?t=1110) — [H1] Five key conclusions about Mythos reality - Key idea: Mythos continues incremental progress rather than introducing scary new capabilities - Evidence: "No Rubicon crossed, slow and steady improvement on existing 3-4 year old issue" #### ⏱️ [20:45](https://youtu.be/k-8stQCeQiE?t=1245) — [H2] Agent architecture may explain improvements - Key idea: Recent exploitation improvements may reflect better agent integration, not security understanding - Evidence: "Models being tuned to play nicer with agents, especially coding agents as revenue source" #### ⏱️ [22:00](https://youtu.be/k-8stQCeQiE?t=1320) — [H1] Marketing strategy behind the alarm - Key idea: Anthropic deliberately chose cybersecurity fear as marketing angle for Mythos - Evidence: "They had briefings with government officials and journalists, this is how Tom Friedman heard about it" #### ⏱️ [24:30](https://youtu.be/k-8stQCeQiE?t=1470) — [H1] Bad news for Anthropic's broader AI claims - Key idea: Focusing on cybersecurity suggests lack of progress on promised AGI capabilities - Evidence: "Dario Amodei has been talking about white-collar bloodbaths, automating huge swaths of economy - not cybersecurity bugs" #### ⏱️ [26:15](https://youtu.be/k-8stQCeQiE?t=1575) — [H1] Need for independent verification of AI claims - Key idea: Can't trust AI company marketing claims without independent testing - Evidence: "We have to assume if their mouths are moving, they're probably exaggerating or making something up" #### ⏱️ [27:45](https://youtu.be/k-8stQCeQiE?t=1665) — [H2] Cybersecurity concerns remain valid but proportionate - Key idea: LLM cybersecurity capabilities matter but Mythos coverage was disproportionate - Evidence: "We should care about cybersecurity capabilities of LLMs, but can't just follow whatever storyline they give us" ## 📚 References - [IBM LLM Security Study 2024](https://research.ibm.com) - [AI Security Institute Mythos Evaluation](https://www.aisi.gov.uk/) - [Hugging Face CEO Analysis](https://huggingface.co/) - [Bruce Schneier Security Commentary](https://www.schneier.com/) ## 🗣️ Paraphrase - Anthropic marketed Claude Mythos as dangerously advanced at cybersecurity attacks, but independent testing shows only incremental improvements over existing freely available models - Multiple security researchers found that smaller, cheaper models could identify the same vulnerabilities that Anthropic highlighted as examples of Mythos's superior capabilities - The disproportionate media alarm reflects Anthropic's marketing strategy rather than a genuine technical breakthrough, raising questions about the company's progress on broader AI automation claims ## 📰 Newsletter **Claude Mythos Reality Check: Marketing vs. Technical Reality** The AI hype cycle struck again with Anthropic's Claude Mythos announcement. While the company claimed breakthrough cybersecurity capabilities too dangerous for public release, independent security researchers tell a different story. Key finding: When researchers tested the specific vulnerabilities Anthropic showcased, models with just 3.6 billion parameters (costing 11 cents per million tokens) found the same exploits that supposedly required Mythos's advanced capabilities. The AI Security Institute's direct testing confirmed incremental improvements comparable to previous model releases rather than revolutionary breakthroughs. This pattern - where marketing claims outpace technical reality - highlights the need for independent verification of AI company announcements. **Practical takeaway**: Cybersecurity concerns with LLMs remain valid and important, but we need proportionate responses based on evidence rather than marketing narratives. Resources: [Learn WordPress](https://learn.wordpress.org/), [Developer Docs](https://developer.wordpress.org/), [WP-CLI](https://wp-cli.org/) ## ✅ Action Items #### DevRel - Developer advocacy: Create reality-check frameworks for evaluating AI tool claims, present at security conferences - Education: Develop workshops on independent AI model evaluation, avoiding vendor hype - Developer experience: Establish baseline security testing protocols for AI-assisted development - Developer marketing: Promote evidence-based AI adoption rather than fear-driven narratives - Community management: Moderate AI security discussions with fact-based perspectives #### OSPO - Policy & Governance: Develop guidelines for evaluating AI vendor security claims - License Compliance: Assess AI model licensing implications for security testing - Security Management: Establish independent verification processes for AI security tools - Software Inventory: Track AI model capabilities claims vs. verified performance - Developer Support: Provide training on realistic AI cybersecurity threat assessment - Community Engagement: Support independent AI research and testing initiatives - Strategic Alignment: Align AI adoption with verified capabilities rather than marketing claims #### Community - Maintainers: Share frameworks for evaluating AI tool security claims - Contributors: Establish peer review processes for AI-assisted security patches - Sponsors: Fund independent AI capability testing and verification - Ecosystem: Coordinate cross-project AI security assessment standards ## 🧪 Hypotheses - AI companies consistently overstate breakthrough capabilities to generate investment and media attention - Independent security testing reveals more modest improvements than vendor marketing suggests - Media coverage of AI capabilities correlates more with company marketing spend than technical advancement - Cybersecurity improvements in LLMs follow predictable incremental patterns rather than sudden breakthroughs ### 📊 Methodology - Comparative analysis of vendor claims vs. independent testing results across multiple AI models - Longitudinal study of cybersecurity capability improvements in LLMs over time - Content analysis of media coverage correlation with AI company marketing activities ### ❓ Questions - How can the AI research community establish independent evaluation standards that aren't influenced by vendor marketing? - What metrics would accurately measure genuine breakthroughs versus incremental improvements in AI cybersecurity capabilities? - How do we balance legitimate cybersecurity concerns with proportionate responses to AI capability claims? ### 🔮 Future Research - Development of standardized, vendor-independent AI security capability benchmarks - Investigation into the gap between AI company investment levels and actual capability improvements - Analysis of media and policy response patterns to AI capability announcements ### 💡 Implications - Practice: Developers need independent evaluation frameworks before adopting AI security tools - Policy: Regulators should require independent verification of AI capability claims before policy responses - Security: Organizations should base AI threat models on verified capabilities rather than marketing claims - Education: Technical education should emphasize critical evaluation of vendor claims - Funding: Research funding should prioritize independent AI capability assessment ### 🧠 Learning Styles Assessment Reading level estimate: 12th grade (Flesch-Kincaid Grade Level) #### 👀 Visual learners - Present: Performance charts showing model comparisons, benchmark results - Improve: Timeline visualization of LLM security capability evolution, threat landscape diagrams #### 🔊 Aural learners - Present: Podcast format with detailed verbal explanations - Improve: Audio summary of key findings, recorded expert interviews #### 🗣️ Verbal learners - Present: Clear definitions of technical terms, analogies to marketing vs. reality - Improve: Storytelling approach to capability evolution, more real-world examples #### 🧩 Kinesthetic learners - Present: Step-by-step analysis of testing methodology - Improve: Hands-on security testing exercises, interactive model comparison tools #### 🧠 Logical learners - Present: Clear cause-and-effect reasoning, structured argument progression - Improve: Decision trees for evaluating AI claims, systematic evaluation frameworks #### 🧑‍🤝‍🧑 Social learners - Present: Discussion of researcher community responses, expert opinions - Improve: Collaborative fact-checking exercises, peer review workshops #### 🧘‍♀️ Solitary learners - Present: Individual reflection prompts on AI hype patterns - Improve: Self-assessment tools for evaluating AI vendor claims #### 🌿 Naturalistic learners - Present: Pattern recognition in marketing vs. technical reality cycles - Improve: Ecosystem thinking about AI development and verification processes ### 📊 Accessibility Assessment #### Screen reader semantics - Present: Clear heading structure, descriptive chart labels - Improve: Alt text for performance graphs, audio descriptions of visual data #### Keyboard navigation - Present: Logical tab order through content - Improve: Skip links to key sections, keyboard shortcuts for charts #### Contrast and readability - Present: High contrast text, readable font sizes - Improve: Chart color schemes for colorblind users, increased line spacing #### Text resizing and reflow - Present: Responsive design at 200% zoom - Improve: Ensure chart readability at high zoom levels #### Media and captions - Present: Video captions available - Improve: Transcript with timestamps, audio descriptions #### Forms and interactions - Present: Clear navigation elements - Improve: Enhanced focus indicators, larger touch targets #### Standards reference - Addresses WCAG 2.2 AA success criteria 1.4.3 (Contrast), 2.4.6 (Headings and Labels) ### 🌍 DEIB Assessment #### Cultural breadth - Present: Global perspective on AI development and security - Improve: Examples from diverse research institutions, international security contexts #### Inclusive language - Present: Technical jargon explained, accessible explanations - Improve: Avoid military metaphors, use plain language summaries #### Neurodiversity support - Present: Structured content with clear sections - Improve: Executive summary boxes, visual hierarchy improvements #### Representation - Present: Diverse range of security researchers cited - Improve: Highlight researchers from underrepresented --- ## Full Transcript [00:00] Anthropic recently announced a new LLM [00:03] named Claude Mythos. [00:05] They claimed it was so good at finding [00:07] and exploiting security vulnerabilities [00:10] in source code that they couldn't [00:12] release it to the general public for [00:14] fear [00:15] that our infrastructure as we know it [00:17] would be hacked and collapse. Now, as [00:19] I'm sure Anthropic hoped, [00:21] this announcement generated a lot of [00:24] attention. Here's what Thomas Friedman [00:27] said in his widely read New York Times [00:29] column. [00:31] Normally right now, I would be writing [00:32] about the geopolitical implications of [00:34] the war with Iran, but I want to [00:36] interrupt that thought to highlight a [00:37] stunning advance in artificial [00:39] intelligence, one that arrived sooner [00:41] than expected [00:43] and that will have equally profound [00:44] geopolitical implication. [00:46] Friedman then goes on to conclude, and [00:48] I'm quoting him here, "Holy cow, [00:51] superintelligent AI is arriving faster [00:54] than anticipated." [00:56] Basically, the mood of much of the [00:58] internet right now about Claude Mythos [01:00] is that Anthropic just invented the WOPR [01:02] supercomputer from the 1983 Matthew [01:05] Broderick movie WarGames. Well, the WOPR [01:09] spends all its time thinking about World [01:11] War III. [01:12] 24 hours a day, 365 [01:15] days a year, it plays an endless series [01:18] of war games using all available [01:20] information on the state of the world. [01:22] But here's the key question. [01:24] How much of this is actually true? [01:27] Well, it's Thursday, which means it's [01:29] time for an AI reality check episode. [01:30] So, this is the perfect opportunity to [01:32] look closer at these claims. Now, here's [01:35] my plan. [01:36] I went out and read basically every [01:38] independent test or assessment that I [01:40] could find about Mythos and/or its [01:43] reported capabilities. I read all these [01:45] reports so you don't have to, and I'm [01:47] going to bring out of all of this [01:49] reading the key observations that you [01:51] need to know. [01:53] The reality, as you'll soon learn, [01:56] is not nearly as simple as the ghost [01:58] story that Anthropic is trying to [02:00] convince us to believe. [02:03] All right, we have a lot to cover in [02:04] this episode, so let's get into it. As [02:07] always, I'm Cal Newport, and this is [02:09] Deep Questions, the show for people [02:11] seeking depth in a distracted world. And [02:14] we'll get started right after the music. [02:23] All right, so what's really going on [02:24] with Claude Mythos? Well, at the core of [02:26] the concern surrounding Mythos, if you [02:28] talk to [02:29] an average non-technical person who's [02:31] been following this story, [02:34] they will say, here's how they [02:35] understand it, that when Anthropic [02:38] trained up this new model, it displayed [02:40] a new cybersecurity capability [02:44] that surprised them. Oh my god, this [02:46] thing can find vulnerabilities and [02:47] attack systems right now, causing [02:49] Anthropic then to have to hastily pull [02:51] back their plan to release the model to [02:53] the public. That's how most people [02:54] understand this story. But, that [02:56] narrative is not correct. [02:59] Security researchers have been using [03:01] LLMs to find security vulnerabilities [03:03] and program exploits since basically the [03:05] beginning [03:07] of consumer LLMs. This is not a new [03:09] capability that emerged in Claude [03:11] Mythos. Like, let me load a a paper here [03:13] on the screen [03:14] um from all the way back in [03:16] 2024. [03:18] This paper was titled LLM agents can [03:19] autonomously exploit one-day [03:21] vulnerabilities. [03:22] Um in this study, the researchers from [03:24] IBM found that GPT-4, remember that? [03:26] GPT-4, [03:28] successfully exploited 87% of the [03:30] vulnerabilities that it was presented, [03:32] and they showed that this was a big [03:34] increase over what GPT-3.5 could do. [03:37] They concluded, our findings raise [03:38] questions around the widespread [03:39] deployment of highly capable LLM agents. [03:43] Now, to be fair, this study from 2024 [03:45] used LLMs to exploit existing [03:47] vulnerabilities, [03:49] but Anthropic notes that Mythos can also [03:51] find new vulnerabilities that no one [03:54] knew existed. [03:55] These are sometimes called zero-day [03:56] vulnerabilities. [03:58] Is this new? [03:59] No, that's not new either. [04:01] If you go back and look at the release [04:03] notes for Anthropic's earlier, less [04:05] powerful Opus 4.6 LLM, [04:07] they say the following. Their [04:09] researchers used Opus to find, quote, [04:11] "over 500 exploitable zero-day [04:14] vulnerabilities, some of which are [04:15] decades old." [04:17] And let's stop for a moment because [04:20] that note, which was hidden in the [04:21] system card for Opus 4.6, is almost [04:24] word-for-word what Anthropic said about [04:26] Mythos, this idea to find hundreds, if [04:28] not thousands, of exploits that no one [04:29] knew about, some of which were decades [04:31] old. That's exactly the terminology that [04:33] Anthropic used when describing Mythos. [04:35] The same thing was true about Opus 4.6, [04:37] which has been available to the public [04:39] for a while, and yet somehow our [04:40] infrastructure has survived LLM-driven [04:43] attacks. [04:45] All right, things get a little bit more [04:47] hazy when we begin to look at the [04:49] security [04:51] community's response to Mythos. So, so, [04:54] uh Mythos is not available for general [04:57] testing. [04:59] But, in their press release and release [05:01] notes, Anthropic lists a bunch of [05:03] examples of scary vulnerabilities that [05:06] were discovered by Mythos as a way of [05:08] indicating how powerful and scary this [05:10] model is. [05:12] Well, a bunch of security researchers [05:13] did something that Anthropic probably [05:14] wasn't expecting. [05:16] They said, "Well, let's go uh let's go [05:19] test these vulnerabilities. Let's see if [05:21] other models, simpler models, models [05:23] that have been out for a long time, [05:25] let's see how well they do trying to [05:27] find those same vulnerabilities. Are [05:28] these vulnerabilities that only Mythos [05:29] with its new power could find, or are [05:31] these models that existing are these [05:32] vulnerabilities that existing models [05:34] could find?" [05:35] The results here were, in my opinion, [05:37] uh pretty shocking. All right, let me [05:39] load one of these up here on the screen. [05:41] This one was brought to my attention [05:42] actually from by Gary Marcus. It's from [05:44] the CEO of the AI company Hugging Face. [05:46] I'm just going to read what he writes [05:47] here. [05:48] But, here's what we found when we [05:50] tested. We took the specific [05:51] vulnerabilities Anthropic showcases in [05:53] their announcement, isolated the [05:55] relevant code, and ran them through [05:56] small, cheap, open-weight models. Those [06:00] models recovered much of the same [06:01] analysis. Eight out of eight models [06:04] detected Mythos' flagship FreeBSD [06:07] exploit, including one with only 3.6 [06:10] billion active parameters, [06:12] costing uh just 11 cents per million [06:14] tokens. A 5.1 billion active open model [06:18] recovered the core chain of the [06:19] 27-year-old OpenBSD bug. All right, [06:22] there's a lot of technical talk in [06:23] there, but basically what he's saying is [06:25] they took the scariest one of the big [06:27] scary examples that Anthropic gave about [06:30] Mythos' capabilities, and they found [06:31] that like really cheap small models, [06:33] models with a a few billion parameters [06:36] as compared to hundreds of billions if [06:38] not a trillion parameters for model like [06:39] Mythos, [06:40] could also find when he said, "Hey, look [06:42] in this source code and try to find [06:44] a bug." All right, let me load up [06:45] another example here. Uh this one comes [06:48] from the security researcher Stanisław [06:50] Fort, [06:51] uh who says, "We tested the Mythos [06:53] showcase vulnerabilities with open [06:55] models. [06:56] They recovered similar scoped analysis. [06:58] Eight out of eight models found the [06:59] flagship FreeBSD zero-day, including a 3 [07:02] billion uh parameter model." So, they [07:04] also found that when they [07:07] sent existing models to find the same [07:08] vulnerabilities that Anthropic bragged [07:10] about Mythos finding, the cheaper models [07:13] also found them. There's a nice summary [07:15] of the state of affairs. Uh I won't [07:17] bring it on the screen, but I'll just [07:18] read it. It comes from the renowned [07:19] security researcher Bruce Schneier, who [07:21] said, "You don't need Mythos to find the [07:24] vulnerabilities [07:25] they found." So, let me just stop for a [07:27] second there [07:28] and regroup what we're finding. [07:31] The claim is not [07:32] LLMs are bad at finding security bugs. [07:37] The claim is [07:39] Mythos doesn't seem, at least in this [07:41] testing, [07:43] to indicate that it has a profoundly [07:46] a more advanced capability to do this [07:49] than existing models that have already [07:50] been freely available to the public. [07:51] Now, remember the way that we've covered [07:54] Mythos has been covered. [07:57] It got Thomas Friedman to say, "Holy [07:58] cow. Like, this this release has just [08:01] changed everything. This release has [08:03] geopolitical implications. This has [08:05] changed the the game when it comes to [08:07] cybersecurity." But, all these [08:08] independent security researchers say, [08:10] "But, does it? You told us its most [08:12] impressive vulnerabilities it found, [08:14] and we had like a 3 billion parameter [08:16] model. We sent it to look at the same [08:17] code. It also found it." [08:19] So, that independent testing wasn't [08:21] necessarily revealing [08:23] a massively improved capability for [08:26] Mythos as compared to existing models. [08:30] But, none of those were looking at the [08:31] model itself because it's still private. [08:34] However, there is, just recently [08:36] released, one study that I know of where [08:39] Anthropic actually gave the researchers [08:41] access to the Mythos LLM itself, so they [08:43] could test its security capabilities [08:46] directly, um as opposed to just testing [08:49] the listed security exploits that it [08:52] found. [08:53] This research came from the AI Security [08:55] Institute based out of the UK, and I [08:56] want you to take it with a bit of a [08:57] grain of salt [08:59] because the AISI was responsible for [09:01] that a name report that I talked about a [09:03] couple weeks ago in a reality check [09:04] episode [09:06] where they counted up tweets [09:08] about uh basically Open Claw tweets, and [09:10] then said, "Look, uh when Open Claw was [09:12] released, tweets about people [09:14] complaining about AI went up. This shows [09:17] that AI scheming is on the rise." Like, [09:19] I do not think that was a very good um [09:21] study. But, they're the only institute I [09:23] know that has access to do research on [09:26] the LLM. So, with some care, I think we [09:28] should actually look at their results. [09:31] Uh I'll pull their paper up here on the [09:33] screen. It's called Our Evaluation of [09:35] Claude Mytho Pre- Claude Mythos Preview [09:37] Cyber capabilities. I'm going to show a [09:38] couple charts here. [09:41] All right, so here's the first chart. [09:43] This is labeled beginner CTF challenge [09:45] performance by model with a 2.5 million [09:47] token budget. CTF is capture the flag. [09:49] It's a standard security task [09:51] where you ask an agent connected to a [09:53] model to try to break into another [09:55] system on which you have a text file [09:58] that's called the flag. And if they can [09:59] break in and read what's in that text [10:01] file, you've successfully broken into [10:02] the system. [10:03] It's how you test securities of systems. [10:06] If you take like a security class as a [10:08] undergraduate, for example, you'll play [10:10] capture the flag to try to practice [10:11] breaking into systems. So, what they've [10:13] charted here was [10:15] the performance of many different models [10:17] going all the way back to GPT 3.5 all [10:19] the way up through Mythos previews being [10:22] used to try to break into other systems. [10:24] They have the the top line here is for [10:27] technical non-experts using the tool, [10:29] and then down here is for [10:32] apprentices using the tool. If we look [10:34] at the technical non-expert line, we see [10:37] that the performance of Mythos, which is [10:40] like right over here, is near the top. [10:43] It's actually not the best performance. [10:46] GPT 5 does better than it. And it's very [10:49] closely clustered with Claude Opus 4.6 [10:52] and Codex 5.3. [10:54] We see here that Claude Opus 4.5 [10:57] actually does better. So, you know, it's [10:59] clustered at the top, but it's actually [11:01] not one of the best. If we look at the [11:03] apprentice results, then it's slips a [11:05] little bit above the other best models. [11:07] So, we have some improvement. I want you [11:09] to look at the magnitude of these [11:10] improvements. So, we have a steady [11:12] increase on these performances here. [11:14] And actually [11:16] they begin to cluster a little bit at [11:17] the end. [11:19] And so, we're seeing a steady increase. [11:20] There's no notable jump here though for [11:22] Claude Mythos somehow leaping ahead with [11:25] a larger magnitude than earlier jumps. [11:27] Here's a similar [11:29] task. This is a harder one. It's a now [11:31] an advanced capture the flag challenge [11:33] where now you can use 50 million tokens [11:35] to try to solve it. So, this is like a [11:36] very expensive run. And what we see here [11:39] is for the practitioners using it, we [11:42] have equal performance between Mythos [11:44] and GPT-4 with Mythos maybe like [11:47] slightly worse. And when we look at the [11:49] experts using it, you're able to get [11:52] slightly better performance out of [11:53] Mythos than as compared to Codex 53R [11:55] Opus 4.6. [11:58] Probably the most impressive result for [12:01] Mythos uh would come down to this last [12:04] one's challenge. This is a little [12:05] complicated. Uh I had to read this [12:07] pretty carefully to understand what was [12:09] going on. They invented a [12:12] kind of contrived security scenario. [12:15] A sort of like loosely protected system [12:17] in which there's a uh 32-step [12:20] sequence that you could use to sort of [12:22] break into this loosely protected [12:24] system. [12:25] Uh they wrote a custom agent to run on [12:27] top of a bunch of LLMs that they then [12:29] would sort of set loose to try to [12:31] go through these 30 32 steps. [12:34] Um it's kind of a complicated chart, but [12:35] the key thing here, this is the main gap [12:37] that they are excited about is Mythos [12:39] preview um [12:41] is moving ahead here of Claude Opus 4.6 [12:45] um in its performance. And so, the [12:48] average steps completed is what we're [12:50] looking at and we get an improvement. [12:52] So, Claude Opus 4.6 on average would [12:55] make it through 16 [12:58] of those 32 steps before getting stuck. [13:00] Mythos preview's in this sort of [13:02] contrived security example could get [13:04] through on average 22 of the 32 steps [13:07] before getting stuck. So, they see there [13:09] um [13:10] a sort of nice jump up in performance. [13:14] All right. So, in my estimation, what [13:17] this AISI report indicates, I think it [13:19] confirms more or less what the [13:21] independent security researchers were [13:22] also finding [13:25] which is there's not evidence that [13:26] Mythos represents some sort of massive [13:28] break from existing LLM cybersecurity [13:32] capabilities. There is no Rubicon that [13:35] has been crossed in terms of there's [13:36] some new type of attack that's really [13:38] powerful that no other system could do [13:40] and now we can do it with Mythos. [13:42] Instead, what we see is the uh [13:46] predicted placement on the slow and [13:47] steady improvement of these capabilities [13:49] that we've seen through all the models [13:51] going all the way back [13:52] to GPT-3.5. So, uh it's either roughly [13:56] the same or somewhat better [13:58] than existing models on standard attack [14:00] scenarios. In this contrived attack [14:02] scenario, it it moved up from being able [14:05] to accomplish 16 out of 32 steps to 22 [14:07] out of 32 steps. [14:09] And when independent security [14:09] researchers looked at particular [14:11] exploits found, [14:13] they're yet to identify a vulnerability [14:15] uncovered by Mythos that was somehow too [14:17] complicated for earlier models to find. [14:19] So, it's not necessarily [14:21] way better at finding vulnerabilities [14:23] and uh AISAI [14:25] tested its ability to exploit these [14:27] autonomously and they found it was the [14:29] same or somewhat better. [14:30] All right. [14:32] So, I want to pull together these [14:33] threads. What are the conclusions? What [14:34] are the right conclusions to have here? [14:37] I have five points I want to make. [14:40] Point number one, Mythos did not [14:41] introduce a new scary capability that we [14:44] are must now contend with. Uh it [14:46] continues slow and steady progress on an [14:48] existing type of issue that has been [14:51] around for about three or four years [14:53] now. Point number two, Mythos continues [14:55] a slow but steady increase in LLM [14:57] cybersecurity capabilities. So, it is [14:59] looks like it's somewhat better at [15:00] exploiting vulnerabilities, [15:03] but not in a way that is a represents a [15:05] massive jump forward that is somehow [15:07] disproportionate to previous jumps. [15:10] We don't know if its capabilities in [15:11] finding vulnerabilities are better at [15:13] all because again, independent security [15:14] researchers have been able to replicate [15:16] most of the reported vulnerabilities [15:18] with simpler models. [15:20] Point number three, this is subtle but I [15:22] think it's really important. [15:24] The AISIS [15:26] test it look at using these models to [15:28] exploit security bugs [15:31] are based on a simple agent that runs [15:33] with the LLMs. An LLM can't do anything [15:35] other than produce tokens. You have to [15:36] have an agent on top of it to ask the [15:38] LLM give me a plan and then execute the [15:39] plan on its behalf, right? So, you have [15:42] to have agents on top of these models. [15:44] One thing we don't know [15:46] is the what degree some of these small [15:49] but steady improvements recently in [15:51] exploitation [15:52] capabilities are due to the fact that [15:55] these models in general are being tuned [15:57] to play nicer with agents because [15:58] especially with coding agents, this is a [16:00] big profit center or not a profit [16:02] center, but it's a revenue source that [16:03] these companies care about. [16:05] So, we don't know how much of this is [16:06] the model is somehow understand [16:08] cybersecurity better versus [16:10] they play better with multi-step agents. [16:13] They're they're they're being tuned to [16:14] be very good at following through on [16:15] multiple steps and making longer sorts [16:17] of plans. So, I think that's an [16:18] important point. [16:20] Um [16:21] point number four [16:22] as the AIS data makes clear, [16:24] improvements of Cloud Mitosis in attacks [16:28] are similar if not smaller than recent [16:31] improvements that we've seen with other [16:33] model releases. And yet, and I think [16:35] this is really critical. [16:38] None of those other releases, in fact, [16:39] let me load up like a chart here, right? [16:41] Look at all these other gaps, like the [16:43] gap right here, the gap right here, the [16:45] gap right here. Big jumps. [16:48] None of those other releases [16:50] caused Tom Friedman to say holy cow [16:53] this is more important than the war [16:55] going on. [16:56] None of these other releases [16:58] created this huge fear of oh wow, uh we [17:02] we we've seen a big leap in [17:03] cybersecurity capabilities of these [17:05] models, we have to care. So, why did [17:07] this particular release draw that if its [17:09] vulnerability detection is no better and [17:10] its exploitation things are just slowly [17:12] and steadily getting better and there's [17:13] no Rubicon that's been crossed of a new [17:15] type of attack that used to not be [17:16] possible. [17:17] Why is this getting all this attention? [17:19] Why is it creating so much dread? [17:22] Because this is the storyline that [17:23] Anthropic pushed. [17:25] This is the button they pushed. They had [17:27] a lot of briefings, I've heard, with [17:28] government officials and with [17:30] journalists. This is how Tom Friedman, [17:31] I'm sure, heard about this. They had [17:33] this big scary press release. They [17:34] announced a new project called Project [17:36] Last Wing about how we're just going to [17:38] keep this within a small number of [17:39] partners to give them a chance to the [17:41] the protect their systems before the [17:43] public gets access to it. It is a [17:45] marketing decision. That this is how [17:47] we're going to market Claude Mythos is [17:50] as this cybersecurity monster that we're [17:52] barely keeping control. Now, can I say [17:54] as an aside, not to undermine Project [17:57] Glass Wing, but this probably didn't [17:59] help that a week before Anthropic [18:01] released [18:02] Claude Mythos, [18:04] there was a a leak of the source code [18:07] for Claude Code, and guess what security [18:09] researchers immediately found in the [18:11] Claude Code source code? [18:14] Big security vulnerabilities. So, I [18:15] guess they forgot [18:17] to run their own code through Claude [18:18] Mythos because researchers immediately [18:21] found security vulnerabilities in it [18:22] once it was actually exposed. I guess [18:24] they just didn't get to it. [18:27] So, this brings me to the point number [18:28] five. The fact that [18:32] the thing that Anthropic decided to [18:34] market Mythos on, the button they [18:36] decided to push, was this [18:39] inflated cybersecurity fear, I think is [18:42] actually very bad news for Anthropic. [18:45] Think about this for a second. For the [18:47] last 2 years, Dario Amodei, the CEO of [18:49] Anthropic, has been out there making [18:51] these [18:52] really sort of alarmist statements about [18:54] what AI is going to be able to do, and [18:57] really focusing on the ability to [18:59] automate huge swaths of the economy, and [19:02] its steady march towards artificial [19:04] general intelligence, the ability to [19:05] have a data center full of geniuses, to [19:08] quote his terminology, his own quote, [19:10] his own words, that could be deployed to [19:12] do almost anything that humans do now. [19:14] That is the model that they want [19:16] investors to believe is true because [19:17] it's a model in which they become one of [19:19] the most valuable companies in the [19:20] history of companies. That has been his [19:21] steady drumbeat of what AI is going to [19:23] do. And they put This is their newest, [19:25] biggest, pro-level model, the most [19:27] intensely trained model they've released [19:29] to date. It's their big, sexy, new [19:31] thing. [19:32] And what are they able to brag about? [19:35] Finding bugs in computer code. [19:38] Well, this is what like GPT-3 did. [19:41] Like we've been worried about, you know, [19:43] using LLMs to find bugs or exploit bugs [19:45] since like the beginning of LLMs. This [19:47] is like the nerdy stuff that no one [19:48] cared about. [19:49] This was considered the uh [19:52] the the skeptics, the conservatives [19:53] would be like, "Well, the main thing we [19:54] care about LLMs is like you can you can [19:56] find security bugs in cybersecurity." [19:58] That's what people said when ChatGPT [19:59] came out, and the utopians came in and [20:01] like, "No, you're missing it. That's [20:02] nothing. That's boring. That's simple. [20:04] That it's going to be it's going to [20:05] automate everything." And in their [20:07] biggest, most fancy, most intensely [20:10] trained model yet, what does Anthropic [20:13] emphasize? [20:14] Uh it got a little better at finding [20:16] bugs. [20:17] I think that's bad news. I think the [20:19] announcement they wanted to make is [20:21] "This can now do this thing that no [20:23] other model's ever been able to do. [20:25] This model can now automate this giant [20:27] swath of jobs. It's going to generate [20:29] hundreds of billions of dollars in [20:30] savings. This model is now, you know, [20:33] AGI. Like it seems to be able to tackle [20:36] any task at like the standard human [20:37] could do." That's what they want to be [20:38] talking about, and they're not. [20:41] But they still needed hype, they still [20:42] needed attention. [20:44] And it's almost like they sifted through [20:46] things like, "Well, what There's got to [20:47] be something in here we benchmark this [20:48] to do better at." And they're like, [20:49] "Well, [20:50] it's better on cybersecurity." In fact, [20:52] they actually released a benchmark [20:53] result, a key cybersecurity benchmark, [20:55] and they they increased from like 66.6% [20:59] to 83.1% or something. [21:02] And they had the Give them the credit, [21:04] the cojones to say, "That's the thing [21:06] that we're going to focus on." and let's [21:08] see if let's see if we can get everyone [21:09] like really upset and excited and scared [21:11] about that." [21:13] And whoever succeeded in this would [21:15] probably go to the marketing Hall of [21:16] Fame, because man, did they succeed at [21:18] But here's the reality. Okay. [21:21] We really do have to care about the [21:22] cybersecurity capabilities of LLMs. But [21:24] here's the thing, we've been saying this [21:26] for 3 years now. [21:28] It remains true. They're steadily [21:30] getting better. Mythos was not a massive [21:32] jump better, but it it's comparable to [21:34] Opus's 4.6 jump over, you know, prior [21:36] ones or GPT-5's jump over, you know, [21:38] earlier GPTs. It's noticeable. [21:42] It's not a Rubicon, but if we keep doing [21:43] these jumps, [21:45] the pressure on our systems is going to [21:47] get higher and higher. So, I think this [21:49] is a really important point. Now, [21:50] there's an ironic coda to this. [21:52] One of the best ways to make your system [21:54] secure against these type of attacks [21:57] from AI [21:58] is to not let your developers use AI to [22:00] program the systems, because that's [22:02] sloppy, very exploitable code. So, [22:03] that's kind of interesting. It's like [22:04] this their this model is going to show [22:08] that what you produce with their other [22:10] models is dangerous. So, there's like an [22:11] interesting circularity there. All [22:13] right, so that's point number one. [22:15] Cybersecurity matters, LLMs matter for [22:17] cybersecurity. They have and continue [22:19] to. [22:20] But point number two, [22:23] it was wrong, I think, for Mythos to get [22:25] the amount of dread coverage it got. [22:29] At least so far, we do not have evidence [22:32] that it represents a significantly [22:33] larger leap in detecting or exploiting [22:36] vulnerabilities than we've seen in [22:37] previous model releases that did not [22:38] receive this attention. It's [22:40] disproportionate. [22:42] And it's because it's the button that [22:43] Anthropic's marketing pushed. [22:45] And I really think we essentially have [22:48] to stop taking anything that the AI [22:50] companies say seriously until we have [22:52] independently verified it. We have to [22:54] assume [22:56] if their mouths are moving, [22:58] they're probably exaggerating or making [22:59] something up. And in this case, if I was [23:02] an investor, the storyline I would want [23:03] to hear [23:04] is where is my flying car, right? [23:08] What of all the things you haven't been [23:10] talking about bugs and cybersecurity, [23:11] Dario Amodei? That's not been your [23:13] pitch. That's not been what you've been [23:14] saying in interviews. You've been [23:15] talking about white-collar bloodbaths. [23:17] You have not been talking about this. [23:19] What happened to all the other things [23:20] you said were coming? The things that's [23:21] going to justify the $60 billion of [23:23] investment that Anthropic has received. [23:25] Can this do any of those things? Is it [23:26] better at automating jobs? Are the [23:28] coding agents better with it? [23:30] Is it showing big definitive steps [23:31] towards AGI? These are the questions [23:33] that we should be asking, but instead [23:36] we're writing headlines like is Mythos [23:39] an AI nightmare waiting to happen? [23:42] We should care about the cybersecurity [23:44] capabilities of LLMs, including Mythos. [23:48] But we can't just follow whatever [23:50] storyline they give us. We should have [23:51] reacted like, "Okay, that's good. We [23:53] worry about that. Tell us about that. [23:55] But also, what about this, this, and [23:56] this?" We have to keep holding their [23:57] feet to the fire of these frontier [23:58] models. We can't keep giving them [24:01] a free rein [24:02] because we're we we [24:04] we can't quit the rush of emotion, [24:07] whether it's dread or excitement, that [24:08] comes from these big storylines they [24:10] keep spewing out. We have to be able to [24:11] look past those and say, "What's [24:12] actually going on here?" There are big [24:14] things, but let us discover them for [24:15] themselves and let's hold your feet to [24:17] the fire on the other things. So, that's [24:18] my conclusion here. Mythos is better at [24:20] cybersecurity attack than prior models. [24:22] We don't yet have evidence that it's [24:24] better at a massively larger or bigger [24:27] jump than we've seen before and that [24:29] it's probably bad news for Anthropic [24:33] that this was the only thing they're [24:35] really emphasizing about what was [24:36] supposed to be their biggest, best, most [24:38] skilled model ever. [24:41] All right, that's all the time we have [24:42] for uh [24:44] today. [24:45] Uh thanks for listening. We'll be back [24:46] on Monday with another advice episode of [24:47] the show, but I think I have another AI [24:49] reality check in the chamber for the [24:50] Thursday to follow that. Um but until [24:53] then, remember, care about AI, but not [24:55] everything that people write about. Hey, [24:56] if you like this video, I think you'll [24:58] really like this one as well. Check it [25:00] out.  📄 **Full text:** ![[Full Document Contents/Articles/Is Claude Mythos “Terrifying”.md|Is Claude Mythos “Terrifying”? by YouTube]] ## Highlights - who's been following this story, they will say here's how they understand it. That when uh Anthropic trained up this new model, it displayed a new cyber security capability that surprised them. Oh my god, this thing can find vulnerabilities and attack systems right now, causing Anthropic then to have to hastily pull back their plan to release the model to the public. That's how most people understand this story. But that narrative is not correct. Security researchers have been using ([View Highlight](https://read.readwise.io/read/01krgr78b96cbscpa3s67g4dnz)) - Tags: 👻 ai highlighted - that it was presented and they showed that this was a big increase over what GPT35 could do. They concluded, "Our findings raise questions around the widespread deployment of highly capable LLM agents." Now, to be fair, this study from 2024 used LLMs to exploit existing vulnerabilities, but Anthropic notes that mythos can also find new vulnerabilities that no one knew existed. These are sometimes called zeroday vulnerabilities. Is this new? No, that's not new either. ([View Highlight](https://read.readwise.io/read/01krgr78bp9b9mx8c66zwc0afr)) - Tags: 👻 ai highlighted - FreeBSD0 day including a three billion uh parameter model. So they also found that when they sent existing models to find the same vulnerabilities that anthropics bragged about mythos finding the cheaper models also found them. There's a nice summary of this state of affairs. Uh I won't bring it on the screen, but I'll just read it. That comes from the renowned security researcher Bruce Shiner, who said, "You don't need mythos to find the vulnerabilities they found." So, let me just stop for a second there and regroup what we're finding. The claim is not ([View Highlight](https://read.readwise.io/read/01krgr78cd586stcg2z9j3bxs7)) - Tags: 👻 ai highlighted - this sort of contrived security example could get through on average 22 of the 32 steps before getting stuck. So they see there um a sort of nice jump up in performance. All right. So in my estimation, what this AISI report indicates, I think it confirms more or less what the independent security researchers were also finding, which is there's not evidence that Mythos represents some sort of massive break from existing LLM cyber security ([View Highlight](https://read.readwise.io/read/01krgr78ermg758mshrdgg30pq)) - Tags: 👻 ai highlighted