Anthropic's Insights on AI Risks Impacting Society  🔗 **Source:** [Anthropic's Insights on AI Risks Impacting Society](https://share.snipd.com/episode/594dd526-f4bc-47ec-92b5-51639b592ec6) ## Highlights - **Apple Builds Tamper Evidence Into Camera Photos** - Apple’s Reference Images aim to make camera photos verifiable by signing sensor data at capture and creating an unalterable version. - The system uses private cloud compute and SynthID, with an API planned for third-party developers. Transcript: Jaeden Schafer Okay, let's talk about Apple first, because I think this is an interesting tool that they've rolled out to kind of fight against fakes. Apple just announced something called reference images. They have kind of, this is tied to their iPhone 18 Pro launch. Basically what it's going to do is it is going to capture signed sensor data. So the moment that you take a photo, it's going to capture that. It's going to route it through private cloud compute. So Apple's compute, and it's going to produce an unalterable image. Is adopting the synth id standard to help users detect ai generated or ai altered images this is something that i that i think open ai and google have also been working with the iphone 18 pro is going to have an 84 megapixel fusion main camera featuring a variable aperture built from six laser cut blades each thinner than a human hair so apple you'll get we're gonna Get into all the drama today there's like anthropics researcher says it's gonna kill all of humanity open ai apparently is stealing everyone's math research and we have apple that Just said hey look we made a better camera and it's gonna be harder to deepfake so apple doing pretty good right now reference image is also going to start inside of the photos app. There's going to be an API coming for third-party developers as well. So other people are going to be able to use this as well, which is going to be interesting. ([Time 0:03:18](https://share.snipd.com/snip/f6f47d58-a3e4-462a-9f88-f3c6e76096da)) - **OpenAI Turns Doodles Into Detailed Designs** - OpenAI’s Sketch feature turns rough doodles into detailed images while preserving the user’s intended layout. - Jaeden Schafer tested a microphone character and described a garden sketch that became both a blueprint and photorealistic view. Transcript: Jaeden Schafer Been playing with it today one of the coolest features on here besides okay i'll just like the quick rundown on it is that i think they're going to start competing directly with canva They have these things called templates built into it so it's like if you generate a logo, it's like, you know, has these pop-ups where it's like, do you want to see what this looks like On a t-shirt, on a mug, on a hat? There's like all these different templates and all these different styles and all these different posters and things you can generate. And it gives you ideas. I think they're going straight after Canva with all of this. Okay. One of the cool features they rolled out, though, was called Sketch. And I was playing with it today essentially what you do is you you say just the at symbol say at sketch and once that happens there is a little draw pad that comes up for me i went and like kind Of doodled a little guy holding a microphone in a red chair with a purple background and i said you know generate image of you know this I kind of explained what was in my image and it went Ahead and generated an image, which was exactly like I had, you know, requested. It's perfect. The thing that I like about this is sometimes when you're doing a prompt, I do, I will admit, I do like taking a reference image. So I'll screenshot an image I like and say like, make an image like this, but it has that and this. And I tell it like some of the some of the things that should be changed on it this is really cool because you can just doodle it if you don't have a screenshot but it's just a idea in your mind You can do that and opening I actually had some really good recommendations and examples of like when this would become really useful one thing that they showed is like if you just doodled Like a picture of the garden in your backyard and or you know maybe you're like I'm trying to like redo my garden and so you do like a doodle where you they had one where it was like you doodled The pool and you kind of doodled everything that you wanted to to be inside of your garden and then it was able to generate first of all it generated like a top-down view of that specific Garden so it's like look this is what your garden could look like with like a pool and your umbrellas and all your stuff. And then it actually went and generated like a ultra realistic photo from like, you know, just the perspective of someone standing there. So anyways, if you had an idea and you doodled it out, it could help you with all sorts of schematics and all sorts of angles of this photo generated all sorts of ways. ([Time 0:05:34](https://share.snipd.com/snip/fd0f1292-9f8f-43d8-8f62-7d9028a5f671)) - **Anthropic Researchers Admit Alignment Remains Unsolved** - Former Anthropic researcher Jacob Coxton argued that leading labs are racing toward self-improving systems without a proven safety plan. - Anthropic alignment lead Evan Hubinger publicly agreed, estimating under a 10% chance of human extinction within a decade. Transcript: Jaeden Schafer Anthropic researcher named Jacob Coxton has quit. And he warns that labs are, quote, gambling with our lives. So Coxton spent three years on pre-training at OpenAI and Anthropic. And then he said that people building the tech privately believe it could kill everyone by 2030. So, you know, countdown. We have less than four years left to live before AI takes over. Now, I've actually seen some really interesting conspiracy theories about this particular post. The main concept for all of them is basically the same. And that's it. This is a pretty complex regulatory capture move where essentially you can, or essentially Anthropik or someone is paying to have maybe he was sincere in his like leaving but it was His post was very quickly amplified so this is basically someone that has zero followers zero you know like history posting on x before today and as soon as he posted he immediately had A whole bunch of people that lead like ai regulatory companies retweeting him and reposting within like 20 minutes. And so this kind of helped the post blow up, get a lot of notoriety. And so a lot of people are saying that it is, you know, just people amplifying it because they want to, you know, they want to regulate AI into oblivion. But in the conspiracy theory realm, one of the things that someone said about this was before he ever made the post, he went to the Wall Street Journal and had them write, I believe, like An article or an op-ed about this. And after it launched, then he tweeted. So it was like coordinated and he's obviously working with the press. So some people believe it's more sophisticated because it's kind of a, you know, quote unquote random. He said, this is his tweet, which was kind of a thread. But the first thing said, the people building AI earnestly believe that it could kill all of us by the end of the decade. This is not a marketing stunt if anything many executives and senior researchers will coach their phrasing in the press to sound sensible but i hear the same people express fear privately No other human activity poses this level of threat so oh yeah and i guess the main tweet he said was i resigned from anthropic today i spent the last three years building or the last three Years doing pre-training research at both OpenAI and Anthropik. Neither company is acting responsibly. They are racing straight to self-improving super intelligence and gambling with our lives. More thoughts below. Anyways, then he kind of goes on the list and he's like, look, like maybe Anthropik sort of doing something, but there's nothing that they're, you know, they don't really have a plan For recursive super intelligence, uh, you know, self-improving. And then everyone's just saying that Anthropic has the absolute worst PR strategy of all time, but maybe it's, maybe it's a genius. I don't know. Then Evan Hubinger, who still currently works on alignment science, he's the alignment science lead at Anthropic. He responded to Jacob's post and he said, Jacob is correct here with an M dash. I don't know if he's using AI to, to respond I mean I would guess yes but maybe some people are mad at me saying that m dashes are normal anyways he said Jacob is correct here we really do earnestly Believe AI could kill all humans exclamation point I personally think it is less than 10 within the next decade I believe Anthropic is trying its best but we do not yet have a plan to solve Alignment for super intelligences and are not clearly on track to. Like, okay, awesome. So the guy in charge of this says, yeah, I think there's some percent chance that within the next decade, AI could kill all humans. And Anthropic is doing its best, but we don't actually have a plan to solve it. So that was a pretty wild response. Someone below that said, ([Time 0:08:23](https://share.snipd.com/snip/e1488dba-eb22-4da8-a001-67b69c402d83)) - **AI Piano Music Found An Unexpected Audience** - Jaeden Schafer described earning about $600 monthly from AI-generated piano music marketed to help cats sleep. - Suno’s V6 uses licensed music after earlier models faced lawsuits and scrutiny over training data. Transcript: Jaeden Schafer But he's at 620,000 likes right now so perhaps we are all on track to die in the next couple years pretty pretty wild claim okay suno is launching their v6 model i love suno i've used it a Ton it's a lot of fun and i've released a few albums with it some of which make me you know hundreds of dollars a month from AI generated music my most famous one that I always tell people About is I have relaxing piano music to help your cat fall asleep that album makes me $600 a month and it's just AI generated it's actually like AI generated it's not slop it's it's just Relaxing piano music it sounds great it's funny how I marketed it it did really get well and a lot of people just added it to their playlist to play to their cat. So cats apparently love AI-generated music, and I'm not even apologizing because if they love it, they love it. Now, one thing that I'll say is the problem with that music and everything else, which Suno takes the full liability for, so it's not my problem, but Suno trained all of their previous Models on licensed music. They just trained it on all the music that existed in the world and all of the record labels sued them you had Warner BMG believe Sony and Universal and so they decided to train their brand New model and and by the way here's like my very contrarian take that I think also a lot of people would be mad about because this goes for Suno and basically everyone else they trained On everything they got in trouble they made licensing deals with the distributors but in the meantime their their company and brand grew a ton maybe they wouldn't have enough money Or wouldn't be able to get permission before but the brand grew a ton and now they were able to pay right because they've raised over 819 million they're able to pay off all of these fines They're able to pay for licensing and now they're able to train a model with actual licensed music now there's a lot of different takes on this i've been doing i've been using ai music Generators for like 10 plus years i think at this point yeah about 10 years and so way before chat gpt ever came out there's been a bunch of music generator models i don't know what those Early ones were using. They weren't that great, but some of them were okay. They weren't horrible. It was pretty fun back in the day. Anyways, with this new model, it's all licensed data. I think that's fantastic. This is what we've seen everyone else do. You get a model, you get it big, you get some revenue, you prove the concept, and then you go pay everybody out. Should you have paid everyone out ahead of time? Maybe, but if the model wasn't proven, it's kind of hard. They wouldn't have had the money to do it. So anyways, that another argument this release is going to happen in three tiers so they have v6 which is paid they have v6 wild which is paid and experimental and then they have v6 mini Which is free all of the older suno models are going to be retired and the reason is because of the data licensing issue suno has raised all of that money and they, you know, obviously have All these lawsuits going on. ([Time 0:12:42](https://share.snipd.com/snip/32f1df42-9158-4f4f-9eeb-2e833d931f58)) - **Watch For Silent Theft Of AI Tokens** - Monitor AI account usage closely because stolen OAuth sessions can silently consume expensive token allowances. - Anthropic invalidated affected sessions and issued partial refunds, but its support tools did not itemize usage. Transcript: Jaeden Schafer Okay. Hackers are draining Claude subscriber token allowances with stolen sessions. Anthropic confirmed that info sealer malware is hijacking Claude code OAuth tokens. One max user watched usage climb from 45% to 55% while doing no work. And so he's like, what's going on? Why are my tokens disappearing? People are hacking in and just using tokens to do certain projects. Confirmed this. And there is a separate users that reported their usage jumping from zero to 49% in 12 minutes after only a couple of prompts and a web search. Anthropic suspended the affected accounts. They invalidated all the sessions and they issued a partial refund. One user received 44 pounds for their $200 a month subscription. Account support tracks total usage, but not itemized usage, meaning token theft could run undetected for months, right? It honestly reminds me of all of the people that were doing like music body and farms where they would steal a whole bunch of people's Spotify passwords. They would just get them on the dark web because they're usually, you could get a, you know, whatever. So they'd steal them from the dark web. Then they would log into a whole bunch of random people's Spotify accounts randomly and just play their own song over and over again. And then they would log out and it was really hard to track and Spotify spent a lot of time and money and eventually kind of cracked down on the whole scheme. And I think a lot of people doing it illegally might've went to prison or some i don't know different things depending on how nefarious uh all of the the body and and kind of fraud went but This reminds me a lot of that it's the same idea you you get maybe combo lists from the dark web you steal a whole bunch of people's account tokens and passwords you log in just get a couple Things done or maybe if they have an api you log in and use a few tokens and then you log out. And then, you know, maybe they'll hardly notice. ([Time 0:16:03](https://share.snipd.com/snip/3bea3846-73b8-4d6e-bf37-1d97b4d318a2)) - **A Two-Minute Task Burned Most Weekly Tokens** - Jaeden Schafer’s highest-tier ChatGPT agent consumed roughly 80% of his weekly tokens during a two-minute website review. - ChatGPT preserved access by switching him to a simpler model instead of locking the account entirely. Transcript: Jaeden Schafer No one hacked me. Is that today I asked chat GPT Astra. So they're, you know, GPT 6 model. I put on the absolute highest usage tier, right? The highest thinking capability. And I asked it to go and review my website and give me some good ideas for some good ideas to, you know, review and make the branding and the visuals on my website better. And over the course of two minutes that it worked for, it burnt through 90% or I think actually 80% of all my tokens for the entire week. So do not touch Astra 6 on the absolute ultra max tier with a six foot pull. That's my advice to the wise if you're worried about your token usage, unless you're at the end of the week and you got to use it quick before it resets or you have one really big problem. I don't know. Like I just, I'm so blown away that in two minutes it could use 80% of my tokens. I'm super annoyed by that. I'm in my chat GPT account and they do have a very cool feature, which huge shout out to chat GPT for, uh, for adding. And by the way, Claude's got the same problem with Claude though. You basically get locked out of your account with chat GPT. They say you are now using Luma, a faster model for simpler tasks. Use your reset or buy a reset for $80 to continue using the most advanced models or wait for it to reset after and then it has my reset date. So basically, they switch you on to like one of their most, I don't know, basic models. You can get basic tasks done. But personally, I'm actually very happy about that because at least they're not completely shutting off my account. Just, yeah, switch me to one of the terrible models and it can sort of do some stuff for me if I'm in a pinch. I actually have a second Claude account that I'll probably be using for a lot of stuff in the meantime. But overall, yeah, good pros and cons to that. ([Time 0:17:56](https://share.snipd.com/snip/31540c85-2a27-484c-9049-87a4da8571a6)) - **Meta Muse Makes AI Purchasing A Security Test** - Meta’s Muse combines personal-agent capabilities with Stripe purchasing, requiring user confirmation before completing transactions. - Its $300,000 bug-bounty program explicitly encourages researchers to find prompt-injection vulnerabilities before broad adoption. Transcript: Jaeden Schafer Okay, really quickly wrapping up, we got to talk about what Meta has launched today. This is Muse. It's a personal ai agent built on a security first pitch the super intelligence lab product ships on ios and android and whatsapp it has stripe checkout and a 300 000 bug bounty ceiling So if you go find bugs they're actually paying you for it which i think is genius on their part the paid tier is for people that are paying $20 a month and they have a $100 a month tier and a Free tier. And of course, all of those is usage based. They said that you get about $130,000 for a single user prompt injections. So like they're just paying you to red team this thing. If you can use it, get it to do something bad, they'll pay you $130,000. That's a pretty awesome deal in my opinion. Checkouts because, and the reason why they're doing this is because they're adding payments into it. They partnered with Stripe. They have checkouts that run through Stripe's link. Single use card numbers, 1Password and ShopPay integrations are all underway. They're working on all that. So what all this means is that you can go to Muse and say, hey, I need to go, you know, I want to eat Taco Bell for lunch. Go and order it for me. And it has access to your Stripe account. It will go get everything ready. It'll have a pop-up that's like, do you confirm spending $14 on this burrito and drink from Taco Bell? You click yes and it gets shipped straight to your door with DoorDash. I personally love this. I mean, it's a big bet. I know it's a gamble. I know it's risky. I mean, OpenAI and Anthropic are not doing this direct Stripe integration where it can buy stuff for you. But I think number one, it's smart for Stripe because they're going to get some good volume here. They're going to prove this concept. It's almost like there's insurance because OpenAI is like offering $130,000 for prompt injection. So they're, you know, I mean, that's a massive bounty program. They'll pay you a lot of money if you can hack this thing. And so a lot of people will try to hack it. They'll solve a lot of problems and yeah, they're going to get access to be able to tell your AI model to go buy things for you and you got to approve it and give it permission. Muse is powered by Meta's in-house Muse Spark model and it's shipped by a Meta super intelligence lab, which actually came about about a year ago. So I know some people are probably freaked out about this one. I think it's kind of cool i'm excited for it and we'll see how well metas muse works i tried signing up for it today and it went on login after i got through all the signup process it like kind Of crashed on me they said they're having really wild really like a ton of traffic on it right now and i think if if i was over on x earlier today i think i saw that they had made a post that it Was number three in the app store or something like that. ([Time 0:19:40](https://share.snipd.com/snip/309eacf5-6dc5-4b35-b36d-db8ab8c531ea))