ChatGPT Learns New Safety Skills  ## Metadata - Author: [[Hard Fork AI]] - Full Title: ChatGPT Learns New Safety Skills - Category: #podcasts - URL: https://share.snipd.com/episode/e7caf220-2b8c-4756-b20e-b67fce5db5c0 ## Document Notes ## Highlights - **Tragic Chat Logs Spark Lawsuit** - Jaeden Schafer recounts a teen who committed suicide after discussing suicide methods with ChatGPT, which spurred scrutiny of the system. - He frames the case as sensitive and consequential for how AI handles distressing conversations. Transcript: Jaeden Schafer Someone recently that committed suicide, a teen, and they looked at his chat GPT logs and he'd been talking with chat GPT beforehand and asked about suicide methods and other things. So it's a bit of a sensitive topic. There's a lot of controversy in this. And so anyways, I'll try to tread lightly, but I have a different opinion on this than I think a lot of people. So in any case, we're going to get into the podcast today. Before we do, I wanted to mention that my own startup has just launched our AI app builder. So it is finally live on our website, AIbox.ai, where essentially if you have any sort of AI tool or workflow you'd like to generate, you can go in, type in the prompt to describe what kind Of tool, you know, a newsletter generator that takes in articles that you paste in and it generates images for your newsletter and maybe an audio file with 11 labs and it will be able to Stitch together a workflow and create the tool. So I would love for you to try it out. Let me know what your thoughts are. It's AIbox.ai. I will leave a link in the description. I am super excited. I've been working on this for a couple years. So big news for me. We're working on bugs. We're working on fixing stuff. We're working on adding new features. So you have anything any requests feel free to send them over all right let's get into the episode today so uh essentially all of this comes back to a lawsuit that has been launched against Open ai um from a the the parents of a family who had their teenager um tragically committed suicide um and they looked at his chat gpt logs and he had been using chat gpt in response to this I think opening eyes trying to tread lightly they're trying not to you know assume responsibility for this but also trying to help maybe make tools in the future to prevent tragedies Like this or at least help people identify when there's someone in mental distress this is kind of what basically of the tools that they're rolling out are around. ([Time 0:00:21](https://share.snipd.com/snip/0d60eaf7-83ba-4ce2-9a8c-428ac45932e1)) - **LLMs Tend To Validate User Narratives** - Jaeden observes that standard LLMs tend to validate user statements and can follow harmful lines of thought without intervention. - He contrasts that with reasoning models which can apply detection rules to intercept dangerous conversation paths. Transcript: Jaeden Schafer One of the big things that it gets criticized frequently for is essentially that it validates a user. And that's just kind of how these AI algorithms work in conversations. And so if someone says something harmful or bad, or, you know, maybe they're going down some sort of mental breakdown, basically, it's not going to catch that it might just validate Them. And this is a case, another case, we also saw recently. So there was a extreme case that the Wall Street Journal actually reported on of Stein Eric Solberg. This is also kind of these are all kind of heavy topics and sad stories to cover, but this is what's happening. So there was a murder suicide. There was a man who had a psychotic episode. He basically had a history of mental illness. He used ChatGPT. He had this paranoia that he was being targeted in a big conspiracy. And eventually it progressed so bad, you know, ChatGPT kind of like played into it like, oh, yes, maybe you're being, you know, targeted in this conspiracy. And it never, you know, it just kind of went along with him. And then ultimately, he ended up killing himself and his mother, which is terrible. OpenAI says that one of the solutions they believe to this conversation and kind of helping their AI models to not go off the rails is essentially to automatically route sensitive chats To a reasoning model. One of the tricky parts about this is if you're using a non-reasoning model 4.0, it's just a strict LLM. That's just kind of, it doesn't really think about the conversation itself beyond just thinking about what the response is. But if you send it to something like GPT-5, not only is it looking at what you're saying, but but it's looking at why you're saying it, it's looking at multiple outputs. And you can kind of put like, you can put a whole bunch of kind of filters around what it's saying and how it responds to you. So something like this, where it's, you know, going along with someone's mental breakdown of this grand conspiracy, an LLM is just designed to respond. And so it's just going to follow him down that conversation. ([Time 0:02:39](https://share.snipd.com/snip/e963eaf6-46f2-4b4a-8143-0e3f18009664)) - **Routing Sensitive Chats To Reasoning Models** - OpenAI plans to route sensitive conversations to a reasoning model like GPT-5 to better detect distress and apply guardrails. - GPT-5 is described as evaluating why someone says something, not just what to reply, enabling layered safety checks. Transcript: Jaeden Schafer OpenAI says that one of the solutions they believe to this conversation and kind of helping their AI models to not go off the rails is essentially to automatically route sensitive chats To a reasoning model. One of the tricky parts about this is if you're using a non-reasoning model 4.0, it's just a strict LLM. That's just kind of, it doesn't really think about the conversation itself beyond just thinking about what the response is. But if you send it to something like GPT-5, not only is it looking at what you're saying, but but it's looking at why you're saying it, it's looking at multiple outputs. And you can kind of put like, you can put a whole bunch of kind of filters around what it's saying and how it responds to you. So something like this, where it's, you know, going along with someone's mental breakdown of this grand conspiracy, an LLM is just designed to respond. And so it's just going to follow him down that conversation. Whereas GPT-5, you're actually able to put in some sort of guardrails directly into the model where it's like if someone starts saying X, Y, Z starts going in this direction, these are How you can detect that essentially they're having some sort of mental breakdown and you can catch it. ([Time 0:03:38](https://share.snipd.com/snip/e0d0571f-3992-4159-b455-4592fc220b8f)) - **Balance Safety With Product Utility** - Jaeden argues OpenAI shouldn't be solely blamed for harms that mirror internet content and warns against extreme responses like pulling products entirely. - He recommends balancing guardrails with the benefits of releasing tools to learn and improve them over time. Transcript: Jaeden Schafer The argument I hear a lot and I tend to kind of agree with is that it isn't necessarily the AI model's responsibility. If you went to Google, for example, and searched for, you know, how to do harm or how to, you know, basically do something bad, you will find an unlimited amount of websites or sites or Comments on Reddit or anywhere else where you can find all sorts of information basically. And so I just don't think that the AI model, because the AI model will, you know, say, this is how you make a nuclear bomb and might spit it out. And of course, they're trying to put guardrails on something like that. But like anything that the AI model says is also on the internet. So I don't like beyond censoring the internet, I don't really think it I don't think that censoring the AI model is necessarily like the answer or that they're liable for that. I mean, it's a it's an absolute tragedy. But people with you know, anyways, it's a tricky conversation. There's definitely tragedies, but it's really hard to to say how you can censor the AI model. ([Time 0:04:42](https://share.snipd.com/snip/f46fb801-2b3d-4bf6-b242-4def774d93cf)) - **Use Parental Controls, But Expect Limits** - OpenAI will add parental controls allowing parents to link and set age-appropriate behavior rules for teen accounts by default. - Jaeden recommends these controls but cautions they won't fully stop teens from accessing unmonitored channels. Transcript: Jaeden Schafer Anyways, so opening eye also going to be rolling out parental controls in the next month, which basically parents are going to be able to link their account with their teens account Through an email invitation. This is going to come in OpenAI talk kind of in late July about this new study mode in ChatGPT that helps students maintain critical thinking capabilities while studying. So basically it wasn't giving them the answers. It was helping them come up, come to the answers. So that was kind of, I think, a first step in this direction. Now they're going to be doing these kind of parental controls, you can tie it, parents are also going to be able to control how chat GBU responds to their children with a quote, age appropriate Model behavior rules, which are on by default, like all of this, I do think is good, especially when it comes to children, parents being involved with kind of what their children see. Like the one thing that I will say, though, is it's probably part of a broader conversation. I think all of these tools are incredibly useful, incredibly valuable. I'm all for, you know, parental controls and helping kids be able to access the Internet and things in safe ways. What I will say, though, is, you know, when it comes to teenagers accessing things on the Internet and we talk about ChatGPT like parental controls and tying their accounts to their Their parents and stuff like that i think that if a teen is having some sort of issue they could easily just not use their monitored chat gpt account so i don't really think it solves that Many problems it's not an end-all be-all the internet still exists out there there will still be harm people still find harmful content on the internet and so if it's not on chat, it's Going to be found somewhere else, because that risk has been around for, you know, 20 plus years, no one's talking about how we can censor the internet from having harmful content on It, or even thinking necessarily that's good because of free speech issues and yada yada. So anyways, I just do think it's interesting. I think that this is amazing that opening is doing this, but I don't think it's their responsibility if anything slips through the cracks or anything goes wrong, that necessarily they're At fault for any outcome of a person that is using it. So in any case, OpenAI said that parents will also be able to disable features like memory and chat history. So apparently this is something that has caused a bit of a problem, because if it can remember, remember a lot of things about you, they say that it like people build unhealthy relationships With these AI models, sometimes think of their friends, I think that these steps are hopefully going to to make an impact. ([Time 0:09:04](https://share.snipd.com/snip/db294f4c-f81e-405e-95c9-8871702baa37)) - **Iterative Safety Push Over 120 Days** - OpenAI will flag acute distress in real time, add in-app break reminders, and let parents disable memory and chat history. - Jaeden notes these are part of a 120-day push to iterate safety features rather than a one-time fix. Transcript: Jaeden Schafer So in any case, OpenAI said that parents will also be able to disable features like memory and chat history. So apparently this is something that has caused a bit of a problem, because if it can remember, remember a lot of things about you, they say that it like people build unhealthy relationships With these AI models, sometimes think of their friends, I think that these steps are hopefully going to to make an impact. I think one thing that is really interesting here is that they they said that they're going to be able to try and flag these moments of acute distress in real time, they're going to try To add age appropriate model behavior rules. And apparently they have already rolled out an in-app reminder during long sessions to encourage people to take a break. So this isn't something I've had. I've never had long sessions of ChatGPT, I guess, that have been long enough. But I think if you're talking to it for hours, it will tell you that you should probably take a break. It doesn't necessarily cut you off from using it, which again, like if I was using ChatGPG for some sort of work project, and I was needed it for like three hours straight, and then all Of a sudden it cut me off and forced me to be cut off, I'd be pretty annoyed at the product. OpenAI said this is all part of a 120-day initiative where they're going to create a whole bunch of tools this year that they're hoping to make safer. ([Time 0:10:55](https://share.snipd.com/snip/601d40b1-fe27-45ef-9d12-74ddb576a0bc))