$555K+ Compensation Clash: OpenAI Safety Lead  ## Document Notes ## Highlights - **Safety Role Signals Renewed Urgency** - OpenAI is hiring a head of preparedness to prevent catastrophic AI risks as models grow more capable. - Jaeden Schafer highlights that safety was deprioritized while product features raced ahead, creating urgency for this role. Transcript: Jaeden Schafer OpenAI is looking for a new head of preparedness, someone that can help the chat GPT maker avoid a lot of the catastrophic events we worry about with AI. Now, this is an interesting story because they previously had a head of preparedness who left. Many people from their safety team have left, and a lot of people say this is due to Sam Altman not prioritizing that part of the company and basically just pushing out models without Doing all of the red teaming. This is a multi-billion dollar company, so I don't think that's 100% the full story here. Today on the show, we're going to get into what they're looking for, how much Sam Altman is personally willing to pay for this new head of preparedness, why now is the time, and everything He's been saying about this on X. We're going to get into all of that, but before we do, I want to say a big thank you to the sponsor of today's episode, which is Delve. If compliance is something that's slowing down your deals, whether that's SOC2, HIPAA, GDPR, I know that it's a ton of work with the screenshots and the spreadsheets and all of the endless Back and forth. And that is why this episode is brought to you by Delve. Delve uses AI agents to automate compliance end to end. They collect evidence, they fill out security questionnaires, and they customize controls to your actual business so you can get compliant in days and not months. You also get one-on Slack support from real security experts who respond fast. Over a thousand fast-growing companies trust Delve to close deals faster and stay compliant as they scale. I'll leave a link in the description to Delve.com if you're interested in learning more or booking a demo. All right, let's get into what's going on Sam Altman himself has personally posted on X saying that, quote, important time. Models are improving quickly and are now capable of many great things, but they are also starting to present some real challenges. The potential impact of models on mental health was something we saw a preview of in 2025. We are just now seeing models get so good at computer security, they are beginning to find critical vulnerabilities. Okay, there's a lot going on ([Time 0:00:00](https://share.snipd.com/snip/fae0d526-9efd-496a-875e-45e543987581)) - **Models Create New, Nuanced Harms** - Sam Altman stresses models now present real challenges like mental-health impacts and cybersecurity threats. - The role must measure nuanced abuse possibilities and limit downsides while preserving benefits. Transcript: Jaeden Schafer Here, and I'll read more of his tweet. He said a couple other things about this role, but essentially what they're looking for is someone that can help, you know, prevent all of the catastrophic events we've been terrified That AI could pose, you know, risk in the future. And I think this is interesting because there are a bunch of things happening happening with security, where we see OpenAI has said that they actually, like basically fine tune and Train models to try to breach security. And then they use those and to try to like harden their models. And then they, you know, put in safeguards so that people can't use open AI models to become like hackers and breach security. What's interesting to me in this is that they are actually going and training the AI to be able to do this in the first place, which I mean is kind of crazy, but maybe you need to be able to Do that to be able to control it. But one shocking thing they found while they were doing this is that AI is getting so good. It's discovering security vulnerabilities. It's discovering ways to, you know, engineer social engineer situations to be able to get data, to be able to get information, to be able to hack and to crack into things that humans weren't Able to do. In other words, they had like literal a red team of hackers that are like trying to use AI and chat GPT to like hack into things. And they and then they had these real people going up against actual just like training the AI model to be a hacker and the AI that was trained to be a hacker was doing better than the actual People. And, and when I say better, it was essentially thinking of new vulnerabilities, really elaborate, complex, multi-step ways to get data and to hack into things that people were not Coming up with. And so this is, you know, definitely part of the big concern that Sam Altman is pointing out here and that he's worried about. He said, quote, we have a strong foundation of measuring growing capabilities, but we are entering a world where we need more nuanced understanding and measurement of how those capabilities Could be abused and how we can limit those downsides both in our products and in the world and in a way that lets us all enjoy the tremendous benefits. These questions are hard and there is little precedent. A lot of ideas that sound good have some real edge cases. ([Time 0:02:00](https://share.snipd.com/snip/ca785dbe-95d1-4e8a-88dd-13cf75f3d9d3)) - **Red Team AI Outperformed Human Hackers** - Jaeden describes an internal red team that trained AI to act as hackers and outperformed human testers. - The AI discovered novel vulnerabilities and social‑engineering chains humans missed. Transcript: Jaeden Schafer Now, the thing that I do think is interesting here and like it's a great point is we obviously all love ChatGPT. We obviously all want this technology to get better. It's helping us a lot with work. It's saving us a lot of time on a number of tasks and menial things that I'm not having to do anymore that I used to do. But at the same time, there are real risks. So we have to figure a way to balance some of the risks. And if we want the AI models to get better, they're also going to get better at areas that are, you know, areas of concern. And so how do we mitigate that? And that's who they're hiring for. Sam Altman finished off his tweet by saying, If you want to help the world figure out how to enable cybersecurity defenders with cutting-edge capabilities while ensuring attackers Can't use them for harm, ideally by making all systems more secure, and similarly for how we release biological capabilities and even gain confidence in the safety of running systems That can self-improve, please consider applying. This will be a stressful job and you'll jump into the deep end pretty much immediately. So this job is not for the faint of heart. But if you go over to their careers page and you look, they give you an outline about the team and the and they say what you're going to be doing in the role. One thing that is interesting is the compensation for this particular job here is $555,000 a year. And they also have equity along with that. So, I mean, it's a, you know, it's a very generous pay package, but it is obviously very stressful and entails a lot. When they say, you know, what you'll actually be doing in the role, they said, own OpenAI's preparedness strategy end-to by building capability evaluations, establishing threat Models, and building and coordinating mitigations. Lead the development of frontier-capable evaluations, ensuring that they are precise, robust, and scale across rapid product cycles, oversee mitigation design across major risk Areas, ensuring safeguards are technically sound, effective, and aligned with underlying threat models. So one thing that I do think is interesting here with this role is, I mean, very publicly, Sam Altman is going ([Time 0:04:00](https://share.snipd.com/snip/0511de3e-60ac-4812-a31e-4af4b7fa9417)) - **Role Combines Evaluations, Mitigations, High Pay** - The head of preparedness role explicitly covers building frontier-capable evaluations and coordinating mitigations across risk areas. - Compensation is high: base listed at $555,000 plus equity, signaling seriousness and visibility. Transcript: Jaeden Schafer One thing that is interesting is the compensation for this particular job here is $555,000 a year. And they also have equity along with that. So, I mean, it's a, you know, it's a very generous pay package, but it is obviously very stressful and entails a lot. When they say, you know, what you'll actually be doing in the role, they said, own OpenAI's preparedness strategy end-to by building capability evaluations, establishing threat Models, and building and coordinating mitigations. Lead the development of frontier-capable evaluations, ensuring that they are precise, robust, and scale across rapid product cycles, oversee mitigation design across major risk Areas, ensuring safeguards are technically sound, effective, and aligned with underlying threat models. So one thing that I do think is interesting here with this role is, I mean, very publicly, Sam Altman is going out saying, look, we're going to pay over half a million dollars a year plus, You know, compensation, which I would imagine this is like a million dollar a year, maybe $750, $800, a million a year, maybe more with the compensation package with the stock options, Probably over a million, if I'm being honest. So, you know, great compensation for this role. But what does that mean? I think because this is so publicly being posted, you could say one of two things. Sam Altman is like, you know, signaling to everyone, look, I'm very serious about this. We really want to hire the best person. Also, it's going to be a very public facing person, I would imagine, just given the nature of the role. And if anything goes wrong at OpenAI, if any of these systems are used for safety, you know that like, whoever gets this role is going to be pointed at like, oh my gosh, XYZ person didn't Do their job because this is what they were supposed to do. Now, one thing that I do think is interesting here is they already had ahead of preparedness. They first announced that they were creating kind of this role in 2023. They said that it was going to be responsible for studying potential catastrophic risks, ([Time 0:05:11](https://share.snipd.com/snip/042fe566-0e9d-4066-9f99-a5d09cefbe76)) - **Safety Leadership Has Shifted Before** - Jaeden notes OpenAI previously created the role in 2023 but reassigned Alexander Madry within a year, suggesting shifting priorities. - Several safety staff left or shifted roles amid a product race against competitors. Transcript: Jaeden Schafer Whether they were immediate like phishing attacks or more speculative like nuclear threats. So that was kind of when it first came out. But less than a year later, they reassigned their head of preparedness, who was Alexander Madri, to a job focused on AI reasoning. Them less than a year later. And now they're trying to fill the role again. And it's interesting, like, I'm not sure if this was because Alexander himself requested to move to something different, or if they just needed help over there. And he had a lot of experience. I think there's, you know, definitely other safety executives at OpenAI that have left the company, or they've taken new roles outside of preparedness and safety and kind of moved It around as I think that got a little bit deprioritized last year. Google Gemini was putting out a lot of updates you know they were concerned about meta they were concerned about Grok they were concerned about like a lot of other players including Claude and it felt like they definitely kind of put this on the back burner because they didn't want to spend all their time on safety when they were it felt like they were kind of falling Behind on some of the model features and they were just focusing on getting the model out as fast as possible now they're going to bring someone in and that's going to be their role. But that person is going to take a lot of like personal, I wouldn't say the personal liability, because obviously everything at the end of the day is opening eyes, you know, responsibility, But that person is going to be very public facing. If anything goes wrong, they will 100% feel a lot of the heat on that. They also recently updated their preparedness framework saying that it might and they said that they're going to adjust it. It's you know, and some of their safety requirements could be adjusted if a competing AI lab released a high risk model without similar protections. So they're like, look, we're make our models really safe. But like if Gemini or Grok come out and their model is crushing it, but they don't have the same protections, then we'll just like dial back the safety on it. So that we can be competitive with them, which is definitely a very interesting point. I know a lot of people will find concerning. Right now, I think Sam Altman, he alluded to this in his post, but a lot of his generative AI chatbots have faced a lot of scrutiny ([Time 0:07:00](https://share.snipd.com/snip/7dbf0824-146f-4ad2-9be1-47fe1cb22c6a)) - **Competition May Erode Safety Standards** - OpenAI may relax some safety requirements if competitors release high-performing models without similar safeguards. - Jaeden warns this competitive tradeoff could undermine safety incentives. Transcript: Jaeden Schafer Of the heat on that. They also recently updated their preparedness framework saying that it might and they said that they're going to adjust it. It's you know, and some of their safety requirements could be adjusted if a competing AI lab released a high risk model without similar protections. So they're like, look, we're make our models really safe. But like if Gemini or Grok come out and their model is crushing it, but they don't have the same protections, then we'll just like dial back the safety on it. So that we can be competitive with them, which is definitely a very interesting point. I know a lot of people will find concerning. Right now, I think Sam Altman, he alluded to this in his post, but a lot of his generative AI chatbots have faced a lot of scrutiny around specifically mental health. There's a bunch of recent lawsuits that allege that ChatGPT enforces users' delusions, it increased their social isolation, and it even led some to suicide. So OpenAI is definitely at a critical moment where they have to get this right. I think it's, you know, not an easy thing. ChatGPT or OpenAI did say that they're continuing to work on improving ChatGPT's ability to recognize signs of emotional distress and to connect users to real world support. I think this is amazing. Obviously, this is one of these situations where you have to learn and figure out what can go like as you see things that go wrong. You have to fix them and try to improve them. I think that that is the exact right approach. I would like to see more and more from OpenAI. So this is going to be interesting to see how this rolls and hopefully to see that their safety platform is robust, but also doesn't slow down the innovation that a lot of us are using in Our day-to lives. Thank you so much for tuning into the podcast today. As always, make sure to go check out AIbox.ai, my own startup, to get access to all of the latest models and build incredible tools if you're not a coder. And also go check out the sponsor of today's episode, which is Delve.com. There's a link in the description to Delve, and I'll see you in the next episode. ([Time 0:08:20](https://share.snipd.com/snip/73f7ca59-4481-40ca-b563-fb30889daffa)) - **Mental-Health Harms Are Immediate Risks** - Generative chatbots face lawsuits alleging harm to mental health and facilitation of delusions. - OpenAI says it is improving detection of emotional distress and connecting users to real-world support. Transcript: Jaeden Schafer Right now, I think Sam Altman, he alluded to this in his post, but a lot of his generative AI chatbots have faced a lot of scrutiny around specifically mental health. There's a bunch of recent lawsuits that allege that ChatGPT enforces users' delusions, it increased their social isolation, and it even led some to suicide. So OpenAI is definitely at a critical moment where they have to get this right. I think it's, you know, not an easy thing. ChatGPT or OpenAI did say that they're continuing to work on improving ChatGPT's ability to recognize signs of emotional distress and to connect users to real world support. I think this is amazing. Obviously, this is one of these situations where you have to learn and figure out what can go like as you see things that go wrong. You have to fix them and try to improve them. I think that that is the exact right approach. I would like to see more and more from OpenAI. So this is going to be interesting to see how this rolls and hopefully to see that their safety platform is robust, but also doesn't slow down the innovation that a lot of us are using in Our day-to lives. Thank you so much for tuning into the podcast today. As always, make sure to go check out AIbox.ai, my own startup, to get access to all of the latest models and build incredible tools if you're not a coder. ([Time 0:08:52](https://share.snipd.com/snip/443a8477-6044-4a10-b9c7-3d8010b18fcc))