Dear AI Companies: Stop the “Doom Trolling” | AI Reality Check  🔗 **Source:** [Dear AI Companies: Stop the “Doom Trolling” | AI Reality Check](https://share.snipd.com/episode/3ea9593b-be5d-43a2-9345-97e9d14e9316) ## Highlights - **Early AI Coverage Shifted From Curiosity To Dread** - Cal Newport identifies early AI discourse as cloaked in dread after ChatGPT's release, driven by alarmist op-eds and company rhetoric. - He contrasts initial curiosity about language-model capabilities with rapid escalation to existential claims like “we have summoned an alien intelligence.” Transcript: Cal Newport The last three years have been sort of exhausting for me. As a computer scientist and a technology commentator, I was excited by ChatGPT when it was first released. I mean, the effectiveness of generative AI at both understanding and producing structured language was cool and unexpected. It was sort of like when you first saw that pinch-to feature on an early iPhone. So it seems self-evident to me that, hey, with enough experimentation, we would for sure find some impressive applications for large language models. And I was curious to learn what they would be. But then almost immediately, the discourse surrounding AI became cloaked in a mantle of dread and hype. A few months after ChatGBT's launch, for example, Yuval Harari, Tristan Harris, and Azaraskan published an alarming New York Times op-ed about what the arrival of this tool foretold. Here's what they wrote. Manipulate the operating system of civilization. By gaining mastery of language, AI is seizing the master key to civilization from bank vaults to holy sepulchres. Later in the article, those authors predicted by 2028, the U.S. Presidential race might no longer be run by humans. And then perhaps most notably in their conclusion, the authors declared, we have summoned an alien intelligence. Now, at the time, I remember that that seemed out of proportion with how large language models actually worked. So a few weeks later, I responded with a long New Yorker piece that I wrote that was titled, What Kind of Mind Does Chat GPT Have? And this led to some media calming, some fear calming media interviews. I got to spend some time up on the Hill explaining auto regression to some senators. But ultimately, it was too little too late because soon after this sort of initial pop of dread and fear around AI, the AI companies themselves embraced the strategy of trying to unnerve And scare their own customers. A month after my New Yorker article, for example, OpenAI CEO Sam Altman signed an open letter that argued, and I'm quoting here, the risk of extinction from AI is on scale with nuclear War. Anthropics Dario Amadei got more specific, claiming on multiple occasions that there was a 25% chance that our AR future would go, quote, really, really badly. He was referring there to the end of the human race. The near ([Time 0:00:00](https://share.snipd.com/snip/a2d97382-600c-48af-97a7-ed3a74d8ba19)) - **High Profile Leaders Issued Existential Warnings** - Cal Newport recounts high-profile claims from OpenAI and Anthropic that framed AI as an existential risk, citing Sam Altman and Dario Amodei. - He lists specific episodes: Altman's open letter equating AI risk with nuclear war and Anthropic's warnings about automation and Mythos. Transcript: Cal Newport A month after my New Yorker article, for example, OpenAI CEO Sam Altman signed an open letter that argued, and I'm quoting here, the risk of extinction from AI is on scale with nuclear War. Anthropics Dario Amadei got more specific, claiming on multiple occasions that there was a 25% chance that our AR future would go, quote, really, really badly. He was referring there to the end of the human race. The near future wouldn't be any picnic either. Amadei argued on multiple other occasions that 50% of entry-level white-collar jobs would be automated in the next one to five years. Sam Altman agreed with his general sentiment and helped fund it out of his own pockets, experiments in universal basic income, because as he explained multiple times, he was convinced That some sort of guaranteed income from the government was the only way that we would avoid having to eat our pets once AI took all of our jobs and cratered the economy. Last summer, while appearing on Theo Vaughn's podcast, Altman's voice cracked as he tried to reckon with the all-powerful technological demons he was summoning. He compared their work to the Manhattan Project. Earlier this spring, Anthropic released a report saying that the new language model Mythos was so good at finding cybersecurity vulnerabilities that they couldn't release it to The public. They gave really anxious briefings to reporters and world leaders about this new danger. This led Tom Friedman, who I think was at one of those briefings, to dutifully write a column that called Mythos terrifying. He said in his column, holy cow, superintelligent AI is arriving faster than anticipated. This is not a publicity stunt, Friedman insisted. Then six weeks later, Anthropic threw some basic guardrails on the model and released it anyway, so maybe it was a little bit of a publicity stunt. Then finally, earlier this month, Anthropic put out a new report that argued that the success of their cloud code, their cloud code software development agent increased the probability That AI may one day soon start improving itself until it gets so powerful that humans can no longer control it. The report featured a scary animation of robots multiplying like cells in a Petri dish and many brightly colored charts with somber captions. They concluded the report by saying that, well, there's nothing they could do about it because, you All right. ([Time 0:02:07](https://share.snipd.com/snip/b92e614e-5601-4614-9cec-f6a184f2aca2)) - **Doom Messaging Harms Public Trust And Mental Health** - Newport links pervasive doom messaging to growing public distrust and anxiety, supported by polling and personal emails. - He reads a developer's email describing daily fear of replacement and mental-health consequences. Transcript: Cal Newport All of this type of fear-mongering has had a massive impact. Polling numbers show most Americans now distrust these technologies and think they're much more likely to cause harm than good. Essentially, anyone now who tries to keep up with AI news, even if they're just sort of casually reading some headlines, will find themselves living a life riven. Know, China. Anxiety and dread. And I'm not exaggerating about this. I want to read you an excerpt from an email that I received just last week from a software developer. Here's what he told me. I am tired. Every day I hear that me as a developer will be replaced. Every day I hear that self-improvement is right around the corner, that everything will be bad, wrong. I don't even know what to believe anymore. These people ruined my mental health and don't even know who I am. Just casually tossing around predictions after predictions after predictions. I'm afraid my son will be living in some dystopian world. Reading messages like this is heartbreaking. ([Time 0:04:29](https://share.snipd.com/snip/575307ce-c347-4a06-a09c-a51ee2459737)) - **Doom Trolling Defined As A Communication Strategy** - Newport coins the term doom trolling to describe companies cataloging grim AI futures while continuing product development. - He analogizes Anthropic leaving a dead bird at the doorstep then returning to raise money and push IPOs. Transcript: Cal Newport I have it here for those who are watching. The title that we gave it was Dear AI Companies, The Doom Trolling Needs to Start. Okay, stop rather. Let's start with the introduction. Here's what I wrote in the very beginning of the piece. Technology revolutions in the digital age are typically accompanied by optimism and excitement. Recall Steve Jobs basking in thunderous applause as he introduced the iPhone in 2007. The major AI companies seem to be following a darker and weirder strategy. They like to solemnly describe the harms that their models will cause while acting helpless to do anything about it. I think this is a big deal, right? This idea that this is new. We're used to it, but it doesn't mean that it's normal. Technology leaders used to say, let me tell you why this is cool and why you should be excited about it. Today, they're doing something so much darker and weirder. Okay, returning to my article, after I give some examples, including in particular that last Anthropic report about recursively self-improvement among their tools, I said the following, Like a cat leaving a dead bird at your doorstep, Anthropic catalogs the grim future that its products might produce, shrugs its shoulders, and then returns to its furious efforts to Make these warnings a reality i then elaborate let's call this strategy doom trolling it's one of the defining and most arresting properties of our current ai moment and i've come to Believe that it is morally indefensible again this is a big claim but i'm saying once we recognize the weirdness of this communication strategy where again like dropping a dead bird At the doorstep, you know, Anthropic being like, hey, we might lose control of AI because of our tools. Carry on. And then they just walk away and get back to trying to raise money for their IPO. It's weird. Let's call it – let's give it a name. Let's call it doom-trolling. And I say this is morally indefensible. ([Time 0:06:56](https://share.snipd.com/snip/0f4a58d1-dc15-4a13-ad64-0b8d83b317d4)) - **Two Moral Explanations For Doom Trolling** - Newport frames two moral possibilities: companies genuinely believe existential risk and must stop, or they cynically weaponize fear for profit and regulatory advantage. - He argues either scenario is morally indefensible. Transcript: Cal Newport So here's what I write. I say there are really only two options for the intentions of AI companies when they engage in doom trolling. The first is that they actually believe that the systems they're building have a non-trivial chance of producing hugely disruptive events, from destroying the economy in the best Case wiping out our species in the worst. If this were true, every reasonable ethical system would argue that there is only one acceptable response to immediately stop working on any product that might accelerate such a future And lobby with all of your resources to help force other AI companies to do the same. From a moral perspective, any other reaction would be monstrous. All right. So if they actually do think the technology they're working on has even a non-trivial chance of causing massive societal or even existential harms, all ethical systems say, yeah, You got to stop and do everything you can in your power to stop other companies from doing the same. Forget your IPO. Forget your stock holdings. Forget your money. This is the future of the human race. Of course that's what you should be doing. So for you to say, yeah, this might lead to the extinction of humans, this trajectory we're on of product development, but, you know, China, and then getting back to trying to do a trillion-dollar Valuation, that's morally monstrous. All right, here's the second option, though. I say the second option is that these AI companies aren't really concerned about these risks and that they're injecting those doses of unresolvable doom for other reasons. They might want to amplify the perceived power of their technology at a time when they're setting up their initial public offerings, or they hope their performative reports and somber Interviews will help them compete for more engineering talent coming from a Silicon Valley culture that's steeped in this type of doomerism. DaVinci Capitalist and AI advisor David Sachs recently suggested that Anthropic was using fear-mongering tactics as a method of regulatory capture, which can impede upstart competitors. Any of these reasons would mean that these companies are laundering the anxiety of millions to improve the financial fortunes of a vanishingly small number of major stockholders. This cynicism would be equally monstrous, right? So the second option is like, they don't really believe that building harnesses on top of fine-tuned language models is going to lead to the automation of our economy or the end of our Species, but they still talk about it like it would. And I say that would be just as bad because that means you're basically alchemizing anxiety into money for a small number of people who have early stock positions in your company. ([Time 0:08:58](https://share.snipd.com/snip/2e8063c9-066d-44ea-988b-42e59c47dec9)) - **Most Engineers See AI Progress As Narrow Not Apocalyptic** - Newport and many East Coast computer scientists view current language-model trajectories as narrow and unlikely to cause economic collapse or extinction. - He emphasizes repeated struggles to make LLMs useful beyond structured language tasks like coding. Transcript: Cal Newport I don't know what they think, but most East Coast computer scientists who aren't sort of infected by the quasi-religious eschatology of Silicon Valley would say, no, language models With harnesses built on top of them are not a trajectory that's going to exponential up to automation of all jobs and certainly not to superintelligence and the death of economy. This technology has its uses. They're more narrow than we think. That's why we're still stuck mainly doing highly structured language type things like coding, which required years of effort to get these custom neurosymbolic harnesses to make It useful. It's why it's not affecting the economy. It's why so many companies now who invested a bunch in AI outside of software development are now moving backwards. And even those who've invested in AI as software developers are now having to reduce the token budgets because they're not getting enough return on their investment. Not magic technology. It's not the chip from Terminator 2 that allowed the sort of cyborgs to come alive. That's what I feel about it. ([Time 0:11:31](https://share.snipd.com/snip/c46a2910-79d7-4791-887c-daf718622ced)) - **Evaluate AI Products By Present Utility Not Future Hype** - Newport tells listeners to stop playing along: ignore future-tense company statements and evaluate products by present utility. - He urges focusing on whether current tools are worth time and money, not sensational forecasts. Transcript: Cal Newport Well, here's the clearest way I can summarize my suggestion. Stop playing along with the doom-trolling game. Don't pay attention to any statements from AI companies that are stated in the future tense. Care about the products they offer now and whether those products are currently worth the time investment and money required to use them. When you encounter scary news articles about AI, remember that most journalists don't have access to some sort of special information or powers or prognostication. They're just replaying or relaying the dark vibes that the AI company leaders are putting off. So you don't have to put special stock in the fact that a major publication is saying something scary about AI. It is just a direct reflection of the doom trolling that the AI companies themselves are doing. ([Time 0:15:55](https://share.snipd.com/snip/e157dd3b-edc6-4256-af66-47623c3c1a3d)) - **Treat Scary AI Headlines As Company Echoes** - Newport advises skepticism toward scary news: journalists often echo company rhetoric and lack unique predictive insight. - He warns that doom rhetoric also helps companies secure outsized valuations and talent. Transcript: Cal Newport It is just a direct reflection of the doom trolling that the AI companies themselves are doing. I want you to remember two things about the current leaders of these major frontier AI labs. First, they are weirder than you think. I think we all seriously underestimated the degree to which the quasi-religious ex-risk singularity culture in Silicon Valley has warped the way that many Silicon Valley leaders Think and talk about human value technology in the future. This is why I think doom trolling got a early foothold, especially in the media, is that we sort of rationally assumed, oh, these are the people who know this technology best. So if they're concerned, we should be concerned too. Now, a few years in, we're realizing that the person that we trusted maybe had a couple screws loose. It reminds me of the classic Simpsons episode where the car company gets convinced that Homer is a genius and they let him design his own car. And it's a terrible disaster and a flop. They realize like, oh, maybe our idea that this is the person who knows best about what someone wants in a car was wrong. It reminds me of this sometimes. Like these guys, these hardcore serious engineers know what they're talking about. And then we see them at their sort of metaphorical robes on as they're doing their demon summoning dance around the GPU on top of an altar like, uh-oh, maybe we shouldn't have been so quick To assume everything they're saying was true. The second thing I want you to keep in mind about the leaders of AI companies right now, this way they've been talking, this sort of doom-drenched rhetoric has been very helpful financially For them. It makes them seem all-powerful and therefore worthy of much more investment capital than the reality of their business model, which is that they are essentially a money-losing natural Language version of Google, an OpenAI's case, and a software development utility, developer utility company in Anthropics case, that reality is not worth a trillion dollar valuation. But the reality of we're going to invent the last tool ever that's going to run the whole economy, there's enough hype around that. You're like, I want to write a check. It puts them in that sort of meme stock territory that SpaceX benefited from. So there's a financial incentive for the way they talk. Keep that in mind. ([Time 0:16:46](https://share.snipd.com/snip/620a0a94-4507-44b1-9472-86a49b16dd56)) - **Make Doomers Prove Their Remarkable Claims** - When friends demand you reassure them, don't feel obligated to disprove Doomers; require them to provide evidence for remarkable claims. - Newport says remarkable claims require remarkable evidence and the burden lies with the Doomers. Transcript: Cal Newport If you begin rejecting doom trolling and show like really any resistance to these gloomy messages, people in your life who are drowning in AI anxiety will begin to use you as a safety Blanket. They will semi-antagonistically challenge you to explain whenever any new sort of stupid feature or benchmark chart is released why this doesn't mean that the Terminator robots Are in fact coming. And here's what I want you to remember once you're in this situation. It's not your job to disprove the remarkable claims of the Doomers. It's the Doomers' job to convince us that the remarkable claims are likely. You do not have to play the safety blanket for people around you. It's this weird asymmetric inversion we've had that instead of the companies having to say why they think we're going to have AI ruling the world, you have to somehow argue why a 10% increase Of Fable 5 on the Epic 6 benchmark doesn't mean that. It's a crazy inversion. It's not your job. It's the Doomer's job. Remarkable claims require remarkable evidence. ([Time 0:18:59](https://share.snipd.com/snip/456d744e-b4c3-4fad-a407-102f2cee3fa6)) - **Use Doom Trolling Label To Undermine Performative Fear** - Newport urges using the term doom trolling to shame and delegitimize the tactic and discourage performative warnings. - He hopes cultural ridicule will make firms less likely to publish dramatic white papers without changing behavior. Transcript: Cal Newport Words have power. My greatest hope for this piece is that by normalizing this new term, by getting it as a part of our lexicon, it will effectively undermine or ridicule the communication strategy of This sort of like, guys, I'm sorry to report, but our latest product, it's probably going to lead to data centers having to devour your pets for organic energy sources. That's probably going to happen. Anyways, our stock will be available next week. See you soon, right? If it kind of ridicules that, it makes it harder for them to do. I want it to be culturally almost impossible for Anthropic to put out one of these stupid white papers with animations and charts where they act all somber, like, oh, we're so safety Aware, and then doing no changes at all in their product offerings or research programs. So let's use that term. ([Time 0:20:11](https://share.snipd.com/snip/12444e40-97b4-4693-9edc-3f0bc786827c))