How to Think About AI Before It’s Too Late  🔗 **Source:** [How to Think About AI Before It’s Too Late](https://share.snipd.com/episode/3e8d721e-6aae-41d7-b98f-26a7724ee0c8) ## Highlights - **Doctorow Uses Local AI As A Practical Writing Plugin** - Cory Doctorow uses a local Ollama model on his laptop as a daily plugin to catch typos and inconsistent references when publishing his self-hosted blog. - He treats AI as a word-processor plugin, valuable for small tasks but not worth massive centralized investment. Transcript: Cory Doctorow Use I make of AI, and it's daily, and I just did it again, is I publish a blog, pluralistic.net. And because I am keenly aware of how the platforms trap us, I control that blog. I publish it on my own self-hosted WordPress site on a server on a shelf that I own. So my prompt for Olama, one of the Olama models that I have running on my laptop is find typos. And that is remarkably good. So not only does it catch a bunch of stuff that my text editor or word processor wouldn't normally catch, things like you type a word that is a correct word, but not the word you meant. So you type shocker instead of shocked. Double punctuation, typing the word and twice in a row, kind of stuff we do all the time. As I was writing, and I pasted in a link to the Wikipedia article on it. And then I referred to it again later by slightly different names. So it was something like saturation theory, and I called it the saturation hypothesis. And the chatbot said, in the URL, it's called saturation theory. Did you mean saturation theory here? Which was amazing. Yeah. Genuinely helpful. Yeah. Super helpful. Look, I don't know where your computer origin stories are, but my first word processor was a program I bought in a magazine that I typed into an Apple II+. My word processors have gotten new features since then on the reg, and an enormous number of them are things I have no use for at all. Many of them are things that I think other people use very badly. And some of them are things that I find indispensable. And AI is a plugin to my word processor, as far as I'm concerned. ([Time 0:04:40](https://share.snipd.com/snip/a5d368d3-8b7a-44f6-9b04-d754c17a0a47)) - **Reverse Centaur Versus Centaur Automation** - Centaur means a human assisted by a machine and reverse centaur means a human conscripted to help the machine. - Cory Doctorow illustrates reverse centaurism as workers forced to be the bottleneck so bosses can 'sweat' depreciating automation assets. Transcript: Charlie Warzel What is a centaur? Not the mythological animal, but the automation theory that you've based the book around. And what is a reverse centaur? Cory Doctorow So in automation theory, centaur is someone assisted by a machine. So, you know, you're wearing glasses, you're riding a bicycle, you're using a spell checker, you're playing a harmonica instead of making mouth noises, you're a centaur. And a reverse centaur is someone who is conscripted to help the machine. So there's a process a machine can almost do, but there's a part of it that it can't do. And someone has inveigled you to do that other part. And because the capital that went into that machine is depreciating, right? It's the machine only lasts so long. You want to sweat that asset as hard as you can. And because machines don't get tired and they typically work faster than we do, you, the reverse centaur, are the bottleneck in the machine. You are Lucy and Ethel trying to get chocolates off the assembly line and get them into the chocolate box. And the boss is running the assembly line at the maximum speed that the humans can conceivably do it at. And they are expecting you to work at two or slightly beyond the point at which you collapse from exhaustion. So someone who is a reverse centaur isn't just used by a machine. They are used up by the machine. Let's ([Time 0:06:52](https://share.snipd.com/snip/da40b5ef-06bf-4b0f-a230-eb92e7f0b6d5)) - **Amazon Drivers Pushed Into Reverse Centaur Labor** - Amazon delivery drivers are pushed into reverse centaurism by routing, sensors, and penalties that remove slack and basic comforts. - Doctorow describes drivers peeing in bottles, vans tracking gaze, and routes that ignore real distances to force faster pacing. Transcript: Cory Doctorow Yeah, it's hard to overstate how much automation in that van is about seeing if you can get the person to work so hard that there is just no slack left for them. So we all know about the drivers not peeing, right? Or rather peeing in bottles. That's not just because they have a hard schedule. The van is determining what route you're going to take and how long it's going to take. And then you have to make the prediction real, irrespective of traffic conditions and so on. You are covered by many cameras and many sensors when you drive an Amazon van. And they are doing things like penalizing you for looking in the wrong direction. So you might be looking at something that you anticipate is a danger, but the van disagrees. And so they are dinging you on that. And then there are the more sort of mundane complaints, like the machine thinks that you can deliver these three parcels after parking your van because they're all in the same housing Complex. And what the machine doesn't know is that one of those drop offs is a mile away. ([Time 0:08:14](https://share.snipd.com/snip/cdf04df1-c913-4007-b35e-bd8d5e55de63)) - **AI Bubble Fueled By Growth Pressure On Mature Firms** - AI bubble growth is driven by firms that must keep showing high growth to justify valuations once markets view them as mature. - Doctorow links executives' incentive to sustain growth to huge CapEx spending on AI to avoid share-price decline. Transcript: Cory Doctorow And one of the things this book wants to interrogate is, first of all, why do we have bubbles? And second of all, why is this bubble so much bigger? And it is so much bigger. I mean, when I wrote the book, it was a $700 billion CapEx expenditure worldwide. Now it's a $1.4 trillion CapEx worldwide. And it's going up and up. Yeah. Charlie Warzel And that $1.4 trillion is Goldman Sachs' prediction for spending on AI infrastructure in 2027. Cory Doctorow So this is a shocking amount of money. We've never set this much money on fire before for anything. It makes tulip bulbs look like rational conduct. You know, South Seas look like an achievable objective, right? So where's this all coming from? Well, first of all, I think we have bubbles because firms that have saturated their market go very quickly from being treated by investors as growing firms to being mature firms. And when that happens, their shares are grossly overvalued because if a share in a company is a claim on its future earnings, if those future earnings are going to be larger than they Are this year, then it makes sense that the share will be worth a lot, because those earnings are going to grow and grow and grow. But if we know how much the company brings in, and if it brings in about the same every year, plus or minus a few points, then it has a much lower what's called a price to earnings ratio, right? It trades at a lower multiple of its annual earnings. And from the perspective ([Time 0:10:29](https://share.snipd.com/snip/c417836e-d24a-436b-865b-a1a21e25204d)) - **Automation Appeals To Bosses For Control Not Just Savings** - Bosses crave automation not only for cost-cutting but to avoid confrontations with skilled workers and keep control. - Doctorow compares prompting bots to giving notes to writers and notes bots simply produce output without pushback. Transcript: Cory Doctorow So excited about AI versus all the other things they've been excited about. And I think a big part of this is that bosses are infinitely horny for firing workers and replacing them with machines. And they have been since forever, right? That's the story of the industrial revolution. There's something about this, and it's not just cutting costs. I think that if you're the boss, you are haunted by the knowledge that if you don't show up for work, the business just ticks over as per normal. But if all the workers don't show up, you're out of business. And so maybe you tell yourself you're driving a car, but secretly you worry that you're in the backseat with a Fisher-Price steering wheel. And one of the things about AI is it dangles the possibility of wiring the toy steering wheel into the car's drivetrain. So the boss has an amazing, cool idea, and then it gets pooped out by the bot, and there are no ego-destroying confrontations with people who know how to do things who tell you that your Idea is dumb. You know, when I was on the picket line with the screenwriters, because I live in Burbank, and I'm in an affiliated union. I'm in the Animators Guild Writers Unit. So when I was on the picket line with the writers near my house, one of them said to me, you know, you give notes to a writer's room the same way you prompt a bot, right? Like give me E.T., but it's about a dog with a love interest and a car chase. And the difference is that you say that to writers and they're like, go make a spreadsheet. The grownups are making a movie. Whereas you say it to a bot and it just gives you a script. ([Time 0:14:50](https://share.snipd.com/snip/ae75385e-23d1-4760-9a83-875de5d95686)) - **Air Canada Chatbot Created An Accountability Sink** - Air Canada used a chatbot that gave a bereavement travel refund process which turned out to be incorrect, leaving a customer fighting for months. - The company benefitted because customers lacked recourse and the chatbot served as an accountability sink. Transcript: Cory Doctorow I have a dog in this fight. Like all the best Americans, I am Canadian. Air Canada's customer service has always been pretty bad. So you could see why, if the only reason to have customer service is to have someone explain over and over again that the company isn't going to solve your problem, why you wouldn't just Ask an AI to do that. And that's what Air Canada did. So there was a man who was going to his grandmother's funeral and he got on the Air Canada customer service portal and he said, how does a bereavement fare work? And the chatbot said, you buy a full fare ticket today, go to your grandmother's funeral, come back, get a copy of the death certificate, send it in, and we will refund you the difference, Which he did. And when he did, the human being who was in charge of issuing refunds was like, I don't know what you're talking about. That's not how we do it. You had to do this before you flew. You don't get any money back. And he appealed it all the way through every level of customer service at Air Canada and eventually got a judgment against Air Canada for on the order of seven or $800 Canadian. So this was like a multi-month process. And you could see that from Air Canada's perspective, doesn't matter if it's wrong. ([Time 0:25:02](https://share.snipd.com/snip/531a8803-9e3b-44bf-9ec5-298884fe9631)) - **Automation Blindness Kills When Humans Rubber-Stamp AI** - Automation blindness occurs when humans rubber-stamp AI outputs because they're overloaded with high-speed review tasks. - Doctorow warns that configuring AI to fire most radiologists and make few rubber-stamp reviewers will increase deadly errors, shifting blame to humans. Transcript: Cory Doctorow Yeah. Well, there's a funny thing that happens at TSA checkpoints. TSA people train all day, every day to find water bottles. And they are the world's water bottle finding as people like that we have ever created. They are the Louis Pasteur of finding water bottles because they get more practice than anyone else. But the number of people who deliberately bring a gun or a bomb onto an airplane, it is indistinguishable from zero. Now, obviously, at the very furthest margin, there are people who do it, which is why we have airport security. But there is every reason to think that without a regular supply of people walking through, deliberately carrying guns and bombs through that have been disguised, that airport security People just cannot remain vigilant for it because they detrain to find one thing when they are training to find the other one, which is water bottles, which is the thing that we all bring Through. Charlie Warzel And so you're suggesting that in this paradigm where all these jobs are, if they're not being fully automated away, if you have a human in the loop, you're suggesting that this type of Automation blindness is going to increase. Cory Doctorow Yeah, especially when you think about this in combination with this reverse center paradigm of work speed up, of sweating the asset. So if your job is to have a new AI judgment or result put in front of you at the maximum speed that you could theoretically evaluate it, and if in the majority of times it's good enough, so Your job the majority of times is to click okay, and you have to do that relentlessly at this incredible pace. It's going to require some very atypical neurological makeup to spot that. I won't say there's no one ever born who couldn't do that job, but most of us are not going to be that person. And there aren't enough of those people to be the human in the loop. So let me give you an example of how you might configure automation in two different contexts. So I have an extremely treatable form of cancer. So I spent a lot of time following the news about cancer treatment and AI. And it looks like AI can sometimes spot solid mass tumors that humans miss on x-rays. And so if there was a sales call between an AI salesperson and the hospital administrator at the Kaiser Hospital where I get my care, and they were saying, right, here's the pitch. Right now, you spend a million dollars a year on radiologists. They're each assessing 100 x-rays a day. I want you to give me half a million dollars a year for my robo-radiologist, over and above the salary of those million dollars that you're spending now. And now your radiologists are going to do fewer x-rays because one or two times a day, the AI is going to say, have another look at that one. And they're going to stop and they're going to look at it again. No risk of automation blindness, right? That is an arrangement as someone with cancer that I would be very happy to see. That's not the pitch because there aren't enough hospitals that want to spend an extra half million dollars a year over and above their payroll. The pitch goes like this, fire nine-tenths of your radiologists. Put the remaining 10% in charge of rubber stamping the output from the x-ray, but they're supposed to look at it closely enough to actually make an assessment about it. But you're going to work them as fast as you possibly can to sweat this asset as much as you can, realize as much of a savings as you can, fire as many workers as you can. In that circumstance, when the AI is usually right about the chest x-ray, but sometimes wrong, and the radiologist is just clicking, okay, okay, okay, okay, okay, five times a minute, That radiologist is going to kill people, and we're not going to blame the AI. We're going to blame them. And that's the accountability sink. That is the accountability sink. ([Time 0:27:32](https://share.snipd.com/snip/86f932bb-aa84-48d1-ad4a-8f2818e68c80)) - **Regulate The AI Bubble Before It Bursts** - Prepare for an AI bubble burst by planning regulation and market-structure changes rather than assuming continued growth. - Doctorow urges using democratic regulation, antitrust, and market-structuring (like past Glass-Steagall) to avert collapse and authoritarian fallout. Transcript: Cory Doctorow It would be really bad for a $1.4 trillion bubble to pop. It would be even worse for a $2 trillion bubble to pop. And so, you know, I do believe that history is ours to make, that the future has not been written. But when I say that, I don't mean that there's a way that AI becomes profitable. I mean that there's a way that we deal with the fact that it is a bubble and what we do when the bubble pops. We know that it's coming, if we understand who created it, if we regulate in response to it. You know, one of the reasons we have this string of bubbles we've had in this century is because of the repeal of Glass-Steagall. Glass-Steagall was a law put in place after the Great Crash and the Great Depression. And it was intended and worked very well to structurally separate investment banks from commercial banks, from consumer banks. And we got rid of that law, right? And this is not all that different from other things that we deregulated in that era. We stopped enforcing antitrust law around the same time, and then we got a lot of monopolies. Those two facts are actually really closely related. And, you know, if I get grumpy or cranky, it's really because it sometimes feels like we have so little object permanence that we couldn't win a game of peekaboo. And we just forget that in living memory, we had a thing we did that stopped a bad outcome. And then we stopped doing that thing. And then the bad outcome happened. And we're like, this is a mystery. What an unfortunate situation for us. But, you know, as Margaret Thatcher told us, there is no alternative. So I guess we just have to live with it. And finance and the end of antitrust law, those are the same people who are inflating this bubble and who are also saying, how dare you ([Time 0:32:02](https://share.snipd.com/snip/030fe1c5-eca6-4fbd-9334-7489c3db6b3e)) - **Attack AI Projects With Multiangle Practical Objections** - When opposing AI deployments, combine environmental, labor, and democratic critiques to avoid conceding the industry's premises. - Doctorow suggests challenging pollution, water/grid impacts, undemocratic siting, and the presumption that the companies' demos equal durable value. Transcript: Cory Doctorow People criticize AI. They stipulate to a bunch of its claims without interrogating them. So when we say this AI data center is going to be polluting, it's going to be noisy, it was undertaken under undemocratic conditions, we have better uses for that land, it's going to be Hard on our water supply and our grid, we should also add, and by the way, this company might be bankrupt before they turn it on and not stipulate to the idea that this thing that can't go On forever will never stop. ([Time 0:36:35](https://share.snipd.com/snip/8d1ead9d-0b38-4cf4-863d-4f5a8c6195a5)) - **Challenge AI Profit Narratives With Unit Economics** - Don’t accept industry narratives that AI will always be profitable; challenge pricing and unit-economics claims. - Doctorow advises reminding investors that training runs cost more than global wages of displaced workers and that $100-bill demos sold for $1 won't scale profitably. Transcript: Cory Doctorow People criticize AI. They stipulate to a bunch of its claims without interrogating them. So when we say this AI data center is going to be polluting, it's going to be noisy, it was undertaken under undemocratic conditions, we have better uses for that land, it's going to be Hard on our water supply and our grid, we should also add, and by the way, this company might be bankrupt before they turn it on and not stipulate to the idea that this thing that can't go On forever will never stop. I think when we criticize AI more broadly, we should remember that as offensive as it is for the AI companies to say to creative workers, our goal is to make you bankrupt. And our tactic for doing that is training our models on your work, that we shouldn't stipulate to the idea that the work is as good as ours, right? That they've done anything good. And we should remind investors who have seen these splashy demos, I think that's what, you know, AI art and AI text gen and AI, you know, AI video and so on. We should remind them that the workers that this stands to displace have a wage bill globally that sums up to less than the kombucha expenses for one training run of mid-journey. And that even if they fire every illustrator in the world and replace them with slop, that it won't turn a buck, right? That this is a money losing proposition that if you charge, if you stop giving or selling $100 bills for a buck a piece, it's always going to be cheaper to hire illustrators. ([Time 0:36:35](https://share.snipd.com/snip/af126b10-98ea-4ea4-b778-a292787eb1da)) - **Oligarchy Prevents Democratic AI Remedies** - Concentrated capital and oligarchy block democratic responses to AI harms, raising the risk of collapse or authoritarian backlash. - Doctorow invokes Piketty to argue unchecked capital accumulation leads to instability and that oligarchs resist fixes to systemic crises. Transcript: Cory Doctorow Well, about 15 years ago, I guess, there was a book by Thomas Piketty, the French economist, called Capital in the 21st Century. And Piketty did this monumental research project going through every ledger of capital flows going back 300 years to trace capital flows and capital concentration over centuries. And their conclusion was that all of the things being equal, capital tends to accumulate. That if you have money, you tend to make money. They summarize this as R is greater than G. The rate of return on capital is always greater than the rate of growth. And so Piketty's point was, if you don't do something about that, rich people get richer and richer and richer and richer and richer and richer. It doesn't matter how big the pie is getting. The share of the pie owned by the rich gets bigger, faster than the pie gets bigger, always, as an iron law. And that eventually, this becomes untenable. Eventually, the rich, because they are unchecked, there is no democratic accountability. They are subject to folly as the rest of us. They look to one another with great jealousy and use us as pawns in bids to take one another's fortunes. Once our fortunes have been stripped bare, that this creates massive instability and that those instabilities tip. And he sees these points over and over again, where the share of income in the top decile, the top 10% reaches, or wealth rather, not income, reaches a certain point. And then you get the French Revolution, and then World War I and World War II, and so on and so on. And his message was that unless we do something about oligarchy, the end state of oligarchy is not stability. The end state of oligarchy is chaos. It's collapse. And that is my worry. Who love the fantasy of AI and love the fantasy of a world without people and that they live in a world without people now where everyone around them is a non-player character who just Glazes them with AI-like nonsense. But there's another group of people who love AI, which is politicians who want to exert their will without having to go through a permanent civil service. Just like there's a dream of a workplace without workers, there's a dream of a government without bureaucrats, right? This is what Trump wants to do with Doge. This was the whole nature of the Doge project was you take away the deep state, which is to say everyone who knows how the government works, and you replace them with chatbots. And then Trump has an idea and it happens. You don't have these collisions with people like that's not just not how the government works. We can't do it that way. There are people who won't be able to fill in that form you want them to fill in. If you take away this support, this other thing will collapse. It doesn't work that way. You just tell the chat bot to do it and the chat bot just does it. You can see how this would be really unstable. I mean, we're living through it. We're living through it with just screw flies. No amount of belief is going to help you when beef is $30 a pound. Like that's politically destabilizing. And so I think that unless we can confront oligarchy by democratic means, that we will find ourselves in the midst of a circumstance where it is being confronted by non-democratic means, Whether or not that's what we want or what we aim for. And I really worry about it. I think oligarchy, it is the force that stops us from addressing the polycrisis of genocide and climate degradation and rising authoritarianism. ([Time 0:43:16](https://share.snipd.com/snip/a221b1f3-1d6f-40be-9a85-045278a214fc))