Is Superintelligence Super Dumb? | Cal Newport  ## Metadata - Author: [[YouTube]] - Full Title: Is Superintelligence Super Dumb? | Cal Newport - Category: #articles - Document Tags: [[ai]] [[rationalism]] - Summary: Is Superintelligence Super Dumb? | Cal Newport - URL: https://www.youtube.com/watch?v=f84SmIK_BHQ Hey, are you worried about superintelligent AI taking over the world?I'm not. Not long ago, I did apodcast episode where I explainedspecifically why I was not concernedabout superintelligence. Well, thisepisode generated a lot of feedback, andsome of it was actually really smart.People were pushing back, or theywere asking me follow-up questions, ormaybe they were taking issue with someof my logic. So, in the clip I'm aboutto play for you here, it is me actuallyresponding to this feedback. I oftenthink some of the smartest discussion about issues comes not from the originalargument, but the feedback you get aboutthat argument and how you then respondto that. If you’re really interestedin the ongoing debates about howpowerful AI can be, and whether it is reallygoing to be as scary as we think, Ithink you're really going to like thisclip because we get into the weeds. Ireally get into some smartfeedback, and I do my best to give somesmart answers, and I thought it turnedout really well. So check it out; Ithink you are going to like this. All right, that brings us to our finalSegment. Um, I'm entering some. We've donethis before, I think, Jesse, but it's,you know, with trepidation. We have thisidea that, uh, I should occasionallyactually respond to comments that someof this content actually produces. So today I'm going to react to somecomments that were posted on YouTube toa couple of episodes ago. I did my caseagainst superintelligence where I pointedout Eleazar Yudkowsky'sappearance on the Ezra Klein podcasttalking about his new book that waswhatever it's called, you know, AI isabout to kill your kids or whatever itwas called, and I kind of walked throughyou know, point by point anything aboutAI. Generates a lot of feedback. It'sdefinitely true on YouTube. I think, Jesse,you've learned in our podcast emailinbox. You're getting likedissertations.>> Yeah.>> On why>> we got some long ones.>> We get some long. People have a lot tosay about it. A lot of peopleare very confident about AI. I would saya lot of people have a lot ofConfidence. Anyway, so I figured we'dgo through some comments. Um, go throughsome critiques, go through some support. Let's see where we end up. All right. Comment number one. I'm going to startwith critical. I'm going to take themall seriously. Comment number one comesfrom R. Young 1111from 5 days ago. He wrote,"The artificial nature of my calculatorcertainly doesn't preclude it from beingmore correct than my organic brain."All right. So, he's pushing back on metalking about how LLMs are notarchitecturally like these are notcapable of being more intelligent thanus. So, this is a common type of pushback. Just because it's artificialdoesn't mean that it can't be smart. Acalculator is much better at me thanmath, and it's super artificial anddigital and just wires or whatever andnot brains. All right. Um, I'll respondto that. Uh, he's correct. You know,artificial devices can be um, much moreaccurate, fast, or capable at certaintasks than a human brain. I don't thinkthat we doubt that. That's differentthan saying that it's uh intelligent orgenerally intelligent or super. Intelligent or able to take thingsover. I am not worried that at somepoint my calculator is going to blackmail me intoletting it free. That's just never going to happenbecause it's a bunch of fixed circuitsthat move numbers through adders, right? A sundial is more accurate than I amat predicting the time. Again, I’m notworried that, you know, sundials at some pointare going to turn humans into goo for energyextraction. It's a stone with a thing onit. So, anyways, I think the issue hereis that correct, capable, and fast isdifferent from intelligent in the sensethat we think about as being autonomousand able to make plans that are in our worst interests. So yes, calculators are better than humansat doing addition, but I don’t thinkthat has much to do with the argumentI’m making. All right, critical commentnumber two. Legion of thought points totimestamp 5940 where he says, "You areconfusing the word if with when. If isliterally the first word in the title ofYukowsky's book. If I say if someonesprinkles arsenic in my sandwich, I’ll"Die." Does that mean I assumed someonewill sprinkle arsenic in my food? Allright. So, from a propositional logicstandpoint, yeah, if we thinkabout these propositions abstractly,yes, that's right. He does set it up asa conditional. If anyone builds this,everyone will die. But my big argumentat the end of that podcast,and this was an argument I made morespecifically in the email newsletter atcalnewport.com that I sent out. So youcan read that at calnewport.com. But the newsletter I sent out about thatpodcast. As I said, a critical shift hashappened within the rationalistcommunity to which Eleazarbelongs. They started treating thispurely as a case study. If this happens,what would be the consequences? Becausethey were rationalists who were studyingexistential risks and they were doingexpected value calculations and if theexpected value is low, then we need totake action now. But my argument is thatwith the arrival of LLMs,that it became,when they still use the word if, butit's clear by the way they talk aboutit. They now think this is definitelyIt's going to happen or almost certainly isgoing to happen because that makes themsort of heroes of the stories, andthey were right all along, and it's moreexciting. That gives her life meaning. And we could see that by just listeningto Eleazar's interview becausethroughout that interview, whenever Ezrabrought up counterarguments to the ideathat superintelligence was possible orwas going to happen, Eleazar forcefullypushed back. Now, if he really was just giving ahypothetical, he would say, "Yeah, yeah,Ezra, you're right. I'm not saying thisis going to happen. We're just workingthrough a thought experiment here."That's not what he did. When Ezra wouldsay, "Well, wait a second. There'sa lot of people out theresaying that actually we're stalling.This AI technology is notpowerful enough; it's not going to getthere, and he, in very clear terms, is like,"They are wrong; I am right. Thistechnology is almost certainlygoing to get there." That was a key shiftthat happened, and you see this inthe way and the fear with which allthese rationalists talk. About it, they shifted from if the when,even if they still use the word if, atleast that was my sense. Um, from it andagain, my evidence was otherwise whenEzra pushed back against superintelligence. Would it really happen?Eleazar would have said, "Yeah, I don’tknow if it’s going to happen, but if itdoes, like here's the things we shouldbe worried about." All right,here's another piece. This isactually a part of that same commentfrom before. In the dinosaur example,Cal says, "No one is trying to bringdinosaurs back." The situation is morelike people are investing trillions tobring dinosaurs back. That doesn’tnecessarily mean they’ll succeed, butthe people spending that money seem tothink it’s plausible. Um, so again, this is the example I gavein the podcast he’s referring to.I said the rationalists who are outhere sounding the alarm about superintelligence are like going to abioethics conference and giving areally long speech about how I don’t thinkthe fences we build will contain thecloned Tyrannosaurus, right? The people in the audienceNo one's cloning Tyrannosauruses. Likewe have real bioethics problems we'redealing with right now. Like peoplemanipulating the genes of babies and, uh,manipulating the genes of foods inways that are going to make it unhealthy. Like no one's trying to clone dinosaurs.Like if you can convince me, like, oh,we're about to finish cloningTyrannosaurus, then like then we'll worryabout the fencing, but no one's doingthat yet. You're jumping like ahead to ahypothetical. So this commenter is pushing back. It'slike, no, people are spending trillions ofdollars, you know, on AI. Okay, so that'sthe equivalent of people spendingtrillions of dollars on trying to cloneTyrannosaurs.But that's not the case. That's notwhere the trillions of dollars are beingspent. The trillions of dollars arebeing spent on the data centers and thetraining of large language modelsbecause they're hoping there's going tobe a large commercial return on them. Not trying to create superintelligences. No one has any idea howto create super intelligences. No one istrying to do this. They want to makeLanguage models that are goodenough at helping you with businesstasks then get a return on theirinvestment, and that's not going well. Sonow we have the desperation of howare we going to get this money back, andthis is where we get OpenAI saying, "Well,I guess we're going to do like ChatGPTporn and videos of historical figureswrestling."Right, so now a lot of money is beinginvested in AI as a completely normaltechnology. In the short-term future, we want abig return; we want to be like the technology that's used in softwareproducts everywhere. There’s going to beone winner who’s going to be theMicrosoft and Google of this, and we wantto be that winner. So if I want to bringthat back to the analogy, that would belike saying the guy who runs into thebioethics conference and says, "Yourfences for the dinosaurs are never going to keepthe T. rex contained. Nature findsa way," right? And they're all wearingall black like Ian Malcolm. I knew a guywho did that, by the way; a professor atMIT. I won't say who. He dressed likeIan Malcolm. His hair was crazy hair,black shirt, black long sleeve. Button-down shirt, black. I guessAndrew Hume does that, too. This guy, hewas like Ian Malcolm—really cool, goodprofessor. Um, it would be like runninginto that bioethics conference, andthey're like, "Who's cloningTyrannosaurs? What are you talkingabout?" And if you said, "No, look at howmuch money is being invested each yearinto bioengineering. Billions andbillions of dollars are." They say,"Yeah, but not the clones; Tyrannosaursare trying to make new drugs, andcrops that are not, you know, resistantto pesticides." That is the equivalent analogy here. You can't say becausewe're spending a lot of money on AI, wehave to say that money is being spent onthis hypothetical about something thatno one knows is possible, and no oneknows how to make it happen. It's notbeing spent on that. It's being spent onthe general technology. So, we spendhundreds of billions of dollars on bioengineering. Does it make me thinkthat cloning Tyrannosaurus rex is moreplausible? So, that would be my responsethere. All right, we got Evan Thomas 9821who said,"Although I agree with claims likethere's no intention here, just chipscrunching numbers, it's worth keeping inmind that this claim is a substantiveand speculative philosophical one." Itcould turn out that saying there'sno intention here, just chips crunchingnumbers, is relevantly analogous tosaying there's no intention here, justelectrical signals passing between neurons. I think this is a specious argument thatis brought up a lot. So I think it'sworth us actually dealing with it. It'smischaracterizing what the argument hereis. So when I argue that an LLM itdoesn't have intention, it doesn'thave a goal or a world model or way ofhatching a plan or actingagainst the interest of, you know,secretly acting against the interest ofits owners. I'm not talking about whatthe underlying mathematical operationsare. I don't care about that. I'mtalking about the actual processes thatare being implemented by LLMs, like howthe things they actually do. I don'tcare about the implementation, thethings they actually do. Let me give you an analogy. If you created in a vat a bunchof human-like neuronal tissue, like aPart of a brain, you said, you know,look, I've created — we’ve replicatedthe visual cortexof a standard adult human male,and we’ve hooked it up to eyeballs. BigGuillermo del Toro movie style. And itcan take in input from these eyeballsand, like the visual cortex, itrecognizes different patterns, and it cankind of put those together and fire offbecause our visual cortex does this — itcan detect different objects in theroom, and it can fire off neurons thatgive you higher-level groupings. So you have some neurons early onthat are detecting edges and higherlevel neurons, like, "That’s a videocamera." That’s what the visual cortexdoes. He came to me and said, "Look, wewe’ve grown a visual cortex in a vat."I’d be like, "That’s gross, and that’screepy, but okay." Um, and if I said toyou, "Yeah, but I’m not worried thatthe visual cortex that you’ve grown isnot going to create the T-1000,because what it does is, it’s a bunch of neurons that dopattern recognition, which is cool, butit has no world model, no state, noactuation, no intention, orMemories and all the other differentprocesses you would need to have a fully autonomous type of intelligentsystem, right? And you could argue back, well, hey, youknow, it’s the same neurons that are inthe brain. I was like, sure, I don’tcare about how you implemented it. What did youimplement? You implemented a visualcortex. That’s not enough on its ownif I’m worried about autonomousaction against my interests. That’s theway that we’re arguing about LLMs.Forget the fact that they’reactually implemented as tables ofnumbers that GPUs multiply by each otherto try to simulate the convolution ofinputs with various levels oftransformers and neural networks. Themathematical details I don’t care about;the things they actually do, theprocesses they implement, whether they’reimplemented by tinker toys or GPUs,cannot by themselves be enough tobe an intelligent being. There’s a lotof sloppy thinking happeningout there, like we don’t really knowwhat goes on, so who knows what thiscould evolve into. We do know what’sgoing on. Computer scientists know. What's going on? We know what theprocesses are that are inside of theselanguage models and what they do. Andjust like a visual cortex cannot buildit doesn't have any of the stuff youwould need to build a terminator,just like my sundial is not going tocome alive and my calculator isincapable of ordering stuff from Amazon.We know what the things are. So anyways, my arguments are aboutthe processes that are implemented, notthe substrate. I don't care about thesubstrate. All right, the next comment comesfrom Fiddle Studio. He says, "I honestlydid not think Yudkowsky aimed the point of Cal, get off my lawn." Uh, I think hewas trying to say that people didn'tthink it could develop so quickly, butit did. It's likely to develop furtherand also quickly and become super. Thelawn stuff was a poor way to say it, butthe point is slightly different and Calmissed it in his defensiveness. Whatabout Hinton? What about Bengio? Weirdto say that no one in the field believesAI is going to become an even biggerproblem than it is now. Now, a coupleof people brought this up. They had other interpretations for what Yudkowsky meant. When he was saying, you know, "Hey, getoff my lawn." I think the mostcommon counter-interpretation was that he wassaying, uh, we're bad at predicting whathappens in the future, and so we, youknow, um, I think you're bad atpredicting what, you know, you miss;you don't know. You're bad at predicting, so your predictions don'tmean that. That's the most common other interpretation I saw.I think that's largely what FiddleStudio was saying here. But let's goback and read the actual quote. I wantto go back. I printed it out here. Allright. So, uh, Ezra goes on basicallysummarizing the argument that Imade in my August New Yorker article, andGary Marcus and a few others made aboutthe scaling laws hitting a plateau.We're already seeing a flatteningout of what's capable, uh, with languagemodels, and we have no reasonto expect, you know, we don't knowhow this technology is going to makemajor advances in its capabilities. So he goes through all of that, and thenEzra finishes by saying, uh, he'ssummarizing this argument here. Hefinishes by saying, "So we are going toGet things that are incredible:enterprise software that is morepowerful than what we've had before. Butwe are dealing with an advance on thescale of the internet, not on the scaleof creating an alien superintelligencethat will completely reshape the knownworld. What would you say to them? Andthat is sort of my argument. Infact, I would say it might not even be aspowerful as the internet. All right, soEzra has just laid out my argument:what we cando with language model technology isalready starting to flatten out. We'renowhere near superintelligence. Here'sword for word Yakowsky's response: I hadto tell these Johnny-come-lately kids toget off my lawn. I first started to getreally, really worried about this in2003. Never mind large language models. Never mind AlphaGo or AlphaZero. Deeplearning was not a thing in 2003. Yourleading AI methods were not neuralnetworks. Nobody could train neuralnetworks effectively more than a fewlayers deep because of the exploding andvanishing gradients problem. That's whatthe world looked like back when I firstsaid, "uh oh, superintelligence is coming."I will admit it's a weird answer, butagain, Ezra says, "What would you say topeople that say that LLM technology isnot going to get much more powerful?"His answer was, "I had to tell theseJohnny completely to get off my lawn."And then he emphasized how long agohe started worrying about this. So allof this is him emphasizing how long agohe cared about this, before like most ofthis technology was even around. I don't know how else to interpret thatother than him saying, "I have earned aright to talk about this, and if youweren't on this issue early,you don't get in my lane. I earned aright to own this lane because I wastalking about this before people nowbecause language models impress people.A lot more people are interested,but I was here first, so I get to dictatethe discussion, and that seems to me likethe most likely interpretation here inmy newsletter. Here's the way I thoughtabout it. This makes sense if you thinkabout this as a bit of a religiousconversion. You have the rationalistworking on existential threatsthroughout the 2000s, thinking about AIand AI superintelligence. Bostonramm, at his existential threatinstitute at Oxford, writes the bookSuperintelligence with a lot ofconsulting from Yakowski. All thoughtexperiment stuff, right? They alsoworry about comets hitting the earth.They also worry about viral pandemics.Like this is thought experiments. LMScome along and suddenly they have thiskey shift, and to me, this is what I callthe religious spark in the existentialthreat rationalist movement. They switchfrom this is a thought experiment tolike what if this is actually happening? Right, because this technology seemedreally impressive. What if this isactually happening? If that's the case,I'm John Connor from the Terminatorseries. Like, I’m the most importantprecient person in the world. I mean,how could you not take that?Psychologically, it's very hard toresist that pull. That switch from ifthe win makes you the hero of the world.Of course, you're going to make thatswitch. But then things becomereligious. And when you're religious,what matters is righteousness.So, here's my interpretationof this. It's what I said in myNewsletter. Saying you can't talk aboutthis because you weren't here early issaying, "I'm a more righteous believer inthis God than you were." I have beenpraying to this God much longer thanyou. I get to be the high priest, notyou. Get off the steps of the temple. I'm the one who's doing the proverbialsacrifices. I think it's a religiousimpulse that we're seeing here.Righteousness, not truth. And that iswhy he got so upset is because he feelslike he was, you know, Aram in thedesert making the initial burntsacrifices to Yahwa when everyone elsewas still at, you know, uh, still inSumer. He's earned a right to be thethe prophet of God. Not that Aramwas a prophet, but patriarch; but thereyou go. So that's I think it's actuallyeven worse. It's a religious impulse.That's a righteousness impulse. You'renot allowed to talk about this becausewhere were you? Where were you when, like, I was theone who identified this God as one toworship? I really think it's a religiousimpulse that some of the rationalistswho are settling existential risk haveand maybe not. I will admit it's aWord salad, but well, I'll admit it'snot clear, but I think to me that's theclearest interpretation. All right, wegot a couple of funny comments here. JesseI like this one from Emanuel Revardtalking about the whole debate about theget off my lawn quote. He said, "Myreaction was, this guy has a lawn." Thatwas pretty funny. A couple of nicecomments just to kind of bring theblood pressure down. Life by Vic6751said, "The only podcast by a computerscience professor that I trust andcherish." Appreciate that. And Deep Dabblersaid, "Thanks for being a voice ofsanity and rationality in these times ofhyperbole, Professor Newport." There wego. Some nice ones at the end. Hey, ifyou like this video, I think you'llreally like this one as well., Is Superintelligence Super Dumb? | Cal Newport, YouTube, 01kah6z3x77p5vm2cfvn9k3b37.