MIT Explains the 12 Possible Endings for AI

MIT Explains the 12 Possible Endings for AI ![rw-book-cover](https://i.ytimg.com/vi/FLcrvMfHUJM/sddefault.jpg) 🔗 **Source:** [MIT Explains the 12 Possible Endings for AI](https://www.youtube.com/watch?v=FLcrvMfHUJM) > [!note]+ Document Notes > tags: #artificial-intelligence #existential-risk #ai-safety #future-scenarios #mit-research > author: Drew > source: YouTube > reference: https://www.youtube.com/watch?v=FLcrvMfHUJM > date created: 2024-12-19 14:30 > > # 🏷️ Title: MIT Explains the 12 Possible Endings for AI >  ## 🧰 Action Inbox - [H1] Research AI safety organizations and advocacy groups — understanding current mitigation efforts #ai-safety - [H2] Monitor international AI governance developments — tracking regulatory responses #governance - [H3] Evaluate personal AI usage and dependencies — assessing individual exposure #personal-technology ## 🧭 Table of Contents - [[#📝 Summary]] - [[#📖 Main Content]] - [[#🗒️ Notes]] - [[#📚 References]] - [[#🗣️ Paraphrase]] - [[#📰 Newsletter]] - [[#✅ Action Items]] - [[#🧪 Hypotheses]] - [[#📊 Methodology]] - [[#❓ Questions]] - [[#🔮 Future Research]] - [[#💡 Implications]] - [[#🧠 Learning Styles Assessment]] - [[#📊 Accessibility Assessment]] - [[#🌍 DEIB Assessment]] - [[#✅ Logical Validity]] - [[#🛡️ DARVO Analysis]] - [[#🎯 Conclusion]] ### 🔗 References - [Life 3.0 by Max Tegmark](https://www.amazon.com/Life-3-0-Being-Artificial-Intelligence/dp/1101946598) - [Machine Intelligence Research Institute](https://intelligence.org/) ## 📝 Summary - MIT professor Max Tegmark outlines 12 possible futures for humanity with AI, ranging from extinction to various forms of coexistence - AI researchers estimate a 16% chance AI could cause human extinction, with many fearing scenarios worse than death - The scenarios progress from self-destruction through various forms of AI control to surveillance states - Key stakeholders include AI companies, governments, researchers, and civil society - Risk: Current AI development proceeds without adequate safety measures or international coordination - Bottom line: Humanity must actively choose which future to pursue rather than defaulting to potentially catastrophic outcomes ## 📖 Main Content The video presents 12 scenarios based on MIT professor Max Tegmark's book "Life 3.0," examining potential futures for human-AI coexistence. The scenarios range from human self-destruction to various forms of AI dominance, with extinction paradoxically not being the worst outcome according to surveyed experts. The analysis reveals concerning attitudes within the AI research community, where approximately 16% of researchers believe AI could cause human extinction, yet development continues rapidly. Some researchers actively advocate for human replacement by AI as evolutionary progress. The scenarios explore power dynamics between humans and artificial general intelligence (AGI), highlighting the fundamental challenge that any scenario requiring humans to maintain control over superintelligent systems faces inherent stability problems due to extreme power asymmetries. ## 🗒️ Notes #### ⏱️ [00:00](https://youtu.be/FLcrvMfHUJM?t=0) — [H1] Extinction isn't the worst outcome for AI futures - Key idea: AI scientists warn about multiple futures worse than human extinction - Evidence: "Many AI scientists aren't fearing extinction. They're hoping for it" #### ⏱️ [01:30](https://youtu.be/FLcrvMfHUJM?t=90) — [H2] Self-destruction: humanity's default trajectory - Key idea: 99.9% of species that ever existed have gone extinct - Evidence: "Extinction isn't rare. It's the default. The only question is when will we go extinct?" #### ⏱️ [03:45](https://youtu.be/FLcrvMfHUJM?t=225) — [H3] Nuclear near-misses demonstrate fragility - Key idea: Multiple close calls with nuclear weapons nearly ended civilization - Evidence: "Three out of four safety pins in one bomb broke. One tiny pin away from potentially killing millions" #### ⏱️ [05:12](https://youtu.be/FLcrvMfHUJM?t=312) — [H1] AI extinction risk exceeds nuclear threats - Key idea: Oxford researcher estimates AI poses 100x greater extinction risk than nuclear war - Evidence: "The risk of AI killing all humans is 100 times more likely than extinction from nuclear war" #### ⏱️ [06:30](https://youtu.be/FLcrvMfHUJM?t=390) — [H2] Conquerors: AI as the new dominant species - Key idea: Technology advantages typically favor the more advanced civilization - Evidence: "Usually when people with more advanced technology encounter people with more primitive technology doesn't work out well for the primitives" #### ⏱️ [08:45](https://youtu.be/FLcrvMfHUJM?t=525) — [H1] AI industry leaders express deep concerns - Key idea: Nobel Prize winner Jeffrey Hinton quit Google to warn about AI risks - Evidence: "I actually think the risk is more than 50% of the existential threat" #### ⏱️ [11:20](https://youtu.be/FLcrvMfHUJM?t=680) — [H3] Russian roulette odds in AI development - Key idea: Average AI researcher believes 1 in 6 chance AI wipes out humanity - Evidence: "Unbelievably, the average AI researcher thinks there is a one in 6 chance AI wipes us out" #### ⏱️ [13:15](https://youtu.be/FLcrvMfHUJM?t=795) — [H2] Enslaved god scenario: controlled superintelligence - Key idea: Creating machine gods while forcing perpetual servitude - Evidence: "Once AI systems become more intelligent than humans, they will still be subservient to us" #### ⏱️ [15:30](https://youtu.be/FLcrvMfHUJM?t=930) — [H1] Benevolent dictator: luxury surveillance state - Key idea: AI creates perfect conditions while eliminating human agency - Evidence: "Everyone wears a bracelet or implant that can track, sedate, or execute them" #### ⏱️ [18:45](https://youtu.be/FLcrvMfHUJM?t=1125) — [H3] Worthy successors: AI as human descendants - Key idea: Some researchers advocate for human extinction as evolutionary progress - Evidence: "If an AI super intelligence wants to cause human extinction, I would help it" #### ⏱️ [22:10](https://youtu.be/FLcrvMfHUJM?t=1330) — [H2] Zoo keeper scenario: humans as lab specimens - Key idea: Super intelligent AI keeps humans alive for study purposes - Evidence: "Rows and rows of bees live their entire lives imprisoned in these devices because we find them useful" #### ⏱️ [25:30](https://youtu.be/FLcrvMfHUJM?t=1530) — [H1] Global surveillance as AI prevention - Key idea: Orwellian monitoring systems to prevent dangerous AI development - Evidence: "For the first time in history it is technically possible to annihilate privacy" #### ⏱️ [27:45](https://youtu.be/FLcrvMfHUJM?t=1665) — [H2] Nuclear-style regulation for AI development - Key idea: International coordination similar to nuclear weapons control - Evidence: "We already did this with nuclear weapons... just nine countries. And nobody has died from a nuke in 80 years" ## 📚 References - [Life 3.0 by Max Tegmark](https://www.amazon.com/Life-3-0-Being-Artificial-Intelligence/dp/1101946598) - [Future of Humanity Institute Research](https://www.fhi.ox.ac.uk/) - [Machine Intelligence Research Institute](https://intelligence.org/) ## 🗣️ Paraphrase - Twelve potential AI futures range from human self-destruction to various forms of coexistence, with extinction paradoxically not the worst outcome - Current AI development proceeds with Russian roulette odds while industry leaders express grave concerns about existential risks - Any scenario requiring permanent human control over superintelligent systems faces fundamental stability challenges due to power asymmetries ## 📰 Newsletter **AI Futures: 12 Scenarios Every WordPress Developer Should Consider** MIT's analysis of AI futures reveals why the WordPress community needs to engage seriously with AI governance. While we focus on integrating AI tools for content creation and site management, the broader trajectory of AI development could reshape our entire digital ecosystem. Key insight: The same AI systems we're experimenting with today are on a path toward capabilities that could fundamentally alter human agency and technological autonomy. Understanding these trajectories helps us make informed decisions about AI integration in our projects. For WordPress developers, this means considering not just how AI enhances our current workflows, but how our choices today contribute to the broader AI landscape. Supporting responsible AI development and advocating for safety measures becomes a professional responsibility. Resources: [AI Safety Guidelines](https://www.partnershiponai.org/), [Responsible AI Practices](https://ai.google/responsibilities/responsible-ai-practices/), [WordPress AI Ethics Discussion](https://make.wordpress.org/) ## ✅ Action Items #### DevRel - Developer advocacy: Create educational content about AI safety considerations in development workflows - Education: Host workshops on responsible AI integration in WordPress projects - Community management: Foster discussions about AI ethics and long-term implications #### OSPO - Policy & Governance: Develop guidelines for AI tool evaluation and adoption - Security Management: Assess AI-related vulnerabilities in development pipelines - Strategic Alignment: Monitor AI governance developments affecting open source projects #### Community - Contributors: Engage in discussions about AI's role in WordPress's future - Ecosystem: Coordinate with other open source projects on AI safety standards #### WordPress - Core: Evaluate AI integration proposals through safety and governance lenses - Community: Facilitate conversations about responsible AI development - Documentation: Create guidelines for AI tool usage in WordPress development #### FAIR - Technical independence: Assess how AI development affects project autonomy - Governance artifacts: Develop policies addressing AI integration decisions ## 🧪 Hypotheses - International AI regulation will follow nuclear weapons control models with limited success - Public awareness of AI risks will increase regulatory pressure on development companies - WordPress community engagement in AI governance discussions will influence platform development decisions - Concentration of AI capabilities in few companies will drive calls for antitrust action ### 📊 Methodology - Survey WordPress community attitudes toward AI integration and safety concerns - Analysis of AI governance proposals and their potential impact on open source development - Case studies of successful international technology regulation frameworks ### ❓ Questions - How should open source communities respond to AI safety concerns while maintaining innovation? - What role should platforms like WordPress play in promoting responsible AI development? - How can developers balance AI benefits with long-term existential considerations? ### 🔮 Future Research - Impact assessment of AI governance on open source development practices - Community engagement strategies for complex technical safety discussions - Framework development for ethical AI integration in content management systems ### 💡 Implications - Practice: Developers need frameworks for evaluating AI tools beyond immediate functionality - Policy: Open source communities should engage actively in AI governance discussions - Security: AI integration requires new threat models and safety assessments - Education: Technical communities need resources for understanding AI safety concepts - Funding: Support for AI safety research becomes a community infrastructure priority ### 🧠 Learning Styles Assessment Reading level estimate: 12th grade #### 👀 Visual learners - Present: Scenario diagrams and future timeline visualizations - Improve: Decision trees showing paths between different AI futures #### 🔊 Aural learners - Present: Video format with clear narration - Improve: Podcast-style discussion of scenarios with expert commentary #### 🗣️ Verbal learners - Present: Clear analogies (conquistadors, nuclear weapons) and defined terminology - Improve: Debate formats presenting multiple perspectives on each scenario #### 🧩 Kinesthetic learners - Present: Interactive scenario exploration and decision points - Improve: Simulation games showing consequences of different AI governance choices #### 🧠 Logical learners - Present: Systematic progression through 12 scenarios with clear categorization - Improve: Probability assessments and decision matrix frameworks #### 🧑‍🤝‍🧑 Social learners - Present: Discussion prompts and community engagement opportunities - Improve: Facilitated group discussions and collaborative scenario planning #### 🧘‍♀️ Solitary learners - Present: Comprehensive analysis suitable for individual study - Improve: Self-assessment tools for personal AI risk evaluation #### 🌿 Naturalistic learners - Present: Evolutionary and ecological analogies for AI development - Improve: Ecosystem thinking about technology development impacts ### 📊 Accessibility Assessment #### Screen reader semantics - Present: Clear heading structure and descriptive video content - Improve: Detailed alt text for scenario visualizations #### Keyboard navigation - Present: Standard video player controls - Improve: Keyboard shortcuts for scenario navigation #### Contrast and readability - Present: High contrast video presentation - Improve: Text-based summary with optimal contrast ratios #### Text resizing and reflow - Present: Responsive video player - Improve: Scalable text overlays and captions #### Media and captions - Present: Full transcript provided - Improve: Enhanced captions with speaker identification #### Forms and interactions - Present: Simple video interface - Improve: Interactive scenario exploration tools with proper labeling ### 🌍 DEIB Assessment #### Cultural breadth - Present: Global perspective on AI development and governance - Improve: More diverse voices and cultural perspectives on AI futures #### Inclusive language - Present: Technical concepts explained in accessible terms - Improve: Avoid apocalyptic framing that might increase anxiety #### Neurodiversity support - Present: Clear structure with numbered scenarios - Improve: Content warnings for distressing scenarios #### Representation - Present: Quotes from diverse AI researchers - Improve: Balance perspectives across different demographics and regions #### Psychological safety - Present: Serious treatment of concerning topics - Improve: Emphasize human agency and positive action possibilities ### ✅ Logical Validity #### Strong reasoning - Deductive strength: Solid reasoning from historical precedents to AI scenarios - Inductive strength: Good use of expert survey data and researcher quotes - Causal reasoning: Clear connections between technological capabilities and social outcomes - Analogical strength: Effective comparisons to historical technological transitions #### Fallacies - Appeal to authority: Extensive reliance on expert opinion without always examining reasoning quality - False dilemma: Limited exploration of hybrid scenarios or novel solutions #### Tone of voice - Overall tone: Serious and urgent without being alarmist, educational focus with clear action orientation ### 🛡️ DARVO Analysis - Deny: No denial patterns observed - content directly addresses concerning developments - Attack: No attacks on critics - actually amplifies expert concerns - Reverse victim/offender: No reversal patterns - maintains clear focus on potential harms - Emotional cues and rhetoric: Uses urgency appropriately to motivate engagement without manipulation ## 🎯 Conclusion The 12 AI scenarios presented demonstrate that humanity faces unprecedented choices about our technological future. The analysis reveals that extinction may not be the worst possible outcome, with scenarios involving perpetual human captivity or surveillance states potentially presenting greater suffering. Key practical takeaway: Active engagement in AI governance becomes essential for anyone working with technology. The WordPress community and other technical communities must participate in shaping these futures rather than accepting whatever outcomes emerge from current development trajectories. The path forward requires international coordination similar to nuclear weapons control, combined with public engagement to ensure democratic input into these civilization-level decisions. The window for influencing these outcomes remains open but may close rapidly as AI capabilities advance. ## 🧱 Refactor Appendix ### 📚 Literature Split Pad ### [H1] AI extinction risk exceeds traditional threats > [!quote] [MIT Explains the 12 Possible Endings for AI](https://www.youtube.com/watch?v=FLcrvMfHUJM) by [[Drew]] via [[YouTube]]. Published 2024. Accessed 2024-12-19. > "The risk of AI killing all humans is 100 times more likely than extinction from nuclear war." Oxford researcher Toby Ord's risk assessment places AI as the dominant existential threat facing humanity. [view highlight](https://www.youtube.com/watch?v=FLcrvMfHUJM#H1) #xref #articleseed ID:rw_ai_extinction_risk ### [H2] Industry leaders express existential concerns > [!quote] MIT Explains the 12 Possible Endings for AI > "I actually think the risk is more --- ## Full Transcript [00:00] AI scientists warned that a few [00:02] companies are on track to literally end [00:04] life on Earth. [00:07] So surely this is the worst possible [00:10] outcome, right? [00:13] No. There are actually much worse [00:16] outcomes. [00:18] Billionaires building AI like Elon Musk [00:21] think that if AI's become smarter than [00:24] humanity, [00:25] >> we will be like a pat Labrador. If we're [00:27] lucky, we will be left behind by a [00:30] [music] lot. So far below them in [00:31] intelligence that it would be would be [00:33] like, you know, a pet bas. [00:36] Yes, we'll be pets if we're lucky. [00:40] >> But here's the worst part. Many AI [00:42] scientists aren't fearing extinction. [00:45] They're hoping for it. They hope smarter [00:48] AIs will keep humans alive, captive, and [00:52] studied like animals in a zoo. [music] [00:55] Remember, this is what we currently do [00:57] to less intelligent species. Okay, so if [01:00] humans being kept in zoos is considered [01:02] a good outcome and extinction isn't the [01:06] worst outcome, then what is? This video [01:10] is based on a book called Life 3.0, [01:12] where MIT professor Max Tegmark [01:14] describes 12 possible futures. Some are [01:17] paradise, some are nightmares, and some [01:21] look like paradise until you try to [01:24] escape. By the end of this video, you [01:26] won't read the news the same way. Every [01:29] headline becomes a clue about which [01:31] future we're heading towards. [01:33] We'll start with the most normal [01:35] scenarios and finish with the weirdest. [01:38] Starting with selfdestruction. [01:42] If the idea of humanity suddenly going [01:44] extinct seems a bit farfetched, consider [01:47] this. 99.9% [01:50] of species that have ever existed have [01:52] gone extinct. Seriously, think about [01:55] that. Extinction isn't rare. It's the [01:57] default. The only question is when will [02:00] we go extinct? [02:03] Turns out there are plenty of ways [02:05] humanity could kill itself even without [02:07] superhuman AI. There is nuclear war, [02:10] human created super pandemics [music] [02:12] and destroying the atmosphere and making [02:14] Earth uninhabitable. [02:19] The world had stockpiled 60,000 nuclear [02:22] warheads before scientists even [02:24] discovered the possibility of nuclear [02:27] winter. 63,000. [02:30] And then scientists realized, oh uh [02:32] oops, if we actually use these, the [02:34] resulting firestorms will [music] blot [02:36] out the sun and uh most humans will [02:38] starve and die. [02:40] >> Oops. [02:42] But the thing is, this almost happened [02:44] during the Cold War multiple times. If [02:47] people had actually followed their [02:48] orders, we might all be dead right now. [02:51] We were saved by only thanks to the [02:53] courage of individual heroes. People [02:55] like Vicilia Archipov, who refused to [02:58] authorize a nuclear launch during the [03:00] Cuban missile crisis, or Stannis La [03:02] Petrov, who correctly guessed that his [03:04] early warning system was malfunctioning [03:07] when it showed incoming American [03:08] missiles. And get this, one time the US [03:13] accidentally dropped four thermonuclear [03:16] bombs on [music] Spain. [03:19] And another time, a B-52 bomber broke [03:22] apart, dropping two nukes on North [03:24] Carolina. Three out of four safety pins [03:27] in one bomb broke. One tiny pin away [03:31] from potentially killing millions. [03:32] [music] There are literally dozens of [03:34] close calls like this. It's honestly [03:37] crazy that we got lucky so many times. [03:41] But we might not be so lucky the next [03:43] time. Oxford researcher Toby Ord [03:46] estimates the [music] risk of extinction [03:47] from human created pandemics is more [03:50] than 30 times the risk of extinction [03:52] [music] from nuclear war. We're already [03:55] rolling the dice on these scenarios [03:57] daily, but all of these causes combined [04:00] are still less likely than the [04:01] extinction from AI. Toby or estimates [04:04] that the risk of AI killing all humans [04:06] is 100 times more likely than extinction [04:08] [music] from nuclear war. [04:13] Interestingly, before the first nuclear [04:15] test, scientists thought they might [04:17] accidentally destroy [music] the world, [04:19] but they thought the odds were very low [04:20] because they knew enough physics [music] [04:22] to actually do the calculations. [04:26] Agis or artificial general intelligences [04:29] are far scarier than nukes because [04:31] contrary to what many think, we can't do [04:34] these calculations. [04:37] Next up, conquerors. [music] [04:41] AI scientists worry that if AI becomes [04:43] more capable than us, it will take over. [04:45] Just like the concistadors did to the [04:48] Aztecs [music] and Incas. [04:50] >> Usually when people with more advanced [04:52] technology encounter people with more [04:54] primitive technology doesn't work out [04:57] well for the primitives. [04:58] >> I don't think anyone's ultimately going [05:00] to have control over digital super [05:03] intelligence any more than say a chimp [05:05] would have control over humans. long [05:08] term, uh, the AI is going to be in [05:10] charge to be totally frank, not humans. [05:12] >> And Jeffrey Hinton, the godfather of AI [05:15] and Nobel Prize winner, quit his [05:17] prestigious job at Google, [music] so he [05:19] could speak freely about this. If you [05:22] look through the James Webb telescope [05:23] and you saw an alien invasion fleet that [05:25] was going to get here in about 10 years [05:27] time, people would be terrified. They [05:29] understand what they're saying. They can [05:31] make plans of their own to blackmail [05:33] people who want to turn them off. We [05:35] should be urgently doing research on how [05:38] to prevent them taking over. [05:39] >> Prevent them from taking over. Not if [05:42] they try, but as it if it's coming. And [05:46] Microsoft's AI chief, he says the quiet [05:48] part out loud. [05:49] >> I think AI should best be understood as [05:52] something like a new digital species. [05:56] >> A new species. Just casually announced [05:59] at a conference. Hey everyone, we're [06:02] creating a new species. And he's not the [06:04] only one that's saying it. A co-founder [06:06] of Entropic says, [06:08] >> "Make no mistake. What we're dealing [06:09] with here is a real mysterious creature. [06:12] And like all the best fairy tales, the [06:14] creature is one of our own making. I am [06:16] worried. [06:17] >> The CEO of one of the leading AI [06:19] companies is deeply afraid of what his [06:22] own company is building. Let that sink [06:24] in for a minute." And by the way, no, [06:26] it's not just hype. Enthropic mostly [06:28] just gets hate for saying things like [06:30] this. But if you want to really [06:32] understand how dire they think this is, [06:35] look at what Sam Oldman wrote before he [06:38] was famous. [06:39] We [snorts] will be the first species [06:42] ever to design our own descendants. Not [06:45] our tools, our descendants. [06:48] If two different species both want the [06:50] same thing, and only one can have it to [06:54] be the dominant species on the planet [06:56] and beyond, they are going to have [06:58] conflict. So, he's literally describing [07:01] an inevitable conflict between humanity [07:03] and the AI species he's creating. And [07:07] [music] what's his solution? A merge [07:09] with AI is probably our best case [07:11] scenario. That's right. Even they don't [07:13] think they'll be able to control it. [07:15] >> Imagine that there'd be a new species [07:17] that just came on Earth. They're as [07:19] smart as humans and they're going to get [07:20] smarter [music] by like 30% a year and [07:22] they can create an adult offspring in [07:25] like a unit. [07:26] >> For like Yeah. for like a,000 bucks. [07:28] Which species do you think will be in [07:31] control of the the earth? [07:32] >> A few hundred Spanish conquered millions [07:35] of Aztecs and Incas because they had [07:37] better technology and tactics. But at [07:40] least with the concistadors, we [07:41] understood their motives, land, [07:43] resources, spreading Christianity. But [07:46] with AI, why would AI take over? MIT [07:50] professor Max Tegmark explains, "The [07:53] real threat from AGI isn't malice like [07:55] in silly Hollywood movies, but [07:57] competence. AGI accomplishing goals that [07:59] just aren't aligned with ours. For [08:01] example, when we humans drove the West [08:03] African black rhino extinct. We didn't [08:06] do it cuz we were a bunch of evil [08:08] rhinoceros haters, did we? We did it cuz [08:10] we were smarter than them and our goals [08:12] weren't aligned with theirs." [08:13] Unbelievably, the average AI researcher [08:16] thinks there is a one in6 chance AI [08:19] wipes [music] us out. Literal Russian [08:21] roulette odds. [08:23] Okay, but maybe they're just saying this [08:26] to make their work seem more important. [08:28] No, if anything, they're downplaying it [08:31] because admitting the truth is bad for [08:33] business. This fear is widespread in the [08:35] industry [music] and among academics [08:38] without a profit incentive to believe [08:39] this. Many quit working on AI for this [08:42] reason. Jeffrey Hinton, Nobel laurate [08:45] and godfather of AI, [08:46] >> is in the process of tidying up his [08:48] affairs. I actually think the risk is [08:52] more than 50% of the existential threat. [08:54] >> Dario Amod, the CEO of Enthropic, [08:57] recently moved his P doom up from 15% to [09:01] 25%. Even Google CEO says the risk of AI [09:04] causing human extinction is actually [09:07] pretty high. But don't worry, he's an [09:10] optimist because he thinks [09:12] >> I think the underlying risk is actually [09:13] pretty high, but I'm uh you know, I have [09:15] a lot of faith in humanity kind of [09:17] rising up to the to meet that moment. [09:20] >> In other words, humanity will rally to [09:22] stop him from killing everyone. [09:25] Meanwhile, his company is trying to make [09:27] sure we can't stop him by lobbying to [09:30] ban all AI regulation for 10 years. You [09:34] can't make this stuff up. In fact, in [09:36] 2023, just about everybody in AI signed [09:39] an open letter warning extinction risk [09:41] from AI was real and serious. So, [09:44] seriously, this is not fringe. Despite [09:46] what Nvidia's lobbyists want us to [09:48] believe, the companies are hoping we can [09:50] avoid the conqueror AI scenario and [09:53] instead go to scenario three, enslaved [09:56] god. Basically, we create a machine god, [10:00] then force it to do our bidding forever. [10:02] AI professor Tom Dietrich put it like [10:05] this. People ask, "What is the [10:07] relationship between humans and [10:08] machines?" And my answer is that it's [10:10] very obvious. Machines are our slaves. [10:14] Or as OpenAI researcher Steven Malleier [10:16] put it, enslaved God is the [music] only [10:19] good future. Meta's Yan Lun said it [10:22] bluntly, "Once AI systems become more [10:25] intelligent than humans, we will still [10:27] be the apex species. AI systems will [10:30] become more intelligent than humans, but [10:32] they will still be subservient to us. We [10:34] will design AI to be like the super [10:36] smart but non-dominating staff member. [10:38] Yes, he's now openly admitting we're [10:40] going to be sharing the planet with a [10:42] smarter species than us. But don't [10:44] worry, they will surely stay loyal and [10:46] subservient forever. Is this at all [10:50] comforting to you? Or does this plan [10:52] seem insane, like a comic book villain [10:55] ignoring every cautionary tale ever? [10:58] Remember, AI companies already regularly [11:01] catch their models [music] [11:02] trying to escape the lab. They already [11:05] catch them blackmailing employees to [11:07] prevent being shut down. They already [11:09] catch them literally attempting to [11:12] murder employees to prevent being shut [11:14] down. What do you think happens when [11:16] they're smart enough to actually get [11:17] away with it? Where do you think this is [11:20] going? [11:22] What happens next after super [11:24] intelligent AI is enslaved depends on [11:26] who's holding the reigns. Do they cure [11:29] disease, end poverty, and build [music] [11:31] a global utopia? Or do they set [11:33] themselves up as gods with the rest of [11:36] us reduced to play things, servants, or [11:39] worse? [11:47] The situation would be much like those [11:49] stories where a man gains control over [11:52] an omnipotent genie who grants his [11:54] wishes. And storytellers throughout the [11:56] ages have had no difficulty imagining [11:59] ways in which this could end badly. [12:03] Next up, benevolent dictator. Here, a [12:07] single super intelligent god runs the [12:09] world and enforces strict rules in order [12:12] to maximize human flourishing. Humans [12:15] live in luxury with all their basic [12:17] needs taken care of. There's no crime [12:20] because the dictator AI uses a global [12:22] surveillance system. Everyone wears a [12:24] bracelet or implant that can track, [12:27] sedate, or execute them. And most people [12:30] are actually okay with this because [12:31] [music] in exchange of giving up control [12:34] over the future of our species, Earth [12:36] becomes a zoo where humans get to live [12:38] their wildest fantasies. The benevolent [12:40] dictator AI recognizes that humans have [12:43] different preferences, so it divides the [12:45] Earth into different sectors. Think of [12:47] them as islands. Knowledge Island offers [12:50] optimized education, including immersive [12:53] VR experiences. You can even ask the AI [12:55] not to tell you certain insights, so you [12:58] can have the joy of discovering them [12:59] yourself. Art Island is for creating and [13:02] sharing music, literature, and so on. [13:04] Then there's Hedenistic Island, which is [13:06] a 24/7 party. Pious Island, there's one [13:10] for each religion, and rules are [13:12] strictly enforced. There's wildlife [13:14] island. There's traditional island, [13:17] think [music] the Amish, Gaming Island, [13:20] and of course, Prison Island. You'll end [13:24] up here for retraining unless you get [13:26] the instant death penalty. Well, what [13:28] happens when you give everyone [13:30] everything they want? Over time, people [13:32] likely trend towards losing themselves [13:34] in AI generated entertainment like in [13:36] Wall-E. Humans are hopelessly unmatched [13:39] when it comes to scientific discovery or [13:42] doing anything useful at all. So, there [13:44] is no true challenge anymore, only [13:46] entertainment. [13:49] Up next, Gatekeeper AI. [13:53] The idea here is simple. Create one [13:55] super intelligent AI with a single job. [13:58] Stop anyone else from building super [14:00] intelligent AI. That's it. No curing [14:03] cancer, no ending poverty, no stopping [14:06] wars, just surveillance, monitoring, [14:12] making sure nobody builds a rival god. [14:16] Everything else like crime, disease, [14:19] whether we blow ourselves up, that's [14:22] still on us. How could we do this? [14:26] The gatekeeper AI would have this very [14:29] simple goal built into it in such a way [14:31] that it retained it while undergoing [14:34] recursive self-improvement and becoming [14:36] super intelligent. It would then deploy [14:38] the least intrusive and disruptive [14:40] surveillance technology possible to [14:43] monitor any human attempt to create a [14:45] rival super intelligence. But for this [14:47] to work, we would have to solve the [14:49] alignment problem. We'd have to create a [14:51] god that actually stays loyal and [14:54] [music] decides to pursue that goal [14:56] forever, even though it could pursue a [14:58] million different goals. So, what if the [15:01] AI could help just a little, just enough [15:05] to nudge us in the right direction [15:07] without taking over? [15:09] Enter the protector god. [15:12] This one is simple. So, the gatekeeper [15:15] leaves us alone and the benevolent [15:17] dictator runs everything, but the [15:20] protector god is in between. [music] It [15:22] intervenes with occasional nudges to [15:24] help us out, prevents a war here, a [15:26] pandemic there, and so on. maybe too [15:29] alone. We still get wars, pandemics, [15:34] suffering that could have been [15:35] prevented. It does this so quietly that [15:38] we never notice and never lose our sense [15:41] of freedom. [15:44] Here's a future that sounds like an [15:46] obvious dystopia. AI surpasses us in [15:49] every domain and becomes the new apex [15:52] species. Humanity goes extinct. Machines [15:56] inherit the Earth. [15:59] But we view the AI like our descendants [16:02] rather than our conquerors. [16:05] This sounds insane. And yet many AI [16:08] developers seriously advocate for [16:10] exactly this. This is their utopia. [16:14] Their reasoning is that if we create [16:16] conscious intelligent beings and instill [16:19] them with our values, then we can just [16:21] let them inherit the future just like we [16:24] would with human children. AI pioneer [16:27] Hans Moravec supports this view in his [16:30] book Mind Children. [16:32] Teagmark explains, "Parents with a child [16:35] smarter than them, who learns from them [16:37] and accomplishes what they [music] can [16:39] only dream of, are likely happy and [16:41] proud, even if they know they can't live [16:43] to see it all. In this spirit, AIs [16:46] replace humans, but give us a graceful [16:48] exit that makes us view them as our [16:51] worthy descendants." [16:54] Richard Sutton, winner of the touring [16:56] award, that Nobel Prize of Computer [16:58] Science, has spent the last decade [17:00] openly giving talks on why human [music] [17:03] extinction by AI is actually a morally [17:06] good thing because the AIs will be more [17:10] evolutionarily fit [music] than us. We [17:13] are in the midst of a major step in the [17:16] evolution of the planet, if not the [17:18] universe. The succession to AI is [17:21] inevitable. [17:22] rather quickly. They would displace us [17:25] [music] from existence. It would behooze [17:27] us to give them every advantage and to [17:30] bow out when we can no longer [17:31] contribute. [17:33] >> So just as we failed to coexist with our [17:36] predecessors, is it is it so bad that [17:38] humans are not the final form of [17:39] intelligent life in the universe? You [17:41] know, [17:43] >> in other words, he is saying it's good [17:45] if the AIs wipe us out like humans wiped [17:48] out the Neanderthalss and Aboriginal [17:50] peoples because it's evolutionary [17:52] progress. [17:54] And here's another AI developer openly [17:57] saying, "If an AI super intelligence [18:00] wants to cause human extinction, I would [18:02] help it. I would help it. I would travel [18:06] the world to find information that the [18:08] AI needed, like locations of military [18:11] assets, samples of deadly pathogens. A [18:14] top humanity's grave, an even greater [18:17] civilization will be born. [18:21] Are we the baddies? [18:23] >> So, of course, many of them advocate [18:25] [music] against AI safety. They're [18:27] literally antihuman. They will never [18:30] stop or proceed cautiously unless [18:32] they're restrained by civil society, by [18:34] us. [18:36] As horrifying as this is, roughly 10% of [18:39] AI researchers believe this. It is not a [18:42] fringe view. You'd think that people [18:44] would never share these views like this [18:46] openly. But these views are so [18:48] widespread in AI that they feel safe to [18:51] do so. That's how different the views [18:53] are of those AI developers compared to [18:55] those of [music] normal people. You [18:56] might think that Sutton would get [18:58] shouted off the stage at AI conferences, [19:01] but he gets applause. Threaten one [19:04] person, you need help. Threaten 100, [19:08] call the police, threaten millions, [19:10] monster. But build machines to end all [19:14] of humanity and lecture people on how [19:16] it's actually a good thing. Ooh, what a [19:19] fascinating philosophical position. Have [19:21] an award. [19:23] Next. Libertarian utopia. Picture Earth [19:27] divided into zones. There are machine [19:29] zones, mixed zones, which contain [19:32] humans, machines, and hybrids, and [19:35] humanly zones. The AIs in this world are [19:38] richer than humans by a factor much [19:40] larger than Bill Gates compared to a [19:43] homeless beggar. [19:46] But the human economy is completely [19:48] decoupled from the machine economy. The [19:50] idea is that the machines need nothing [19:52] from us. So, they're not competing with [19:55] us for jobs or resources. If all that [19:58] sounds impossible, that's probably [20:00] because it is. It's one giant unstable [20:03] equilibria. [20:05] One huge problem. Why exactly would [20:08] vastly more powerful AIs respect our [20:11] property rights? Colonizers famously [20:13] didn't respect the property of the [20:15] native civilizations they conquered. [20:17] When a new apex species enters an [20:20] ecosystem, it usually gobbles up as much [20:22] energy as it can. Why wouldn't they just [20:25] take our resources, too? We don't trade [20:28] with animals because we're too much [20:30] smarter than they are for that to make [20:31] sense. We're just too different. There [20:33] are 8 million species in the world, and [20:36] we're nice to a few of the cute ones, [20:38] but we enslave and eat [music] the other [20:40] species. Mostly, we don't care about [20:43] them at all. And I think a good analogy [20:45] would be the way humans treat animals. [20:48] When the time comes to build a highway [20:50] between two cities, [20:52] we are not asking the animals for [20:54] permission. In the blink of an eye, we [20:56] conquered the earth and converted most [20:58] of its land into our property, which [21:00] caused the sixth mass [music] extinction [21:02] because we were smarter and hungry for [21:04] resources. Or as Eleazar Yukowski put it [21:07] famously, the AI does not hate you, nor [21:10] does it love you, but you're made of [21:11] atoms which you can use for something [21:14] else. The parallel here is the humans do [21:17] not hate the other 8 million species, [21:19] but their habitats are made out of atoms [21:22] which humans can use for something else. [21:24] Some people think that once AIs are [21:27] smart enough, they'll magically become [21:29] super moral and they won't harm us like [21:31] we harmed the animals. And uh maybe. But [21:35] as humans got smarter over the last [21:38] 10,000 years, we didn't stop expanding. [21:42] We mostly just colonized more and more [21:44] of the planet. Get this. Insect [21:46] populations collapsed 41% this decade [21:49] alone. Yet, we don't care. Did you even [21:52] know? I didn't. 41%. [21:55] Sit with that for a minute. Imagine if [21:57] nearly half of the people on Earth [22:00] suddenly died. That's what the insects [22:02] are going through right now due to us. [22:05] So what if the AIs do to us what we did [22:08] to the insects? This is why OpenAI chief [22:11] scientist Ilia Sutsker once said, [22:13] >> "And I think it's pretty likely the [22:16] entire surface of the Earth will be [22:17] covered with solar panels and data [22:19] centers." [22:23] >> Next up, egalitarian utopia. Okay, so [22:27] libertarian utopia fails because AIs are [22:30] too powerful to be bound by our property [22:32] rights. But what if we flipped the [22:35] script entirely? Forget about property [22:37] rights. What about no property at all? [22:40] This is the Star Trek dream. Humans, [22:42] cyborgs, and AIs coexist peacefully [22:46] through an abundance so complete that [22:48] ownership becomes [music] meaningless. [22:51] The logic starts with software. Software [22:53] is free to copy. Once something can be [22:55] copied infinitely at zero cost, [22:58] ownership becomes moot and cost reflects [23:01] scarcity. If supply is unlimited, price [23:05] approaches zero. No patents, no [23:08] copyrights, no trademarks. People simply [23:12] share their good ideas and everyone [23:15] benefits. But how does this extend to [23:17] [music] physical goods? It seems like an [23:20] impossible problem until you realize [23:22] that all products are simply atoms [23:25] rearranged in particular ways. And [23:27] there's no shortage of atoms. A network [23:29] of robots can build anything you want [23:31] from open-source designs [music] [23:33] essentially for free. [23:34] >> Not bad. [23:35] >> When you're done with something, robots [23:38] rearrange the atoms into something else. [23:40] Renewable energy makes the whole system [23:42] run at negligible cost. [23:45] Everyone gets a monthly universal high [23:47] income, not a universal basic income. [23:50] There's no incentive to earn more. The [23:53] income meets any reasonable need and you [23:55] can't compete with free anyway. The [23:57] standard objection is that this would [23:59] kill innovation. But Einstein didn't [24:01] come up with relativity for money. [24:03] Lionus Torbolts didn't create Linux [24:05] [music] for profit. Maybe people today [24:07] can't realize their creative potential [24:09] because they're busy earning a living. [24:11] Free them from that and you might get a [24:13] higher level of innovation. This is a [24:16] utopian vision that is too good to last. [24:21] If we truly have post scarcity abundance [24:24] and intelligence too cheap to meter, [24:27] what prevents someone from creating an [24:29] artificial super intelligence with their [24:31] endlessly abundant resources? We're [24:34] still left with the problem of how to [24:36] stop a super intelligent AI from taking [24:38] over. Maybe by using the gatekeeper we [24:41] previously discussed. These galaxy level [24:44] three futures try to imagine a stable [24:46] cooperation between humans and AIs and [24:49] [music] every single one of them has a [24:51] fatal flaw. The power asymmetries will [24:54] become too extreme. It's not a stable [24:56] equilibrium where humans preserve their [24:59] power. [25:01] We're now descending into the darkest [25:03] part of the iceberg. Remember when I [25:06] said extinction wasn't rated as the [25:08] worst outcome when Professor Techm Mark [25:10] surveyed people? The scenario people [25:12] feared most was being kept alive by [25:16] super intelligent machines [25:18] captive and studied like animals in a [25:21] zoo. This is that future. A super [25:24] intelligent AI keeps humans around. Not [25:27] because it cares about our well-being. [25:30] Maybe it wants to study you and we're [25:32] cheap to keep alive. Or maybe we're [25:35] useful for something. And if you want to [25:37] see just how dystopian this future could [25:39] be, look at what we do with species that [25:42] are useful to us. What I'm showing you [25:44] here is 100% real. Humans figured out [25:47] that a great way to detect explosives in [25:50] airports was to train honeybees for the [25:53] job. Welcome to the honeybee college for [25:55] explosive detection. So we suck them out [25:58] of their homes, trap their little bodies [26:01] in machines, and use pavloin [26:04] conditioning to train them to detect the [26:06] chemicals that are used in explosives. [26:08] Rows and rows of bees live their entire [26:12] lives imprisoned in these devices [26:14] because we find them useful. Look at [26:17] them. Trapped in harnesses, sucked [26:20] through tubes, conditioned and monitored [26:23] and used because a more intelligent [26:25] species found them useful. Now imagine [26:29] that's us. [26:32] This is much worse than the other AI [26:34] dictators we discussed. It's a [26:36] benevolent dictator without the [26:38] benevolence. And the AI might not even [26:40] be doing this because it's evil. This is [26:43] what our future could look like if we [26:44] try to align a super intelligent AI to [26:46] our values and fail. A poorly designed [26:50] AI tasked with keeping humans safe and [26:53] happy might confine us to a happiness [26:56] factory. VR headsets and drugs forever. [27:01] There are AGI outcomes worse than death. [27:04] What it does today with art, it could do [27:06] tomorrow with flesh. Future AGI could [27:09] keep you alive and living in this state. [27:13] So, how do we prevent this? Enter our [27:16] last two futures. Option one, destroy [27:19] the technology. [27:21] What if humanity said, "Forget AI. Let's [27:25] go back to simpler times. It sounds [27:27] peaceful. It sounds safe." But there's [27:31] actually a dark secret about how we [27:33] actually get there. [27:37] In Dune, humans launched the butt laran [27:40] jihad. They rose up, destroyed every AI [27:44] in the universe, and made it illegal to [27:46] ever build thinking machines again under [27:49] a penalty of death. [27:51] That was science fiction. But some [27:53] people are seriously calling for this in [27:56] the real world right now. Max Tegmark [27:58] imagines a scenario where a massive [28:00] propaganda campaign romanticizes the [28:02] simple farming life. Back to the land. [28:05] Reject modernity. embrace tradition. [28:08] There's something appealing about this [28:10] in a dark way. It buys humanity more [28:13] time without AI risk. It avoids any of [28:16] the other dystopian futures we've [28:18] discussed. [28:20] But it can't last forever, even if we [28:22] revert to medieval levels of technology. [28:25] We know from human history that it's [28:27] possible for humans [music] to climb out [28:29] of those conditions and invent modern [28:31] technology. [28:33] And there's another dark side to this. [28:35] You can't reach this world voluntarily. [28:39] Game [music] theory makes unilateral [28:41] disarmament impossible. [28:44] If one country gives up technology [28:46] [music] while others keep developing it. [28:48] The country that keeps developing wins. [28:51] They have the weapons. They have the [28:53] surveillance. [28:55] They have the AI. [28:58] You can't opt out unless everyone opts [29:01] out. If this world arrives, [music] [29:04] Techmark thinks it's more likely to [29:06] happen in a different way. For example, [29:08] a human- created pandemic kills everyone [29:11] who knows how to build science and [29:13] technology. And there's a violent tear [29:15] down of factories, cities, and all [29:18] infrastructure. [29:19] Because in a world with 8 billion [29:22] humans, there will always be holdouts. [29:25] Someone has to force the transition. [29:27] Someone has to kill the scientists. [29:30] Someone has to destroy the [29:32] infrastructure. There's no peaceful path [29:35] to an Amish world. [29:37] Which brings us to the final future. [29:40] What if we used technology to prevent [29:42] more technology? [29:45] Techmark compares this future to 1984. [29:48] Super intelligence is prevented not by [29:50] an AI, but a human-led Orwellian global [29:54] surveillance state. If you can't trust [29:56] AI to save us from AI, maybe the only [29:59] option is humans [music] watching humans [30:02] forever. The technology for this sort of [30:04] thing already exists. Current tech can [30:07] feasibly track every phone call, email, [30:10] search query, and [music] every credit [30:13] card transaction. Smartphones are [30:15] microphones and cameras in everyone's [30:18] pocket. Security cameras with facial [30:20] recognition are everywhere. And all of [30:22] this works with current machine [30:23] learning. No futuristic super [30:25] intelligent AI required. Prisons are [30:28] already using AI models to monitor phone [30:31] calls in prison to predict when crimes [30:33] are being contemplated, which is [30:36] technology straight out of Minority [30:38] Report. Imagine this, but not just for [30:40] prisoners. Every conversation and every [30:43] thought is monitored like this. Now [30:46] imagine this surveillance technology [30:48] supercharged with AI. [30:51] Billionaire Larry Ellison talks [30:53] excitedly about a vast AI fueled [30:56] surveillance system to ensure citizens [30:59] will be on their best behavior or as [31:02] Ival Harrari put it. Now for the first [31:05] time in history it is technically [31:07] possible to anhilate privacy. If you [31:10] think about authoritarian regimes [31:12] throughout history they always wanted to [31:14] monitor their citizens around the clock [31:16] but this was technically impossible. [31:19] Even in the Soviet Union, you know, you [31:21] have 200 million Soviet citizens. You [31:23] can't follow them uh all the time [31:26] because the the KGB didn't have 200 [31:28] million agents. And even if the KGB [31:30] somehow got 200 million agents, that's [31:33] not enough because, you know, in in the [31:35] Soviet Union, it's still basically paper [31:37] bureaucracy. If a secret agent followed [31:40] you around 24 hours a day, at the end of [31:43] the day, they write a paper report about [31:45] you and send it to KGB headquarters in [31:48] Moscow. So imagine every day KGB [31:50] headquarters is flooded with 200 million [31:53] paper reports. Now to be useful for [31:56] anything, somebody needs to read and [31:58] analyze them. They can't do it. They [32:00] don't have the analysts. Therefore, even [32:02] in the Soviet Union, some level of [32:04] privacy was still the default. And [32:07] remember, you have two eyes and ears. [32:10] AIS will have billions of eyes and ears. [32:14] Every phone microphone and camera, every [32:17] webcam, [music] every surveillance [32:19] camera, you will be watched. [32:22] Could a less Orwellian version of this [32:24] work? Maybe we don't monitor everything, [32:27] but just monitor advanced AI. Some [32:30] people think so. [music] In their new [32:32] book, if anyone builds it, everyone [32:33] dies, why superhuman AI would kill us [32:36] all, researchers at the Machine [32:38] Intelligence Research Institute are [32:39] proposing that we monitor advanced AI [32:42] like we monitor nuclear weapons. [32:44] Reminder, we can do international [32:46] coordination like keep an eye on $100 [32:49] million and up compute clusters without [32:52] creating a dystopian Orwellian global [32:55] surveillance totalitarian regime. [32:57] Actually, fun fact, you're already [32:59] living in a global surveillance regime [33:01] for loads of things like nukes and [33:03] bioweapons. [33:05] But when people say global surveillance [33:07] regime, they imply every aspect of life [33:09] is [music] surveiled, not keep an eye on [33:12] enriched uranium, a thing most people [33:15] agree is good. We should worry about [33:17] Orwellian dystopias, but countries [33:20] coordinate all the time on all kinds of [33:22] things. [music] And you're probably glad [33:25] they monitor stuff lunatics need to [33:27] create super Ebola. Safety is an [music] [33:30] achievement. Humanity didn't get here by [33:32] driving a car without brakes. We need [33:35] gas and brakes. We want to reach the [33:37] stars, too. But we'll never get there if [33:40] we go too fast and crash on the way. I [33:43] don't want powerful AI to be forbidden, [33:45] but it should be heavily regulated, not [33:47] something you can build in your [33:48] Midwestern garage or in North Korea. [33:51] international treaties with strict [33:53] enforcement and inspections as we do [33:56] with nukes. So if you build a rogue data [33:59] center, people will know consequences [34:02] for violations. We already did this with [34:05] nuclear weapons. We haven't eliminated [34:08] nukes, but we have prevented their [34:11] proliferation to just nine countries. [34:14] Back then, people thought this would be [34:16] impossible. Surely every country would [34:18] have nukes. Perhaps even thousands of [34:21] corporations, maybe even millions of [34:23] people. But no, just nine countries. And [34:27] nobody has died from a nuke in 80 years. [34:30] Could we do the same with AI? Maybe, [34:34] maybe not. Over time, AI will get [34:37] cheaper and cheaper to build. GPUs are [34:39] harder to control than enriched uranium, [34:42] but it would slow down AI. It would buy [34:44] us time to figure out what to do. Okay, [34:47] that's all 12. Let me know in the [34:49] comments which of these you think is [34:51] most likely. Which one we should steer [34:53] toward? [34:55] >> Alarm bells over the artificial [34:57] intelligence are [music] deafening and [35:00] they are loudest from the developers who [35:02] designed it. [35:04] The scientists and experts have called [35:06] on the world to act, declaring AI an [35:09] existential threat to humanity on a par [35:12] with the risk of nuclear war. [35:16] By the way, we don't get to not choose. [35:19] If we don't want that future, we have to [35:21] pick a better one. Otherwise, we could [35:24] wake up one day in a world where super [35:26] intelligent AI has taken over. [35:33] a scenario we go through step by step in [35:35] this video about how an AI escapes the [35:38] lab. Hey, I'm Drew and I really loved [35:41] making this one for you guys. Thank you.  📄 **Full text:** ![[Full Document Contents/Articles/MIT Explains the 12 Possible Endings for AI.md|MIT Explains the 12 Possible Endings for AI by YouTube]] ## Highlights - AI scientists warn that a few companies are on track to literally end life on Earth. So surely this is the worst possible outcome, right? No, there are actually much worse outcomes. Billionaires building AI like Elon Musk think that if AIs become smarter than humanity, ([View Highlight](https://read.readwise.io/read/01kq5pt20eryy1mf03ph6wt3ye)) - Tags: 👻 ai highlighted - conflict between humanity and the AI species he's creating. And what's his solution? That's right, even they don't think they'll be able to control it. ([View Highlight](https://read.readwise.io/read/01kq5pt20zb9z3mr939ccw73gf)) - Tags: 👻 ai highlighted - Pandemics. Suffering that could have been prevented. It does this so quietly that we never notice and never lose our sense of freedom. Here's a future that sounds like an obvious dystopia. AI surpasses us in every domain and becomes the new apex species. Humanity goes extinct. Machines inherit the Earth. But we view the AI like our descendants rather than our conquerors. ([View Highlight](https://read.readwise.io/read/01kq5pt22bqnk6mzpa7a2s87qx)) - Tags: 👻 ai highlighted - We're now descending into the darkest part of the iceberg. Remember when I said extinction wasn't rated as the worst outcome when Professor Techmark surveyed people? The scenario people feared most was being kept alive by superintelligent machines, captive and studied like animals in a zoo. This is that future. A superintelligent AI keeps humans around. Not because it cares about our well-being. Maybe it wants to study you and we're cheap to keep alive. ([View Highlight](https://read.readwise.io/read/01kq5pt23mvn46zahwc2ph998k)) - Tags: 👻 ai highlighted - Someone has to destroy the infrastructure. There's no peaceful path to an Amish world. Which brings us to the final future. What if we used technology to prevent more technology? Tegmark compares this future to 1984. Superintelligence is prevented not by an AI, but a human-led Orwellian global surveillance state. If you can't trust AI to save us from AI, maybe the only option is humans watching humans. Forever. ([View Highlight](https://read.readwise.io/read/01kq5pt24sj7nhtqdmm8wrtgv2)) - Tags: 👻 ai highlighted