Can We Preserve Knowledge … Forever?  🔗 **Source:** [Can We Preserve Knowledge … Forever?](https://share.snipd.com/episode/9105fa98-6ae6-409d-956d-1c0ab204fbcc) ## Highlights - **Creation and Rescue of Caper in the Castro** - CM Ralph created the 1989 game Caper in the Castro as a love letter to their LGBTQ community during the AIDS crisis. - The original Mac Plus disk sat in a drawer until 2017 when Adrienne Shaw helped extract and upload it to the Internet Archive so people can play it online. Transcript: Manoush Zomorodi I'm Manoush Zomorodi, and we are going to take you back to 1989 and to the first gay video game ever. CM Ralph Yeah, it was a big deal. The sounds had to be kept very small. Everything had to fit on that 800 megabyte diskette. It was very limited. The basic premise of the game is that you assume the role of a lesbian detective named Tracker McDyke. And you are searching for your friend who is a drag queen, and her name is Tessie LaFemme. This is CM Ralph, the creator of Caper in the Castro. She's got some inside information about something that's going on in this little neighborhood of the game, which is the metaphor for Castro Street in San Francisco. But before she can get the information to you, the phone suddenly hangs up. And this is where you're left. You're left, oh my God, where's my friend? In the late 80s, CM was working in Silicon Valley by day and at night making caper in the Castro on their Mac Plus. Tessie, thankfully, if you finish the game, you free her. ([Time 0:01:05](https://share.snipd.com/snip/64a8f872-f3a3-45bc-88fe-d55f30aba5fc)) - **Wayback Machine Preserves Ephemeral Web History** - Brewster Kahle built the Internet Archive to preserve ephemeral web content and aims to be a digital Library of Alexandria. - The Wayback Machine crawls around a billion URLs a day and saves snapshots so journalists and lawyers can retrieve removed or changed web content. Transcript: Manoush Zomorodi This is Brewster Kale. Brewster Kahle The average life of a webpage before it's either changed or deleted is 100 days. That's it. I think it was kind of a cruel joke to call webpages pages. Because you think of them as lasting a long time. You know, Gutenberg Bibles and all of that kind of thing. And nope. Manoush Zomorodi Brewster knew this was going to be a problem. Disappearing websites, missing chapters of the internet. He knew this way back in 1996. And that is why he created the Internet Archive. As we heard, it's where Caper in the Castro can still be found. And over the years, the archive's mission has expanded, saving old books and movies, TV shows and music. Brewster Kahle The idea is to try to build the library of everything, the Library of Alexandria for the digital age. We can make all the books, music, video, web pages, software, everything ever published by people available to anybody curious enough to want to have access to it. That was the dream of the Internet. And the Internet Archive is a part of making that dream come true. Manoush Zomorodi Which is a lofty goal and a huge endeavor. Because how does someone even begin to back up the Internet? Brewster started by building something he called the Wayback Machine. Brewster Kahle Yes, probably the most used and important part of the internet archive right now is the Wayback Machine, where we have collected web pages by going and basically clicking every web Link on every web page every two months. So if you go to archive.org and type in a URL, we'll show you different versions of that URL over time. We collect about a billion URLs every day. And we're finding that it's really important to journalists that are trying to find what actually happened. Lawyers love it because they can use it to say, hey, you said this before and now you're not saying that it's the only record often. Manoush Zomorodi Yeah, and you can dig up information that someone's deleted. Brewster Kahle Yes. Take Donald Trump's tweets. A large part of the policy of the impact on our country while Donald Trump was first president was through his Twitter feed. And then they turned it off. So it all just kind of disappeared. So we have a copy that we have made available through the Wayback Machine that makes it so we can see what it is. Or when a company goes under GeoCities and just everybody's sites go away, they're endless of these sites that go away or make different business decisions. And so people say, gosh, I'm glad that I can get a hold of that. But we run into things like locked files, databases you can't get through to. Some of those make things real challenges. We're working back and forth with the different websites to try to make things available. And the web also has parts that go obsolete so that the old websites, you can't replay them anymore. So there's challenges every day. ([Time 0:07:10](https://share.snipd.com/snip/4eff3859-cd86-4153-a026-7babfd8fd7e0)) - **Use Save Page Now To Preserve Webpages** - Anyone can help archive web pages by using archive.org's Save Page Now feature to preserve content at risk of disappearing. - Brewster notes it's used about 80 times a second and is practical for saving obituaries or articles you care about. Transcript: Brewster Kahle Yes. So we have robots that are going around and collecting a billion URLs. And fortunately, there are over 100 people that work for the Internet Archive that are trying to work on keeping it all alive. Too big, but we try to collect ones that are referenced a lot or the linked to from Twitter pages, say. So we can't collect everything, but we collect a lot. And if we're not collecting the right things, go to archive.org. There's a save page now feature and you can put in a URL and people do this all the time. It's used at about 80 times a second. So even anybody can go and participate in making things permanently available. ([Time 0:12:00](https://share.snipd.com/snip/d9778abd-d7c4-42b4-accc-79d054e6308b)) - **Digital Licensing Undermines Traditional Ownership** - Digital files complicate ownership because publishers license e-books rather than sell them, allowing content to be altered or revoked. - Brewster frames this as a social choice about whether digital knowledge will be public or locked by private companies. Transcript: Manoush Zomorodi And it's getting harder to proceed because e-books present very particular problems. For example, that e-book you downloaded, you don't actually own it. Brewster Kahle So it turns out the big publishers don't sell e-books. They license them. So your e-book that's on your Kindle or whatever, you don't actually have that. Not in the same sense you had a physical book. You can't pass it down to your kid. And anytime they want to change it, they can change it at any time or make it go away. Manoush Zomorodi It's a licensing issue. And Brewster and the Internet Archive started trying to get around it by buying physical copies of books, scanning them, and making their own e-books to lend out. Brewster Kahle So we started that in 2011, and in the beginning of the pandemic, four of the largest publishers decided to sue the Internet Archive to say that you aren't allowed to digitize and lend. Manoush Zomorodi What the Archive calls equal access, publishers say is digital piracy. Brewster Kahle And that suit is ongoing. We'll hear probably next year from the district court and it'll probably be appealed. But we'll see. The big concept that I never really would have imagined would be at play is digital ownership. When you buy a digital file, do you own it in the same sense that you owned a physical thing? You can't just go and post it and give it to everybody. That's understood. Fine. But do you get to keep it? And what the big publishers are saying is, no, there's no such thing as digital ownership anymore, ever. So that's the absolute opposite of what we were doing with the Internet in the earliest days when we were trying to democratize access, democratize creation. Manoush Zomorodi I actually went back into the TED archives and watched your talk from 2007 where you laid out your vision and you knew then that there would be conflicts, even if you didn't know what they Were. Brewster Kahle There's a political and social question out of this. All of this, as we go digital, is it going to be public or private? There are some large companies that have seen this vision that are doing large-scale digitization, but they're locking up the public domain. The question is, is that the world that we really want to live in? What's the role of the public versus the private as things go forward? How do we go and have a world where we both have libraries and publishing in the future, just as we basically benefited as we were growing up? ([Time 0:15:07](https://share.snipd.com/snip/c6976dea-046d-418d-9c18-a1a43c4bbad0)) - **A Museum Stored On DNA In A Fridge** - Dina Zielinski stored a digital museum encoded into DNA and keeps the vial in her refrigerator as a demonstration project. - She and an artist collaborator produced a DNA-encoded archive to show DNA's capacity for high-density, long-term storage. Transcript: Dina Zielinski Will we keep it safe? Absolutely. One of the biggest challenges is the sheer amount of data that we're generating. And I don't think we really realize, most of us, where all of that data is being stored. Okay, maybe I'll pay a few more bucks a month so that I can store my data in the cloud. But what is that cloud? You know, it's this huge server facility. Manoush Zomorodi This is molecular biologist Dina Zielinski. Dina Zielinski So it's projected that by the year 2025, the global data sphere, as in like all of the digital data that is out there, is projected to exceed more than 175 zettabytes. And one zettabyte is equal to a trillion gigabytes. So it's really become an urgent problem. You know, we've come a long way in digital data storage and computing, but we're running out of storage devices, quite literally. Manoush Zomorodi Dina says there is a possible solution to our data storage problem, a microscopic solution that's been around for billions of years, DNA. I actually have some in my fridge here. Dina Zielinski You can store, yeah, you know, that was a collaboration I had with a local artist, but we stored a digital museum on DNA. And so it's currently in my refrigerator. Manoush Zomorodi Wait, so you have a museum on DNA in your refrigerator right now? Yes. ([Time 0:20:10](https://share.snipd.com/snip/d05f69e8-17fe-495e-bf1c-8deffcd49d7d)) - **How DNA Encodes Digital Data** - DNA can encode binary data by mapping bit pairs to A, C, G, T, enabling any digital file to be written as sequences of nucleotides. - Labs synthesize the sequences into dried DNA powder that can later be hydrated and sequenced to recover the original bits. Transcript: Manoush Zomorodi Dina says you can use DNA just like a hard drive or a floppy disk, it does an even better job. Dina Zielinski It's true. I mean, it's absolutely true. It can beat any man-made device in terms of longevity, density. It has a very small ecological footprint. I mean, it's biocompatible. It's something that we'll always be interested in. And we can think of DNA as just another storage device. So instead of a compact disk or a floppy disk or a hard drive, we simply use the molecule of DNA. Manoush Zomorodi Yeah, and when it comes to DNA's durability, I have heard you use the example of Utsi the Iceman. Dina Zielinski This was a 5,000 man whose body was found frozen by hikers in the Alps in the 90s. And these are sort of the ideal storage conditions. And all that is to show that even after thousands of years, we're still able to extract meaningful information from the DNA. So DNA in itself is very fragile, actually. If I just have DNA in a tube, for example, it's very susceptible to UV radiation. It is really only stable at room temperature for a relatively short period of time. And then it starts to sort of degrade and accumulate errors. But if you store it very cold and very dry, we could theoretically store data that's critical to humanity in such a place, in a naturally existing cold and dry place, like Utsi the Iceman. Here's Dina Zielinski on the TED stage. Storing data on DNA is not new. Nature's been doing it for several billion years. In fact, every living thing is a DNA storage device. But how do we store data on DNA? We first learn to sequence or read DNA, and very soon after, how to write it or synthesize it. This is much like how we learn a new language. And now we have the ability to read, write, and copy DNA. We do it in the lab all the time. So anything, really anything, that can be stored as zeros and ones can be stored in DNA. To store something digitally, we convert it to bits or binary digits. Each pixel in a black and white photo is simply a zero or a one. And we can write DNA much like an inkjet printer can print letters on a page. We just have to convert our data, all of those zeros and ones, to A's, T's, C's, and G's. And then we send this to a synthesis company. So we write it, we can store it, and when we want to recover our data, we just sequence it. ([Time 0:22:19](https://share.snipd.com/snip/b0777ea4-5c80-468e-b353-37136e2d55c4)) - **Reserve DNA For Critical Cold Archives Now** - Use DNA storage only for cold archival data today because synthesis costs remain extremely high. - Dina says stability is strong if DNA is dehydrated and stored cold or in silica, with proven longevity across thousands of years in tests. Transcript: Dina Zielinski So if you store it, for example, in your refrigerator, in your kitchen, I mean, it can easily last years, even tens of years. Ideally, you would freeze it and that would further extend the life of the DNA. Even for thousands of years, like Ötzi? Yeah, so, I mean, when we synthesize DNA, it's still fragile on its own. But if you keep it protected, so we can store DNA in silica beads, so glass beads, and some researchers at ETH in Zurich, Robert Grass's team, have shown that the DNA is stable for thousands Of years. And they've replicated sort of a European climate. And they showed that by inducing temperature and other stress, they were still able to recover the data from the DNA. Manoush Zomorodi And let's say you were like, today's the day. I'm going to get the data out of the DNA. How would you go about doing that? So just add water. Dina Zielinski So you take your, you just take your tube of DNA, you add water to it, and then you load it onto the sequencer. And then you simply convert your A's, T's, C's, and G's back to zeros and ones and decode the data. Manoush Zomorodi Okay, so this is something that you need highly specialized computers and machines to do right now, right? Dina Zielinski Yes. Even though I have DNA in my fridge, I cannot do much with it. Manoush Zomorodi Okay, got it. So just being at home, it's not like you're going to store your data in the fridge like you store data on your desktop, on your laptop or something. What we're talking about is really putting critical pieces of humanity's collective knowledge in a safe place. Dina Zielinski Yeah, so DNA storage is mostly useful for archival or cold data. One of my favorite examples is here in France. They store the rights of man in DNA, and they actually keep it in the French National Archives here. Wow. And you can make copies very easily and very cheaply, but it is currently stored safely in a vacuum tube in the archives here. So why aren't more people doing this now? I would say in the next decade or so, the synthesis costs will drop down sufficiently in order to make this a scalable and affordable alternative. But for the time being, it's simply too expensive. But if we can store critical archival data, that's really the ideal solution for DNA, at least in the next decade or two. ([Time 0:26:47](https://share.snipd.com/snip/d50dd77e-ec10-4ccf-b2e7-0fe4c5e87648)) - **LiDAR Turned Years Of Fieldwork Into 45 Minutes** - Chris Fisher used airborne LiDAR to map Angamuco in Mexico and collected in 45 minutes what would have taken 20 years of traditional fieldwork. - The LiDAR returned dense 3D data that revealed thousands of building foundations and transformed his approach to archaeology. Transcript: Manoush Zomorodi Ancient city that Chris found was massive. Chris Fisher So it had served to preserve thousands of building foundations. Manoush Zomorodi And that was so exciting, but also terrifying because Chris would need years to excavate and map this site. Chris Fisher I mean, this would have been my career, surveying this one site. I'm super impatient and I didn't want to do that. Manoush Zomorodi So Chris asked a colleague for advice. Chris Fisher I'm like, dude, there's got to be a better way. And he's like, well, have you heard of this LiDAR technology? Manoush Zomorodi LiDAR stands for Light Detection and Ranging. It's the sensing technology that self-driving cars use to scan and make maps of the terrain around them. Chris Fisher But what we do is we use something called airborne LIDAR. Manoush Zomorodi Using helicopters, planes, or increasingly drones. Chris Fisher And basically you have some sort of instrument. That instrument fires off a pulse of infrared beams. You know, hundreds of thousands of pulses in some instances. When one of those beams strikes something, it could be the ground surface, it could be buildings, pyramids, roads, all of that sort of stuff. It returns back to the sensor and it gives you a measure of distance. Manoush Zomorodi All that data creates a full 3D scan, a digital map. And that first map that Chris saw absolutely delighted him. Chris Fisher In 45 minutes of flying, the LIDAR had collected 20 years worth of normal archaeological fieldwork. ([Time 0:34:16](https://share.snipd.com/snip/07fdd0c6-d127-463e-9539-4b8b95fce147)) - **Documentation Can Alter The Things It Records** - Surveying and publicizing sites can unintentionally change them, so the original LiDAR scans may be the only record of how a place once looked. - That fragility and rapid environmental change motivated Fisher to start the Earth Archive to scan threatened regions now. Transcript: Chris Fisher The tiny gravel bar where we first landed our helicopter was gone. Manoush Zomorodi Chris Fisher continues from the TED stage. Chris Fisher The brush had been cleared away and the trees removed to create a large landing zone for several helicopters at once. Without it, after just one rainy season, the ancient canals that we had seen in our LIDAR scan were damaged or destroyed. And the Eden I described soon had a large clearing, central camp, lights, and an outdoor chapel. In other words, despite our best efforts to protect the site as it was, things changed. Our initial LIDAR scan of this city of the Jaguar is the only record of this place as it existed just a few years ago. Broadly speaking, this is a problem for archaeologists. We can't study an area without changing it somehow. And regardless, the earth is changing. Archaeological sites are destroyed. History is lost. We take for granted that our cultural and ecological patrimony will be around forever. It won't. Manoush Zomorodi So the Honduran government brings in soldiers to make sure that other people don't come in looking for treasures. But in the process, they destroy other things that probably the naked eye would never know was there unless we were an archaeologist. And so when you realized that that was happening, did you say to yourself, well, thank goodness we have this original mapping of this area with LIDAR? Chris Fisher Yeah. So my initial reaction was, oh yeah, we've got these. We have the LIDAR images of them. But then my secondary reaction was like kind of an intense sadness. By documenting this place and understanding it and going to it, we'd actually fundamentally changed it. Manoush Zomorodi So what do you do with sadness is always the question that I have. And I feel like, you know, do you just say, well, this is what my life's calling. This is what I do. Yeah. Or do you think, huh? Chris Fisher Yeah. That made me realize how fragile the earth is, how rapidly the earth is changing due to the climate crisis, and how much stuff there is left to discover. It took me several months to figure this out. But eventually I did come to the realization that I could do something. That due to the climate crisis, we're losing cultural and ecological treasures that we don't even know exist. And unless we document them, we will never know they exist. We've lost 50% of our rainforests. We lose 18 million acres of forest every year. And rising sea levels will make cities, countries, and continents completely unrecognizable. Unless we have a record of these places, no one in the future will know they existed. Looking at my scans from Honduras and Mexico, it's clear that we need to scan, scan, scan now as much as possible while we still can. ([Time 0:38:05](https://share.snipd.com/snip/a41a5248-1f81-45db-8670-b430d07f0a8e)) - **Scan Now To Create Baseline Planet Records** - Prioritize creating baseline high-resolution scans now to measure future change and enable many scientific studies. - Chris says LiDAR freezes the earth in time and can reveal vegetation, hydrology, archaeology, and human impacts for future research and mitigation. Transcript: Chris Fisher Number one, create a baseline record of the Earth as it exists today to more effectively mitigate the climate crisis. To measure change, you need two sets of data, a before and an after. Right now, we don't have a high-resolution for much of the planet. Number two, create a virtual planet so that any number of scientists can study our Earth today. Archaeologists like me can look for undocumented settlements. Ecologists can study tree size, forest composition, and age. Geologists can study hydrology, faults, disturbance. Possibilities are endless. Number three, preserve a record of the planet for our grandchildren's grandchildren so they can reconstruct and study lost cultural patrimony in the future. Manoush Zomorodi Okay, so your mind-boggling project is called the Earth Archive, and it requires, as you put it, that we scan, scan, scan the planet. Chris Fisher Yeah, so this was the big idea. That's where I landed, in that we have this technology to scan your surface and everything on it and curate it in perpetuity for our grandchildren's grandchildren. And these LIDAR records represent the ultimate conservation records. Because not only do they record the ground surface, but they record all the vegetation on it, every tree. It would record the underlying topography. It would record the hydrology of places. We'd be able to see archaeological sites in those zones. We would be able to see modern settlement change and roads and deforestation features and countless, countless other things. But the archaeology piece of this is probably the least important. The ecological piece is probably a lot more important. And so it really would be the ultimate baseline data set from which you could do other significant analysis. Manoush Zomorodi Layer and layer and layer upon layer of information. I mean, I love the way you described it. It's like a research sandwich that it's capturing. Chris Fisher Exactly. And that's the amazing thing about these LiDAR records. Once you do the scan, it freezes the earth and everything on it in time. And since it's a digital record, you can curate it in perpetuity. ([Time 0:41:32](https://share.snipd.com/snip/4cdb08a6-07ec-479c-a1be-0777c935e3b8)) - **Give Scans To Communities To Protect Their Land** - Use LiDAR scans to empower local communities to defend land and heritage by creating undeniable records of burial sites, forests, and coastline. - Fisher's Earth Archive worked with the Yakutat Tlingit to scan an island threatened by logging so the tribe could monitor and use data for stewardship. Transcript: Chris Fisher So we've just completed a really exciting project that we hope facilitate with the Yakutak Tlingit tribe in Alaska. Speaker 6 We live in a dynamic, shifting environment. We find that the landscape is actively moving all around us. Manoush Zomorodi This is Chris's partner on the project, Andrew Gildersleeve. Speaker 6 I'm the chief executive officer of the Yakutak Tlingit tribe. We're really the top of southeast Alaska on the Gulf. What makes us so unique is the fact that we have one of the few glaciers in the world that is actually growing a tidewater glacier. Manoush Zomorodi Nearby, lots of logging is causing deforestation, especially on one particular island. Speaker 6 It's made of gravel, and the trees are literally what hold the gravel together. Manoush Zomorodi And that is jeopardizing the tribe's livelihoods, their land, and their burial sites. Speaker 6 So when Chris came onto the scene, we decided to move quickly, and we immediately moved to have a LIDAR scan of this barrier island. Now, this island is home not only to burial grounds and ancient village sites, it's also literally our protection from the Gulf of Alaska, from potential tsunamis. And with LiDAR, it immediately meant that this island could not only speak for itself and tell us its stories, it also meant that the logging operation was not going to get away with any Of the destructive tendencies that they had in the past. We had recorded and collected everything with regards to the island from the roots to the tops of the trees. Chris Fisher They're seeing the changes in their landscape. They're concerned and they want to use every technique they possibly can to ensure that they are practicing best standards for their continuing stewardship. Speaker 6 LIDAR gives us an immediate and clear history. And as we live in this dynamic time and this dynamic landscape with the climate crisis, with the deforestation, with fisheries issues, tourism issues, we need serious data to really Start answering big questions of the tribe. ([Time 0:44:58](https://share.snipd.com/snip/419280a5-88d6-4aad-8e9e-6dc49761833e))