Legal Consequences of Generated Content  ## Metadata - Author: [[Changelog Master Feed]] - Full Title: Legal Consequences of Generated Content - Category: #podcasts - URL: https://share.snipd.com/episode/ad518c2b-7256-40a3-b3e1-ee228fe27129 ## Document Notes ## Highlights - **Brute Forcing All Melodies** - Damien Riehl and his friend brute-forced every possible musical melody, generating 471 billion melodies in total. - They copyrighted and placed all melodies into the public domain to protect users in melody copyright lawsuits. Transcript: Speaker 1 Yeah, absolutely. So I have both my current state with my job that pays me money. That is with VLEX, where I'm doing lots with large language models right now. We have a billion legal documents that were running large language models across and doing embeddings to be able to do outputs, for example, a legal research memorandum and eventually Speaker 4 Be able to provide motions, briefs, pleadings, that sort of thing. Speaker 1 So that's my job to pays me money. For the job that doesn't pay me money at all, which you referenced is my all the music project. This is a project that I started with my friend Noah Rubin, who one thing your listeners might find interesting is that around 2018, I did cybersecurity. The biggest thing I did was that Facebook hired me and my company to investigate Cambridge Analytica. So I spent a year of my life on Facebook's campus with Facebook's data scientists and my former FBI, CIA, NSA people that worked with me to figure out how bad guys use Facebook data. Speaker 4 Did that for about 50 some weeks in a row. The stuff we would do on Monday would make the New York Times in the times of London by Friday. Speaker 1 So at the end of a 14-hour day on the Facebook campus, I retreated to my hotel with Noah Rubin, my friend. We were in the lounge and over a beer, I said, you know, Noah, how we can brute force passwords by going A, A, B, A, C. I said, what if we could do that with music where we would go da da da da da da de re, da da da da da, dada da da da, dada da da da, until we mathematically exhausted every melody that's ever been And every melody that ever can be. And so he said F, yeah, but he didn't say F, yeah. So within a few hours, he had a prototype where he cranked out 3,000 melodies. To date, we've now cranked out 471 billion melodies with a B. Mathematically exhausting every melody that's ever been and ever can be. We've written all those to disk. Once they're written to disk, they're copyrighted automatically. So we've copyrighted 471 billion melodies. And then we placed everything in the public domain to be able to protect the use to all my melody lawsuit defendants. ([Time 0:04:22](https://share.snipd.com/snip/9b91217d-fdae-4367-82de-efafb5f1b593)) - **Questioning Human Creativity** - Human creativity might not be as unique as we think, as machines can brute force melodies that humans create. - Courts have recognized unoriginal melodies as uncopyrightable, challenging traditional copyright assumptions. Transcript: Speaker 1 And so this really gets to the heart of what we think human creativity is, right? Is it creative as in brute forcing? Or is it truly lightning in a bottle of creativity that we want to protect with intellectual property laws and other things like that? I think what we're learning from my project and from the large language models is that maybe human creativity ain't as special as we think it is. As an example, on a Tuesday, a jury found that Katy Perry had violated a copyright of a melody that sounded like this. Speaker 4 Dah, dah, dah, dah, dah, dah, dah, dah, dah, dah, dah, dah, dah, dah, dah, dah, dah, dah, that was on a Tuesday. Speaker 1 On my talk on a Saturday, I said that my, that particular melody shows up in my dataset 8,128 times. So Katy Perry got dinged for $2.8 million over something that I had thousands of times just through brute force. So was that melody creative? No, it was just a brute force. After my talk was made public, the judge actually went back and reversed the jury verdict, saying that melody was so unoriginal as to be uncopyrightable, essentially what I was arguing My TED talk. I think going to the heart of what is human creativity, it's good that we have large language models and projects like mine to be able to go to the heart of there are some things that we should Protect and ([Time 0:08:59](https://share.snipd.com/snip/c13ebe07-fcd7-42b2-a3eb-d8a722256204)) - **Use AI as Writing Assistant** - Use AI tools to greatly speed up writing by generating drafts from outlines and then editing them. - Treat AI-generated output as an assistant, not a final product, to enhance creativity and productivity. Transcript: Speaker 1 I was reached out by the editor of a large legal magazine to say, hey, Damien, I want you to write the cover story on GPT. I said, how long do you want it to be? He said about 17 double space pages. And I was like, man, I don't have time because my rule of thumb is one hour per double space page. So that's 17 hours that I just don't have time to do. But then I realized, oh, wait, the topic is GPT. So what I did is I created an outline of headings and subheadings. That's about two pages worth. And I said to GPT, for each of the bullet points, give me four sentences, essentially a paragraph for each of the bullet points. Speaker 4 And it's out. That was my one for the day. Speaker 1 Craped out the 15 pages worth. And then I spent the next three hours editing, moving, adding, working with the text, not accepting the 15 pages outright, but working with the text and regenerating. And then I got it out the door three hours later. And the editor is like, oh, this is perfect. I don't need any edits. Let's get it out the door. So that took a 17 hour project down to three hours. ([Time 0:12:21](https://share.snipd.com/snip/a256e3cf-4d4a-45b3-88fc-cf5aac590111)) - **Transformative Use of Copyrighted Works** - Ingesting copyrighted works into large language models constitutes transformative fair use by extracting ideas, not expressions. - Model outputs create new expressions that are likely uncopyrightable, aligning with copyright office views on machine-generated content. Transcript: Speaker 2 What's gray area? What's kind of being maybe being litigated right now? Speaker 1 I'm going to answer your question and it's going to be a fun walk. So take a walk with me. Okay. So this walk, it's going to begin with the Google Books project. So you might remember the Google Books ingested every book that ever existed, perhaps also including the Harry Potter book. A bunch of publishers said, hey, no fair because you can't ingest all these things because these things are copyrighted. Every one of these books is copyrighted. They sued. And then the district court and the appellate court, the second court of appeals said, yes, all those things are copyrighted, but Google's use of that is actually fair use. And the particular type of fair use is called transformative use that the use that Google was using was transformative to what the original purpose of the book was. Purpose of a book is to read it, enjoy it, et cetera. A Google's purpose was to index it, to be able to create a word index, to be able to then search all the books and to be able to provide the end user with a snippet, say maybe a page or two of That. So because it was not, you couldn't eyes a user couldn't use Google Books to be able to essentially replicate the book process. But instead, I'm using a disserge that is a transformative use, therefore fair use, that is not an infringement copyright. So that was back in the day. Now think about how large language models work. So a large language model, if you have the input, it is ingesting, say, the entirety of Harry Potter. But really what it's doing is placing those in vector space, right? It's saying that these words are similar to those words in vector space. And once that happens, largely it jettisons the thing, right? In copyright law, there is the idea, expression dichotomy. Ideas are uncopy-readable. So if I have the idea of a man in a black hat fighting a man with a white hat over a woman who is tied to a railroad track, those are ideas that are uncopy-readable. You've seen lots of movies like that. But the expression of the idea, any particular movie that has that in there, that is copyrightable. So ideas uncopy-readable, expressions of the ideas are copyrightable. So if you apply that to what's happening when the large language model ingests all the books, it's essentially putting all the words into vector space. So it's saying, you know, this is a Bob Dylanism, or this is an Ernest Hemingwayism, or this is a Harry Potterism. Each of those are ideas, not expressions of ideas. And so really it's taking the expressions and effectively jettisoning those in favor of the ideas. So that's on the input side. And then on the output side, one can imagine that it's taking those ideas, Bob Dylanism, Ernest Hemingwayism, and then it's outputting them in a new expression. And if you believe the copyright office, machine generated output is similarly uncopy-writable. So we're kind of faced with an idea that inputs, ideas uncopy-writable outputs, the expressions of ideas created by machines, similarly uncopy-writable. To your particular point, this has not been tested in court. So a judge, who by the way may not know what he's talking about, or she is talking about, might rule against what I'm about to say right now. But at least I would make the really good argument that the ingestion of the thing is extracting the ideas from the book. And that is a transformative use, because if you think about Google Books, they were printed three pages or so of verbatim of these books. That's way more bulk than just think about the vector space. It's not reproducing any expression. It's really taking the ideas. So if Google Books is permissible, almost certainly the large language model should also be. And that's really what is being argued right now in the cases that are happening with the GitHub copilot case happening in the West Coast, and then the stable diffusion case in Delaware, And there's others like it, where if I were the lawyers and that I would be arguing exactly what I just argued right now. So to your specific use case, you are kind of interrogating this copyrighted work, but I would make the argument, if I were representing you, that this would be a transformative use. And just like you as a human would have read the book and you can, as a human, could provide output based on that book. ([Time 0:16:42](https://share.snipd.com/snip/5dea41ed-4df5-400e-8a52-707c02aa109c)) - **Smooth vs Jagged Content** - Machine-generated content is "smooth," statistically likely, unlike the "jagged" unpredictable human creativity. - The rise of machine content might limit truly human-created content on the internet beyond late 2022. Transcript: Speaker 1 First, I think it should stand because otherwise my all the music project would essentially carpet bomb all the music and we would just have machines creating new expressions that Would essentially make human expression obsolete. So number one, I think it should stand because if not, we are in a world of hurt with a copyright. Thing number two is you're right that never before in human history have we had a machine that creates new things. We've had the printing press where we take my ideas and then I can replicate it a whole bunch of times. We have the digital revolution in the 70s, 80s, 90s, 2000s where now we can replicate human stuff, but never before have we had a way that the machine itself is making new expressions Of ideas. And so as a result of that, some of the smartest people I know that are thinking about this is said that the web as it stands is probably large language models are going to stop right around November of 2022 because anything after that you're going to have a whole bunch of machine generated content that is going to be essentially large language model created things. If you know about the tech and I assume your audience does because it's statistically likely it is smooth. Humans are jagged in the way that they write text and that's what detects you BT and others to see the jaggedness of humanness. Machine generated content is smooth, not jagged. So that's what detects you BT says. Could you define that real quick what jagged versus smooth means in this context? Sure. Yeah. So jagged means random. Smooth means statistically almost deterministic. So the idea is that we as humans say random things and we put things in a way that maybe hasn't been said before, whereas machine, at least an LMM machine is going to be able to say, you know, What's the most statistically likely word and therefore that is smoother than our jagged randomness. The idea is that as the machines are creating the smooth deterministic, statistically likely next word, that is essentially as new large language model in just that smooth text, it's Going to further smooth the corpus. And we're going to miss all of the human created jaggedness that is going in there. ([Time 0:21:14](https://share.snipd.com/snip/a73a3476-07f1-457c-9fbf-75ed028275f2)) - **Copyright on Machine vs Human Work** - Fully machine-generated works are generally uncopyrightable, as they lack human creativity. - However, humans adding original creative elements on top of machine outputs can obtain copyrights for those additions. Transcript: Speaker 1 If I were to create this machine created coloring book, for example, and that which under the US Copyright Office today, that entirely machine created thing is therefore uncopyrightable. This really goes to the heart of what is copyright in the first place. And all copyright is is a monopoly. It is a government sanctioned monopoly giving you the author a monopoly of life of the author plus 70 years on the thing you created. But as an exchange for that monopoly, the government says this has to be original. That is has to be your creative work that does this. And if it is truly original and it is truly creative, we will give you that monopoly of 70 years, a life of the author plus 70 years. So really the question is, is there anything copyrightable in the machine generated work? Well, probably not because there was no human creativity in that thing. So that's thing number one. But then let's look at another scenario. But if somebody else did a human created coloring book that was identical to what the machine had done. Does that turn it from unoriginal therefore uncopyrightable with the machine created one? Tell us in the face, human does it? It is copyrightable even though they're identical. Speaker 2 Yeah, essentially because the machine wasn't copyrightable and you're recreating it, even though there might be no creativity on the humans part because they're literally looking At the output of the uncopyrightable machine output, but they can steal the idea without the creativity involved and then copyright. Am I understanding correctly? Speaker 1 That's right. And we've dealt with this. The courts have dealt with this for a few hundred years. And you can imagine the Shakespeare is in the public domain. It's not been in copyright for hundreds of years. You can build a top Shakespeare, say with West Side Story that was based on Romeo and Juliet, right? The writers of West Side Story don't get copyright in the underlying story of Romeo and Juliet, but they do get copyright in whatever they put atop Romeo and Juliet. So the fact that it's New York City and all of these things. So they just get what's called thin copyright on top of the public domain thing. So you could imagine it going back to our coloring book example, right? If someone then copies that and then adds a little human touch on that, they don't get the underlying copyright thing because that's public domain. They only get what they've added atop the machine created thing. And really the question is how much can you really add a top that is really creative enough to make it worthwhile? ([Time 0:25:21](https://share.snipd.com/snip/698cd75f-6343-4fe9-ba19-76fc566bd86e)) - **Provenance Challenges in Model Training** - Training models on data with restrictive licenses may taint the resulting model's legality. - Provenance of training data is crucial but can become murky, leading to complex legal challenges. Transcript: Speaker 1 I think that what you're asking really about is the provenance of everything that comes downstream, that is, what is the provenance of the input and what is the provenance of the essentially Being able to manipulate that input to create a thing that's called a model. Of course, the output of the model could train new inputs to be able to go into new models. Speaker 2 Yes, the cyclical sort of thing. Speaker 1 That's right, it's like a snake eating its own tail. Within the law and criminal law, there's a thing called the fruit of the poisonous tree. The first act might be innocuous, but then it leads to a chain reaction, a bunch of dominoes that leads to the end. This is the fruit of the poisonous tree, it's a legal concept that you could imagine is similar for the questions you asked. If there is input data that maybe has questionable licensing, for example, the llama, it was released for open source, but only for academic purposes. You could imagine if someone were to be able to create a model for commercial purposes that is based on that ostensibly licensed for academic purposes, that is maybe a fruit of the poisonous Tree question to be able to say, is that model now tainted because it was ingested in opposition to the license? Speaker 2 Yes, I've wanted to use some of these llama based models quite recently and just haven't because it makes me ask a lot of questions. Am I right with that hesitation or is it yet to be determined, but you would make certain assumptions or what do you think? Speaker 1 Yeah, so I should clarify that I'm a lawyer, but I'm not your lawyer. So nothing I'm saying is going to be legal advice, okay? Yes, correct, correct. So I would say that yes, anytime that you are ingesting items that are licensed and then you're using them in a way that is maybe against that license or not permitted by that license, I think anyone should be worried when that happens, speaking generally. I would also say that yes, anyone who does that should be worried. Also proving such things is tricky, right? Because there is the law and then there's what can be proved in the preponderance of the evidence in the court of law. As the dominoes fall and as the snake keeps eating its tail, the provenance of what data did you actually use and where did it come from? ([Time 0:29:01](https://share.snipd.com/snip/1d8cc824-5adc-42f2-83eb-e581a6cbb708)) - **IP Laws Strained by AI** - Intellectual property laws are under strain as AI-generated content makes patents and copyrights less valuable. - Mass machine-generated claims threaten to flood patent offices, potentially causing legal and business challenges. Transcript: Speaker 1 Think. What I think you've described and what I think we're saying in a society is the intellectual property laws that we've created since the beginning of our founding history, the constitution Says that we will protect inventions, that's constitutional. So what I think we're seeing is our intellectual property regime that has existed since the 1700s creaking under its own weight with this new large language model generating in the Way that's never been done in human history. I think you're right, the value of patents, what is the value of a patent if I can use a large language model to much like I described with all the patents, we with all the patents are saying Every patent that's ever been done and every idea, every claim and every patents, let's recombine those. But you could imagine, and I've heard that there are companies out there that are doing not what's been done before, but new ideas and then making a ton of new claims and filing those new Claims with the US patent office. And to be able to say, here's a new idea. And if you carpet bomb the US patent office with all of these new things that are just machine generated, there's currently a case, the Thaller case TH L E R that he created, I had said a Machine created this patent. And the patent office said, machine creative patents are not a thing, you can't do it. But they only knew that because Thaller told him that it was machine created. How many of these things are being filed that nobody's told anybody that has been patent? And then is that fraud on the patent office? Probably. ([Time 0:32:38](https://share.snipd.com/snip/b630b5dd-adc5-4732-a071-94b7cfe07164)) - **AI Productivity Reshapes Work** - AI boosts productivity massively, causing shifts in employer expectations and workforce sizes. - Companies might demand 10x output with fewer employees, reshaping future work environments. Transcript: Speaker 1 Regard? Yeah, so I'm going to talk about four worlds. First world is 2022 world before the large language bottles. And in that world, I would work 40 hours a week full time. And I would give 40 hours a week of 2022 productivity as a result of that. And as a result of that, an employer would hire a workforce like me to do that. So that's world number one. In world number two, I know of people anecdotally that have working three full time jobs because they're getting at least 100% or so productivity gains, maybe 10x productivity gains Based on the code that you said. So they have three full time jobs. So he's essentially working 30% of the time for each, but still providing 100% of the output for that. And their employer is saying, wow, that's great output. They don't care, right? So that's world number two. I think what we're in today. World number three is probably the employer is going to say, hey, hey, hey, hey, don't give me 30% of your time, give me 100% of your time, and maybe give me 10x output of 2022 level output, Right? I want that productivity gain from you. So that's world number three. But I think shortly thereafter is going to come world number four where the executives are going to say, wait, if we lay off two thirds of the workforce and then still require them to work 40 hours a week with their 10x productivity, I can say to my shareholders, look at all the costs that we cut by laying off two thirds of the workforce. And we're still getting five X productivity on top of our 2022 productivity. We've cut costs. We've increased productivity. Aren't we great? ([Time 0:36:52](https://share.snipd.com/snip/6b83bb2b-cfb6-41df-92d8-fb76ac8a31a0)) - **Outrun the AI Tsunami** - Learn to use AI tools effectively to stay ahead in a rapidly changing environment. - Leveraging AI assistants can help you outperform competition as AI reshapes industries. Transcript: Speaker 1 I think about AI as largely a tidal wave or a tsunami and we are running faster than the tsunami. How do we run faster than tsunami? You learn how to use co-pilot to be able to go faster. You learn how to be able to do things that the machine cannot yet do. That's running faster than tsunami. Really, I say to lawyers that are worried about AI, that AI will not take a lawyer's job, but a lawyer that uses AI will take the job of a lawyer that does not use AI. Really I would say the same thing for coders who are listening, that learning to use the tool to run faster than the tsunami, there's another joke, there was a bear and a campground and Two guys and the one guy gets out of his tennis shoes and the other guy says, you can't outrun a bear and he said, I don't have to, I just have to outrun you. In that sense, I learned how to use the large language models to outrun your competition because as the wave crashes over them, it's not going to crash over you. ([Time 0:40:24](https://share.snipd.com/snip/fcbce117-7c1d-4799-8dcd-e8d51bee9417))