We Are Not Ready for Better Deepfakes  ## Metadata - Author: [[Decoder with Nilay Patel]] - Full Title: We Are Not Ready for Better Deepfakes - Category: #podcasts - URL: https://share.snipd.com/episode/34e30c79-0017-48d4-a556-ce5742c4c83a ## Document Notes ## Highlights - **Face Swap Deepfakes Use Body Doubles** - The first generation deepfakes involved face swaps using a body double and footage of the real person. - TikTok accounts recently used this technique to impersonate celebrities like Tom Cruise. Transcript: Alex Heath Mirage.app. The title is We Build Synthetic Humans. Here's What's Keeping Us Up at night. And I want to get into that. I like the way you construct this with the four generations of deepfakes and the technology there. So maybe we can start with that. Can you kind of walk us through when deepfakes started, how they worked, and then getting to where they're going? Totally. Gaurav Misra Yeah. And, you know, completely, obviously, agree with you on how quickly the technology is developing. It feels like every week, every month, you know, there's like actually miles and miles of progress that's happening, you know, in this space. And it's not the type of thing that you really talk about too much, especially in this context, right? And people have been quiet about it in some ways, right? Obviously, everybody wants to talk about the positive sides, but we wanted to make sure that we're also like, you know, being honest and open about like what we're discovering in terms Of like, what are the negative side effects and use cases? And just so people know, you know, what's out there and how we might address it, right? Both our own company and others in the world as well. That being said, you know, on the deep break front, like with the report, we really tried to structure it as like, not from a technology perspective, more from the user perspective. Like how do people perceive these deepfakes, right? And how they might classify these things, right? To like the everyday person, they might look at two or three different videos generated from different models across four years and feel like, hmm, they all look kind of real. Like, I don't understand what the difference is, but they actually end up being slightly different in the way they're generated. And based on that, there's tells of like how you can tell, okay, this is generated versus this is real just by looking at it. And with our own company internally, like we look at these videos all the time, right? Whether it's our own technology or whether it's other companies that are out there, and there's many at this point, right? So we've come to the point where there's many people on our team who can look at a video and just by looking at it for a few seconds, they can be like, oh, this was generated by this version Of the model of this company, probably around this time period, right? Because you just know that there's certain things off about it or certain things that, you know, are just the telltale signs. But, you know, that's starting to happen. The interesting part of that is like, I see that actually across, you know, all the different generative models. These days, like I can look at a piece of text made by ChatGPT and immediately know that it's ChatGPT, like within three lines or four lines, because it says one of the things that it always Says, like, it's not just X, it's Y, you know, like, or there's an M dash, the M dash is exactly the classic. Alex Heath Yeah. Gaurav Misra So I think those same things are happening in the video space as well, where you can kind of tell, but at the same time, as the technology is developing, the whole goal is kind of to make Sure that there's no tell, right? Like that's kind of the point of the technology itself. Alex Heath And you increasingly can't tell right now. That's, I think, the thing that freaks me out is, and you see it almost reflexively, where if something looks iffy now, you look at the TikTok comments and it's people just saying AI, AI, AI, AI, over and over and over. So like people are starting to see this, but I think it's also getting harder. And that's, it's interesting that that's happening at the same time. So I want to get into the state of things right now, but I guess, I think this audience will find this interesting. These four generations of deep fakes, can you explain kind of the first one and then how that progressed to the fourth one that you all talk about that is coming? Gaurav Misra We classified it in terms of how much of the video is actually generated, right? And that is actually what has been evolving, right? Now, within each generation, there is development of the technology of that generation that still happens. So, for example, the first generation we see as like face swaps, right? So let's say you want to impersonate Tom Cruise, something that actually happened. If you wanted to do that, like four years ago, you would find a Tom Cruise impersonator, basically like a real person who looks and talks like Tom Cruise says similar things has a similar Voice, acts in a similar way, probably is the same same build height and weight. And then you would have them basically, you know, act a bunch of stuff, you would record it. And then you would use this face swap technology to basically place Tom Cruise's face on top of that. Now, when this technology first emerged, you actually needed to train on a bunch of Tom Cruise videos. So you probably would like download a bunch of his publicly available videos and train your model specifically to be able to emulate Tom Cruise. And then you would copy that, the likeness of him on top of this body double. And that's kind of what you get, right? And that tech got pretty good pretty quickly. There's actually accounts that got pretty viral for doing exactly this. Alex Heath There's a viral TikTok, the deep Tom Cruise that you have in the report that I remember at the time. Yeah. Gaurav Misra And actually today there's accounts that are impersonating Leonardo DiCaprio and a bunch of other people. These are on Instagram. I mean, if you go find them, they have like, some of these videos have 50 million to a hundred million views on them, which is crazy, but they're using the same method that the generation One method, but that's happening today. One of the things that's changed on that. And I think that's evolving is today's methods of face swap, like don need to train on a particular person. They're kind of generalized. So they're using more recent technology that lets them basically just take a picture and face swap somebody without actually having to train specifically on someone. So even if there's no footage available, sometimes that can be the case. There's people who are still abusing this technology, right, today. Now, the problem with this particular generation is that it's actually much harder to tell than other ones. The reason is that most of the footage is real, right? And if most of the footage is real, then obviously it's going to look real, right? What's not real is just the elements, the features of the face, right? Which is a small component, right? The voice is real. The lip movement is real. The expressions are real. The hand gestures, the body, everything is real. What sometimes can be an obvious ([Time 0:05:18](https://share.snipd.com/snip/83f1a8a0-24e8-49f0-ac28-12f84f8c351d)) - **Lip-Sync Deepfake Risks** - Lip-sync deepfakes let making people say things they never said, a big leap from first generation. - Such models are widely available globally, often without moderation, enabling misuse. Transcript: Alex Heath We saw some of this with even the candidates in the last U.S. Election. And you all say in the report, this is the quote, first time you could make someone that exists say something that was never said. Exactly. Which is a wild statement. And this is relatively recent. Gaurav Misra Yeah. And this is being actually, like, you know, many articles have talked about this. There's been a lot of press about this as well. But, these technologies are being abused. I mean, go look at any of Elon Musk's like Twitter posts. Right. And you'll see like the first few replies are going to be these fake videos of Elon Musk saying buy cryptocurrency or something. You know, I'm giving out cryptocurrency. Come to my website. Like something like that. Right. All of those. It looks like he's speaking it, but these are actually lip synced with one of these models. And there's actually like 60 to 70 providers of these models today, you know, which is also wild. A lot of them, most of them are not American based. They're like in China or other countries where, you know, there's not really much we can do about how they police it. Alex Heath Well, there's not much we can do in the US either, let's be clear. That's fair. Gaurav Misra At least we have some level of control here. Maybe we can pass a law or something. But yeah, there's very little we can do with some of the foreign ones. And a lot of these ones have zero moderation, right? And they have no intent to moderate. No, they don't even have a positive. They want it to be used for fraud and other types of purposes. Like that's okay for them, right? Because it makes money for them. They don't care. Alex Heath Can we name in shame? What are the worst ones? Gaurav Misra I think the worst offenders are the Chinese ones because there's no consequences for them doing ([Time 0:12:06](https://share.snipd.com/snip/4444661c-75a0-4a14-b1e5-b433ebf2cd39)) - **Diffusion Models Generate Whole Videos** - New diffusion-based models can generate entire videos from nothing, without original footage. - This removes original references, making fake videos harder to identify and moderate. Transcript: Alex Heath Frame generation and short form video. And these are the newer diffusion-based models that for the first time you can generate everything from nothing basically in the frame exactly with no existing footage that you're Relying on and i think for a lot of people this is vo3 this is google's latest model that they've seen you can obviously tell that it's fake yeah but you all i think outline how for the first Time this is what's really kind of potentially scary about this and also cool, but definitely scary is that it's this generating everything from nothing concept. Exactly. Gaurav Misra And there's no base footage, right? Essentially, especially from a press perspective, if you want to like prove that something was generated, right, and show that, oh, this was like a deepfake created of like XYZ candidates Saying ABC that they didn't say, right, you can always find the base footage and be like, here's the original. So clearly, you know, this was manipulated, right. But with these sort of transformer models now, which are generating the entire frame, right, they're not, there's not, there's no original footage to rely on, right. So there's nothing at the base of it. There's no thing you can find to prove that, oh, look at that, like this is the original, which is what makes these more difficult to detect in the long run. Some of these technologies are still developing. And so the fidelity may not always be there in all the models that kind of work in this domain, but that's just a matter of time, right? More data and more training that will get really good. And once that's there, once the fidelity is there, right, then you kind of end up in the situation where it's really hard to tell. And you can not only have people say stuff they didn't say, you can put them in locations they weren't in or doing stuff they never did while saying things they never said, right? So it's a whole other level of complexity from like a detection perspective, from a moderation perspective. And, you know, obviously that applies to celebrities and public figures and some of those more abused personas, I would say. Right. But it also applies to like everyday people. You don't want like someone to take your friend and. Alex Heath We'll be right back. Support for this show comes from Salesforce. You might remember a time not long ago when AI wasn't all that helpful. But today, Agent Force, the powerful AI from Salesforce, can analyze, decide, and execute tasks autonomously, operating at speeds and scales no human workforce could match. These AI agents represent a new world of digital labor that not only handles monotonous, low-value work, but orchestrates and carries out high-value, multi-step tasks. This isn't just another step forward. It's an enormous leap, redefining how work gets done and what's possible for businesses and their employees. AgentForce is adaptable, autonomous, and proactive, and, of course, totally integrated into Salesforce, so they're truly part of the team. That way, you and your employees can focus on the tasks that actually move your work forward. AgentForce, what AI was meant to be. Learn more at salesforce.com/agentforce. Gaurav Misra AI is moving fast. So fast, it's hard to keep up. Alex Heath In fact, in ServiceNow's latest AI Maturity Index, scores dipped 20% from last year. But that's okay, because AI isn't a sprint. It's a marathon. Gaurav Misra You may be behind today, but tomorrow, you could be a pace setter. Dive into ServiceNow's AI Maturity Index and see how you can innovate as fast as your ambitions. Visit servicenow.com/AIMaturity. Alex Heath Fox Creative. Gaurav Misra This is advertiser content from Adobe. As the chief marketing officer for a tech company, I lead a team that makes fresh content on a daily basis. But this month has been particularly nuts because we're planning a conference. Alex Heath There are dozens of people working together on all the documents, slides, posters, flyers, and merchandise. Gaurav Misra Fortunately, our event marketer is the best in the business. And she said two words, Adobe Express. Alex Heath Adobe Express has been a lifesaver. Brand kits have kept everything consistent. Templates mean whatever content we need to create looks slick and professional from the start. Gaurav Misra And easy syncing of assets means that anytime a change comes in, like a last minute swap out of our new app logo, I know that every team member will instantly have access to the latest version. That way I know when the curtain goes up. Isn't that new logo gorgeous? Alex Heath Everything will go off without a hitch. Gaurav Misra Adobe Express, the quick and easy app to create on-brand content. Learn more at adobe.com/express slash business. Alex Heath We're back with Caption CEO Gaurav Misra. The tell for this right now is basically just that it, besides the fact that you can clearly tell like with VO3 that this is, it looks crazy. It looks fantastical. Usually it's AI, but there's this eight second mark limit, right? Because that's the model output limit because of, I'm assuming, compute constraints. But that's like for VO3, for example, you can't do more than eight seconds. Gaurav Misra Right. So this is all about the window that the model is trained on. So these models, the diffusion model specifically, like they're trained on a specific length, or at least this generation that exists right now. So that means that they're just trained to produce eight second videos. They can't produce nine second, seven second, nothing else, right? Trained so there is compute constraints so essentially as that window increases so if it's nine seconds 10 seconds 12 seconds there's a quadratic or like a square increase in the amount Of compute needed so eight seconds versus 16 seconds is going to be four times the compute to run it also to train it probably and then same if like you double that you quadruple the amount Of compute right so that's why all of them kind of turn out about this eight to 10 seconds, because it just like scales too fast after that. And there's not GPUs that can like, you know, run that quickly enough for it to be viable. So those things are being solved right now. I mean, we're working on, you know, obviously a next generation model that is not going to have any of these constraints. And we can bypass basically all the compute ([Time 0:15:00](https://share.snipd.com/snip/830af3f3-02bd-439f-afec-a1e53ff55f7f)) - **Unlimited Length, Multi-Person Deepfakes** - Fourth generation deepfakes will have unlimited length and multi-person interaction with consistent settings. - This transforms content creation into a software-like process, enabling entire movie scenes without physical sets. Transcript: Gaurav Misra And so this is kind of where the fourth generation will start, right? And with the fourth generation, we think there's properties that make models of models, the fourth generation, none of the existing models actually capture all of those properties, But there are models today that capture parts and individual elements of what a fourth generation model would look like for this. So for example, unlimited length is a critical one. And we actually just released unlimited length a couple of days ago. It's not publicly announced right now, but it's available to certain users in the app. And so you won't have like eight second or 10 second limits or anything like it can do continuous. I mean, the social post that I shared with you was a continuous shot, right? There was no cuts in the middle of it. Once I was speaking, I was speaking throughout the video, I think it was 50 seconds or so. And I've posted a couple more of those types of videos recently where you'll see there's no cuts. It just like continuously speaks for the video. And even in the deepfake report, when we share some of the fourth generation results, you'll see that they're continuous videos with no cuts in between. So they can go to arbitrary lengths, essentially. And then you'll see models like VO3 excel at another property that we think is going to be important for these fourth generation models, which is multi-person interaction. Because that kind of is the next frontier of a lot of this type of stuff. I would say on top of that, another property that we would probably see is not just character consistency, which, you know, is already being achieved pretty well. It's pretty close, but also location consistency. So let's say there's two, three shots, different camera angles, looking at the same person, the same location, the location remains consistent across those shots, right? So if there's a glass on the table, when the camera cuts to the left, the glass still is on the table at the same location, right? And it's the same glass, right? Those types of things are all being worked on. Like those things we're working on, I'm sure other people are working on as well. You can imagine the positive value of something like that. Like you can literally create completely new environments. You can literally like shoot a movie if you wanted to, without even moving a single item, without even leaving, you know, the room, you can come up with a location, tables, chairs, whatever You want in there, the people that are sitting in there, what they're saying, all of it just like created out of nothing. Just like, you know, in civil engineering, you can imagine a bridge, you can't build it, right? In software engineering, you can, whatever you can imagine, you can build it, right? Essentially, movies and TV and video becomes like software engineering. Anything you can imagine, you can just do, right? There's nothing, nothing physical to stop you from getting it done, right? And so that is the power of what's going to get unlocked in sort of this next generation of models. Obviously, there comes the downsides. Alex Heath Yeah, and you all have this example where ([Time 0:21:50](https://share.snipd.com/snip/fd11b0eb-1e49-4b2e-b8fd-4de9fa24b2e5)) - **Audio Integrity Lags Visuals** - Audio remains a key tell for advanced deepfakes due to lack of contextual background noise and casual tone. - Synchronized audio-video models like Google's are emerging but still imperfect and undertrained. Transcript: Alex Heath Frontier synthetic models. But if the audio was off and it was just the captions and I scrolled past this, I would definitely think this was Jon Stewart. Gaurav Misra Totally. That, I think, is the biggest tell of the fourth generation borderline models that exist today is the audio. Right? Because audio models are obviously making progress of their own, but none of them yet focus enough on sort of like casual everyday voice that sounds like it was recorded very ad hoc. Like a lot of them come off a little bit too professional, a little bit too advertising or podcast or something like that. These models struggle with that because they were just never built for this type of, you know, casual use case. And then secondarily, you know, there's an issue for matching audio and video, essentially, where let's say you're on the streets of New York City in the video, you're walking down, You expect there to be some sort of noises happening, you know, you expect there to be like people talking and walking in the background, maybe cars honking and driving by and things Like that. And so audio models just don't have that context today, right? Like when you're generating a piece of audio, you just can't tell it like, actually, this is a New York City on the street. So make it right for that, right? It also carries through into things like, you know, if you're walking down the street in New York city, like you're probably don't not having like a professional microphone in front Of you. Right. So audio is not going to sound perfect. It's going to be like, kind of like sound like it's recorded at a distance. And so those properties of the audio can't be generated right now. And so because of that is very obvious. Sometimes you listen to the clip and it looks like it's recording a podcast studio, but you watch it and it's walking down the streets in New York City and your brain is just like, this Doesn't connect, right? And so that's like the key problem. I would say Google's VO3, from what we know about it, what people know about it, they haven't really released any details about this, but supposedly they're generating audio and video Simultaneously in a single model, which is definitely a technical achievement if true, because people haven't done that before. And if that is true, then it is doing the right thing. The audio is being generated with context of what the video is going to be. That's why it's able to generate background noises and things like that. It just feels to me a little bit under-trained right now because it has a little bit of the, you can hear sort of cracking in the audio. As if you listen to it closely, you can hear like, you know, almost cracks between the voice. You can, you know, hear it creak a little bit. ([Time 0:24:40](https://share.snipd.com/snip/38000cd3-8b13-464b-aadc-71a5af208a77)) - **Deepfake Technology Is Inevitable** - Deepfake technology is inevitable due to its commercial value and global development. - Engaging in its development helps steer outcomes toward more positive use. Transcript: Alex Heath This is being used already and how I feel, even as a relatively, I would say, informed internet user, which is that I am increasingly unable to quickly decipher what is real or not, why Is this worth it? Why is it worth building this when you're seeing the impacts of it already? It's a great question. Gaurav Misra So, I mean, I think I fundamentally believe that the technology itself is inevitable, right? Like we cannot escape the development of this technology. It's kind of like trying to stop LLMs from happening, right? They will happen. We may or may not be a part of it. We can choose not to be a part of it, but the technology itself is, it's already on its way. And just by the nature of capitalism, it will get built, it will get adopted, right? If the value is there, and it seems like the value is there. And that's what we're seeing. Like we have millions of users using this stuff to literally do, you know, stuff like getting their business off the ground, right? Like it's a one person shop, like a lot of our users are like, by the way, millions of people who are small businesses, like nail salons and like, you know, oh, it's like a plumber or an electrician Or it's like a home cleaning service or, you know, trash removal, lawn services, you know, these types of users, right? Like everyday people that are using our technology to help get leads, sell stuff, make a name for themselves, grow their business, things that are just creating a ton of value in the Economy. So it's hard to argue with this type of value, I think, right? On the flip side, you get these negative effects. And the way I think about that is like, I mean, so many times in today's world, like you look at the state of the world and you imagine, like, I wish things were different. I wish there was something I could do about some war in some country or like some political thing that's happening. I wish I could contribute to it, but all I can do is I can vote or I can donate money. And that's really the extent of anything that I can actually truly achieve, right? As an individual. And so when I get a chance to be actually at the front and center of helping decide where a certain inevitable technology is going to go, how society might change and actually have an input Into it, I'm going to take a seat at the table, right? Like, I'm not going to walk away and be like, you know, not for me. I'm going to take a seat at the table and be like, I would love to help figure this out, right? I would love to work ([Time 0:29:40](https://share.snipd.com/snip/ebb12fdf-e063-45e8-a400-0988cde6c3c0)) - **Potential Regression to Trust Era** - Worst case is society rejecting all digital media and trusting only people, reverting to a trust-based era. - This regression risks misinformation traveling faster but might revive importance of trusted journalism. Transcript: Gaurav Misra Things along the way that were learned? And how can we make sure that mistakes are not made and that things are done the right way? Alex Heath You've done that exercise, like thought out that far? Yeah, definitely. Lay it out. What is the worst scenario? Gaurav Misra I would say one possible scenario, right, which seems most likely given the current information that we have, right? We don't know what technology might develop around, you know, identifying these videos or so on and so forth, right? So given a lack of that information, let's say nothing changes in any of those areas. I think the most likely scenario is that we kind of go back to how things were a hundred years ago. Essentially, video didn't exist. Audio recordings didn't really exist. Photos barely existed. And so let's just call it, they didn't exist. Everything was practically just a matter of trust, right? Like what you believed and what you saw just depended on, oh, well, who wrote it? Well, it's this publication, the XYZ publication. And they're very reputable, you know? It's like, oh, okay, so I believe it, right? Or maybe it's like a trial. And it's like, well, did they do it? Well, we have three witnesses. Do we believe the witnesses? Well, yeah, they're upstanding people, you know, so we believe them. That's sort of the worst case of where things end up, right? Alex Heath You mean like society wholesale rejects this and goes back in time? Gaurav Misra Kind of information digital, right? Like whether it's audio, video, written, right? Like nothing can be actually recorded. So the only thing that can be recorded is what people have seen, right? And that's the only thing that can be trusted, but that can be trusted only as much as you trust the person, right? And everybody has motives to manipulate the truth. So that's obviously something that's the way, but that is how the world worked a hundred years ago, right? That's exactly how the world worked. Not to say that that was a better world. I don't think so. I think we've come a long way from that. But that is the worst we could regress to, right? So that's like the base case, I would say, right? Is that world. I would say the one difference between that world and today is that information can travel much faster today than it could back then, which means that damaging information or information That could potentially have any harm can make its way through a lot more people today and probably in the future even faster than it could 100 years ago. By the 100 years ago, it might have been limited. But at the same time, there's a flip side of that too. 100 years ago, if something made it out, if, you know, some damaging information made it out across the U.S., like, oh, something about a candidate or an election or something, it would Be really hard to counter that because you don't have a way of reaching all those people that quickly anymore either, right? Which is flipped now where you can counter that and be like, hey, this is incorrect information, right? Like, this is the reality. And so, you know, it kind of falls in a balance of pros and cons between, you know, where things are today and where things were back then. But I would say that's like the worst case scenario. Alex Heath And then building on top of that. Sorry, the worst case scenario is not just like society collapses because we go into echo chambers and we don't trust each other and we go to war with each other. That seems reasonable. Gaurav Misra Fair, yeah. I believe in people in society enough to think that if this scenario has played out in the past, and it has literally, right? Like there was a time where you couldn't trust anything but what people said, then it will probably play ([Time 0:33:05](https://share.snipd.com/snip/c5dae0a9-bb29-4139-9a5e-be12fc2c22d4)) - **Watermarking Fails as Authenticity Tool** - Watermarking AI-generated videos is ineffective because they can be easily removed or altered. - Cryptographically signing videos on-device can provide more trustworthy authenticity than metadata. Transcript: Gaurav Misra Crypto signing and things like that. I'm not talking about the blockchain, but like just signing cryptographically videos that are recorded on device as they're recorded with the identity of the person who recorded It. So you know where it was recorded, when it was recorded, who recorded it, as opposed to like an unsigned video, which is like, hmm, it could have come from anywhere. There's no source. There's no crypto signed sort of like signature to say that this was recorded by any particular person. Right. So you might have some concept of like trusted videos, right. Which were recorded on crypto signing devices, right. Where maybe on an iPhone, there's an inbuilt like chip that automatically is signing the video as, as it's being recorded with your, you know, your name or something like that. So, you know, who recorded it and location where it was recorded. So things like that could emerge as, you know, a possible outcome in the future. Alex Heath Isn't that just fancier metadata that can be scraped just like metadata can today? Gaurav Misra Well, it is metadata, but it's cryptographically signed. So it's one step further. I think the problem that I have with today's metadata, like SynthID and like some of those other solutions. Yeah. Alex Heath You're not a fan of watermarks. It says in this report, you don't think that's going to work. Gaurav Misra I think it's just a very thinly veiled attempt at like avoiding the problems. Like it's like the answer you want to give to a journalist when you're like, well, what are you doing? Oh, we put a watermark on it. It's like, yeah, it's so easy to remove the watermark. It takes like five seconds. And by the way, it's easy to accidentally remove the watermark. You may not even be wanting to remove it. You crop the video and it removes the watermark. And so I don't know, what's the point? It doesn't seem to be actually doing anything besides just like pretending that it's solving a problem. And so I'm not really a big fan of that. And, you know, I'd be curious to see, like, I can't come up with how exactly they would make it better than what it is right now. Maybe they have a roadmap. And I'd be curious to see like the companies that are working on that kind of stuff, you know, where something like that might go. But it's not to my satisfaction, I would say. I will say that maybe there are cryptographic solutions though, right? Like that might be an interesting direction. It's still watermarking, but it's like crypto watermarking. Maybe there's something to that. There's obviously like things there too that could go wrong in terms of like on device, people could hack stuff and all kinds of things. But it seems much more reasonable to me if it was crypto signed rather than just like, oh, I just threw some metadata on there that anybody can change literally, right? Or remove, or I could myself literally remove proving that it's false, right? I would also say a big part of it is also like, do you want to mark videos that are real or do you want to mark videos that are fake you know which one is more important and ([Time 0:38:20](https://share.snipd.com/snip/34819c1a-65bc-494d-a026-149bfd2fa612)) - **Human Moderation Against Abuse** - Employ human moderators to review deepfake videos at current scale. - Constantly update safeguards as abuse tactics evolve to stay ahead. Transcript: Alex Heath Having millions of creators use them, as you just explained? What are you doing to keep your models from being used to do bad? Gaurav Misra Yeah. I mean, so obviously we've run into these problems before. Like we talk about in the report how, you know, people have tried to abuse our models before. And it's not a thing that happens like, you know, every once in a while. There's often people who are trying to abuse our models all the time, right? And the thing is, though, in today's world, with a few different options available on the market, and some of the options being completely devoid of any kind of moderation or any attempt To even moderate, if you make it just hard enough, people will abandon it and just go to the place where it's absolutely easy to do it, right? So that's kind of why we haven't dealt with this problem as publicly and as obviously as other companies might have, because we make it just hard enough where they just choose to leave And go to the platform where it's really easy. That's kind of what we've seen more recently. More specifically on what we are doing at this moment with some of most recent models that are actually mentioned in the report. We're starting off with like the most basic approach, which is let's just let's just look at all the videos created. Let's just have human look at all the videos created. Alex Heath And if something can do that, it's you're at the scale where you can have humans look at every video at this moment. Gaurav Misra Yeah, but that may not, know, reasonably be possible at some point in the future. But at this moment, even though it might be time consuming and operationally complex, that's where we want to start. Just so we can understand the patterns, right? Like we might not even understand all the types of abuse that are possible, right? Like there's things that people come up with that we wouldn't even have thought about. One of the ones that I've been thinking about recently is like ID verification abuse, right? Where you kind of take a person's likeness, have them hold up an ID that's like, obviously digitally created one of themselves and use that video and have them say something like, this Is me on this date or something like that. Use that video to get verified on, you know, some sort of verification system. Many companies have these, right? Like clear type stuff, basically. And yeah, I mean, these are the types of things that everyday people will come up with new ways to abuse, right? And it's an ongoing and constant battle, right? It's like, we do things, we wait and watch, we see five new things appear, and then we do more things, and then we wait and watch, and then five new things appear, and people figure out how To bypass stuff, people figure out how to, like, you know, get squeezed past the defenses, basically, right? And then we have to continue improving, basically. And that's, like, never ending, you know, there's no end to that, today or ever, really. But I think this is a ([Time 0:43:40](https://share.snipd.com/snip/1f44843d-e275-4ce8-a75f-bc481eee523c)) - **Non-Public Figure Deepfake Challenges** - There are big challenges stopping misuse of deepfakes of private non-public people. - Current defenses often rely on user complaints and legal actions post-misuse. Transcript: Gaurav Misra Who they might know in real life, but is not a public figure. So we wouldn't be able to identify them and then have them be able to say something, you know, that they didn't say. And in those cases, we have to depend on like people complaining. And that has happened where people reach out to us and said, Hey, like someone made a video. I never said these things. And we have to go figure out who did it, ban the accounts, reach out to them, you know, share the... Usually there's a lawsuit involved between the parties. So we're obviously happy to share whatever information is necessary for that. We have to take those on a case-by basis, basically, right? But it always comes out as like a user complaint or, you know, something of the nature. Alex Heath Yeah, I mean, in your video, your deep fake video accompanying this report, you say, you know, please use this ([Time 0:47:30](https://share.snipd.com/snip/474f3993-b36d-4fe4-891f-b5c01e2c04fe)) - **Awareness Eases Deepfake Danger** - Raising awareness about deepfakes is the fastest way to mitigate their impact. - Educate people on recognizing fake content to reduce misinformation’s power. Transcript: Alex Heath You know, potentially unlimited synthetic video with audio that matches. How do we navigate this? How do we discern what is real or not? Gaurav Misra So I would say the number one thing to do in the short term and the fastest way to kind of mitigate this problem, at least, is to create awareness around it, right? Because I think a lot of people just don't know. Matter of fact, like a lot of that is already happening through platforms like TikTok. I mean, you said it yourself, like anytime there's something off in the video, there's 10 comments there like AI, AI, AI. And that's a small number of people who can tell. A vast majority probably can't, but there's some people who can. And TikTok essentially is giving them, you know, likes for being able to do that. You know, that's just the nature again of how TikTok works, but that is creating awareness. And like, you know, you'll also see in those videos, probably a lot of comments being like, oh my God, I, I could never have told like, this is crazy. Right. I don't even believe it. And now they believe it. And now they know what to look out for, right? Because if they didn't even know it was possible, that leaves them kind of defenseless with this type of technology. So I would say step one is awareness, which is why we're talking about it. And we're open about like, obviously the problems, you know, both for us, for other companies. We're also open about, you know, solutions that we're exploring and where things might go. We're also talking about like what's coming next and not just like how amazing the technology is going to be because that it will, right? But also what the downsides are going to be of these next generation of models too, ([Time 0:54:20](https://share.snipd.com/snip/0e43a820-c3c7-459d-a88b-bc8ce24862bc)) - **Verify Real Videos, Not Fakes** - Instead of marking fakes, focus on cryptographically verifying real videos for trust. - Verified videos should be highlighted on platforms to counter misinformation effectively. Transcript: Gaurav Misra Right? So we can be ready and prepared with solutions for what is to come, right? Even before. And by the way, there's companies exclusively working on identifying this type of stuff on, you know, finding solutions to like detective fakes and things like that. We also want to give those companies like a heads up of like, Hey, by the way, like these are the things that are coming and, you know, we should figure out how to detect these X, Y, Z new things That are going to be happening. Right. So that's the short-term plan. I would say in the longterm, a solution is needed. I think one nice step would be to formalize sort of the legal process around this, right? Like, I think we should have laws that kind of address this area. It's a new area. I get it why these things move slowly, but I think it will get addressed. I think there's motion around that in the government, but I think that's just part of the puzzle because it's a global economy, right? Like just having laws in the U.S. Will not do anything, especially if the laws are like hindering progress and things like that, right? So they have to be done correctly. But I would say beyond that, there's like a more global solution needed. And I actually think there needs to be more of a focus on marking and detecting videos that are known to be real and known to be sourced, known to be correct, right? Rather than trying to like mark every fake video, right? Because I mean, you can literally make fake videos with CGI, right? Like you can, you don't even need AI for it. You can just use CGI to make fake videos. They look super real. You know, you can put them in Hollywood movies. You can make anyone have, you know, people have been brought back to life in movies and stuff before. So this is not something that can't be done if someone, a bad actor, a government, or, you know, wanted to do it today, they could do it with no AI at all, right? And so we should be prepared for all scenarios, including ones that are non-AI on that front, right? But I think the right solution is focusing on the videos that are real, marking them and actually surfacing them on the platforms that are distributing, ([Time 0:55:45](https://share.snipd.com/snip/7a984f1b-1f0d-48f8-902e-ac30b4120d4b))