The Future of AI Video Arrives With Sora 2  ## Metadata - Author: [[Hard Fork AI]] - Full Title: The Future of AI Video Arrives With Sora 2 - Category: #podcasts - URL: https://share.snipd.com/episode/8ec89b8c-40dc-4093-9522-4cd70f702566 ## Document Notes ## Highlights - **Sora 2 Launches As An App** - OpenAI released Sora 2 as a major step in AI video generation and it's packaged as an app for broader access. - Jaeden Schafer found the model "absolutely incredible" after reviewing it for hours. Transcript: Jaeden Schafer OpenAI has just announced Sora 2, their latest video model. I've been looking over it for the last couple hours, and it is absolutely incredible. Today on the podcast, I'm going to be breaking down everything that they have announced with it, basically the capabilities of where it's at today, what it's able to build. I'll show off their launch video. I'll show off a bunch of examples of videos created with it. Also look at the response to this getting over on X in the comment section, which I find is really insightful a lot of the time. And what people are saying over on my own social media. There's so much going on with this and this is an absolutely crazy model. So let's get into it. The first thing I wanted to mention is that this is going to be an app. This is kind of the big thing that they're pushing right now. With the original version of Sora, there was a couple different places. You used to be able to go to Sora.com and then they kind of pulled it into the chat GPT experience. It was for just the highest pain tier and then they had different tiers. They brought it down for more regular people. They did a bunch of things. One of the biggest criticisms I actually got of people when I posted about this on LinkedIn was from my Tom, who said, cool, still no minute long Sora 1, like they said though, which is True. When they did the first Sora 1 announcement, they're like, and it's gonna be a minute long, and blah, blah, blah, and they never actually rolled that out. So I think a lot of people are kind of upset about that. Okay, so the upset stuff aside, let's talk about what this thing is capable of, because I have been really impressed. And the first thing I wanted to do to launch this off was to show off their launch video. If you're watching on YouTube or Spotify, you can see this. If you're on Apple, the launch video is talking. So it's explaining all the capabilities. So you'll be able to hear it. And for anything that's just text on the screen, I'll explain. But let's jump into this. ([Time 0:00:01](https://share.snipd.com/snip/b51fea86-8b72-4ab7-8251-775a317bf239)) - **Integrated Audio-Visual Generation** - Sora 2 generated the entire launch demo including voice, sound effects, and visuals, showcasing integrated audio-video synthesis. - Jaeden highlighted voice cloning and likeness cloning as especially striking features. Transcript: Jaeden Schafer It says everything you're about to see and hear was generated by sora 2 so that's including the sound effects and one year ago sora 1 redefined what was possible with moving images today We're announcing the sora app okay i'm also just going to say really quickly before we do this all of this is voice voiceover by sam altman and there's a an like an animation of him actually Talking it looks like him actually talking but all of it was generated by sora including his voice so really really crazy powered by the all-new sora 2. It's the most powerful imagination engine ever built. And it's packed with new features. I'll pass it to Bill for more details. One thing that I think is really impressive with this whole demo video is they're showing a bunch of interesting video clips that we've created, of course, which is great, but it's the Sound effects that are mind-blowing to me. This is something that these typically have struggled with. A lot of these video generators, they just do pure video. You've got to add all the sound effects after the fact. So the fact that it has sound effects, it can do voices, it can apparently do voice cloning and likeness cloning, like we're seeing Sam Altman, an AI clone of him, is really, really impressive. ([Time 0:01:43](https://share.snipd.com/snip/dd692704-0b5b-4f3c-b244-30d78fdfc8d3)) - **Improved Motion And Physics** - Sora 2 improves motion, physics, and body mechanics for more realistic movement across complex scenes. - The demo shows diverse scenes like ice skating and flying on a dragon to illustrate those gains. Transcript: Jaeden Schafer Now every video comes with sounds. Sora 2 is also the state of the art for motion, physics IQ, and body mechanics, marking a giant leap forward in realism. We're watching a really impressive ice skating video where the figure skater is twirling, which just shows off really complex movement. And we're introducing Cameo, giving you the power to step into any world or scene and letting your friends cast you in theirs. He's flying on a dragon while he says that. Now we have Sam Altman again. On the path to AGI, the gains aren't just about productivity. It's about creating new possibilities. It's also about creativity and joy. One, two, three, four. We now have a sports arena with giant racing ducks. That's why we're launching Sora 2, inside the Sora app, allowing everyone to push the limits of their imagination and create in ways we never thought possible. ([Time 0:02:56](https://share.snipd.com/snip/3fa3c57a-f7af-491e-9d50-36f24d13d15c)) - **Persistent World State Reduces Errors** - Sora 2 models persistent world state allowing consistent objects and characters across multiple shots. - This reduces bizarre artifacts like teleporting objects and enforces realistic physics like rebounds. Transcript: Jaeden Schafer Okay, so what's interesting to me with all of this, oh, they have like a blooper final scene at the end, driving a fancy car, and then it says Bill will return in Sora 3. So I'm assuming Bill is like an AI avatar person that they're going to use to be their mascot for all of their video announcements. Okay, I think this is really interesting. They said, uh, in their post, they said the original SOAR model from February, 2024 was in many ways, the GPT one moment for video. I would tend to agree with this. Um, it was interesting. It was like, it could do a lot of stuff, but I didn't see a lot of people actually using it. Uh, they said the first time video generation started to seem like it was working and similar behaviors like object permanence emerged from scaling up pre-trained compute. They said, since then, the Sora team has been focusing on training models with more advanced world simulation capabilities. We believe such systems will be critical for training AI models that deeply understand the physical world. A major milestone for this is mastering pre-training and post-training on large-scale video data, which are in their infancy compared to language. So I think what's interesting here, we're going to be seeing the advancements are going to get really impressive and they're going to move forward a lot. We're at the very early stage of what we can do with the video, but they have a really clear path forward to how they can do this better. Some people think like, oh, we've hit a peak or we've hit a plateau with AI capabilities, especially when they're talking about like text models and how smart they are. But we're really, and whether that's true or not, I don't think anyone is saying that about video. We're just at the very tip of the iceberg with what we can do with video and what it's capable of. And we definitely haven't hit a plateau on making these models much, much better, which is really, really impressive. They said prior video models are over optimistic they will morph objects and deform reality to successfully execute upon a text prompt. For example if a basketball player misses a shot the ball may spontaneously teleport to the hoop. In Sora 2 if a basketball player misses a shot it will rebound off of the backboard. ([Time 0:04:26](https://share.snipd.com/snip/5b3ec5fd-0a5a-4093-8f4f-15db14e3ea8c)) - **Styles From Cinematic To Anime** - The model supports multiple visual styles, including cinematic realism and anime, enabling full animated movies. - Jaeden emphasized this as a first real path to easily creating full-length animated content. Transcript: Jaeden Schafer Interestingly mistakes in the model makes frequently appear to be mistakes of the internal agent that SOAR2 is implicitly modeling. So this is really interesting because it gets to this whole question about like the laws of physics, how these AM models obey laws of physics. And apparently this is a lot better. They have a video of like a dog jumping around a dock. It's into objects and it's doing a much, much better job than what you would have seen from other models. And so they say that this model is a big leap forward and control ability, which is basically the ability to follow intricate instructions. You can do it across multiple shots with an accurate persisting world state, meaning you can have kind of like the same environment and you can have a person from multiple angles and Multiple places inside of that environment. It is super realistic, cinematic. They can do anime styles. They can do a lot of really impressive things. And the cool thing for me is like with the anime and the, you know, they have, they have like a video of like a dragon. It looks kind of realistic, but kind of cartoony. You could literally make a full-on animated movie. ([Time 0:06:28](https://share.snipd.com/snip/aff25531-bf47-4201-9190-7558c149b08d)) - **Early Stage With Visible Flaws** - Jaeden notes Sora 2 is like "chat GPT 3.5" for video, meaning it's an early but significant stage. - He pointed out small imperfections in demos, like an unnatural hand pose in one clip. Transcript: Jaeden Schafer They have tons of these really cool animation-type videos that they've created, and you could create full-on movies with this, which is quite exciting, I feel like, finally for the First time. Now, of course, this is like chat GPT 3.5. We'll have to get to 4, and then we'll have to get to GP 5 on the video eventually. So it's definitely not perfect. Even in some of their demo videos, there's like one that they showed with an Asian guy that is like in this pool, and he's spinning this stick around. And at the very end of the video, when his stick is in resting position, his hand looks kind of like twisted in a weird way that's not natural. So even in their demo videos, it's not perfect, but you definitely feel this is coming leaps and bounds ahead. They have one where an ostrich stole a guy's hat and it's running away with it. The guy's trying to chase it and get it back. ([Time 0:07:35](https://share.snipd.com/snip/112f26bf-6291-48af-bf01-9f4f41617567)) - **App Focused On Creator Inspiration** - Sora 2 includes a social app with discovery, remixing, and a customizable feed focused on inspiring creation. - The app emphasizes showing content from people you follow to encourage creative use, not just engagement clicks. Transcript: Jaeden Schafer And it's pretty funny. So they said, on the road to general purpose, this is over on their announcement page. They say on the road to general purpose simulation and AI systems that can function in the physical world, we think people can have a lot of fun with models we're building along the way. We first started playing with this upload yourself feature several months ago on the Sora team, and we all had a blast with it. So basically in this new announcement, it's going to be on the app for iOS. Inside it, you can create and remix other people's generations. You can discover new videos. They have a customizable Sora feed, and you can bring yourself or your friends into video cameos. So they said with cameos, you can drop yourself straight into any source scene with remarkable fidelity after a short one-time video and audio recording in the app to verify your identity. And we've seen this with other video apps like this, like Hey Jen, where basically they're going to make you prove that you're not just like deep faking somebody else. So you got to do an audio recording and you got to do like a video of yourself and they have ways to basically verify that you're the actual person. And after that, you can upload videos of yourself or pictures of yourself and it's going to be able to animate them which is really really cool they said um they launched the app internally Last week and they've heard from a lot of people that are are loving it so yeah it's very interesting they said there's definitely concerns because it's sort of a social media platform At this point that they're rolling out with this video thing they said there's concern about doom scrolling addiction isolation and real-time sloppedmized feeds are top of mind. So here's what we're doing about it. I think it's hilarious that they're calling this sloptimized feeds, basically this AI slop. They're worried that everything's going to turn into AI slop. I think as the models get better, this is less of a problem. But they said we're giving users the tools and optionality to be in control of what they see in their feed. By default, we show you content heavily biased towards people you follow, interact with, and prioritize videos that the model thinks you're most likely to use as inspiration for your Own creation. So it's interesting. They're explaining their algorithm here, and it's not just like what's going to get the most engagement or what's going to be maybe the most sensational, but what's going to make you Want to create more, which is interesting. And of course, that goes into the creation loop. So it's in their best interest, but I find it interesting that that was kind of their, one of their big, um, philosophies and they have a whole bunch more details on what they call their Feed philosophy. You can go into overall, um, this is super phenomenal. There's a ton of great responses. Of course, it's all on an app. They said, Andrew users will be able to get it once they have an invite code from someone who already has access. Otherwise, it's available. And they said, we also plan to release Sora 2 in the API. So Sora 1 Turbo is going to keep remaining available. And everything that people have created with it are going to continue to live on the Sora.com library. But now they've created this new one. ([Time 0:08:24](https://share.snipd.com/snip/767b75a0-fcb4-43cb-88c3-1355967b0a12)) - **Mixed Public Reaction And Creative Promise** - Public reaction mixes amazement and concern about addiction and whether video models are getting enough hype. - Jaeden sees major creative potential for creators if Sora 2 reliably solves video production pain points. Transcript: Jaeden Schafer And they said, we also plan to release Sora 2 in the API. So Sora 1 Turbo is going to keep remaining available. And everything that people have created with it are going to continue to live on the Sora.com library. But now they've created this new one. People on X are saying, can't believe video models are not getting as hyped as LLMs. This is just insane. Anyways, someone said, what problem does it solve? You're selling digital cigarettes at this point, which is funny because, you know, maybe people are going to get addicted to AI videos. But at the end of the day, this is obviously a very useful tool. If this can solve this kind of image or this video creation problem with AI that people have been looking for, this could be amazing for creators making all sorts of content. So really excited for the creativity that comes out of this one. Let me know what you think. ([Time 0:10:58](https://share.snipd.com/snip/31247f36-91b5-4e0e-8e55-27c0de86c3dc))