OpenAI Scrambles for Computing Power - DTNS 5119  ## Metadata - Author: [[Daily Tech News Show]] - Full Title: OpenAI Scrambles for Computing Power - DTNS 5119 - Category: #podcasts - URL: https://share.snipd.com/episode/73ca1bbc-2617-4134-bf5c-7257a0ccaf53 ## Document Notes ## Highlights - **OpenAI's Portable Voice-First Device** - OpenAI is developing a palm-sized, screenless device with cameras, mic, and speakers to be always-on and conversational. - The product faces challenges in personality, privacy, and scaling compute for many users. Transcript: Robb Dunewood The Financial Times reports on the status of OpenAI's hardware effort. You might recall that OpenAI acquired one of Johnny Ives' companies in May and the team around that has been building OpenAI's hardware strategy. FT sources say that it's a palm sized device. There have been some other leaks that say this as well about the size of a smartphone, but no screen. It has a camera or possibly multiple cameras, a microphone, some speakers, and it would be meant to sit on a desk and then respond to you kind of like an Amazon Echo, but a lot smaller. And you could also carry it around if you want. It would always be on to collect data and would not need a wake word to interact with. It had previously been reported that the device might arrive by the end of next year, but Financial Times says that's probably going to be later. Maybe we're looking at 2027. One challenge is deciding on the voice and the mannerisms of the device's personality. If you don't got a wake word, it needs to know when to talk and how to be useful and not annoy you. Another challenge is how to balance privacy. If it's collecting data all the time, how do you balance privacy with the utility of having the model know a lot about you? And another problem is, of course, how to get enough computing power to make these models work well for a large number of users. ([Time 0:02:40](https://share.snipd.com/snip/d564d338-d012-4816-ba5d-99021ba26dd5)) - **Learning To Use Voice Devices** - Tom Merritt shared his initial confusion about the purpose of voice assistants until he learned useful routines for them. - He compared his Echo-to-Nest switch to how user habits determine whether a device becomes valuable. Transcript: Tom Merritt But it wasn't clear when I first got it. So I'm looking at this. And even though we haven't seen anything yet, I'm wondering what am I going to use this for? But it's Johnny I, so it'll probably be awesome. Oh yeah. Robb Dunewood It'll be a big blank white room. Uh, I actually, he probably won't do that anymore. That's his Apple signature. But, uh, I, I get a little more from this financial times article about what this might end up being. Uh, it's very much around the personality. It's very much sounds like a voice assistant, but not one that's just meant to sit in a room that you talk to, one that you could easily move with you, takes up less space. That makes me wonder, well, could I just do this on the phone? But my guess is when OpenAI can control the hardware stack, they can make it faster. They can make it do things that it couldn't do on Android or iOS where they're having to negotiate with the operating system and the lockdowns and the privileges and all of that. But then I, as a consumer, do I want to carry around two of these? How expensive are they going to be? Maybe I'll buy a couple of them. And I don't know. ([Time 0:04:28](https://share.snipd.com/snip/81fd0688-ed58-488d-ae12-b438e5a0e80d)) - **Sora Reveals Inferencing Bottlenecks** - OpenAI's Sora app exploded in popularity, straining infrastructure and prompting stricter copyright checks. - High demand revealed that inferencing needs are acute and unpredictable for new multimedia products. Transcript: Robb Dunewood OpenAI's Sora app for generating 10-second videos that you can put yourself and your friends in shot at the top of the Apple App Store over the weekend. Speaking of needing compute power, as usage increased, OpenAI has had to make adjustments to the app, including tightening restrictions against copyright infringement. They had a restriction that said no public figure should be allowed to be used unless that public figure had opted in. But rights holders would need to opt in to blocking their products from being used. In practice, it was blocking a lot of stuff that I don't think people had opted in, but they've tightened it up now. It is a lot harder to get it to output public figures and copyrighted material than it was. And they are now saying that they have added controls for rights holders so that they can decide, yes, you can use our property as long as you're not doing something political or something Negative or this or that, right? OpenAI also added a feature that lets you restrict what kind of content your cameo, that's the video representation of yourself, can be used for. For instance, Justin Robert Young could stop all of us from putting him in videos with mayonnaise because he hates mayonnaise. And there's been about a dozen made of him eating mayonnaise. Because of that, he could put in a little thing that says, don't let my cameo be used along with mayonnaise. And obviously, you could have other restrictions as well. You can already restrict who uses your cameo. Rob, I know you've been playing around with this, but you're on Android. So you've got a few restrictions on what you can do because of that. Tom Merritt Well, thank you for my invite. You gave me an invite last Thursday, but this is the first time in a long time that I've actually felt like, oh, I'm an Android user. Therefore, I'm a second-class citizen because I cannot partake in all of the new hotness that Sora 2 offers because I don't have the ability to actually scan my own likeness and put my Own voice in there and make stuff of myself. That being said, I've been playing around with it. Now, I haven't really posted much yet just because if I want to post stuff, I probably want to post stuff with me in it. But I have been playing around with it. And just before we did the show, I made a video and it was just make a Samsung commercial for a new smartphone with African-Americans that are Gen X. Okay. Robb Dunewood And it did it. Tom Merritt It did it. That was the prompt. It just did it. It did a commercial that would have been passable. I mean, it could be a 10 second ad that you would skip when you're watching YouTube videos. It was it was it was that good. And then I started playing around and, you know, changing camera angles and you can actually make it zoom in so that that sore logo is kind of obscured a bit. This is impressive. You know, I I'm on the record. I think this thing is going to be huge. I mean, like tick tock I think this is going to be one of the biggest social media platforms that we've seen since probably TikTok. Robb Dunewood Yeah, I was really having fun with it on Friday and Saturday. And then Saturday morning, they said, you've done too many requests. And they put me in timeout for like eight hours. Where I couldn't create anything. I could still view stuff and like it and all of that, but they definitely are having a compute problem because this is more popular than they thought it would be. They thought it would be kind of popular, but they have found that it is more popular and I was not the only one complaining that like, oh, I'm on timeout, I can't post anything. So I think it's a widespread issue as they grow that user base and need to add that compute. I also noticed when they added the copyright lockdown. Now, before they added the lockdown, I would say about 50% of the stuff I was doing, they would say, that might infringe on a public likeness. That might infringe on a copyright. Once they added the lockdown, it went to like 70%. And wasn't trying to make stuff that was infringing copyright. I was just like, oh, it'd be fun to do this. So it was interesting to me to hear that your Samsung went through, which implies either that slipped through the cracks or Samsung may have said, yeah, it's okay if they're making commercials Of us, we'll let them do that. Tom Merritt Well, here's the thing. I said Samsung, but it just showed me a non-iPhone smartphone. It didn't necessarily look like a Samsung phone. Robb Dunewood Gotcha. Tom Merritt So, so, so I did notice that. I also noticed that literally an hour before we are recording this show, I don't have any governors telling me that I've created too much, but the website is choppy. It's like, it's, it's, it's taking his time to do stuff. And it's not just taking time to generate the video, just clicking on different areas, looking at my profile, going to look at other people's videos. ([Time 0:06:24](https://share.snipd.com/snip/33075413-c8e6-4826-a7b9-180dc16dc15c)) - **Android Limitations With Sora** - Tom Merritt tried Sora on Android but lacked features like scanning his likeness, making him feel like a second-class user. - He still generated acceptable demo videos and praised the app's video generation quality. Transcript: Tom Merritt Well, thank you for my invite. You gave me an invite last Thursday, but this is the first time in a long time that I've actually felt like, oh, I'm an Android user. Therefore, I'm a second-class citizen because I cannot partake in all of the new hotness that Sora 2 offers because I don't have the ability to actually scan my own likeness and put my Own voice in there and make stuff of myself. That being said, I've been playing around with it. Now, I haven't really posted much yet just because if I want to post stuff, I probably want to post stuff with me in it. But I have been playing around with it. And just before we did the show, I made a video and it was just make a Samsung commercial for a new smartphone with African-Americans that are Gen X. Okay. Robb Dunewood And it did it. Tom Merritt It did it. That was the prompt. It just did it. It did a commercial that would have been passable. I mean, it could be a 10 second ad that you would skip when you're watching YouTube videos. It was it was it was that good. And then I started playing around and, you know, changing camera angles and you can actually make it zoom in so that that sore logo is kind of obscured a bit. This is impressive. You know, I I'm on the record. I think this thing is going to be huge. I mean, like tick tock I think this is going to be one of the biggest social media platforms that we've seen since probably TikTok. Robb Dunewood Yeah, I was really having fun with it on Friday and Saturday. And then Saturday morning, they said, you've done too many requests. And they put me in timeout for like eight hours. Where I couldn't create anything. I could still view stuff and like it and all of that, but they definitely are having a compute problem because this is more popular than they thought it would be. They thought it would be kind of popular, but they have found that it is more popular and I was not the only one complaining that like, oh, I'm on timeout, I can't post anything. So I think it's a widespread issue as they grow that user base and need to add that compute. I also noticed when they added the copyright lockdown. Now, before they added the lockdown, I would say about 50% of the stuff I was doing, they would say, that might infringe on a public likeness. That might infringe on a copyright. Once they added the lockdown, it went to like 70%. And wasn't trying to make stuff that was infringing copyright. I was just like, oh, it'd be fun to do this. So it was interesting to me to hear that your Samsung went through, which implies either that slipped through the cracks or Samsung may have said, yeah, it's okay if they're making commercials Of us, we'll let them do that. Tom Merritt Well, here's the thing. I said Samsung, but it just showed me a non-iPhone smartphone. It didn't necessarily look like a Samsung phone. Robb Dunewood Gotcha. Tom Merritt So, so, so I did notice that. I also noticed that literally an hour before we are recording this show, I don't have any governors telling me that I've created too much, but the website is choppy. It's like, it's, it's, it's taking his time to do stuff. ([Time 0:07:57](https://share.snipd.com/snip/9bc44f8e-bdf8-4eec-a423-487a2d37eb29)) - **Avoid Single-Vendor Compute Risk** - Diversify infrastructure vendors instead of relying on a single supplier for massive AI workloads. - OpenAI is buying AMD-powered data centers to supplement NVIDIA for inferencing capacity. Transcript: Robb Dunewood And then speaking of compute, OpenAI continues to strike deals to add more computing power, the latest being its intent to purchase six gigawatts worth of data centers powered by AMD Chips, starting with the MI450 in the coming year. They're going to do one gigawatt in the coming year with the MI450. These will be for inferencing. I see everybody, including the Wall Street Journal, Bloomberg, saying this is a shot over the bow of NVIDIA. But I think what OpenAI is saying is no one can provide all the compute we think we need. So we're going to buy AMD chips to supplement the inferencing. NVIDIA is still the preferential partner and will do almost all of the training that they do. OpenAI will also receive stock options that could see it take up to a 10% stake in AMD. OpenAI has announced deals in the last few weeks with NVIDIA as its preferred chip provider, with Oracle, and with Broadcom to start designing its own chips. And CNBC reports OpenAI has now narrowed down sites for its new data centers in the SoftBank-funded Stargate project to 20 sites. So they're making some progress there as well. ([Time 0:11:14](https://share.snipd.com/snip/41971389-51f0-4da8-b6b8-4cc0970074ad)) - **OpenAI's Multi-Pronged Hardware Push** - OpenAI will use AMD MI450 chips for inferencing and keep NVIDIA for training while seeking chip design help from Broadcom. - The company also secured potential equity in AMD and is narrowing data-center sites to scale fast. Transcript: Robb Dunewood The coming year. They're going to do one gigawatt in the coming year with the MI450. These will be for inferencing. I see everybody, including the Wall Street Journal, Bloomberg, saying this is a shot over the bow of NVIDIA. But I think what OpenAI is saying is no one can provide all the compute we think we need. So we're going to buy AMD chips to supplement the inferencing. NVIDIA is still the preferential partner and will do almost all of the training that they do. OpenAI will also receive stock options that could see it take up to a 10% stake in AMD. OpenAI has announced deals in the last few weeks with NVIDIA as its preferred chip provider, with Oracle, and with Broadcom to start designing its own chips. ([Time 0:11:29](https://share.snipd.com/snip/2b2a0e2c-d278-4093-9254-de79bdf69801)) - **Heir Apparent Signaling At Apple** - Apple is managing succession signals by leaking potential successors to test market and investor reactions. - Jeff Williams' reported departure highlights the importance of naming visible successors even years ahead. Transcript: Tom Merritt An investigation into Apple's collection of user recordings for Siri has been referred to the Office for Combating Cybercrime in Paris. Apple requires users to opt into a program to let Apple retain recordings for up to two years to help improve Siri. The complaint says sensitive conversations can be listened to by contractors. And Bloomberg's Mark Gurman says his sources say that Apple COO Jeff Williams is leaving at the end of the year. Apple needs to find a new potential successor to CEO Tim Cook. Gurman believes the most likely candidate is hardware engineering chief John Ternus. Robb Dunewood We are thrilled to introduce the impossibly thin iPhone Air. At 5.6 millimeters, it's the thinnest iPhone we've ever made. And it's also exceptionally light. It's unlike anything you've ever held before, and it's packed with our most advanced features. Tom Merritt Ternus introduced the iPhone Air in the latest Apple announcement. He's 50 years old and trusted by Cook to contribute to product maps beyond the scope of the hardware chief. Robb Dunewood Yeah, it's still early, but Gurman seems convinced. ([Time 0:16:20](https://share.snipd.com/snip/b721631a-af0a-4bc0-b417-78dab266ac52)) - **Never Skip External Backups** - Always keep off-site backups and cold storage for critical systems to avoid catastrophic data loss. - Relying solely on a large centralized cloud without external backups risks permanent loss after disasters. Transcript: Robb Dunewood South Korea's Ministry of the Interior and Safety reported Wednesday that a fire at the National Information Resource Service's Daejeon headquarters destroyed a G-Drive cloud Storage system that contained files for more than 750,000 civil servants. The system had been in use since 2018 and gives each worker up to 30 gigabytes of storage. However, because of that large capacity, they had not done external backups. Now, not all ministries relied on the system exclusively, thankfully, but some did. And some official documents were also stored in the government's ONARA system and may be recoverable once that system is restored. But yeah, local, on-device, local, and external backup. External backup, very important in that system. Tom Merritt It is kind of crazy to me that they're not backing this stuff up. 750,000 people's work is gone. Maybe you can get some of it back. There's a potential chance that you're not going to get a lot of it back. That just, that violates so many principles of good data recovery strategy. There's that you have to have an off-site back, or have some you know some offsite cold storage or something cold storage right so maybe you lose a little but not everything but ([Time 0:18:23](https://share.snipd.com/snip/b4005d9a-1a0d-46d4-9a7c-bb0dd5ae7c8e)) - **Design + Messaging Drive Privacy Fears** - Instagram added clearer location-sharing indicators and removed profiles from maps to reduce user confusion. - The feature is off by default, showing how design and communication affect public perception. Transcript: Tom Merritt I'm guessing the procedure will change. Instagram will update its map feature to make it easier to see if you're sharing location. An indicator whether you're sharing location or not will show up at the top of the map under the profile photo in the notes tray and on the DMs page. Location is off by default. Instagram has also removed profiles from the map to make it clear that locations are not real time. And a reminder will show up when you add location to a post, if you have location on for that post. The map feature, which is available in Canada and the United States, is expanding to India as well. Man, people lost their minds when they launched this feature. Robb Dunewood It was off by default. That was the only thing I needed to see. It was like, oh, okay, so if I don't do anything, nothing happens. But even if I did turn it on, it was making a map of places I've been, not tracking me at the time. And people didn't understand that. So Instagram is still doing spin control here. ([Time 0:20:12](https://share.snipd.com/snip/7ece3daf-1be8-47f0-bb4e-97916ef10b1d)) - **Europe Attracts Data Center And EV Interest** - Europe is attracting data center investment and Chinese EVs/autonomous vehicle makers, signaling supply and demand shifts. - Power infrastructure stocks rose as investors expect increased data-center builds across Europe. Transcript: Robb Dunewood Europe is getting some increased attention from multiple tech sectors. A track of shares for European data center and infrastructure operations is up 23% on the year. So this is like a bundle of shares in those sectors, power generators in particular leading the charge, as investors expect there to be a lot of data centers built in Europe and new power Capacity needed for them. Reuters also reports that Chinese autonomous car makers now targeting Europe for expansion of their sales. Chinese EVs have been moving into Europe and autonomous cars are coming on their heels or I guess on their bumpers. Tom Merritt Yeah, EVs made in China, they are there. Tesla is still Tesla, but they are there. ([Time 0:21:41](https://share.snipd.com/snip/8d71f734-8364-4ea5-8482-8f3d803523f4))