Why Is DeepSeek So Popular? - DTNS 4943  ## Metadata - Author: [[Daily Tech News Show]] - Full Title: Why Is DeepSeek So Popular? - DTNS 4943 - Category: #podcasts - URL: https://share.snipd.com/episode/2e5b1522-4cd8-4d88-af0b-243efd6a4705 ## Document Notes ## Highlights - **DeepSeek's Rise** - DeepSeek, a Chinese AI model, is gaining popularity due to its open-source nature and lower cost than competitors. - It uses innovative techniques like chain of thought and multi-head latent attention for better accuracy and efficiency. Transcript: Tom Merritt I'm Tom Merritt. I'm Rob Dunwood. Let's start with what you need to know in the big story. And yes, it's about DeepSeek. Last week, we mentioned the release of China's DeepSeek R1 model. It is open source, runs at much lower cost than competitive models and over the weekend DeepSeek's app rose to the top of Apple's App Store charts and it has had so many signups that it Has had to limit new accounts. It also says it has come under a denial of service attack which complicates matters all of this attention has caused some investors to begin selling off their stock in companies like Nvidia and asml who are chimp companies that are expected to do well with people having to need a lot of chips because deep seek claims it can be trained on a lot fewer chips than the models From open and meta need. Deep Seek was founded by hedge fund manager Liang Wen Feng as a fintech company. He was a quant. It was a hedge fund thing. But China began cracking down on fintech, so they pivoted to AI. U.S. Restrictions on what chips they could use encouraged them to create a model that could run well on lower end chips. And that's what we've got today. DeepSeq is using what is called chain of thought. This is similar to OpenAI's O1 model. Anthropic has a version of this. Meta's working hard on a version of this. Chain of thought helps reduce inaccuracies and so-called hallucinations. As we've talked about a lot, large language models simply try to make a very good guess on what the next bit of data should be, what the next letter in the word is, what the next pixel in the Image is. Chain of thought lets the model backtrack and try again. So I'm oversimplifying, but it basically checks its work and then uses the results that come most often and fit the facts well. If you know the space of machine learning, it's a bit of reinforced learning inserted into large language models. And that means the responses take a little longer. So maybe 30 seconds instead of a second or even up to five minutes, but they're more accurate. They're also more expensive because you have to use more chip power to do the more tries uh the more tries you give a cot model the better the result but the more money you spend supposedly In some cases a response from open ai's o1 model could cost three thousand dollars of compute power now that's an outlier but it gives you an idea of what we're dealing with here. DeepSeek, on the other hand, was forced to optimize to work less on cheaper hardware. So there's a very good description of this at the end of a post about NVIDIA on Jeffrey Emanuel's YouTube transcript optimizer.com blog. But the highlights are that it uses 8-bit floating point numbers instead of 32-bit. That may tell you everything you need to know, but that saves memory. In exchange for a little precision, but you've got the chain of thought model that keeps it accurate. This means it needs far fewer GPUs to train. There are several other tricks. One of the most impressive is called multi-head latent attention. The MLA system from DeepSeq can compress how individual tokens of information are represented so that only the most important are captured. Again, that's going to use less memory. It's wasteful to store a whole index when you only need part of it. So DeepSeq doesn't do that. They do some good load balancing and how the GPU communicates. They also load balance the actual large language model transformers that create answers to queries. And again, I'm gonna oversimplify here, but they create multiple smaller models that are good at specific types of queries, and have created a smart way to route the queries to those Models as needed. Emmanuel describes it as having small committees of experts on various topic matters, and they know which ones to pick. Those models can be smaller and more efficient to run. And even though the aggregate number of parameters across all those models would be very high, you don't have to run all the models at any given time. So the idea is it can run on two NVIDIA 4090 GPUs, which cost you a couple thousand bucks, rather than a bunch of expensive H100 GPUs, which would cost you 40,000 bucks. And of course, it is against the rules right now for DeepSeek to get H100 GPUs from Nvidia. Now, there's a lot more in the details, but the upshot here is that DeepSeek can charge 95% less money for inference requests on its API than say OpenAI or Anthropic can. This is all impressive unless DeepSeek is lying to us and secretly using a bunch of H100s that it can't admit that it has and then trying to play it off. That would probably be revealed in the short term, but there are some smart people out there who think that's what's going on. Emmanuel notes on his blog that, and here I'll quote him, On the AIME 2024, one of the most challenging high school math competitions, R1 achieved 79.8% accuracy, matching OpenAI's O1 model. On Math 500, it hit 97.3% and achieved the 96.3% on CodeForce's programming competitions. So whatever it's doing behind the scenes, it is doing it well. It has received a lot of praise for not only being open source, but also making it easy to see the iterative process the model uses to come to its answers. The company has not fully documented its energy costs, though. So that's where some people go, what are they really doing to train these models? It's clear that the models work well once they're trained. As we mentioned before on Friday's show, DeepSeek also complies with Chinese content restrictions. So don't ask it about Tiananmen Square. It's not going to give you an answer. But because of that transparency, the app can show its reasoning. And it has showed in a couple of cases that it is trying to follow government aligned responses now that shouldn't be a surprise this chinese company they're not going to be allowed to Operate out of china without doing that but something to keep in mind this may all end up being a scare that does not have depth that is quite possible it may end up just being a simply a new And more efficient way to handle models and open ai anthropic and meta and everybody else could do this as well. ([Time 0:02:32](https://share.snipd.com/snip/5a7a28dc-e5d9-4bed-86e6-37093ba38488)) - **AI Industry Volatility** - The large language model industry is still new and volatile, as seen by DeepSeek's impact on the market. - One company's innovation can significantly shift the landscape, indicating the industry's immaturity. Transcript: Robb Dunewood What do you make of all of this? It has definitely caused panic. And what's interesting about this and what it reminds me is that we're just over two years into the world revolving on large language models. I mean, this is, you know, we're just, you know, 28 months into this. I don't even think it's that at this point. So the fact that it was able to move the markets as much as it was able to kind of let you know that this is still a burgeoning industry. It's still brand new. We don't know, you know, I think we kind of know who the winners and losers are going to be. At least we think we know. But I think ultimately this is it shows you this is still a brand new industry and that one company coming out with something can move the market as much as it did kind of illustrates that. Tom Merritt Back in September. It has not been out for very long. I think the fact that this took off over the weekend caused a lot of people to flash back to chat GPT when it first came out, suddenly rocketing to the top of awareness. So it looks like it's a solid model on the parts that we can see. Maybe it is secretly trained in a different way than they're saying, but it does work well. Right. And I think that that has put the spook in a lot of people. A lot of folks are a lot of folks who know what they're talking about are saying there's maybe less going on here than it looks like. Tomorrow's show to kind of give us the insider perspective. ([Time 0:08:50](https://share.snipd.com/snip/3862c8bd-c081-4282-bd34-3118db6d1efb)) - **YouTube's Ad Blocker Crackdown** - YouTube viewers report hour-long unskippable ads, potentially related to ad blocker usage. - YouTube's response suggests it may be disabling playback for ad blocker users, though their policy limits non-skippable ads to 15 seconds. Transcript: Tom Merritt It's kind of hard to keep track of all those name changes oh that's orb right okay that's altman's thing uh and essentially they're saying you can sign people up you just can't pay them For that which is interesting a lot of youtube viewers are out there saying they are getting served hour-long ads on YouTube without an option to skip them. Meaning if they want to keep watching YouTube, they have to wait an hour or more, or more likely just close the app or reload the page. But it's annoying. YouTube does serve hour-long ads. You may not have ever run across them, but usually they have an option to skip them. YouTube's policy is that non-skippable ads can't be longer than 15 seconds. So what's going on here? Android Authority reports that Google responded to its question about the issue with a statement that it has, quote, launched a global effort to urge viewers with ad blockers enabled To allow ads on YouTube or try YouTube Premium for an ad-free experience. It also said it reserves the right to disable playback controls for users violating its terms of service by using ad blockers so rob it stops short of confirming the hour-long unskippable Ads are intentional but it sort of left it out there like yeah they're probably using ad blockers probably using ad blockers and maybe something kind of broke and trying to figure out Who's using the ad blockers yeah and Google doesn't care if it broke for people using the ad blockers. Now, if we get a bunch of people saying, I'm not using an ad blocker and this still happens, that's a whole different story. But it made me think of our friend, Chris Ashley, complaining about working out and the hour-long ads coming up. And he's like, yeah, I know I can skip them, but I had to stop my workout and go over to do ([Time 0:14:14](https://share.snipd.com/snip/1ce4faa6-ecd0-4cfd-ad4a-6837907a49ad)) - **Matter's Growing Adoption** - Matter is becoming more prevalent in smart home devices, even among smaller manufacturers. - Apple's adoption of Matter certification in place of its own will further streamline compatibility. Transcript: Robb Dunewood CES is huge. My feet can definitely attest to this. Tom decided to follow up on one of the categories that may not have gotten as much attention as it deserved. Tom Merritt I'm talking with Richard Gunther, host of the Smart Home Show and Home On. Richard, thanks for joining me, man. Hey, Tom. Richard Gunther Good to be here. Tom Merritt Good to have you. We have finally put CES behind us, perhaps maybe not fully unpacking from CES, literally and figuratively. Richard Gunther Yeah, I think all of us have that problem. Tom Merritt Yeah. But you do a great job covering smart home. So I thought it would be good to kind of, now that we have some distance between us and the show, wrap up some of the smart home stuff that you saw at CES, because I think it's one of the more Interesting parts of the show and doesn't get quite as much spotlight as the big announcements and the chip announcements and all that. So, so talk to us about some of the highlights in Smart Home that you saw there. Richard Gunther Yeah, I think that's a fair point about coverage. And interestingly enough, though, it's taking up more and more acreage on the floor there. So we have lots of vendors, lots to see. I saw three big themes that caught my interest this year. The first of them is really the continual rollout of Matter, which is that platform or protocol that was designed to help smart home products work together, whether they were originally Intended for Google ecosystem or Amazon's ecosystem or someone else's ecosystem. Now, Matter will help these devices work kind of with everything. That's the goal anyway. And so a bunch of devices in the past years from the major manufacturers like Signifies, or yeah, like Signifies, Philips Hue, like Nest, some of the big brands have already incorporated Matter into their products. But we were seeing kind of the tier B level vendors, including it in theirs now. So Acara, which is a well-known manufacturer of switches and cameras and stuff like that, they're starting to incorporate it in their devices. We saw LIFX incorporating in new announcements that they had. And we're even seeing it in devices like Razer's Lighting that's designed to work with their gaming lighting ecosystem. So I like seeing that sort of adoption because it's kind of a reinforcement that this is becoming commonplace now. Tom Merritt Yeah. And it's still going to be a while before it will fulfill its promise of you not having to think about it. But this is one of those steps along the way. And I was encouraged like you to see like, oh, everything I'm seeing in the smart home space has matter on it. It felt like I didn't run across anything that didn't have it. It might not have Apple. It might not have Google or Amazon, but it had it had matter on it. Richard Gunther And one of the announcements that came out of the show was that I believe it's Apple is going to start supporting the matter certification in place of their own. So if you're Matter certified, then you'll be HomeKit certified. And Google and Amazon had talked about that. That's the promise of Matter ultimately. And that makes life easier on manufacturers too. Yeah. Tom Merritt All right. What's your second trend? Richard Gunther So second up is energy management. And this is something that may not specifically be smart products. Sometimes it is, but how to handle managing energy better in your home or for your smart home products. So a lot of us use a bunch of batteries, sensors and locks and all these things all have batteries that need to be replaced pretty often. So rechargeables are a great way of doing this. One of the innovation awards that the CTA gives out each year went to a company called Olite for their device called the O-Station X. And the best way that I can describe this is as a battery charger hopper. It's this bin that you put your spent recharge batteries into, and it will automatically feed them in, determine if they need to be charged or if they're already kind of dead, can't be Charged anymore, or if they're the wrong kind, the kind that can't be charged. It doesn't matter which way you put them in. It figures it out automatically, charges four at a time, and puts them in a bin ready for you to use again. Real time saver, this is going to be coming out later this year. I think it's pretty cool. Another product that I thought was ingenious was a battery system intended for specific appliances. So, you know, a lot of times you end up with a generator if you lose power because you want to keep your freezer alive or your refrigerator alive. And companies are starting to think of power systems to address those things without having to get out a gas generator or have a whole home generator. You can just have a battery behind your refrigerator or above your refrigerator that'll keep it running for a day and maybe keep things OK for you. While if you live in an area where you're dealing with power outages regularly. Tom Merritt Yeah, that's good stuff. I still am waiting for those companies and they're still at CES that promise ambient charging to make headway. But in the meantime, that bin that you're talking about is compelling if I can just not have to think, take one less thing to think about off my plate. I like that. Richard Gunther Right, right. Absolutely. And my favorite in this category was an induction oven from Copper. You may know I'm also interested in the smart kitchen space. And this is an induction oven that incorporates a five kilowatt hour battery. So a battery that's going to you do a lot of cooking for multiple meals. It includes an oven and a cooktop, and it can operate off grid if you lose power. But the best part of it is that one of the hardest things with induction ovens is that you need 220 to get it to install it in your home. Lot of homes don't have 220 behind their range. This just runs on a 110 plug because all it's doing is charging the battery. And then you're running the oven off the battery. So it makes induction cooking available and accessible to people who otherwise wouldn't have it as an option. Tom Merritt I would love to see that become more commonplace. I know it's going to raise the price a little bit, but, uh, you know, it's makes so many things more convenient and, and yes, if you've got a whole home battery, maybe you don't necessarily Need the backup, but that's, that's nice to have too, and prevents it from, could prevent other devices from things like brownout damage and all of that. Yep. For sure. All right. Let's get to your last one. What's the, piece of the three? Richard Gunther Yeah, my last category is like outdoor robots, robots to help with your outdoor work, whether it's yard work or pool work. So my two favorites here were a yard robot system ([Time 0:20:26](https://share.snipd.com/snip/3c4daa27-ec08-4f86-abf5-7b7fc4820c10)) - **Car Makers vs. Tech Giants** - Car manufacturers resist Apple and Google's infotainment systems partly due to legacy issues and long design lifespans of cars. - Security concerns arise from infotainment systems needing access to the car's CAN bus data. Transcript: Robb Dunewood We end every episode of DTNS with some shared wisdom. Today, Carl is helping us understand why car manufacturers are resisting letting Apple and Google run their infotainment services. Tom Merritt Yeah, Carl has a good email here. He says, these are selections from it. He wrote a lot of good stuff. One is a legacy problem. A new car might have a seven-year design life, but in reality, manufacturers know that their reputation is based on what's driving around that is 10 to 15 years old. The other part of that is the design compromise. That is the CAN bus system that basically everyone uses. If the infotainment needs access to the CAN bus data, say to set the internal car temperature or monitor speed, then it can also talk back and who handles the safety, you know, if there's An injection attack attempt or something. The second problem is Apple doesn't want to allow someone else access to their serial number locking tech, and the automakers don't want to share theirs either. And I suspect that this is a factor as well, since those systems have to be able to authenticate and play nice to each other. Last, if a bad Android update bricks a few thousand cars, who's responsible for that? Or if Apple suddenly gets a restriction and older cars are shipped to Mexico or the former Eastern Bloc countries then no longer work, that's a big reputation hit for the manufacturer. Tesla already is getting raked for taking away paid-for options at the point of resale. It would only get worse. I'm not saying Samsung would release a patch that only worked for Kia and Hyundai and maybe bricked Fords and Hondas, at least intentionally, but it's a massive trust problem, mostly Stemming from decisions made in the early 90s, and that design legacy is not quickly changed. Thanks for the hard work you put into the update. I'm still working out how it fits in my day since my commutes have changed, but I'm liking the variety. Cheers, Carl. ([Time 0:31:15](https://share.snipd.com/snip/22f56a24-c852-4789-bc10-17da3ff17143))