AI Chips: Hard Forks and Innovations  🔗 **Source:** [AI Chips: Hard Forks and Innovations](https://share.snipd.com/episode/d9af90c2-4c00-4e81-9b42-a7a089f52b3b) ## Highlights - **Anthropic Is Building Custom Chips For Claude** - Anthropic is building a custom silicon team to design chips tailored to Claude's workloads to escape Nvidia dependency. - Custom chips take 18–24 months and cost hundreds of millions, but efficiency gains compound across billions in API usage. Transcript: Jaeden Schafer The first story I wanted to talk about is the fact that Anthropic is building their own AI chip design team, and it's going to be for creating custom hardware for Claude. It's the same thing that OpenAI, Google, and Meta, they're all doing it right now. And the big reason is everyone wants to get off of Nvidia. It's these companies are incredibly dependent on Nvidia. Nvidia basically has quotas and they dole out their GPUs and their chips, and they only give a certain amount to, you know, different companies and nobody wants to feel bottlenecked By one player that they're obviously spending a ton of money on. So everyone's building their own chips. Right now, they're also scouting Samsung as a manufacturing partner, is ending Anthropic status as the basically the last major AI lab without a public silicone program program they're Like all right it's time to get serious on that one. Anthropic currently relies on a bunch of different suppliers so there's AWS, Google, NVIDIA, AMD all of them give them compute but of course that means that they don't get to control The pricing allocation or any of the roadmap timing. OpenAI shipped their Broadcom designed chip, which is called Jalapeno. It's an inference chip. And that was in June of this year. Google now runs their own TPUs. Meta is building MTIA accelerators. And Anthropic basically was the only one that wasn't working on this custom silicone typically takes between 18 to 24 months to design and it's going to cost hundreds of millions of Dollars per chip version but every single efficiency gain that they can get on this compounds across billions of dollars that anthropic and is you know using for these clod api calls So i think this is something that is kind of a play on margin it's not a shortcut this is going to take a long time anthropicropic can tune memory and math formats to Claude's exact workloads In a way that no off the shelf GPU can really match. But they're also betting on a two year runway before any of that investment is going to get paid back. ([Time 0:01:27](https://share.snipd.com/snip/d98cc67c-3ec7-4628-9524-183abb9a4c80)) - **Anthropic Scouted Samsung And Joined The Silicon Race** - Jaeden notes Anthropic scouted Samsung and was the last major lab without a public silicon program. - He frames this as a margin play: tuning memory and math formats to Claude's specific workloads. Transcript: Jaeden Schafer It's the same thing that OpenAI, Google, and Meta, they're all doing it right now. And the big reason is everyone wants to get off of Nvidia. It's these companies are incredibly dependent on Nvidia. Nvidia basically has quotas and they dole out their GPUs and their chips, and they only give a certain amount to, you know, different companies and nobody wants to feel bottlenecked By one player that they're obviously spending a ton of money on. So everyone's building their own chips. Right now, they're also scouting Samsung as a manufacturing partner, is ending Anthropic status as the basically the last major AI lab without a public silicone program program they're Like all right it's time to get serious on that one. Anthropic currently relies on a bunch of different suppliers so there's AWS, Google, NVIDIA, AMD all of them give them compute but of course that means that they don't get to control The pricing allocation or any of the roadmap timing. OpenAI shipped their Broadcom designed chip, which is called Jalapeno. It's an inference chip. And that was in June of this year. Google now runs their own TPUs. Meta is building MTIA accelerators. ([Time 0:01:37](https://share.snipd.com/snip/6b53b5de-15f2-45ba-9586-d7cdd7d84398)) - **AMD's Helios Rack Challenges Nvidia At Scale** - AMD unveiled Helios, a rack system positioned as a direct competitor to NVIDIA for AI data centers. - Five frontier AI labs committed at gigawatt scale and shipments start later this year, signaling real multi-vendor competition. Transcript: Jaeden Schafer After announcing a strategic partnership with Anthropic, AMD has just unveiled Helios, which is a rack system for AI data centers. And they're saying that this is basically a direct competitor to NVIDIA's products that they have right now. Microsoft, OpenAI, Meta, Oracle, and Anthropic have all committed to deploying it at a gigawatt scale. And it's going to be starting shipments later this year. Anthropic and AMD both announced a strategic partnership to deploy up to two gigawatts of GPUs on Helios, which is, you know, the exact same scale of NVIDIA's largest AI training cluster. So they're, you know, they're really being quite competitive in the market. AMD also introduced Venice X, which is a data center CPU designed to pair with Helios. That's going to be launching in 2027. And it's going to compete on kind of this full stack alternative to a lot of the different competitors that are out there. The CEO Lisa Su projected the AI accelerator market will reach 1.4 trillion by 2030, approaching the size of today's entire semiconductor industry. So you know, this one segment of the market she's saying is going to be as big as the entire semiconductor industry. This is the first time that AMD has landed all five major frontier AI labs on a single rack platform, they have everyone signed up for it. I think this is showing that they're really like a strong contender for a quote unquote, place option for a lot of these massive AI infrastructure build outs that are coming up the scale Of all of these different deals that they've done with these these five hyperscalers. I think they're all measured in gigawatts. And I think that shows that the hyperscalers are all of them are ready to diversify away from Nvidia's almost complete dominance. ([Time 0:03:22](https://share.snipd.com/snip/6e32cbb6-b2a7-4982-9377-d756b01b598e)) - **Claude Voice Moves To More Powerful Models** - Claude's voice mode will move from Haiku to the more powerful Opus and Sonnet models, improving quality. - Jaeden worries higher-quality models might slow the previously near-instantaneous transcription speed he valued. Transcript: Jaeden Schafer Anthropic is upgrading Claude's voice mode. It's now going to work with their most powerful models, Opus and Sonnet. So I think in the past it was only working with Haiku. And I will say one thing that I actually did appreciate about this was I've been going back and forth between open AI and anthropic as far as using like Claude co-work and chat GPT work And I did find that chat GPT work after they did their upgrade and everything's running on soul 5.6 extra high when I go and ask it to complete a project for me and I'm doing the voice mode On it it takes a while for it to kind of transcribe and write it all down and it's actually kind of annoying the the leg it's a few second leg and I noticed that it doesn't always happen but When you have a sole 5.6 extra high on it does and so I actually appreciated that um anthropic when I was using clod it was like instantaneous transcription of what i was doing it was all On haiku now is it perfect is it the is it the best like there's a couple things that it might mess up but it was instantaneous and super fast well they're going to upgrade it because evidently They think that it needs to be a little bit higher quality so now they're putting it on opus and sonnet i hope they don't sacrifice too much of the speed because honestly that was my favorite Thing they're going to be able to they're allowing you to plug that directly into Gmail, Google Calendar, Slack, Canva, Notion. So you can now tell Claude by voice to do your emails. I mean, everything else that you've probably been doing, I even have like a literal, I have this device from OpenAI. It's like a little keyboard. It is, I think, built with work louder. And it literally just has a microphone button that I push. So this is the way I've been working a lot lately is just talking to these models and having them go and complete tasks for me instead of typing it out. I think it's what a lot of people are doing. There's a lot of software like Whisperflow that's out right now that does that kind of stuff. So a lot of different options. But Claude obviously decided they wanted to put a big upgrade here. Mode is now going to default to whatever Claude model you last used in text chat so a sonnet workflow is automatically going to have that for the audio which is kind of what opening eyes Doing right now and I'm not going to lie that's actually annoying to me I wish you could choose the audio honestly what I find is when it does the transcription and when it types it out I Would actually rather a faster, worse model because you can like many times I'm literally using Apple's like voice to text, which is really ghetto, not very good. And when I when I do it, there's like a bunch of typos, but Claude understands and it literally fixes it and does everything the way I want. So I find that Claude is very good at getting the meaning out. I'd rather just have the underlying model better and having the voice transcription faster because I absolutely hate talking and then waiting five seconds for it to type out before I can hit send. But I mean, that's just me being impatient. So I guess the quality improvement is probably good, but I wish they would let you choose in the settings what model was powering your voice. Free users are going to keep getting haiku. So maybe I got to go switch to being just a free user. Just kidding. And there's going to be paid users that get the Opus and Sonnet and all of the rest of it. ([Time 0:05:15](https://share.snipd.com/snip/96cd6658-6404-4918-aec7-905352dd84a6)) - **Host Prefers Faster Voice Transcription Over Higher Quality** - Jaeden describes his personal workflow tradeoff: he values instant, slightly-lower-quality transcription over slower, higher-quality output. - He often uses voice-to-text and prefers faster turnaround so he can send or act immediately. Transcript: Jaeden Schafer So I think in the past it was only working with Haiku. And I will say one thing that I actually did appreciate about this was I've been going back and forth between open AI and anthropic as far as using like Claude co-work and chat GPT work And I did find that chat GPT work after they did their upgrade and everything's running on soul 5.6 extra high when I go and ask it to complete a project for me and I'm doing the voice mode On it it takes a while for it to kind of transcribe and write it all down and it's actually kind of annoying the the leg it's a few second leg and I noticed that it doesn't always happen but When you have a sole 5.6 extra high on it does and so I actually appreciated that um anthropic when I was using clod it was like instantaneous transcription of what i was doing it was all On haiku now is it perfect is it the is it the best like there's a couple things that it might mess up but it was instantaneous and super fast well they're going to upgrade it because evidently They think that it needs to be a little bit higher quality so now they're putting it on opus and sonnet i hope they don't sacrifice too much of the speed because honestly that was my favorite Thing they're going to be able to they're allowing you to plug that directly into Gmail, Google Calendar, Slack, Canva, Notion. So you can now tell Claude by voice to do your emails. I mean, everything else that you've probably been doing, I even have like a literal, I have this device from OpenAI. It's like a little keyboard. It is, I think, built with work louder. And it literally just has a microphone button that I push. So this is the way I've been working a lot lately is just talking to these models and having them go and complete tasks for me instead of typing it out. I think it's what a lot of people are doing. There's a lot of software like Whisperflow that's out right now that does that kind of stuff. So a lot of different options. But Claude obviously decided they wanted to put a big upgrade here. Mode is now going to default to whatever Claude model you last used in text chat so a sonnet workflow is automatically going to have that for the audio which is kind of what opening eyes Doing right now and I'm not going to lie that's actually annoying to me I wish you could choose the audio honestly what I find is when it does the transcription and when it types it out I Would actually rather a faster, worse model because you can like many times I'm literally using Apple's like voice to text, which is really ghetto, not very good. And when I when I do it, there's like a bunch of typos, but Claude understands and it literally fixes it and does everything the way I want. So I find that Claude is very good at getting the meaning out. I'd rather just have the underlying model better and having the voice transcription faster because I absolutely hate talking and then waiting five seconds for it to type out before I can hit send. But I mean, that's just me being impatient. ([Time 0:05:21](https://share.snipd.com/snip/fae2f7f7-05f1-4ace-8f0d-5a7c845770b9)) - **Claude Audio Integrates Directly Into Productivity Apps** - Claude's audio can now plug into Gmail, Google Calendar, Slack, Canva, and Notion for voice-driven workflows. - This enables telling Claude by voice to draft emails or manage calendar tasks without manual typing. Transcript: Jaeden Schafer So you can now tell Claude by voice to do your emails. I mean, everything else that you've probably been doing, I even have like a literal, I have this device from OpenAI. It's like a little keyboard. It is, I think, built with work louder. And it literally just has a microphone button that I push. So this is the way I've been working a lot lately is just talking to these models and having them go and complete tasks for me instead of typing it out. ([Time 0:06:28](https://share.snipd.com/snip/3af1ecaa-3577-4e12-b86b-d0564c707460)) - **Google Cloud Revenue Surges On AI Demand** - Google Cloud revenue jumped 82% to $24.8 billion, driven by AI infrastructure deals and multi-year contracts. - Gemini usage rose to 950 million MAUs and Alphabet posted huge quarterly profits, justifying heavy datacenter and chip spending. Transcript: Jaeden Schafer Google Cloud revenue is up 82% to 24.8 billion dollars last quarter that absolutely crushed the expectations that wall street had i think it's proving that the company's huge ai spending Is actually paying off the 514 billion dollar backlog of signed customer contracts i think is showing that there is a huge demand for ai infrastructure and service is real and it's locked In for years. So a lot of these deals that Google Cloud's doing, it's not just for this year or for this month, but these are multi-year deals. Cloud growth is accelerating really fast. It's up from 63% last quarter to 82% this quarter. It's mostly driven by a lot of different companies renting compute power and buying pre-built AI products from them. Gemini reached 950 million monthly active users. That's up from 750 two quarters ago. So the usage of their own products is accelerating, but also the amount of companies signing deals to power their AI on their infrastructure is also increasing. And Alphabet posted 112.1 billion in quarterly profit. That's almost four times the $28.1 billion from the same quarter last year. That is wild. $112 billion up to $21 billion just one year later. That's absolutely insane. Google's $180 billion to $190 billion in annual spending on data centers. And chips is now, you know, it's something that kind of crashed their stock a couple of weeks ago when it came out and everyone was like, oh my gosh, Google's overspending. But I think this is showing there is serious demand. The demand is accelerating like crazy. They know they're going to get that money back. And so I think they expect that 2027 is going to be when this massive build out starts turning into some real cash generation. ([Time 0:08:20](https://share.snipd.com/snip/f90ea8e0-6f0d-49c0-8a9d-dd377c5b0541)) - **Closed Labs Lobby To Restrict Chinese Open Models** - OpenAI and Anthropic lobbied US regulators to restrict Chinese open-weight models citing national security concerns. - Jaeden highlights this occurs when it benefits large closed labs, and he remains skeptical of purely protectionist motives. Transcript: Jaeden Schafer OpenAI and Anthropic are both lobbying US regulators to restrict Chinese openweight AI models like Moonshots, Kimmy, you know, K3 came out a while ago, and it was all the rage. And then we have the latest Quen model came out a few days ago. What they're saying here is that there are national security concerns. They, of course, are saying this at a time when it would be quite convenient. It would push these companies towards, you know, being the winners in this space. No one could really dethrone them. None of these open source models could go scorched earth on them. Opening eyes, ball argued the u.s should create regulatory fear uncertainty and doubt around open weight models to weaken their competitive position um and then he kind of walked That argument back but that was like his first thing we've heard this in in crypto a lot back in the day the fud um that he's you know openly saying that they should create some fud and he's Like no no actually it's serious it's not it FUD for FUD's sake. Kimi's launch at the end of last month, I think a lot of people were panicking. There was a viral demo showing it generating a Mac OS-like UI mockup in 30 minutes. I think a lot of the hype faded after that. A lot of these demos are like, oh my gosh, look at this crazy, incredible thing. And then once it actually comes out and people actually play with it, they're like, okay, it's not as crazy as it seems, which is why I'm actually a little skeptical. I mean, whatever. You can say I'm dumb, but it's why I'm a little skeptical of Anthropoc always hyping up their models to an insane degree, only giving them to enterprises, not to the regular people. And saying, look, it can do all these crazy things. I don't know, makes me skeptical when they won't give it to you. But maybe it's truly dangerous. I don't know. This is the second time that we've had in the last under a year, basically, where we've had this kind of cycle where there's a huge cheaper model that comes out and it's super capable. Deep Seek had something that that kind of was along these lines. And then afterwards, there was a bunch of policy restrictions that people were calling for after it came out. So I think we're seeing basically the same pattern play out over and over again. Chinese labs are going to ship like a super cheap open model. It's going to be really competitive in the benchmarks. A lot of the American executives are going to start warning about security and regulation and we need to have restrictions. And of course, the people that are really winning are those big closed model companies, OpenAI, Anthropic, etc. So I think whether those security concerns are legitimate or not, it doesn't change the fact that all of the people asking for these policy changes happen to be the people that are going To be winning from it. ([Time 0:10:12](https://share.snipd.com/snip/c7561c39-6b02-44bd-a0c7-2fd579d77f39))