715: Would You Like a LLM With Your Browser?  🔗 **Source:** [715: Would You Like a LLM With Your Browser?](https://share.snipd.com/episode/d1e22b45-8417-4967-ad71-4927ce0a0f4b) ## Highlights - **Local Browser Models Offer Private Fast LLM Features** - Local browser models can offer privacy and speed by running entirely on-device without network requests. - Chris Coyier highlights Chrome's plan to download a model on first use, giving instant summaries and other LLM features without sending data off-browser. Transcript: Chris Coyier Like there's API, like JavaScript APIs, not DOM ones, but APIs that you ask, you know, in the browser for, I forget what they, what them off of. It's like navigator.askai or whatever. But then the idea is the first time you ask one of them, then it downloads a model. Like Chrome doesn't ship with an AI model. You have to ask it to please download one. And then you would have access to a rudimentary LLM. So you could ask it stuff like, oh, I don't know, summarize this. All the normal stuff you would ask an LLM model, but you're asking an internal model. So it doesn't go out to the internet. It just happens right on your browser. I'm actually pretty like, I don't know, like, I think that's good. Like it's private, it's super fast. It's private in the sense that there's no network transaction and stuff. There's like some stuff to like about that as conceptually. There's stuff to not like to meaning. I don't know. How was this thing trained? Are we ever going to talk about that? It sounds like no. Dave Rupert Like the ([Time 0:01:35](https://share.snipd.com/snip/dfd4078b-5fc6-44e3-bc0e-c2818daad6ce)) - **Bundled Models Could Recreate Chrome Only Web** - Shipping a single vendor's model in browsers risks 'model calcification' where developers tune for that model's quirks and create effectively browser-specific sites. - Dave Rupert warns this recreates the old Chrome-only website problem because prompts get tuned to one model's behavior. Transcript: Dave Rupert Let me go. I also wrote a little post on this. So it's kind of fresh in my brain. But one is model calcification. That was kind of the big one, right? Like, which sounds weird. And, and, but if Google is shipping this and everyone writes code for Google's model, right? You know how you like tune prompts. You're like, Oh, Claude wants to be talked to this way. Oh, uh, chat GPT likes it when I say please. And thank you. Or what, you know, like you kind of tune the prompts to get the response from that model. And then when you switch to another model, you'll get something totally different. Right. That like is maybe like like it basically is a reboot of like the Chrome only websites problem. Like I'm only coding for Chrome because and due to like Chrome's like you know just dominance in the field like the browser share that now this Gemini model becomes the default model Right ([Time 0:10:13](https://share.snipd.com/snip/74a765fe-554b-4e8a-bfa7-7505e2194fab)) - **Use Local Models For Summaries Not Heavy Generation** - Use small on-device models for focused tasks like summarizing content rather than broad generation to avoid unnecessary cloud costs. - Dave Rupert suggests browser summarizers let users get gists locally without paying third-party API fees. Transcript: Dave Rupert Like I shouldn't have to pay Sam Altman money to like just, hey, what's this about? Or like, give me the gist of this, you know, like, like, you know, Apple used to have a little summarizer feature in like, you know, their little AI or like, you know, automator or whatever, Apple script or something like that, you know, like it's, you know, I feel like that's reasonable tech, you know, I feel like, you know, a lot of these, like, you know, a lot of like accessibility Things, like people are like, oh, we're going to use AI to solve things, you know, alt text and stuff like that. You know, it's like, I don't think that's good. Like, I don't think that's sufficient. I don't think that solves accessibility. But like, I think if somebody comes across an image with no alt text and they can ask their browser, hey, describe this image to me. It's missing alt text. I think that's cool. You know, like, like, I think that's a tool that empowers and not like, not like just basically. In fast or something. Yeah. Like, doesn't, doesn't it just like, like if I'm just generating alt text, like I'm spamming, you know, basically. And so like, that's not useful, but like if somebody needs this information, they can maybe get it, you know? So I think that's cool. ([Time 0:15:09](https://share.snipd.com/snip/70461542-3858-43fb-bbe6-548ee8b44cf6)) - **AI Cut Email Time For Someone With Cerebral Palsy** - Dave Rupert recounts meeting a person with cerebral palsy who shaved email composition time from 30 minutes to five minutes using AI. - The example shows on-device assistance can meaningfully empower people with limited mobility to communicate faster. Transcript: Dave Rupert Yeah. And he's just like, you know, now that 30 minutes of writing an email painfully with like his hands have limited mobility and stuff like that. But like now he can like send out an email in five minutes, you know? And so like, just, and I think like that to me is empowering. ([Time 0:17:10](https://share.snipd.com/snip/0aaba6fd-6133-4156-9a16-f9d45dd674c5)) - **Built In Models Bring Provider Usage Rules To Sites** - Browser-built-in models bring new policy surfaces because websites using them must follow the model provider's usage rules. - Rupert notes embedding Google's model may force site developers to abide by Google's generative AI policy, affecting allowed content and liability. Transcript: Dave Rupert So the model calcification, like we're basically redoing Chrome's the Chrome only website problem. The there's a model neutrality issue like where Google says to use this, you have to agree to our privacy or our use policy. Right. And generative AI use policy, which I don't know, I read this and I'm like, not like, you know, you know, it's like, don't do hate speech. Don't do CSAM. Don't do extremism. Chris Coyier Don't do, you know, non-consensual intimate imagery, like all this stuff I pretty tend to agree with of course it's just trying to get them not to be sued yeah well and that's i mean like Dave Rupert But what if you want to like i guess it's not open i guess at that point right like you have to like from a standards perspective it's now like a there's usage rules around these things you Know like i'm not like advocating for those things and it does read kind of like a Grok's whole entire business plan. But like, what is like, you know, I think like, you know, it would take a lawyer to figure out, I think, but like, what is appropriate basically, you know, like what is, I don't know, what, What, what, like, do we have to agree to? And then because Google's model is here and my prompt API code is running on Google, Google's terms of service there are everybody's terms of service. Does that make sense? Like if I want to make a robot that really is cussy and yells at people and, you know, makes them do pushups like, you know, a dommy mommy kind of robot. Like I have, you know, like that creates this, like I violated Google's policy and now I, do they sue me or do they, can they take my website down or what, you know, or block me or I don't know, You know, like what can, what's the impact there? I don't think that's like fully thought through, you know? So, and then like, I don't know the, and then overstated developer position, like Google's like, yeah, 80% of people love this. And it's like, where's the data? Oh, we too, we'll get it to you. We got it. You know, and it's like, you know, it's clearly like they're ramping up to a Google IO thing and they want to release this. Like, that's why it's coming out. So it's not, it's not like, anyway, it's not like because there was this overwhelming developer, positive developer response. Or if there was, it's like, which developers did you talk to? The people using the origin trial? Or did you talk to like people, normal people? Chris Coyier That's funny. 80% of the people who opted into the origin trial love this. Dave Rupert Who jumped through the hoop, flipped the switch, and downloaded a four gigabyte model. ([Time 0:20:46](https://share.snipd.com/snip/16745a33-ed95-47bd-aceb-98db7b8e52a0)) - **Local Models Lower Cost But Data Origins Remain Hidden** - Local small models may be more energy efficient and ethically sourced, but training data provenance remains opaque. - Both hosts repeatedly note major AI vendors refuse to fully disclose training data sources, leaving lingering trust questions. Transcript: Dave Rupert Can use fetch. I do what Google is going for, or people in this space, is you were saying it earlier. It's like these models are local, right? They're private. They don't phone home. That's cool. I think we all like that, right? If we're having to use AI, they cost less energy to run. Like, you know, like I'm not burning a hole in the sky to use it, right? They have a, I'm going to say a higher probability of ethically sourced content. Although, to your point, where'd they get the data? Where'd you get that data? Who knows? Chris Coyier You know, I know you didn't pay me for that data, but where'd you get that i've asked a lot of people and it is the number one mandate of every single one of these companies to not talk about It i don't know if it's number one i'm emphasis i'm being um i'm emphasizing that just for fun but never have i have i heard any ai company be like oh let me talk about where we got all the data. Yeah. Well, I don't want to dwell on it too much. It's just like, I don't know. I don't even know ([Time 0:28:34](https://share.snipd.com/snip/837b6880-c9fd-4297-ae28-0361676e06b3)) - **A Quarter Of Recent Web Pages Disappeared** - The web experiences significant content loss: about 26% of pages from 2013–2023 are gone, and the Wayback Machine saved ~56% of them. - Dave Rupert cites Internet Archive stats to show large-scale internet rot and incomplete archival coverage for media like MP3s. Transcript: Chris Coyier So a quarter of pages from the last decade, a little before the last decade, 2013 to 2023 are gone. Wow. So a quarter of them in the last decade, we can basically say, and that the Wayback Machine, which is the Internet Archive's tool for archiving these things in surely very expensive Web servers and cloud infrastructure somewhere, saved about 50% of them. It says 56% of them. Pretty cool. So they're on it. Although I think we all know if you've ever used the Wayback Machine, it's not a perfect system. You know, you get kind of like a dom snapshot of the page sort of, but they'll probably be broken images and stuff. I think they do their best. It's probably a very sophisticated and powerful system. But I would think, Dave, that if we turned off ShopTalkShow.com and that the Internet Archive would have it. But would they have like the MP3s? Probably not. Right. They probably don't grab those because they're a little big. ([Time 0:40:07](https://share.snipd.com/snip/eddf32b4-059a-4a16-bbb0-8e9791f14fd9)) - **Statify Sites Before You Walk Away** - When retiring a dynamic site, convert it to static files to minimize maintenance and hosting costs. - Chris Coyier recommends exporting WordPress or Rails apps into flat HTML/Markdown and hosting on cheap static providers like Netlify. Transcript: Chris Coyier But you'll still have the domain cost. Yeah. I wish there was better. It's hard. Dave Rupert I wish there was better like statification of, you know, like tooling, like, like just, Hey, I'm taking this down. Just make this static zero maintenance to run. I don't need a WordPress server. I don't need, you know, like, I just need a like flat file HTML or Markdown or something that builds it, you know, like once and then it's done, you know, like, I think that would be kind Of cool. I know like you can do like importers and all that stuff, but like, what if it was like stupid, simple, right? Like that would be pretty cool. Um, you know, I've like, we had, you know, we had like that app day trip, you know, and, uh, it, you know, basically what'd you do with day trip it's just gone right it's gone like it was you Know it was like on like a whatever rails three with a heroku bamboo server and so like you know rails three became a security issue and you know uh heroku basically like went out of you Know or well went out business quit doing free hobby to tier servers but then they like, you know, the bamboo, they're like, we're decommissioning and you have to move. But then like that, like broke stuff. And so it was just like, it's just gone. We just have to wipe it and, you know, and delete people's data. Because I feel like that's the responsible thing. Right. Like, I mean, you know, more responsible would be give people a window to get data out, you know, but, you know, that's like, also we kind of, like I was saying, like we hit like the end of The roadmap for our tech, you know, or like end of support. And so, you know, so it was a very natural decommissioning. And whereas like, you know, I think, yeah, I, you know i don't know it would be cool just if there was like an easy way to be like click this rails app make it static or something you know like Slurp all these views and put them into mustache you know but we also put a lot of crap into websites and so that's also makes it difficult but um i i honestly think like 90% of all websites Would benefit from that. ([Time 0:43:06](https://share.snipd.com/snip/b005a8da-a0e4-4cb6-9d87-b922a63a1a36)) - **DayTrip Died When Host And Stack Were Decommissioned** - Dave Rupert describes shutting down his DayTrip app when Heroku decommissioned free tiers and Rails dependencies became insecure. - The app's data and functionality were lost, illustrating fragility of hosted services tied to specific stacks and vendors. Transcript: Dave Rupert Um, you know, I've like, we had, you know, we had like that app day trip, you know, and, uh, it, you know, basically what'd you do with day trip it's just gone right it's gone like it was you Know it was like on like a whatever rails three with a heroku bamboo server and so like you know rails three became a security issue and you know uh heroku basically like went out of you Know or well went out business quit doing free hobby to tier servers but then they like, you know, the bamboo, they're like, we're decommissioning and you have to move. But then like that, like broke stuff. And so it was just like, it's just gone. We just have to wipe it and, you know, and delete people's data. Because I feel like that's the responsible thing. Right. Like, I mean, you know, more responsible would be give people a window to get data out, you know, but, you know, that's like, also we kind of, like I was saying, like we hit like the end of The roadmap for our tech, you know, or like end of support. And so, you know, so it was a very natural decommissioning. ([Time 0:43:43](https://share.snipd.com/snip/2f9547ef-977d-4b28-bda9-bf45b8d22ad4))