1025: The Open Web's Second Chance  🔗 **Source:** [1025: The Open Web's Second Chance](https://share.snipd.com/episode/93dfbb11-7ea9-447a-ac2d-dc8deb155d65) ## Highlights - **AT Protocol Is A User-Owned JSON Layer** - AT Protocol is a people-centric JSON storage layer that lets each user own a folder of JSON records as the source of truth. - Apps become consumers that sync and present that per-user folder so your content stays with you even if you change apps. Transcript: Dan Abramov Don't know there's so much pressure in this question like i'm like i don't know if i can condense it like to me i think it it fixes a lot of stuff. Like, you know, like, you know, like in The Matrix, in the movie, like, there's this question, like, have you ever felt that, like, something is wrong with the world? And it's kind of like this, but for the internet, right? Like, internet, especially, like, if you grew up with the internet, I think you kind of maybe remember it as this maybe lighthearted thing where, you know, like you, you like create A website, you like post something to it. Like you can link to each other. It's very permissionless. It's very much kind of like, there's a spirit of like open, you know, openness and no like walled gardens. And then with time, in a sense, it's gotten more social, right? Like social networks and so on. So like we've gotten like shared spaces, but then these shared spaces, they feel like internet inside of internet, like, you know, different walled gardens, like Facebook or Twitter, Where we're going to put our stuff there. You know, like I put like my answers on Stack Overflow or my issues on GitHub, or like my posts on Reddit or somewhere else. But then with time, these platforms, they kind of, you know, they fatten up with like everyone's content. And then they start doing, you know, they're like shutting down alternative clients, start showing ads everywhere, trying to milk, you know, everything that, and like, we can't really Like walk away from them because then we're going to erase everything that we created there. So in a sense, it's kind of like we put our stuff into somebody else's box and then this box is just, you know, it's not on the open web anymore. It's kind of, there's like a ghost of it on the open web, you know, like the, the projections, like the HTML that you can hit through the servers. But like, in a sense, it's like, it's, it's locked somewhere, somewhere else. And the big thing about that Prado is, it's kind of like a data layer for the web is how I like to think about it. So it's kind of like, what if that JSON that we usually put into uh like other products into their boxes what if the json just you know hung on the web like the json was part of the web uh kind Of like in the personal sites era but um you know you still can use apps of course so like apps are kind of the consumption and production model but it lives sort of in everybody's... It's like if everybody had a Dropbox folder that all apps can kind of interact with, and then you can always leave an app, but your stuff is still going to be in that folder. And so other apps can kind of present it differently or even write to it differently, and different apps can see each other's content and kind of interoperate. ([Time 0:03:01](https://share.snipd.com/snip/26a6e870-ae69-4d0f-adec-411a0171539d)) - **Identity Is A Portable Web Account** - Identity in AT Protocol is an upstreamed web account that you can host yourself or switch providers, avoiding dependence on Google-style single sign-on. - The identity document includes your current username handle, hosting location, and a public key for verification. Transcript: Dan Abramov So you know like you have a Google account, right? Like maybe you have like a lot of sites that you log into, but you only have like maybe several Google accounts, right? Because maybe you have like separate identities for like work stuff and personal stuff, or maybe like even a couple different identities for personal stuff. But really, you don't want to have 400 password pairs and so on. It makes sense to manage them from a single identity piece. And a lot of us use Google for that. And so with Prado, the first thing that it introduces is it really upstreams that concept into web itself. So the idea is that identity should actually be a legible concept in the web. So you should have a web account. And you don't have to be logged in all the time, of course, or you can have multiple. But ideally, we upstream this so that you don't depend on Google. And so, you know, it could be hosted by Google, but you should be able to switch it to another provider or to host it yourself on your own server. But basically like the thing that you log into apps with, that should be something that, you know, that is like under your control and that is a web standard. So that's kind of the first idea. Wes Bos Is that like the idea of like owning your own domain or does it go even further than that? Dan Abramov It goes further than that. So Adprado uses domains as handles. So this is a part of this. Your identity has a handle, but it's not tied to a domain because you want to change handles over time. Like that's pretty common, right? Like people want to be able to change their, oh, like you could lose a domain, right? Like you don't want to lose your identity. So it's like a layer above domains. There's like a identifier that's associated with you. And then there's like a JSON document that says three things. One is like what your current username is, which is a domain, but it can change over time. Kind of like your passport. So like it says who you are, like what's your name? You know, what is your handle? The second thing it says is where is your data stored? So what is your current hosting? And that is the novel concept, right? Like in today's web, we don't really have a concept of per user hosting, right? So like I only have a hosting if I host a website. Whereas here it's again, kind of more like Google account or Google drive. It's like everybody who gets, you know, a web account has some hosting and it could be provided by like a big company, but you could switch it over time to, you know, run it yourself or with A community and so on. It's just like a web standard. So everybody gets a host thing where some JSON can be stored. And the third thing is just your public key so that the things that you do can be verified. ([Time 0:08:28](https://share.snipd.com/snip/ebaff55e-daa6-4b85-82e2-fed0b0a0c68b)) - **Apps Write And Subscribe To User Repos** - Apps act like CMSs that write JSON records into a user's hosted repo; follows, likes, and posts are files in folders on the user's host. - Apps also subscribe to streams of relevant events and sync them into their own databases for features like search and recommendations. Transcript: Dan Abramov Yeah. So the key separation is really between hosting and apps, right? So apps are these like user facing thing. So this, this doesn't change. You just, you know, you just use an app, use the blue sky app or, uh, you know, tangled or like for get, you know, for Git repos, or there's like a bunch of blogging, like macro blogging apps, Like Leaflet, Offprint, Pocket. So these are all, you know, they work like normal sites. You just log into them, you have an account, you can like write something and publish. But these apps, like the way they work is they're kind of like CMSs in a sense, because when you write something on Leaflet, it actually creates a file in your folder on your hosting. So it knows where your accounts folder is and you just put some JSON there. And it's the same, like if I follow you on Blue Sky, I'm actually creating adjacent, like the app will create adjacent file in the Blue Sky Follows subfolder of my folder, on my hosting. So like every action you do, you know, when I, like, you like my post, you're actually creating a JSON file in your folder under blue sky likes that has a link to a post in my blue sky posts Folder in my repo. It's kind of like a web of JSON with links. And so this is, this is like what happens on the hosting layer is it's like the source of truth for like everybody's stuff that we created that is like available publicly, you know, with No permission. It's just, it's just like, you can just, you can just like grab it as JSON and explore it. And then all apps listen to events from all hosts. So this is how you actually update your database, right? Because like apps, you know, you don't want to go to a thousand hostings to like calculate the number of flags, like that would be stupid and it wouldn't work, right? It's too slow. Your WebSocket listener in the app, it just puts it into your database. So you just filter out, you know, stuff that is the data that is relevant to your app. You know, like the BlueSky server listens to BlueSky events about BlueSky records. And like Tangled server listens to events about Tangled records. And some of them may actually like, you know, they may also care about something else, right? Like Tangled this decide to listen to blue sky follows and put the blue sky social graph into its own database and then show me what my you know friends are pushing to to tangled so it's Wes Bos Kind of like they all can see each other's and it's at that app level where you would implement like an algorithm or um spam protection or or any of that stuff that people would like because I think people coincide like at protocol with the people that are like remember the good old rss days where you just had a list in chronological order you know you see that on instagram As well people are like just go back to photos and show me the people who i follow and people say that, but I think that what the data shows of people actually doing, you probably know, you Worked for Facebook, people like to consume the algorithm a bit more. That's why every single app now has a for you page. Yeah. Dan Abramov Yeah. And in that sense, I think like, I really like RSS as an analogy because, you know, there's like, in a sense, like RSS gives a sense of ownership over content, right? Like I publish, like the source of truth is really like on my site or in my, like in a product where you don't have to have a site, like the hosting is kind of like a behind the scenes, like It's kind of more like Dropbox per user. But it's really, you know, like it's the source of truth for my stuff is just this JSON that lives somewhere, like wherever my hosting is. And then apps are downstream from that. And in that sense, RSS and Google Reader, right? So like sources of truth is like RSS, it's like hosting. But then there are apps like Google Reader that can aggregate it into their own databases, do algorithms, do recommendations, do global search, and so on. And so in that sense, a Proto app is really like a normal app. The only difference is that the database is not the source of truth. It's where stuff gets synced. ([Time 0:14:13](https://share.snipd.com/snip/414b3a93-b8a6-4b31-8799-7d274bca6d00)) - **At Protocol Is Public-First Today** - Currently AT Protocol focuses on public data: PDS (personal data server) records are publicly indexable and visible to tools like pds.ls. - Extensions for protected groups or DM-like closed data are being worked on but are not part of the core yet. Transcript: Scott Tolinski So that tracks those every in that same regard is everything on a protocol public uh right now yes so it's currently focused on public data consumption so an aggregation so everything Dan Abramov That you put there is public uh there's actually like a really nice tool called pds.ls, personal data server, pds, and then.ls, like Unix, ls, you know, list files. And you can go to that tool and you can input any handle and you can literally see, you know, kind of like index of a server, like you can see all the folders. So you can see what's in your hosting for your user, like your kind of JSON records. So that is all public for everyone. They're working to like extend the protocol for kind of protected or like closed, you know, like closed, like not closed accounts exactly, but kind of like. Think like dms or something right not exactly like dms dms is like e2e is the best thing for that so which is like a completely different beast but you know there are already good protocols For that but they're working on like extension of protocol for things like closed groups or you know forums that are closed and so on but it's it's like a separate thing that is still being Worked on. It's like it's not out yet. ([Time 0:19:12](https://share.snipd.com/snip/bc0a7b70-a094-46c8-9e26-aae88b0dbb25)) - **Lexicon Lets Apps Define Their Own Schemas** - AT Protocol doesn't force a single post schema; instead it provides Lexicon, a JSON schema language where apps define their own types. - Lexicon generates typed tooling so different apps can standardize organically or define custom formats for their content. Transcript: Dan Abramov Yeah. So the product itself, app agnostic so it doesn't specify you know it doesn't try to specific what i think is like a mistake a lot of these things do is like try to specify what a post is or Like what a comment is and so like it probably doesn't try to do any of that it gives you like a language for schema definitions. So instead of specifying, like, you know, trying to get everyone to agree, like how many characters there should be in a post, instead it lets every app developer define their own types. So you can say like, you know, my blue sky, like blue sky says, like a post is an object with like text field that's at most 300 Unicode segments or whatever. And a language field and created that date time field. And it's kind of like TypeScript definitions. They're written in JSON, so it's not exactly TypeScript. But there's tools for... LLMs are actually really good at writing this JSON now. It's called lexicon, like this format. But this lexicon definitions, there's like tooling that code gens type script types plus parsing utilities. So for any format that you define, you get like a fully typed, you know, blue sky post or tangled repo or like leaflet blog or whatever, you know, object that is like fully typed and can Parse that JSON. Yeah. And so, and all the, like in the user's folders or repositories, as they're technically called, all data is broken down into folders that are, again, like per type. So each application can define their own types. And then that means that, you know, like Blue Sky Post can be different from some other app post. And, you know, they can just keep using completely different data formats, kind of like file formats, right. And extensions, like you can just define your own for your app. It's also interesting, like some apps decide actually to standardize on the same one. So there's like an example where multiple blogging apps in the atmosphere decided that they want to keep their social graph unified so that they're, you know, they have like a reader, Each of them has like a reader page, kind of like Google Reader. And they were like, it would be cool if you could follow a blog from the other app, right? But it still shows up for you in the reader. And so they kind of standardized on, okay, let's use the same lexicon the publication itself, so that we can have like a brand color, for example, like a theme there, the publication Name, the author, and so on. And let's also standardize for subscriptions. So how you follow each other is the same data format and posts or like, you know, declarations, like here's a post. And so this is like now they all can show or link to posts from each other, like aggregate posts from each other. And you can actually, even if you don't use any of those blogging platforms, you can use tools that integrate with your SSG or whatever you use for your blog and also publish records in That format. And then they will also show up in these apps, which is kind of cool. It's kind of like a standard, but it's grassroots. It's just like three developers came together to unify their data model. ([Time 0:21:47](https://share.snipd.com/snip/59a43a58-274e-408c-a8c1-08a78919b391)) - **Store Large Media Off Or Use Specialized Hosts** - Expect media to be handled flexibly: use host-provided blobs for images/videos referenced by hash or store large binaries off-protocol and reference them in records. - Plan hosting choices: large media may require paid or specialized hosts while JSON stays lightweight on any PDS. Transcript: Dan Abramov Yeah, I think I mean, you know, like one thing about it, Prado is that it's, you can take as much of it as you want, or as little as you want, right? Like nobody forbids you from, you know, you could use it just as identity layer, but like all the actual stuff lives in your database. And so this is kind of like, okay, this, you know, this doesn't, this, this isn't like in the spirit of like full-length Proto, but it, you know, it's like a step. And there are things that don't like protected data that like aren't even a part of the protocol yet. So like you would have to store it separately like in your database or whatever and so this is kind of similar so there's there's a built-in thing in the Prado for media uh with it which Is uh blobs so in your repository or like alongside with your repository on your hosting, you can store these binary things, like images or videos and so on. And the records, your JSON can reference them by the hash. And so in that sense, they're like a Prado native, right? But again, it depends on your hosting. So your host might not be happy with storing, storing gigabytes just for your personal because it's much bigger than JSON, right? Yeah. So I think this depends, like one way is like, maybe some hosts would, you know, specifically like video oriented things, like would allow larger storage, maybe it's like a paid thing. ([Time 0:28:57](https://share.snipd.com/snip/e36f7ffc-e8ba-448c-9162-d512b6cb065e)) - **The Protocol Is The API And Enables Global Indexes** - AT Protocol functions as the API itself: you can fetch a user's repo directly instead of asking an app's API, enabling agents and AI to cross-query open JSON. - A public archive called Hubble ingests network data so you can query aggregated historical content without hitting individual hosts. Transcript: Dan Abramov So so in that sense uh you know the like with the product like the protocol is the api so if you want to get some data from you know if you want to get my like blue sky data, you don't ask the blue, Sure, you could go to the blue sky API server. But the point of what product is you don't need to do that. You can just go to my hosting and literally download my repo. Scott Tolinski And my repo is just, you know, like, that's, that's, that's where my files are. Dan Abramov And the thing is, like, if you want, you can download the entire network, it's going to be terabytes, but you can like there's, you know, there's a person who's working on a public archive. So you can quickly access the current version of like, you know, aggregated from like old hostings. It's called Hubble. That's like this like read-only archive that's like off to date and you can query anyone's data without going to their hosting. So that thing is currently being like, you know, it's ingesting like all the, you know, all the historical data, like everybody's, everybody's content. And it's like, it's going to be like terabytes, but you can actually download the whole thing and like crunch the numbers on it or like analyze it. Like you can, it's kind of like you can download the entire internet as JSON, put it into your database and run queries across products. Right? Like you can make like a SQL query across multiple products and like do joins between them. Like it's pretty crazy. ([Time 0:33:40](https://share.snipd.com/snip/65f399ca-e599-44af-b306-cef0a57cde6f)) - **AT Protocol Keeps Hosting Separate From Apps** - AT Protocol differs from Mastodon: Mastodon runs app instances that each own accounts, while AT Protocol separates hosting from apps so your identity and data live at your host, not inside a single app instance. - This makes the user experience feel like a unified web app while keeping portability of hosting and data. Transcript: Dan Abramov So the way to think, the way I like to think about Mastodon is Mastodon is kind of like, it's kind of like WordPress. So it's, it's kind of like, you know, it's an, it's like a mini Twitter that you can install on your server. And that server is that's the entire thing right like it's it's like a like a hosted Twitter so you just you just spun out the copy of the app and so you can imagine there's like you know like If you if there's a copy running at like instance1.com you sign up there your account is there. That is the Twitter you signed up for. And then if there's another instance, it's completely independent hosting, completely independent app. And if your friend makes an account there, it's like a different service. They have nothing to do with each other. It's just somebody else's app who's like administered by whoever is running it. And then the problem is like, okay, first problem is like your identity is actually inside that app, right? Like if somebody removes that server from the internet, like your, you know, your identity was there. So it's kind of um like it's not really like a network i think of it the similars to um kind of like kingdoms and like ancient china or like you know like fiefdoms where like you belong to This clan or whatever like there is you know there is like part of the map that is like your instance. And then maybe if your friend lives somewhere else, like these instances have to talk to each other, right? Because if I follow you, but you're in another instance, what happens is there's like copies of Twitter start sending like emails to each other in a sense, right? Like the, like ActivityPub is kind of like a mailbox abstraction where these instances start messaging each other and saying, actually, Wes has posted and this instance was interested In, somebody here follows him, so I'm going to send really all Wes posts here. So you have this N squared divided by two connections between instances where they like send stuff to each other that you may have missed, but it's very fragmented because like, even If you try to like patch it with like loading stuff from other instances or like, you know, assembling like, it's kind of, you know, it's, it's many apps that are disconnected that try To sync to each other via email is like my kind of mental model. And you're tied to a specific one. And like, for example, if admins of one instance decide to have beef with admins of another instance, they may just stop sending messages to each other. And then you're not gonna not gonna get your friends posts right because like your friend is on another instance which is super weird right like it's yeah yeah i always found that that Scott Tolinski Disconnect to be super odd trying to like pick which instance i was going to join where my account lives it was like immediately like this is what's supposed to upsert uh usurp twitter Like this this experience like no one's gonna no one's gonna latch on to this where i think blue sky people can be unaware of the protocol behind it and still get a twitter-like experience Out of it which is one of the the pros of that design yeah. Dan Abramov Yeah. That's, that's like the, the paper that they popular, you know, with the first version of the protocol is called usable decentralized social media. So that was very much a constraint that, you know, behind the scenes, you know, there's this like credible exit, there's this like app agnostic hosting that like can change at any time. But from the user perspective, you interact through apps, and developers can build other apps, you know, that have kind of like projections of the entire internet, but filtered down To some specific type of content. But like from the user perspective, it feels like the same web, except that you have, you know, you can see stuff from other apps, like the boundaries between apps dissolve a little bit And like stuff from one app can appear in another app. ([Time 0:37:15](https://share.snipd.com/snip/9f6923cc-41a4-4824-a745-ec30fd2fc6fe)) - **Blogging Apps United Around Standard Site** - Several blogging apps (Leaflet, Offprint, Pocket) deliberately agreed on a Standard Site lexicon so posts flow across apps and appear in readers. - Blue Sky added a special embed card that uses the publication brand color when it detects a Standard Site record. Transcript: Dan Abramov So one example I mentioned before is, there were three different blogging platforms on Osprato. One is called Leaflet, another Offprint, and another, what is it, Pocket. And so they kind of teamed up, you know, let's use the same format, and then stuff from each app will appear in another app. And so this is called Standard Site. So they published it as a format. They called it standard site. And this let posts appear in different places. But then other developers started using this format as well. So there's integrations for your blog to appear in the atmosphere. If you start on deploy publishing standard site records under your account, then any app that listens, you know, like these blogging apps will see your posts. But people also started building like more stuff around it. So somebody built an app called Standard Reader, which is an aggregator of all standard site records. And it kind of shows like, you know, what's, what standard site articles are popular, like in your network, like if you log in, like which of them are being shared by your like Blue Sky Follows, and so on. So it's kind of like, nobody had to coordinate, you know, like, you can just like build an app and just an existing data on the network. And then another cool thing was, uh, sky added like a special embed card if you share a post or like a link that has a standard side record associated with it it's going to get like a special Visual treatment that uses your brand color of your publication so it's kind of like a special embed that's cool and again like i think that's like it's interesting here that BlueSky App did not need to, like, ask someone's permission to do this. It's just like this data already exists in the open web. So they just started ingesting it and using it in their app. And I think that, I think that's like one interesting example. ([Time 0:50:02](https://share.snipd.com/snip/a0ab3958-91dd-4197-a348-1728fe24b448)) - **Last.fm Style Data Emerged Before The App** - Teal.fm is a lexicon for scrobbles where users post play events; third-party tools already write TLFM play records from Spotify to users' repos. - The Teal app itself barely exists, yet millions of play records are available across the network. Transcript: Dan Abramov So there's like a Prado version of that called Teal.fm. Teal is in color. Except that the app doesn't actually exist yet. So it's like a few people's like hobby project. So they, you know, they're like, I think there's some private beta maybe, but the app itself is just like a landing page. However, they published their lexicon on GitHub. So there's a lexicon for TLFM play that just represents like one scrovel, right? Like one play of a track, like it has the artist name, the track name and so on. And so people started building tools that, you know, sync your Spotify or Apple music or whatever, and write your TLFM play records to your repo. And as a result of that, the data for a lot of people is already there. Like I'm posting TLFM records, when I listen to Spotify. And so people started building like there are third party TLFM apps that show what everybody's like, there's already like 6 million, I think, like place across like everybody. And they're showing like, which are like popular artists, like who's listening to what, but the deal FM doesn't even exist. ([Time 0:52:21](https://share.snipd.com/snip/f769cc38-b5c5-4787-b828-d1b2d91bd89f))