What Dia’s Skill Gallery Means for Agent Design Insights  ## Metadata - Author: [[Hard Fork AI]] - Full Title: What Dia’s Skill Gallery Means for Agent Design Insights - Category: #podcasts - URL: https://share.snipd.com/episode/4e0a51da-3214-4635-9793-4b93dcc10570 ## Document Notes ## Highlights - **Experts Needed for Effective AI Agents** - AI agents require expert knowledge to perform specialized tasks well. - Non-experts need expert-crafted prompts or skills to empower AI effectively. Transcript: Jaeden Schafer But if you had the help of a lawyer to craft the prompt and to read it over, it would be no problem, right? So what is the solution? How do you get an expert's advice or prompts into your agent to complete tasks? Now, there are a ton of companies tackling this. And this is actually something that myself with my company, AI Box, are currently building something low key in stealth towards. So today on the podcast, I want to talk about a couple companies, Dia, and also Comet from Perplexity that are tackling this. I'll give you basically my thesis on why their approach isn't the best and what they should be doing. And I'll explain a little bit about what I'm doing with AI Box. I hope at the end of the day, this episode basically helps you understand what is going on with AI agents and how we can get these things to perform better, no matter what solution you use. So to break this down, first of all, I wanted to talk about the first company, which is Dia. So Dia is a company that it's put out by the browser company, and they basically created something that they call the first AI browser. So we've heard this a couple times. I think OpenAI is even working on an AI browser. It's kind of interesting, like the difference between an AI browser versus now we just have ChatGPT with agents or operator built in, what the need is, what the difference is. We have Anthropic that has computer use. But in any case, the browser company has just launched its first browser called Dia. ([Time 0:00:27](https://share.snipd.com/snip/bd2eb4b5-f37f-47c8-890d-9bc34ab3e278)) - **Challenges of AI in Specialized Tasks** - AI agents often struggle with domain-specific tasks without clear, expert guidance. - Non-expert users risk mediocre results due to lack of precise instructions and jargon knowledge. Transcript: Jaeden Schafer Basically, it's kind of an interface to work with an agent. So I do love the concept of browsers with AI and AI agents in them. I think this is something that's really powerful. So what is going on with Dia? How are they trying to automate these small tasks? Basically, the problem you run into, and I found this myself while using ChatGPT operator frequently, is that you'll ask it to do something and it can get through like a bunch of the process, But it just doesn't know enough. It's not an expert in that particular area. You know, I'll try to get it to help me with like podcast editing. And so I'll say like, hey, log into my podcast production platform, go to my latest episode, take like trim the beginning silence of the episode, trim the end silence of the episode, Look for any anomalies. And you know, I'm trying to like explain like basically what a podcast producer would do to try to get it to go through our entire workflow of post production editing on an episode. And the problem with this is that there are many places where it can kind of glitch up. But also the other problem is like when even if I tell it to do the same thing, it might do it once great. And the second time it might not completely understand it gets confused by some buttons, it goes off in the wrong direction. More or less, though, I'll say something like, go and look for anomalies in the recording and fix them. And it has no idea what the heck anomalies are, right? You got to be very specific. So because I'm, you know, I do this all the time. ([Time 0:01:51](https://share.snipd.com/snip/d975ba9d-7399-491b-acba-e01d64aaa780)) - **Inconsistent AI Podcast Editing** - Jaeden Schafer tries to automate podcast editing with AI and faces inconsistent results. - The AI misses domain-specific knowledge, needing highly detailed guidance to work well. Transcript: Jaeden Schafer It's not an expert in that particular area. You know, I'll try to get it to help me with like podcast editing. And so I'll say like, hey, log into my podcast production platform, go to my latest episode, take like trim the beginning silence of the episode, trim the end silence of the episode, Look for any anomalies. And you know, I'm trying to like explain like basically what a podcast producer would do to try to get it to go through our entire workflow of post production editing on an episode. And the problem with this is that there are many places where it can kind of glitch up. But also the other problem is like when even if I tell it to do the same thing, it might do it once great. And the second time it might not completely understand it gets confused by some buttons, it goes off in the wrong direction. More or less, though, I'll say something like, go and look for anomalies in the recording and fix them. ([Time 0:02:17](https://share.snipd.com/snip/2d4562bc-ed6b-4776-8a25-694186505a5b)) - **Personal Struggle with Audio Editing** - Jaeden Schafer shared his struggle with mixing and mastering music for his wife's album without expertise. - He had to turn to professionals to get useful expert input and complete the task. Transcript: Jaeden Schafer I recently had this experience. I was helping my wife. She released an album of music. And she got kind of sick of all the mixing and mastering on the album. And was like, hey, can you help me do the mixing and mastering? She worked for months on this project and really got burnt out by the end. And I'm like, okay, sure. How hard could it be? I opened up the software. Was very overwhelmed by mixing and mastering, to say the least. Tried my hand in a couple songs. They didn't turn out great. I had to turn to some professionals to to help me with this so why do i have to turn to professionals why do we all have to turn to professionals uh what's the and and i guess what is the solution When we're using these ai agents whether it's dia or perplexity that has a tool how do we basically get the input of a professional because once i you know i called a professional on the Phone got a bunch of tips and examples, sent them over one of my recordings. ([Time 0:03:41](https://share.snipd.com/snip/4c26a7ef-c501-4b3b-8c14-8a857f4071bc)) - **Dia’s Skills Gallery Concept** - Dia’s skills gallery lets users save successful AI task workflows as shareable "skills." - This captures expert processes making AI outputs reliable and reproducible. Transcript: Jaeden Schafer Dia has launched something called a skills gallery. So basically, once you get an AI agent to do like a task, if it's something that you got to do repetitively, if it doesn't correct one time, you basically save this as a skill. So they have something that they've basically like given three examples of quote unquote skills that these things could kind of have saved in there and could be replicated. And the beauty of this, I'm not exactly sure how their platform works, but you could imagine if I accomplished something successfully, whatever prompt I gave it, however I got it to Do that particular task, if I can save a snapshot of that skill where it essentially turns what it did into code, so it's like, it's not gonna mess it up in the future. If you can save that snapshot and share it with other people, then someone else that might not be, you know, an audio engineer or a podcast engineer or a lawyer or a doctor or a plumber or An expert in any domain that you are an expert in can use that to get part of the process done correctly, you know, as created by a professional who understands what's actually going on And has a basically gives the the enough direction to the ai agent to get the job done correctly because i think uh it's kind of going back to my example of like trying to mix and master some Music for my wife it was like i had no idea how to level out the audio and i was like going on youtube and like how to level audio on music track. I didn't realize what I should have been searching for was how to get an EQ plugin for Logic Pro and apply it to a file. Honestly, if I'd done that, I probably could have found a tutorial and figured it out very quickly. Had no idea what EQ was, had no idea how to add a bus, had no idea, basically all this, the industry jargon. And so I was a million miles away. I watched so many videos that were barking down the wrong tree, right? I was looking at audio equalization, not the right thing. So in any case, I think this applies to basically every industry. There's industry jargon. There's just different terms and things that you do inside of an industry. If you're not familiar with them, you don't even know the right questions to ask. And so just having a professional come in and explain it, or in this case, giving you one of these snapshots is literally the difference between like, mediocre or kind of bad crappy output By an AI agent versus an expert level good one, right? ([Time 0:04:46](https://share.snipd.com/snip/6730114d-1ebf-4c20-8aa9-ae68c6be090e)) - **Leverage Expert Knowledge in AI** - Use expert-created workflows or "skill snapshots" to overcome industry jargon barriers. - Sharing expert knowledge in AI tools boosts output quality for novices. Transcript: Jaeden Schafer If you're not familiar with them, you don't even know the right questions to ask. And so just having a professional come in and explain it, or in this case, giving you one of these snapshots is literally the difference between like, mediocre or kind of bad crappy output By an AI agent versus an expert level good one, right? And basically, this is what companies are making all their money on nowadays. Everything's a quote unquote, GPT wrapper. And they're like, people say that is like, oh, you know, these companies are just GPT wrappers, Like, they're not that great. It's like, yeah, because they like deeply understand what their customers need. They deeply understand the issue and the solution more than the customer understands the issue and solution. So the output they're giving is way better than what other people could just get themselves with ChatGP, even though, yes, they're using the technology of ChatGPT, they have domain Expertise. ([Time 0:06:38](https://share.snipd.com/snip/be751a75-a1a9-4988-a214-ed200902e852)) - **Comet Browser’s AI Workflow Approach** - Perplexity’s Comet browser introduces natural language shortcuts and customizable scripts for repetitive tasks. - It aims to become a personal AI-powered operating system for customized workflows. Transcript: Jaeden Schafer So basically, the shortcuts, the examples that they've given are organize my tabs, teach me comment, trending on social, evaluate this deal, prep meeting manage my feed so maybe things You do frequently or maybe i feel like what they're saying is things that like people on their platform do frequently i'm not sure if it's personalized to you if it is personalized to You that's way more useful but in any case um you basically can can do these repeated tasks he then goes on to say uh you will also soon be able to create your own tamper monkey like scripts That are natural language generated for ensuring repetitive usage of custom workflows so everyone's comment will feel personal and something you built for yourself like your own Apps scripts workflows widgets dashboards with perplexity labs the browser feels more and more like an operating system that way there's a reason we purchased os.ai the roadmap is To Comet feel like your own mini customized computer within your existing computer or phone and compute running across client and server with the ability to run local models to do it. ([Time 0:08:53](https://share.snipd.com/snip/7d585f09-f998-48ce-aa17-4d29162e6647)) - **Building Expert AI Agent Tools** - AI Box builds an agent builder platform allowing experts to create replicable AI task tools called agent boxes. - This improves AI agent performance with consistent expert-designed workflows and integrations. Transcript: Jaeden Schafer So what I'm building at AIbox to essentially try to help solve this problem that a lot of people are having with agents is something called an AI agent builder platform. And essentially what you're able to do is build what we're calling agent boxes. So these are tools that basically AI agents can use. You can think about it sort of like the, sort of like how Dia is creating their skills gallery. You can imagine like a skills gallery, but it's much more comprehensive. It's not just like it accomplished a task and you kind of save that accomplished workflow as a snapshot, you can very manually go and get very granular with what AI models you want to accomplish Different parts of the task, what prompts are used to do it, what tools are used to do it, in all the integrations you need to like build out one particular function or task in a very replicable Way. And it's basically built for experts to go in there, automate different elements of a workflow, and for agents in the future to go and be able to basically, you could just go to your agent And say, you know, do XYZ task, go use tools from AI Box to help you accomplish that. The agent can then go and pick different agent boxes on our platform to help them accomplish a task in a very efficient manner. ([Time 0:10:33](https://share.snipd.com/snip/e9160eda-7125-4df6-b2ce-d522dcacbca4))