Claude Code Can Now Control Your Browser - CourtneyR.dev's 2nd Brain

Claude Code Can Now Control Your Browser ![rw-book-cover](https://img.youtube.com/vi/P7JrP57AxR0/maxresdefault.jpg) 🔗 **Source:** [Claude Code Can Now Control Your Browser](https://www.youtube.com/watch?v=P7JrP57AxR0) > [!note]+ Document Notes > tags: [ai-agents, browser-automation, developer-tools] > author: Coding with Lewis > source: YouTube > reference: https://www.youtube.com/watch?v=P7JrP57AxR0 > date created: 2024-12-19 14:30 > > # 🏷️ Title: Claude Code Can Now Control Your Browser (Thanks to Vercel) >  ## 🧰 Action Inbox - [H1] Test agent browser with multimodal model — better screenshot analysis #ai-agents - [H2] Compare agent browser vs browser-use for current projects — architecture differences 05:30 #tooling - [H3] Set up agent browser CLI for automated testing workflows — dark mode validation example #testing ## 🧭 Table of Contents - [📝 Summary](#-summary) - [📖 Main Content](#-main-content) - [🗒️ Notes](#️-notes) - [📚 References](#-references) - [🗣️ Paraphrase](#️-paraphrase) - [📰 Newsletter](#-newsletter) - [✅ Action Items](#-action-items) - [🧪 Hypotheses](#-hypotheses) - [📊 Methodology](#-methodology) - [❓ Questions](#-questions) - [🔮 Future Research](#-future-research) - [💡 Implications](#-implications) - [🧠 Learning Styles Assessment](#-learning-styles-assessment) - [📊 Accessibility Assessment](#-accessibility-assessment) - [🌍 DEIB Assessment](#-deib-assessment) - [✅ Logical Validity](#-logical-validity) - [🛡️ DARVO Analysis](#-darvo-analysis) - [🎯 Conclusion](#-conclusion) ### 🔗 References - [Agent Browser GitHub](https://github.com/vercel/agent-browser) - [Browser Use](https://github.com/browser-use/browser-use) - [Playwright MCP Server](https://github.com/modelcontextprotocol/servers/tree/main/src/playwright) ## 📝 Summary - **What happened**: Vercel released an open-source headless browser CLI called "agent browser" that allows AI agents to control browsers through simple command-line interfaces - **Why it matters**: Enables automated testing and interaction for agent-generated code, reducing manual developer work in the era of AI-driven development - **Key stakeholders**: Developers using AI agents, automation testers, the broader AI tooling ecosystem - **Risks and tradeoffs**: Limited to Chromium browsers, requires external agents, simpler than alternatives like browser-use - **Bottom line**: A lightweight weekend project that could significantly streamline agent-browser interactions for specific use cases ## 📖 Main Content Agent Browser is a headless browser CLI built by Chris Tate at Vercel in a single weekend, written in both Rust and TypeScript. The tool allows AI agents to interact with web browsers through command-line interfaces, performing actions like clicking, typing, taking screenshots, and creating accessibility snapshots. The architecture uses a Rust binary for fast command parsing that communicates with a Node.js daemon managing Chromium browsers through Playwright. This setup enables multiple browser sessions and efficient resource usage. Key features include accessibility tree generation, reference-based actions, semantic locators for finding elements, and JavaScript evaluation capabilities. The tool competes with more feature-rich alternatives like browser-use and Playwright MCP server, but offers simplicity and ease of installation as its main advantages. ## 🗒️ Notes #### ⏱️ [00:00](https://youtu.be/P7JrP57AxR0?t=0) — [H1] Introduction to Agent Browser - Key idea: Open-source headless browser CLI built in a weekend by Vercel employee - Evidence: "lets your agent do anything in the browser from dragging and dropping to uploading an image" #### ⏱️ [00:30](https://youtu.be/P7JrP57AxR0?t=30) — [H2] 2026 Vision: AI Agents Everywhere - Key idea: Future where AI agents handle all code writing, reviewing, and testing - Evidence: "developers are even moving away from IDEs entirely in favor of doing everything in the terminal" #### ⏱️ [01:15](https://youtu.be/P7JrP57AxR0?t=75) — [H2] Core Features Overview - Key idea: Agent browser provides accessibility snapshots and reference-based actions - Evidence: "accessibility tree and references to elements of a page, reference based actions" #### ⏱️ [01:45](https://youtu.be/P7JrP57AxR0?t=105) — [H2] Dark Mode Demo Setup - Key idea: Testing agent browser on a React login page with broken dark mode - Evidence: "I'm blinding my users because it's in light mode" #### ⏱️ [02:30](https://youtu.be/P7JrP57AxR0?t=150) — [H2] Agent Browser in Action - Key idea: Agent uses CLI commands to test and fix dark mode functionality - Evidence: "agent hyphen browser-help to see the available commands because there's no slash commands, no skills" #### ⏱️ [03:45](https://youtu.be/P7JrP57AxR0?t=225) — [H3] Validation Testing Demo - Key idea: Agent creates bash scripts to test form validation automatically - Evidence: "it's made an even better bash test down below, which we can see over here that makes use of agent browser eval" #### ⏱️ [05:30](https://youtu.be/P7JrP57AxR0?t=330) — [H1] Architecture Deep Dive - Key idea: Rust binary parses commands and sends JSON to Node.js daemon managing Chromium - Evidence: "gets sent to a Rust binary that passes that command and converts it to JSON" #### ⏱️ [06:15](https://youtu.be/P7JrP57AxR0?t=375) — [H2] Why Rust and TypeScript - Key idea: Rust provides speed and resource efficiency for the CLI component - Evidence: "The benefit of it being Rust is that it's fast, resource efficient, and stays running" #### ⏱️ [06:45](https://youtu.be/P7JrP57AxR0?t=405) — [H2] Daemon and Browser Management - Key idea: Node daemon manages multiple browser sessions through Unix sockets - Evidence: "a demon is run on each session, meaning multiple browsers can be controlled" #### ⏱️ [07:30](https://youtu.be/P7JrP57AxR0?t=450) — [H1] Comparison with Browser-Use - Key idea: Browser-use has full agent reasoning loop and more features - Evidence: "can run the full agent reasoning loop. So, plan, action, observe, and replan all on its own" #### ⏱️ [08:15](https://youtu.be/P7JrP57AxR0?t=495) — [H2] Agent Browser Limitations - Key idea: Simpler tool requiring external agents, only supports Chromium - Evidence: "much simpler and can only be used with an external agent" #### ⏱️ [08:45](https://youtu.be/P7JrP57AxR0?t=525) — [H2] Playwright MCP Server Comparison - Key idea: Playwright MCP supports all browsers but may confuse agents with too many tools - Evidence: "if you already have loads and loads of MCP tools, then adding a bunch more could confuse the agents" #### ⏱️ [09:30](https://youtu.be/P7JrP57AxR0?t=570) — [H3] Personal Recommendation - Key idea: Choose based on use case; agent browser offers simplicity and easy installation - Evidence: "I like the simplicity and ease of installation from agent browser" ## 📚 References - [Agent Browser GitHub Repository](https://github.com/vercel/agent-browser) - [Browser-Use Project](https://github.com/browser-use/browser-use) - [Playwright MCP Server](https://github.com/modelcontextprotocol/servers) - [Vercel](https://vercel.com) ## 🗣️ Paraphrase - Vercel created a lightweight browser automation tool that lets AI agents control web browsers through simple command-line instructions, making it easier for agents to test the code they generate - The tool uses a dual-architecture approach with Rust handling fast command processing and Node.js managing the actual browser interactions through Playwright - While simpler than alternatives like browser-use, agent browser focuses on ease of use and quick setup, making it ideal for developers who want straightforward browser automation without complex configuration ## 📰 Newsletter **AI Agents Get Browser Control: Vercel's Weekend Project Changes Testing** This week, a Vercel engineer built something remarkable in just a weekend: Agent Browser, an open-source tool that lets AI agents control web browsers through simple CLI commands. **What This Means for You:** - **Automated Testing**: Your AI agents can now test the interfaces they build, reducing manual QA work - **Faster Development**: No more switching between terminal and browser to validate agent-generated code - **Simple Setup**: Install with npm and start using immediately with any AI model The tool shines in its simplicity - while competitors like browser-use offer more features, Agent Browser focuses on ease of use. It creates accessibility snapshots, performs reference-based actions, and can even generate bash scripts for automated testing. **Quick Start:** ```bash npm install -g agent-browser agent-browser --help ``` Whether you're using Claude, GPT-4, or local models, this tool bridges the gap between AI code generation and browser validation. **Resources:** [Agent Browser GitHub](https://github.com/vercel/agent-browser), [Browser Automation Guide](https://playwright.dev/), [AI Agent Best Practices](https://docs.anthropic.com/) ## ✅ Action Items #### DevRel - Developer advocacy: Create demo showcasing agent browser with popular AI models, track adoption metrics - Education: Workshop on browser automation for AI agents, office hours for troubleshooting - Developer experience: Documentation improvements for CLI installation and usage patterns - Developer marketing: Case study campaign showing before/after development workflows - Community management: Monitor feedback on simplicity vs feature-richness debate #### OSPO - License Compliance: Review Rust/TypeScript dual-license compatibility for internal use - Software Inventory: Add agent browser to approved automation tools SBOM - Developer Support: Training materials for CLI-based browser testing - Community Engagement: Evaluate contributing to upstream agent browser development #### Community - Maintainers: Template for integrating browser testing into agent workflows - Contributors: First-issue program for browser automation tooling - Ecosystem: Cross-tool coordination between agent browser, MCP servers, browser-use #### WordPress - Core: Browser testing automation for Gutenberg agent contributions - Playground: Integration with agent browser for automated playground testing - Community: Meetup kit on AI agents for WordPress development - Training: Lesson plan on automated browser testing for WordPress sites ## 🧪 Hypotheses - Agent browser adoption will be higher among developers prioritizing simplicity over comprehensive features - Rust/Node.js hybrid architecture provides optimal performance for CLI browser automation - Weekend projects from major companies can drive significant ecosystem adoption - Chromium-only support is sufficient for majority of agent browser automation use cases ### 📊 Methodology - Survey comparison: Agent browser vs browser-use adoption rates and user satisfaction - Performance benchmarking: Rust CLI vs pure JavaScript browser automation tools - Feature analysis: Track which agent browser capabilities see highest usage ### ❓ Questions - Will Vercel expand agent browser into a full browser automation platform? - How will the tool handle complex multi-step workflows that require state management? - What security implications arise from agents having direct browser control? ### 🔮 Future Research - Integration patterns between agent browser and existing CI/CD pipelines - Multi-browser support roadmap and technical challenges - Agent reasoning capabilities when provided with browser automation tools ### 💡 Implications - **Practice**: Shifts browser testing from manual developer task to automated agent workflow - **Policy**: Need for guidelines on agent-controlled browser security and data handling - **Security**: Browser automation by AI agents creates new attack vectors requiring monitoring - **Education**: Training developers to write testable interfaces for agent validation - **Funding**: Investment in lightweight, focused developer tools over comprehensive platforms ### 🧠 Learning Styles Assessment Reading level estimate: 8-9 (Flesch-Kincaid Grade Level) #### 👀 Visual learners - Present: Screenshots of dark mode testing, architecture diagrams implied - Improve: Browser automation flow diagrams, before/after interface comparisons #### 🔊 Aural learners - Present: YouTube video format with narration - Improve: 3-minute audio summary of key CLI commands and use cases #### 🗣️ Verbal learners - Present: Clear definitions of technical terms, CLI command explanations - Improve: Analogies comparing browser automation to physical testing #### 🧩 Kinesthetic learners - Present: Step-by-step CLI demos, hands-on testing examples - Improve: Interactive tutorial with reproducible login page example #### 🧠 Logical learners - Present: Architecture explanation, comparison matrix of tools - Improve: Decision tree for choosing between agent browser alternatives #### 🧑‍🤝‍🧑 Social learners - Present: Community discussion prompts about tool adoption - Improve: Collaborative testing scenarios, peer review of automation scripts #### 🧘‍♀️ Solitary learners - Present: Individual setup instructions, self-contained examples - Improve: Personal project ideas, reflective questions on automation benefits #### 🌿 Naturalistic learners - Present: Tool ecosystem comparisons, organic development stories - Improve: Evolution narrative of browser automation, tool lifecycle thinking ### 📊 Accessibility Assessment #### Screen reader semantics - Present: Clear heading structure, descriptive link text - Improve: Alt text for screenshot descriptions, ARIA labels for code examples #### Keyboard navigation - Present: Standard YouTube video controls, logical tab order - Improve: Skip links for demo sections, keyboard shortcuts for timestamps #### Contrast and readability - Present: Standard contrast ratios in video player - Improve: High-contrast code examples, larger text in CLI demonstrations #### Text resizing and reflow - Present: Responsive video player - Improve: Scalable code font sizes, flexible layout at 200% zoom #### Media and captions - Present: YouTube auto-captions available - Improve: Professional captions, downloadable transcript with timestamps #### Forms and interactions - Present: Standard video controls - Improve: Accessible CLI command input examples, labeled interactive elements ### 🌍 DEIB Assessment #### Cultural breadth - Present: English-language content with global CLI tool access - Improve: Multi-language examples, diverse developer workflow scenarios #### Inclusive language - Present: Gender-neutral technical terms, accessible jargon definitions - Improve: Plain-language CLI explanations, avoid weekend work assumptions #### Neurodiversity support - Present: Structured content with clear timestamps - Improve: Visual CLI command organization, consistent heading patterns #### Representation - Present: Focus on technical tool demonstration - Improve: Diverse developer use cases, varied project types beyond React #### Psychological safety - Present: Non-judgmental tool comparison, respectful framing - Improve: Acknowledge different skill levels, encourage experimentation ### ✅ Logical Validity #### Strong reasoning - Deductive strength: Architecture explanation logically follows from performance requirements - Inductive strength: Tool comparisons based on observed feature differences - Causal reasoning: Links weekend project timeline to simplicity design choice - Analogical strength: Compares browser automation tools through consistent criteria #### Fallacies - Appeal to novelty: Assumes weekend project timeline makes tool impressive without considering quality - False dilemma: Presents choice between simplicity and features as binary when hybrid approaches exist #### Tone of voice - Overall tone: Enthusiastic and educational with technical depth, maintains objectivity in tool comparisons ### 🛡️ DARVO Analysis - Deny: No denial patterns present - straightforward technical explanation - Attack: No attacks on competing tools, presents balanced comparison - Reverse victim/offender: Not applicable to technical content - Emotional cues and rhetoric: Positive framing of all tools mentioned, excitement about technological progress ## 🎯 Conclusion Agent Browser represents a pragmatic approach to AI-browser integration, prioritizing simplicity and ease of use over comprehensive features. Built in a weekend by a single developer, it demonstrates how focused tooling can address specific pain points in the emerging AI agent development workflow. The key takeaway for practitioners is that different browser automation tools serve different needs: agent browser for simplicity, browser-use for full agent reasoning, an --- ## Full Transcript [00:00] This is agents browser, an open- source [00:02] headless browser CLI built in a weekend [00:04] by a single VEL employee that lets your [00:06] agent do anything in the browser from [00:08] dragging and dropping to uploading an [00:10] image and even toggling offline mode. [00:11] But why would anyone use this over [00:13] something like a browser use which has [00:15] way more features? And is Versel getting [00:17] into the agent browser space? Hit [00:19] subscribe and let's get into it. [00:22] 2026 is the year of AI agents writing, [00:25] reviewing, and testing all of your code. [00:27] No more tab completions. In fact, [00:29] developers are even moving away from [00:31] idees entirely in favor of doing [00:33] everything in the terminal since all [00:35] we're really doing now is reviewing [00:36] code. And to help with this movement, we [00:39] need the agents to actually interact [00:40] with and test the code they've written. [00:42] Because the last thing you want to do as [00:44] a developer is to open up the browser to [00:46] test each feature an army of agents have [00:49] written one by one because that's just [00:51] tedious. This is where Vel's new agent [00:54] browser comes in handy. Written by Chris [00:56] Tate in both Rust and Typescript. I'll [00:58] explain why later. This tool makes it so [01:00] easy for an agent to interact with the [01:02] browser using CLI commands that do a [01:05] bunch of things like creating an [01:07] accessibility snapshot that provides an [01:09] accessibility tree and references to [01:11] elements of a page, reference based [01:13] actions that takes the references from [01:15] the tree and apply relevant actions to [01:17] them. There's also semantic locators if [01:19] you don't want to use references that [01:21] allow you to find an element based on [01:23] its area ro its text content its label [01:26] and so much more. In fact, let's go [01:28] through a quick demo of how it works. [01:30] Now, here is a little login page with an [01:32] email and password. And it's built with [01:34] Shadien React MV not because of a cell [01:37] or anything. It just happened to be [01:38] built that way. Now, there's one problem [01:40] with this whole page and it's that right [01:43] now I'm blinding my users because it's [01:45] in light mode. So I want there to be a [01:47] dark mode which I've actually gone ahead [01:49] and asked the agent to do but as you can [01:51] see it hasn't done it correctly. I mean [01:53] okay this text changes but nothing else. [01:56] So let's go ahead and get the agent to [01:58] fix this using agent browser. So right [02:00] now I'm using open code with the GLM 4.7 [02:03] model but of course agent browser can [02:05] work with any agent and any model. I've [02:07] gone ahead and told it that dark mode is [02:09] broken and it should test it with agent [02:11] hyphen browser on the specific port. [02:13] What's important is this part of the [02:15] command to run agent hyphen browser-help [02:19] to see the available commands because [02:21] there's no slash commands, no skills. [02:22] I've just installed agent browser [02:24] globally with npm. I'm going to hit [02:26] enter and then it checks the available [02:28] commands, uses the agent browser [02:30] snapshot functionality to create a [02:32] snapshot of the page which shows [02:33] documents, heading, paragraph, text, and [02:35] even images. It's then clicked on the [02:37] relevant element and taken a screenshot [02:39] to see if dark mode is working. And this [02:41] is the screenshot if you're curious. [02:43] From here, it's gone ahead and fixed the [02:44] issue before taking another screenshot [02:46] of the fixed dark mode. And it's finally [02:48] finished the task, which we can test by [02:50] clicking up here. And we have a page [02:52] with perfect dark mode. Let's try [02:54] another test. Actually, while this was [02:56] running, I had another agent in the [02:58] background fix another issue. You may [02:59] have noticed that if I press the login [03:01] button, it will take me straight here [03:03] without any validation, which of course [03:05] isn't good. So I went ahead and asked it [03:07] to fix the issue with the validation in [03:09] this project and it did something [03:11] actually really interesting. It first [03:13] checks the available commands from Asian [03:15] browser and then if we scroll down it [03:17] fixes the issue and even makes a bash [03:20] script. So it's over here to test that [03:23] it works. So it echoes the first test, [03:25] adds an empty input, clicks the login [03:27] button and then expects these errors. [03:30] It's made a few tests here, but it's [03:32] actually made an even better bash test [03:34] down below, which we can see over here [03:36] that makes use of agent browser eval to [03:38] run some JavaScript code. So now we can [03:40] see if I press the login button, we get [03:42] some validation. This looks like an [03:44] email, but it's actually a placeholder. [03:46] If I give it an email over here, just a [03:47] made up one, and hit login, it says [03:49] enter a valid email, which I can do like [03:51] this. And then I can enter a password [03:53] before it takes me to this dashboard. [03:55] So, basically, agents have addressed two [03:57] issues with this app and tested it [03:59] themselves to validate it works using [04:01] the agent browser plug-in. I would say [04:03] if I could do it again, I'd go with a [04:05] model that has multimodal support so you [04:07] can actually read the screenshots it [04:09] takes instead of using GLM 4.7. But now, [04:12] let's get into some architecture. How [04:14] does agent browser actually do all of [04:16] these things? Well, after an agent runs [04:17] a command like agent browser click at [04:20] E2, this gets sent to a Rust binary that [04:23] passes that command and converts it to [04:25] JSON. The benefit of it being Rust is [04:27] that it's fast, resource efficient, and [04:30] stays running after it's spawned. This [04:32] JSON then gets sent to a node demon [04:35] through a Unix socket, and this demon [04:37] manages the Chromium browser. What's [04:38] cool about this is that a demon is run [04:41] on each session, meaning multiple [04:43] browsers can be controlled. Once the [04:44] demon validates the output, it launches [04:47] the browser which is headless by default [04:49] and executes the action using playright. [04:52] And once it's completed, it sends the [04:54] output also in JSON back to the agent. [04:57] So the CLI and from here the agent can [04:59] do whatever it wants. So it can send [05:01] even more commands to the ROS passer or [05:04] it can end the whole loop. All of this [05:06] is super impressive for a weekend's [05:08] project. But how does it compare to [05:09] browser use or even using the Playright [05:12] MCP server? Well, browser use can be [05:14] used with or without an external agent [05:17] because it can run the full agent [05:19] reasoning loop. So, plan, action, [05:21] observe, and replan all on its own [05:24] without the use of anything else. It [05:26] also has a Python and TypeScript SDK for [05:29] fine grained control. It has a skills [05:31] marketplace, an MCP server, and uses [05:33] better stack to track the API status, [05:35] which is very cool. Agent browser on the [05:37] other hand is much simpler and can only [05:40] be used with an external agent. So you [05:43] the developer have to provide an agent [05:45] like cursor, claw code, open code and [05:48] the agent only interacts with agent [05:50] browser through CLI commands. When it [05:52] comes to comparing agent browser with [05:54] the Playright MCP server right now, [05:57] agent browser only supports Chromium [05:58] browsers. So no Firefox or Safari, which [06:01] I believe the Playright MCP server does [06:04] support. In fact, it supports everything [06:06] that Playright can do, but with an MCP [06:08] server that is perfect for agents. The [06:10] only kind of downside is if you already [06:12] have loads and loads of MCP tools, then [06:14] adding a bunch more could confuse the [06:16] agents since it has more tools available [06:19] to choose from autonomously, and it [06:21] might not choose the right one the first [06:22] time. Basically, choosing between [06:24] browser use, the Playright MCP server, [06:26] and agent browser really depends on your [06:28] use case and what you want from your [06:30] agent. For me personally, I like the [06:33] simplicity and ease of installation from [06:35] agent browser. And I primarily use [06:37] Chromium browsers, so not too fussed [06:39] about not having Firefox or Safari [06:41] access. So, I'll use it for now and see [06:43] how it goes. Also, while I've got your [06:45] attention, we are so close to hitting [06:47] that 100K subscriber mark. So, if you [06:49] haven't already, go ahead and hit the [06:51] subscribe button.  [[3. Resources/Readwise/Full Document Contents/Articles/Claude Code Can Now Control Your Browser|Full Document Contents]] ## Document Notes --- tags: #ai-agents #browser-automation #developer-tools author: Coding with Lewis source: YouTube reference: https://www.youtube.com/watch?v=P7JrP57AxR0 date created: 2024-12-19 14:30 --- # 🏷️ Title: Claude Code Can Now Control Your Browser (Thanks to Vercel) ## 🧰 Action Inbox - [H1] Test agent browser with multimodal model — better screenshot analysis #ai-agents - [H2] Compare agent browser vs browser-use for current projects — architecture differences 05:30 #tooling - [H3] Set up agent browser CLI for automated testing workflows — dark mode validation example #testing ## 🧭 Table of Contents - [[#📝 Summary]] - [[#📖 Main Content]] - [[#🗒️ Notes]] - [[#📚 References]] - [[#🗣️ Paraphrase]] - [[#📰 Newsletter]] - [[#✅ Action Items]] - [[#🧪 Hypotheses]] - [[#📊 Methodology]] - [[#❓ Questions]] - [[#🔮 Future Research]] - [[#💡 Implications]] - [[#🧠 Learning Styles Assessment]] - [[#📊 Accessibility Assessment]] - [[#🌍 DEIB Assessment]] - [[#✅ Logical Validity]] - [[#🛡️ DARVO Analysis]] - [[#🎯 Conclusion]] ### 🔗 References - [Agent Browser GitHub](https://github.com/vercel/agent-browser) - [Browser Use](https://github.com/browser-use/browser-use) - [Playwright MCP Server](https://github.com/modelcontextprotocol/servers/tree/main/src/playwright) ## 📝 Summary - **What happened**: Vercel released an open-source headless browser CLI called "agent browser" that allows AI agents to control browsers through simple command-line interfaces - **Why it matters**: Enables automated testing and interaction for agent-generated code, reducing manual developer work in the era of AI-driven development - **Key stakeholders**: Developers using AI agents, automation testers, the broader AI tooling ecosystem - **Risks and tradeoffs**: Limited to Chromium browsers, requires external agents, simpler than alternatives like browser-use - **Bottom line**: A lightweight weekend project that could significantly streamline agent-browser interactions for specific use cases ## 📖 Main Content Agent Browser is a headless browser CLI built by Chris Tate at Vercel in a single weekend, written in both Rust and TypeScript. The tool allows AI agents to interact with web browsers through command-line interfaces, performing actions like clicking, typing, taking screenshots, and creating accessibility snapshots. The architecture uses a Rust binary for fast command parsing that communicates with a Node.js daemon managing Chromium browsers through Playwright. This setup enables multiple browser sessions and efficient resource usage. Key features include accessibility tree generation, reference-based actions, semantic locators for finding elements, and JavaScript evaluation capabilities. The tool competes with more feature-rich alternatives like browser-use and Playwright MCP server, but offers simplicity and ease of installation as its main advantages. ## 🗒️ Notes #### ⏱️ [00:00](https://youtu.be/P7JrP57AxR0?t=0) — [H1] Introduction to Agent Browser - Key idea: Open-source headless browser CLI built in a weekend by Vercel employee - Evidence: "lets your agent do anything in the browser from dragging and dropping to uploading an image" #### ⏱️ [00:30](https://youtu.be/P7JrP57AxR0?t=30) — [H2] 2026 Vision: AI Agents Everywhere - Key idea: Future where AI agents handle all code writing, reviewing, and testing - Evidence: "developers are even moving away from IDEs entirely in favor of doing everything in the terminal" #### ⏱️ [01:15](https://youtu.be/P7JrP57AxR0?t=75) — [H2] Core Features Overview - Key idea: Agent browser provides accessibility snapshots and reference-based actions - Evidence: "accessibility tree and references to elements of a page, reference based actions" #### ⏱️ [01:45](https://youtu.be/P7JrP57AxR0?t=105) — [H2] Dark Mode Demo Setup - Key idea: Testing agent browser on a React login page with broken dark mode - Evidence: "I'm blinding my users because it's in light mode" #### ⏱️ [02:30](https://youtu.be/P7JrP57AxR0?t=150) — [H2] Agent Browser in Action - Key idea: Agent uses CLI commands to test and fix dark mode functionality - Evidence: "agent hyphen browser-help to see the available commands because there's no slash commands, no skills" #### ⏱️ [03:45](https://youtu.be/P7JrP57AxR0?t=225) — [H3] Validation Testing Demo - Key idea: Agent creates bash scripts to test form validation automatically - Evidence: "it's made an even better bash test down below, which we can see over here that makes use of agent browser eval" #### ⏱️ [05:30](https://youtu.be/P7JrP57AxR0?t=330) — [H1] Architecture Deep Dive - Key idea: Rust binary parses commands and sends JSON to Node.js daemon managing Chromium - Evidence: "gets sent to a Rust binary that passes that command and converts it to JSON" #### ⏱️ [06:15](https://youtu.be/P7JrP57AxR0?t=375) — [H2] Why Rust and TypeScript - Key idea: Rust provides speed and resource efficiency for the CLI component - Evidence: "The benefit of it being Rust is that it's fast, resource efficient, and stays running" #### ⏱️ [06:45](https://youtu.be/P7JrP57AxR0?t=405) — [H2] Daemon and Browser Management - Key idea: Node daemon manages multiple browser sessions through Unix sockets - Evidence: "a demon is run on each session, meaning multiple browsers can be controlled" #### ⏱️ [07:30](https://youtu.be/P7JrP57AxR0?t=450) — [H1] Comparison with Browser-Use - Key idea: Browser-use has full agent reasoning loop and more features - Evidence: "can run the full agent reasoning loop. So, plan, action, observe, and replan all on its own" #### ⏱️ [08:15](https://youtu.be/P7JrP57AxR0?t=495) — [H2] Agent Browser Limitations - Key idea: Simpler tool requiring external agents, only supports Chromium - Evidence: "much simpler and can only be used with an external agent" #### ⏱️ [08:45](https://youtu.be/P7JrP57AxR0?t=525) — [H2] Playwright MCP Server Comparison - Key idea: Playwright MCP supports all browsers but may confuse agents with too many tools - Evidence: "if you already have loads and loads of MCP tools, then adding a bunch more could confuse the agents" #### ⏱️ [09:30](https://youtu.be/P7JrP57AxR0?t=570) — [H3] Personal Recommendation - Key idea: Choose based on use case; agent browser offers simplicity and easy installation - Evidence: "I like the simplicity and ease of installation from agent browser" ## 📚 References - [Agent Browser GitHub Repository](https://github.com/vercel/agent-browser) - [Browser-Use Project](https://github.com/browser-use/browser-use) - [Playwright MCP Server](https://github.com/modelcontextprotocol/servers) - [Vercel](https://vercel.com) ## 🗣️ Paraphrase - Vercel created a lightweight browser automation tool that lets AI agents control web browsers through simple command-line instructions, making it easier for agents to test the code they generate - The tool uses a dual-architecture approach with Rust handling fast command processing and Node.js managing the actual browser interactions through Playwright - While simpler than alternatives like browser-use, agent browser focuses on ease of use and quick setup, making it ideal for developers who want straightforward browser automation without complex configuration ## 📰 Newsletter **AI Agents Get Browser Control: Vercel's Weekend Project Changes Testing** This week, a Vercel engineer built something remarkable in just a weekend: Agent Browser, an open-source tool that lets AI agents control web browsers through simple CLI commands. **What This Means for You:** - **Automated Testing**: Your AI agents can now test the interfaces they build, reducing manual QA work - **Faster Development**: No more switching between terminal and browser to validate agent-generated code - **Simple Setup**: Install with npm and start using immediately with any AI model The tool shines in its simplicity - while competitors like browser-use offer more features, Agent Browser focuses on ease of use. It creates accessibility snapshots, performs reference-based actions, and can even generate bash scripts for automated testing. **Quick Start:** ```bash npm install -g agent-browser agent-browser --help ``` Whether you're using Claude, GPT-4, or local models, this tool bridges the gap between AI code generation and browser validation. **Resources:** [Agent Browser GitHub](https://github.com/vercel/agent-browser), [Browser Automation Guide](https://playwright.dev/), [AI Agent Best Practices](https://docs.anthropic.com/) ## ✅ Action Items #### DevRel - Developer advocacy: Create demo showcasing agent browser with popular AI models, track adoption metrics - Education: Workshop on browser automation for AI agents, office hours for troubleshooting - Developer experience: Documentation improvements for CLI installation and usage patterns - Developer marketing: Case study campaign showing before/after development workflows - Community management: Monitor feedback on simplicity vs feature-richness debate #### OSPO - License Compliance: Review Rust/TypeScript dual-license compatibility for internal use - Software Inventory: Add agent browser to approved automation tools SBOM - Developer Support: Training materials for CLI-based browser testing - Community Engagement: Evaluate contributing to upstream agent browser development #### Community - Maintainers: Template for integrating browser testing into agent workflows - Contributors: First-issue program for browser automation tooling - Ecosystem: Cross-tool coordination between agent browser, MCP servers, browser-use #### WordPress - Core: Browser testing automation for Gutenberg agent contributions - Playground: Integration with agent browser for automated playground testing - Community: Meetup kit on AI agents for WordPress development - Training: Lesson plan on automated browser testing for WordPress sites ## 🧪 Hypotheses - Agent browser adoption will be higher among developers prioritizing simplicity over comprehensive features - Rust/Node.js hybrid architecture provides optimal performance for CLI browser automation - Weekend projects from major companies can drive significant ecosystem adoption - Chromium-only support is sufficient for majority of agent browser automation use cases ### 📊 Methodology - Survey comparison: Agent browser vs browser-use adoption rates and user satisfaction - Performance benchmarking: Rust CLI vs pure JavaScript browser automation tools - Feature analysis: Track which agent browser capabilities see highest usage ### ❓ Questions - Will Vercel expand agent browser into a full browser automation platform? - How will the tool handle complex multi-step workflows that require state management? - What security implications arise from agents having direct browser control? ### 🔮 Future Research - Integration patterns between agent browser and existing CI/CD pipelines - Multi-browser support roadmap and technical challenges - Agent reasoning capabilities when provided with browser automation tools ### 💡 Implications - **Practice**: Shifts browser testing from manual developer task to automated agent workflow - **Policy**: Need for guidelines on agent-controlled browser security and data handling - **Security**: Browser automation by AI agents creates new attack vectors requiring monitoring - **Education**: Training developers to write testable interfaces for agent validation - **Funding**: Investment in lightweight, focused developer tools over comprehensive platforms ### 🧠 Learning Styles Assessment Reading level estimate: 8-9 (Flesch-Kincaid Grade Level) #### 👀 Visual learners - Present: Screenshots of dark mode testing, architecture diagrams implied - Improve: Browser automation flow diagrams, before/after interface comparisons #### 🔊 Aural learners - Present: YouTube video format with narration - Improve: 3-minute audio summary of key CLI commands and use cases #### 🗣️ Verbal learners - Present: Clear definitions of technical terms, CLI command explanations - Improve: Analogies comparing browser automation to physical testing #### 🧩 Kinesthetic learners - Present: Step-by-step CLI demos, hands-on testing examples - Improve: Interactive tutorial with reproducible login page example #### 🧠 Logical learners - Present: Architecture explanation, comparison matrix of tools - Improve: Decision tree for choosing between agent browser alternatives #### 🧑‍🤝‍🧑 Social learners - Present: Community discussion prompts about tool adoption - Improve: Collaborative testing scenarios, peer review of automation scripts #### 🧘‍♀️ Solitary learners - Present: Individual setup instructions, self-contained examples - Improve: Personal project ideas, reflective questions on automation benefits #### 🌿 Naturalistic learners - Present: Tool ecosystem comparisons, organic development stories - Improve: Evolution narrative of browser automation, tool lifecycle thinking ### 📊 Accessibility Assessment #### Screen reader semantics - Present: Clear heading structure, descriptive link text - Improve: Alt text for screenshot descriptions, ARIA labels for code examples #### Keyboard navigation - Present: Standard YouTube video controls, logical tab order - Improve: Skip links for demo sections, keyboard shortcuts for timestamps #### Contrast and readability - Present: Standard contrast ratios in video player - Improve: High-contrast code examples, larger text in CLI demonstrations #### Text resizing and reflow - Present: Responsive video player - Improve: Scalable code font sizes, flexible layout at 200% zoom #### Media and captions - Present: YouTube auto-captions available - Improve: Professional captions, downloadable transcript with timestamps #### Forms and interactions - Present: Standard video controls - Improve: Accessible CLI command input examples, labeled interactive elements ### 🌍 DEIB Assessment #### Cultural breadth - Present: English-language content with global CLI tool access - Improve: Multi-language examples, diverse developer workflow scenarios #### Inclusive language - Present: Gender-neutral technical terms, accessible jargon definitions - Improve: Plain-language CLI explanations, avoid weekend work assumptions #### Neurodiversity support - Present: Structured content with clear timestamps - Improve: Visual CLI command organization, consistent heading patterns #### Representation - Present: Focus on technical tool demonstration - Improve: Diverse developer use cases, varied project types beyond React #### Psychological safety - Present: Non-judgmental tool comparison, respectful framing - Improve: Acknowledge different skill levels, encourage experimentation ### ✅ Logical Validity #### Strong reasoning - Deductive strength: Architecture explanation logically follows from performance requirements - Inductive strength: Tool comparisons based on observed feature differences - Causal reasoning: Links weekend project timeline to simplicity design choice - Analogical strength: Compares browser automation tools through consistent criteria #### Fallacies - Appeal to novelty: Assumes weekend project timeline makes tool impressive without considering quality - False dilemma: Presents choice between simplicity and features as binary when hybrid approaches exist #### Tone of voice - Overall tone: Enthusiastic and educational with technical depth, maintains objectivity in tool comparisons ### 🛡️ DARVO Analysis - Deny: No denial patterns present - straightforward technical explanation - Attack: No attacks on competing tools, presents balanced comparison - Reverse victim/offender: Not applicable to technical content - Emotional cues and rhetoric: Positive framing of all tools mentioned, excitement about technological progress ## 🎯 Conclusion Agent Browser represents a pragmatic approach to AI-browser integration, prioritizing simplicity and ease of use over comprehensive features. Built in a weekend by a single developer, it demonstrates how focused tooling can address specific pain points in the emerging AI agent development workflow. The key takeaway for practitioners is that different browser automation tools serve different needs: agent browser for simplicity, browser-use for full agent reasoning, an --- ## Full Transcript [00:00] This is agents browser, an open- source [00:02] headless browser CLI built in a weekend [00:04] by a single VEL employee that lets your [00:06] agent do anything in the browser from [00:08] dragging and dropping to uploading an [00:10] image and even toggling offline mode. [00:11] But why would anyone use this over [00:13] something like a browser use which has [00:15] way more features? And is Versel getting [00:17] into the agent browser space? Hit [00:19] subscribe and let's get into it. [00:22] 2026 is the year of AI agents writing, [00:25] reviewing, and testing all of your code. [00:27] No more tab completions. In fact, [00:29] developers are even moving away from [00:31] idees entirely in favor of doing [00:33] everything in the terminal since all [00:35] we're really doing now is reviewing [00:36] code. And to help with this movement, we [00:39] need the agents to actually interact [00:40] with and test the code they've written. [00:42] Because the last thing you want to do as [00:44] a developer is to open up the browser to [00:46] test each feature an army of agents have [00:49] written one by one because that's just [00:51] tedious. This is where Vel's new agent [00:54] browser comes in handy. Written by Chris [00:56] Tate in both Rust and Typescript. I'll [00:58] explain why later. This tool makes it so [01:00] easy for an agent to interact with the [01:02] browser using CLI commands that do a [01:05] bunch of things like creating an [01:07] accessibility snapshot that provides an [01:09] accessibility tree and references to [01:11] elements of a page, reference based [01:13] actions that takes the references from [01:15] the tree and apply relevant actions to [01:17] them. There's also semantic locators if [01:19] you don't want to use references that [01:21] allow you to find an element based on [01:23] its area ro its text content its label [01:26] and so much more. In fact, let's go [01:28] through a quick demo of how it works. [01:30] Now, here is a little login page with an [01:32] email and password. And it's built with [01:34] Shadien React MV not because of a cell [01:37] or anything. It just happened to be [01:38] built that way. Now, there's one problem [01:40] with this whole page and it's that right [01:43] now I'm blinding my users because it's [01:45] in light mode. So I want there to be a [01:47] dark mode which I've actually gone ahead [01:49] and asked the agent to do but as you can [01:51] see it hasn't done it correctly. I mean [01:53] okay this text changes but nothing else. [01:56] So let's go ahead and get the agent to [01:58] fix this using agent browser. So right [02:00] now I'm using open code with the GLM 4.7 [02:03] model but of course agent browser can [02:05] work with any agent and any model. I've [02:07] gone ahead and told it that dark mode is [02:09] broken and it should test it with agent [02:11] hyphen browser on the specific port. [02:13] What's important is this part of the [02:15] command to run agent hyphen browser-help [02:19] to see the available commands because [02:21] there's no slash commands, no skills. [02:22] I've just installed agent browser [02:24] globally with npm. I'm going to hit [02:26] enter and then it checks the available [02:28] commands, uses the agent browser [02:30] snapshot functionality to create a [02:32] snapshot of the page which shows [02:33] documents, heading, paragraph, text, and [02:35] even images. It's then clicked on the [02:37] relevant element and taken a screenshot [02:39] to see if dark mode is working. And this [02:41] is the screenshot if you're curious. [02:43] From here, it's gone ahead and fixed the [02:44] issue before taking another screenshot [02:46] of the fixed dark mode. And it's finally [02:48] finished the task, which we can test by [02:50] clicking up here. And we have a page [02:52] with perfect dark mode. Let's try [02:54] another test. Actually, while this was [02:56] running, I had another agent in the [02:58] background fix another issue. You may [02:59] have noticed that if I press the login [03:01] button, it will take me straight here [03:03] without any validation, which of course [03:05] isn't good. So I went ahead and asked it [03:07] to fix the issue with the validation in [03:09] this project and it did something [03:11] actually really interesting. It first [03:13] checks the available commands from Asian [03:15] browser and then if we scroll down it [03:17] fixes the issue and even makes a bash [03:20] script. So it's over here to test that [03:23] it works. So it echoes the first test, [03:25] adds an empty input, clicks the login [03:27] button and then expects these errors. [03:30] It's made a few tests here, but it's [03:32] actually made an even better bash test [03:34] down below, which we can see over here [03:36] that makes use of agent browser eval to [03:38] run some JavaScript code. So now we can [03:40] see if I press the login button, we get [03:42] some validation. This looks like an [03:44] email, but it's actually a placeholder. [03:46] If I give it an email over here, just a [03:47] made up one, and hit login, it says [03:49] enter a valid email, which I can do like [03:51] this. And then I can enter a password [03:53] before it takes me to this dashboard. [03:55] So, basically, agents have addressed two [03:57] issues with this app and tested it [03:59] themselves to validate it works using [04:01] the agent browser plug-in. I would say [04:03] if I could do it again, I'd go with a [04:05] model that has multimodal support so you [04:07] can actually read the screenshots it [04:09] takes instead of using GLM 4.7. But now, [04:12] let's get into some architecture. How [04:14] does agent browser actually do all of [04:16] these things? Well, after an agent runs [04:17] a command like agent browser click at [04:20] E2, this gets sent to a Rust binary that [04:23] passes that command and converts it to [04:25] JSON. The benefit of it being Rust is [04:27] that it's fast, resource efficient, and [04:30] stays running after it's spawned. This [04:32] JSON then gets sent to a node demon [04:35] through a Unix socket, and this demon [04:37] manages the Chromium browser. What's [04:38] cool about this is that a demon is run [04:41] on each session, meaning multiple [04:43] browsers can be controlled. Once the [04:44] demon validates the output, it launches [04:47] the browser which is headless by default [04:49] and executes the action using playright. [04:52] And once it's completed, it sends the [04:54] output also in JSON back to the agent. [04:57] So the CLI and from here the agent can [04:59] do whatever it wants. So it can send [05:01] even more commands to the ROS passer or [05:04] it can end the whole loop. All of this [05:06] is super impressive for a weekend's [05:08] project. But how does it compare to [05:09] browser use or even using the Playright [05:12] MCP server? Well, browser use can be [05:14] used with or without an external agent [05:17] because it can run the full agent [05:19] reasoning loop. So, plan, action, [05:21] observe, and replan all on its own [05:24] without the use of anything else. It [05:26] also has a Python and TypeScript SDK for [05:29] fine grained control. It has a skills [05:31] marketplace, an MCP server, and uses [05:33] better stack to track the API status, [05:35] which is very cool. Agent browser on the [05:37] other hand is much simpler and can only [05:40] be used with an external agent. So you [05:43] the developer have to provide an agent [05:45] like cursor, claw code, open code and [05:48] the agent only interacts with agent [05:50] browser through CLI commands. When it [05:52] comes to comparing agent browser with [05:54] the Playright MCP server right now, [05:57] agent browser only supports Chromium [05:58] browsers. So no Firefox or Safari, which [06:01] I believe the Playright MCP server does [06:04] support. In fact, it supports everything [06:06] that Playright can do, but with an MCP [06:08] server that is perfect for agents. The [06:10] only kind of downside is if you already [06:12] have loads and loads of MCP tools, then [06:14] adding a bunch more could confuse the [06:16] agents since it has more tools available [06:19] to choose from autonomously, and it [06:21] might not choose the right one the first [06:22] time. Basically, choosing between [06:24] browser use, the Playright MCP server, [06:26] and agent browser really depends on your [06:28] use case and what you want from your [06:30] agent. For me personally, I like the [06:33] simplicity and ease of installation from [06:35] agent browser. And I primarily use [06:37] Chromium browsers, so not too fussed [06:39] about not having Firefox or Safari [06:41] access. So, I'll use it for now and see [06:43] how it goes. Also, while I've got your [06:45] attention, we are so close to hitting [06:47] that 100K subscriber mark. So, if you [06:49] haven't already, go ahead and hit the [06:51] subscribe button. ## Highlights - the agent browser plug-in. I would say if I could do it again, I'd go with a model that has multimodal support so you can actually read the screenshots it takes instead of using GLM 4.7. But now, let's get into some architecture. How does agent browser actually do all of these things? Well, after an agent runs a command like agent browser click at E2, this gets sent to a Rust binary that passes that command and converts it to JSON. The benefit of it being Rust is that it's fast, resource efficient, and ([View Highlight](https://read.readwise.io/read/01keyeg1jm13kmmx7zwe352rk1)) - Tags: 👻 ai highlighted