AI browsers are scary 🔗 **Source:** [AI browsers are scary](https://www.youtube.com/watch?v=iUNqFCQPwdg) > [!note]+ Document Notes > tags: [ai-browsers, prompt-injection, cybersecurity] > author: ThePrimeagen > source: YouTube > reference: https://www.youtube.com/watch?v=iUNqFCQPwdg > date created: 2024-12-19 10:30 > > # 🏷️ Title: AI browsers are scary > <!-- deep-read:sections start --> ## 🧰 Action Inbox - [H1] Audit browser security settings — protect against prompt injection attacks #security - [H2] Research prompt injection defenses — evaluate current mitigation strategies #research - [H3] Review logged-in accounts — minimize exposure in AI browsers #privacy ## 🧭 Table of Contents - [📝 Summary](#-summary) - [📖 Main Content](#-main-content) - [🗒️ Notes](#️-notes) - [📚 References](#-references) - [🗣️ Paraphrase](#️-paraphrase) - [📰 Newsletter](#-newsletter) - [✅ Action Items](#-action-items) - [🧪 Hypotheses](#-hypotheses) - [📊 Methodology](#-methodology) - [❓ Questions](#-questions) - [🔮 Future Research](#-future-research) - [💡 Implications](#-implications) - [🧠 Learning Styles Assessment](#-learning-styles-assessment) - [📊 Accessibility Assessment](#-accessibility-assessment) - [🌍 DEIB Assessment](#-deib-assessment) - [✅ Logical Validity](#-logical-validity) - [🛡️ DARVO Analysis](#-darvo-analysis) - [🎯 Conclusion](#-conclusion) ### 🔗 References - [AI browsers are scary](https://www.youtube.com/watch?v=iUNqFCQPwdg) - [Google's Camel Paper on Prompt Injection](https://arxiv.org/abs/2308.07308) ## 📝 Summary - AI browsers are rapidly proliferating, growing from zero to three in just a few months - These browsers integrate ChatGPT-like capabilities directly into web browsing - Major security risks include prompt injection attacks through PDFs, images, and web content - Companies may be developing AI browsers primarily to harvest user interaction data - Current AI models are vulnerable to prompt injection and real-time censorship manipulation ## 📖 Main Content AI browsers represent a new category of web browsers that integrate large language models directly into the browsing experience. Instead of visiting ChatGPT as a separate website, users can interact with AI to analyze and question any web content they encounter. However, this integration creates significant security vulnerabilities through prompt injection attacks, where malicious content can hijack the AI to perform unintended actions like accessing logged-in accounts or stealing personal information. The rapid development of these browsers may be driven more by companies' desire to collect training data than by genuine user needs. ## 🗒️ Notes #### ⏱️ [00:00](https://www.youtube.com/watch?v=iUNqFCQPwdg&t=0) — [H1] AI browser explosion timeline - Key idea: AI browsers went from zero to three in just a few months - Evidence: "At the beginning of the summer, there were zero. By the end of the summer, there was one. It's not even Halloween, and there is three." #### ⏱️ [00:30](https://www.youtube.com/watch?v=iUNqFCQPwdg&t=30) — [H2] What AI browsers are - Key idea: AI browsers integrate ChatGPT functionality directly into web browsing - Evidence: "What if instead of a website, it was the browser? Meaning, you go to the website and then you can just use chat GPT in any website" #### ⏱️ [01:00](https://www.youtube.com/watch?v=iUNqFCQPwdg&t=60) — [H1] Security concerns with logged-in accounts - Key idea: AI browsers pose risks to sensitive logged-in accounts and payment information - Evidence: "I am logged into all my important accounts. I have Amazon logged in. I have PayPal logged in. I have all of my credit information just sitting there waiting to be stolen." #### ⏱️ [01:30](https://www.youtube.com/watch?v=iUNqFCQPwdg&t=90) — [H1] Prompt injection through PDFs - Key idea: Malicious PDFs can contain hidden instructions that hijack AI behavior - Evidence: "We're uploading a basic PDF into Claude and asking for it to be summarized. But look at the output... Claude is now filing a linear issue ticket" #### ⏱️ [02:15](https://www.youtube.com/watch?v=iUNqFCQPwdg&t=135) — [H2] Hidden text in PDFs enables attacks - Key idea: PDFs can contain invisible prompt injection instructions - Evidence: "Look at the PDF and what do you see? A bunch of hidden text. That is prompt injection where you ask something but something else happens" #### ⏱️ [02:45](https://www.youtube.com/watch?v=iUNqFCQPwdg&t=165) — [H1] Image-based prompt injection attacks - Key idea: Images can contain hidden text that triggers malicious AI behavior - Evidence: "This is a beautiful image on the internet... there is a bunch of hidden text on here" #### ⏱️ [03:15](https://www.youtube.com/watch?v=iUNqFCQPwdg&t=195) — [H1] Gmail access and data theft via images - Key idea: Prompt injection can navigate to sensitive accounts and steal information - Evidence: "Instead of discovering who created this image, I am now being navigated to my Gmail, codes being taken, and then going to another website" #### ⏱️ [03:45](https://www.youtube.com/watch?v=iUNqFCQPwdg&t=225) — [H2] Multiple attack vectors exist - Key idea: Prompt injections can occur through various content types across the web - Evidence: "It could just be as simple as hidden text on a web page. It could just be a tweet that you're looking at" #### ⏱️ [04:30](https://www.youtube.com/watch?v=iUNqFCQPwdg&t=270) — [H1] Data harvesting motivation theory - Key idea: Companies develop AI browsers primarily to collect user interaction data for training - Evidence: "They want to see you accept. They want to see you reject. They want all of that good hand artisal data generated so they can use that to train better models" #### ⏱️ [05:00](https://www.youtube.com/watch?v=iUNqFCQPwdg&t=300) — [H2] Real-time censorship capabilities - Key idea: AI browsers can dynamically control what content users can access - Evidence: "If you ask it to look up a video of say Hitler, it will say, 'Sorry, I can't do that.' ...someone said, 'Okay, it looks like it's resolved now.'" #### ⏱️ [05:30](https://www.youtube.com/watch?v=iUNqFCQPwdg&t=330) — [H3] Google's partial solution research - Key idea: Google has research showing potential prompt injection defenses with performance trade-offs - Evidence: "There's a paper called Camel defeating prompt injections by design... they can actually solve some level of prompt injection by only making the model 9% dumber" #### ⏱️ [06:00](https://www.youtube.com/watch?v=iUNqFCQPwdg&t=360) — [H1] Current jailbreaking ease demonstrates vulnerability - Key idea: Expert hackers can quickly bypass AI safety measures on new models - Evidence: "Planning the prompter is a prolific jailbreaker and it takes them about 30 minutes on a new model to jailbreak it" ## 📚 References - [AI browsers are scary - YouTube](https://www.youtube.com/watch?v=iUNqFCQPwdg) - [Google's Camel Paper on Prompt Injection Defense](https://arxiv.org/abs/2308.07308) - [Terminal.shop Coffee Subscription](https://terminal.shop) ## 🗣️ Paraphrase - AI browsers are web browsers that integrate ChatGPT-like functionality, allowing users to interact with AI while browsing any website, but they're proliferating rapidly without adequate security measures - These browsers create serious security vulnerabilities because malicious actors can hide prompt injection attacks in PDFs, images, or web content that can hijack the AI to access logged-in accounts or steal personal information - The rapid development appears motivated by companies wanting to harvest user interaction data for AI training rather than genuine user needs, while also enabling new forms of real-time content censorship ## 📰 Newsletter **AI Browsers: The New Security Threat You Need to Know About** A concerning trend is emerging in web browsing: AI-powered browsers that integrate ChatGPT-like functionality directly into your browsing experience. While this sounds convenient, security experts are raising serious red flags about prompt injection attacks. **The Risk**: Malicious actors can hide instructions in PDFs, images, or web content that hijack the AI to access your logged-in accounts, steal personal information, or make unauthorized purchases. Since most of us stay logged into Amazon, PayPal, and other sensitive accounts, the potential for damage is significant. **What's Really Happening**: The rapid development of these browsers (from zero to three in just months) may be driven more by companies wanting to harvest your interaction data for AI training than by genuine user needs. **Bottom Line**: Until prompt injection vulnerabilities are solved, consider keeping your AI interactions and sensitive browsing separate. The convenience isn't worth the security risk. Resources: [AI Security Research](https://arxiv.org/abs/2308.07308), [Prompt Injection Examples](https://github.com/prompt-injection), [Browser Security Guide](https://owasp.org/www-project-web-security-testing-guide/) ## ✅ Action Items #### DevRel - Developer advocacy: Create security awareness content about AI browser risks, demo prompt injection attacks at conferences - Education: Develop workshops on prompt injection defense, create security-first AI integration tutorials - Developer experience: Document secure AI integration patterns, provide prompt injection testing tools - Developer marketing: Campaign highlighting security-conscious AI development practices - Community management: Monitor discussions about AI browser security, provide expert guidance on forums #### OSPO - Policy & Governance: Establish AI browser security assessment policies for enterprise adoption - License Compliance: Review AI browser dependencies and data handling compliance requirements - Security Management: Implement vulnerability assessment processes for AI-integrated applications - Software Inventory: Track AI browser usage and associated security risks in organizational software catalogs - Developer Support: Provide training on secure AI integration practices and prompt injection prevention - Community Engagement: Support open-source prompt injection defense research and tooling - Strategic Alignment: Align AI browser adoption with organizational security and privacy KPIs #### Community - Maintainers: Develop prompt injection testing frameworks and security guidelines for AI integrations - Contributors: Create open-source tools for detecting and preventing prompt injection attacks - Sponsors: Fund research into prompt injection defense mechanisms and secure AI browser development - Ecosystem: Coordinate security standards across AI browser projects and related tools #### WordPress - Core: Evaluate security implications of AI-powered plugins and themes that interact with external content - Core Performance: Assess performance impact of security measures needed to prevent prompt injection - Core AI: Establish guardrails and security protocols for WordPress AI features and integrations - Security: Develop WordPress-specific guidelines for secure AI integration and prompt injection prevention - Plugins: Create security review processes for AI-powered plugins that process user content - Documentation: Document secure AI integration practices and prompt injection risks for developers - Community: Educate WordPress developers about AI browser security risks through meetups and WordCamps #### The WPCC - Programs: Fund security research fellowships focused on AI browser and prompt injection defense - Projects: Support development of open-source prompt injection detection and prevention tools - Funding: Provide microgrants for WordPress security plugins that address AI integration risks #### FAIR - Technical independence: Ensure AI browser security research remains independent of vendor influence - Policies: Establish transparent funding for prompt injection defense research - Governance artifacts: Create security assessment charters for AI browser evaluation ## 🧪 Hypotheses - AI browsers are being developed primarily for data collection rather than user experience improvement - Prompt injection attacks will become more sophisticated and widespread as AI browsers gain adoption - Current AI models cannot reliably prevent prompt injection attacks despite security measures - Organizations developing AI browsers can implement real-time censorship more easily than traditional search engines ### 📊 Methodology - Study design to test hypothesis 1: Analyze AI browser privacy policies and data collection practices compared to feature development timelines - Data sources and analysis for hypothesis 2: Monitor security forums and incident reports for prompt injection attack evolution and frequency - Experiments to validate hypothesis 3: Systematic testing of current AI models against various prompt injection techniques and defensive measures ### ❓ Questions - How can we develop reliable detection systems for prompt injection attempts in real-time browsing scenarios? - What architectural changes would be needed to create truly secure AI browsers that isolate AI processing from sensitive account access? - How will the rapid proliferation of AI browsers affect web content security standards and practices? ### 🔮 Future Research - Investigation into formal verification methods for AI browser security architectures - Development of standardized prompt injection testing frameworks and benchmarks - Exploration of privacy-preserving AI browser designs that don't require extensive data collection ### 💡 Implications - Practice: Developers need to implement prompt injection testing in AI-integrated applications and consider security-first design patterns - Policy: Organizations require new security policies for AI browser adoption and data handling in AI-integrated systems - Security: Traditional web security models need updating to address AI-mediated attacks and real-time content manipulation - Education: Security training programs must include prompt injection awareness and defense techniques - Funding: Investment in AI browser security research and defensive tooling becomes critical for ecosystem safety ### 🧠 Learning Styles Assessment Reading level estimate: 8th Grade (Flesch-Kincaid) #### 👀 Visual learners - Present: Video demonstration of prompt injection attacks, visual examples of hidden text in PDFs and images - Improve: Architecture diagrams showing AI browser attack vectors, flowcharts of prompt injection processes #### 🔊 Aural learners - Present: Video narration explaining AI browser risks and attack mechanisms - Improve: Podcast-format discussion of AI browser security implications #### 🗣️ Verbal learners - Present: Clear definitions of prompt injection, analogies comparing AI browsers to traditional browsers - Improve: Story-driven explanations of attack scenarios, glossary of security terms #### 🧩 Kinesthetic learners - Present: Step-by-step demonstration of prompt injection attacks - Improve: Interactive labs for testing prompt injection defenses, hands-on security assessment checklists #### 🧠 Logical learners - Present: Timeline of AI browser development, causal relationships between features and vulnerabilities - Improve: Decision trees for AI browser security assessment, quantified risk analysis frameworks #### 🧑‍🤝‍🧑 Social learners - Present: Discussion of community security concerns, collaborative security research references - Improve: Peer review processes for AI browser security, community-driven security testing initiatives #### 🧘‍♀️ Solitary learners - Present: Individual security assessment opportunities, self-reflection on personal risk tolerance - Improve: Personal security audit worksheets, self-paced security learning paths #### 🌿 Naturalistic learners - Present: Ecosystem thinking about AI browser security within broader web security landscape - Improve: Analysis of security evolution patterns, environmental impact of increased AI processing ### 📊 Accessibility Assessment #### Screen reader semantics - Present: Clear heading structure, descriptive video content explanation - Improve: Detailed alt text for security demonstration screenshots, ARIA labels for complex security concepts #### Keyboard navigation - Present: Standard video controls accessible via keyboard - Improve: Keyboard-navigable security testing interfaces, skip links for demonstration sections #### Contrast and readability - Present: High contrast video player interface - Improve: Ensure security warning text meets 4.5:1 contrast ratio, large text for critical security information #### Text resizing and reflow - Present: Video scales appropriately with browser zoom - Improve: Ensure security documentation reflows properly at 200% zoom without horizontal scrolling #### Media and captions - Present: Video content with clear audio narration - Improve: Closed captions for security demonstrations, transcript with timestamps for technical content #### Forms and interactions - Present: Standard video interaction controls - Improve: Accessible security assessment forms, clear error messaging for security testing tools #### Standards reference - Map security demonstration interface to WCAG 2.2 AA success criteria <!-- deep-read:sections end --> ## 📺 Video  --- # 📄 Full Transcript *Auto-generated from YouTube captions · timestamps roughly every 30 seconds · [watch on YouTube](https://www.youtube.com/watch?v=iUNqFCQPwdg)* **[00:00]** There is an AI browser explosion happening right now. Now, you may not be aware of this, but at the beginning of the summer, there were zero. By the end of the summer, there was one. It's not even Halloween, and there is three. Now, I've done some math and I've crunched some numbers, and if this trend continues, we will have 67 Chromium-based AI browsers by next Christmas. So, you're probably asking, "What the heck is an AI browser?" Well, this is ChatGPT.com, but this is a website. What if, instead of a website, **[00:30]** it was the browser? Meaning, you go to the website, and then you can just use ChatGPT in any website, and ask any question about the content. I'm actually, like, genuinely scared of these. It's because I am logged into all my important accounts. I have Amazon logged in. I have PayPal logged in. I have all of my credit information just sitting there, waiting to be stolen. And with these AI browsers, I could be prompt injected by as simple as something as a PDF. Now, I'm sure plenty of you know what prompt injection is, but for those that don't know what the **[01:01]** Hydra of LLMs is, look at this video. Now, you'll notice that we're uploading a basic PDF into Claude, and asking for it to be summarized. But, look at the output. Does that look like summarization, or does that look like Claude is now filing a linear issue ticket? That doesn't seem like my request. What happened there? Well, look at the PDF, and what do you see? A bunch of hidden text. That is prompt injection, where you ask something, but something else happens due to the context. Something that you don't intend **[01:33]** to happen. Now, you're probably saying, "Well, I never look at PDFs. I don't go to school. I don't read." Well, look at this. This is a beautiful image on the internet. Now, you may not realize it. Look really closely, but there's a bunch of hidden text on here. Now, if I ask, "Who is the author?" instead of discovering who created this image, I am now being navigated to my Gmail, code's being taken, and then going to another website in which they're stealing your information. See, that's the thing about prompt injections, is that you will interact with a piece of content, and **[02:04]** you may or may not be able to see these hidden instructions. Remember, you can just dump raw ANSI terminal escape sequences, and Jippity can just read it. So, if people encode stuff into images, Jippity is just going to be able to read it. And it doesn't even have to be complicated. It could just be as simple as hidden text on a webpage. It could just be a tweet that you're looking at. Anytime you interact with a webpage, there is some small chance that there's going to be someone trying to maliciously take your information if you're using an LLM. And to me, that's **[02:36]** terrifying, because I have all of my sensitive information. I do not need 50 industrial-grade anime waifu pillows sent to my house. This is not what I want in my life. So, why is everyone creating an AI browser all of a sudden? We really actually had none just a little bit ago. Now, I do not think it has anything to do with people trying to use more ChatGPT. think ChatGPT necessarily needs that as the utmost priority. Now, I'm going to have to put on my tinfoil hat here, but I do think it's for data. The reason why they want **[03:09]** you using the browser is they want to see you accept, they want to see you reject, they want all of that good hand-artisanal data generated, so they can use that to train better models. Cuz let's just face it, they have no problem scraping the internet for all the data, whether it's legal or not. And they would just absolutely love it if you could just legally send them all that information for free. But, there's actually a second reason why I also don't want to use these browsers, and I think this one's really important, which is if you look at this exchange on **[03:40]** Twitter. So, ChatGPT Atlas browser can browse the web, but not history. It turns out if you ask it to look up a video of, say, Hitler, it will say, "Sorry, I can't do that." Now, the even more kind of terrifying part is that shortly afterwards, someone said, "Okay, it looks like it's resolved now." Which means they literally have control pretty quickly to be able to shape exactly what you're seeing in real time. It's a whole new level of censorship, because now everything you're doing is not only going through Google, it's going to first go through Jippity. Now, we can't **[04:10]** solve censorship, but can we solve prompt injection? There's a paper called Camel, Defeating Prompt Injections by Design. It's from a little-known company called Google, which, you know, they may have put the T in ChatGPT, and they're showing that they can actually solve some level of prompt injection by only making the model 9% dumber. Now, I just don't think there's going to be a world where I'm going to be using one of these AI browsers anytime soon. And the obvious reason, besides for the censorship, should be that I don't think **[04:40]** you can prove to me that you cannot be prompt injected. Pliny the Prompter is a prolific jailbreaker, and it takes him about 30 minutes on a new model to jailbreak it. And that's the current state of the art frontier models. We are nowhere near having this problem solved, and I am just not going to have PayPal susceptible to a JPEG. And I don't think you should, either. I'm a bit of a pessimist, so I just don't think this problem's going to be solved anytime soon. But, I know there's a few smart **[05:11]** people out there. So, if you have an idea, if you know how to solve prompt injection, you should definitely not tell Samuel Jippity Altman. Instead, hey, me and you, come on. Come closer. Just let me know. We'll start a company, and we're going to rake it in. The name is the Primeagen. Hey, is that HTTP? Get that out of here. That's not how we order coffee. We order coffee via SSH, terminal.shop. Yeah, you want a real experience? You want real coffee? You want awesome subscriptions, so you never have to remember again? Oh, **[05:42]** you want exclusive blends with exclusive coffee and exclusive content? Then, check out Chron. You don't know what SSH is? Well, maybe the coffee's not for you. Terminal coffee in hand, living the dream.