Drupal AI Learners Club From Autocomplete to Autopilot

Drupal AI Learners Club: From Autocomplete to Autopilot ![rw-book-cover](https://img.youtube.com/vi/lNnbJQ6l2B4/maxresdefault.jpg) 🔗 **Source:** [Drupal AI Learners Club: From Autocomplete to Autopilot](https://www.youtube.com/watch?v=lNnbJQ6l2B4) > [!note]+ Document Notes > tags: #drupal #ai #coding #agents #development-workflow #automation > author: Drupal AI Learners Club > source: YouTube > reference: https://www.youtube.com/watch?v=lNnbJQ6l2B4 > date created: 2024-12-19 15:33 > > # 🏷️ Title: Drupal AI Learners Club: From Autocomplete to Autopilot >  ## 🧰 Action Inbox - [H1] Try one annoying local task with agents — Scott's advice for beginners #ai-agents #development - [H2] Set up DDEV/Lando sandbox for agent testing — Required for Drupal development #drupal #development-environment - [H3] Create agents.md file with project conventions — Context for agent behavior #documentation #ai-agents ## 🧭 Table of Contents - [[#📝 Summary]] - [[#📖 Main Content]] - [[#🗒️ Notes]] - [[#📚 References]] - [[#🗣️ Paraphrase]] - [[#📰 Newsletter]] - [[#✅ Action Items]] - [[#🧪 Hypotheses]] - [[#📊 Methodology]] - [[#❓ Questions]] - [[#🔮 Future Research]] - [[#💡 Implications]] - [[#🧠 Learning Styles Assessment]] - [[#📊 Accessibility Assessment]] - [[#🌍 DEIB Assessment]] - [[#✅ Logical Validity]] - [[#🛡️ DARVO Analysis]] - [[#🎯 Conclusion]] ### 🔗 References - [Drupal AI Best Practices Project](https://github.com/drupal-ai/drupal-ai-best-practices) - [Cursor IDE](https://cursor.com/) - [Claude Code](https://github.com/anthropics/claude-code) - [Codex CLI](https://codexcli.com/) ## 📝 Summary - Scott Falconer demonstrates progression from simple AI autocomplete to autonomous agent loops - Shows real example of agent debugging Drupal issue, creating 28 Jira tickets and 25 pull requests - Covers practical setup requirements, safety habits, and tool recommendations - Key shift: moving from typing assistance to task management and architectural guidance - Bottom line: AI agents change the unit of work from functions to stories/epics ## 📖 Main Content This session bridges the gap between basic AI usage (ChatGPT conversations, IDE autocomplete) and advanced agent workflows where AI can autonomously execute code, run tests, and create pull requests. Scott demonstrates how agents work in loops - inspecting context, planning, editing code, executing, and revising based on results without human intervention at each step. The core difference is that agents have access to your local machine, can use tools, maintain memory, and follow multiple steps toward an objective. This shifts the bottleneck from typing/writing code to providing guidance, reviewing output, and managing ambiguity. The unit of work changes from individual functions to complete user stories. Key requirements include API access to models (Claude, OpenAI), local development environments (DDEV/Lando), and safety practices like working in branches. Popular tools include Claude Code, Codex CLI, and Open Code for terminal-based agent management. ## 🗒️ Notes #### ⏱️ [00:00](https://youtu.be/lNnbJQ6l2B4?t=0) — [H1] Session Introduction and Ground Rules - Key idea: Bridging gap between ChatGPT usage and agent orchestration - Evidence: "Scott's over here orchestrating agents to write code written by other agents" #### ⏱️ [05:23](https://youtu.be/lNnbJQ6l2B4?t=323) — [H2] This Week in AI Updates - Key idea: AI security tools finding 17-year-old vulnerabilities, 84% developers use AI but only 29% trust output - Evidence: "Claude Mythos found OpenBSD vulnerability that was 17 years undetected" #### ⏱️ [12:45](https://youtu.be/lNnbJQ6l2B4?t=765) — [H3] Scott's Weekend Agent Example - Key idea: Agent autonomously filed 28 Jira tickets and 25 pull requests while managing three children - Evidence: "It was like having four kids instead of three - you give them guidance, they do ridiculous things" #### ⏱️ [18:30](https://youtu.be/lNnbJQ6l2B4?t=1110) — [H1] Agent vs Model Definitions - Key idea: Models predict next words, agents work in loops with memory and tools - Evidence: "Think of agent as another user with permissions, roles, goals and implications" #### ⏱️ [22:15](https://youtu.be/lNnbJQ6l2B4?t=1335) — [H2] Risk Assessment and Safety - Key idea: Handing laptop to agent is like handing it to another person - Evidence: "Same risk as handing your laptop to somebody else - they won't pour orange juice but will cause chaos" #### ⏱️ [26:40](https://youtu.be/lNnbJQ6l2B4?t=1600) — [H3] Evolution Ladder: Browser to Agent Loops - Key idea: Progression from browser chat → autocomplete → repo-aware chat → agent loops - Evidence: "Unit of work changes from functions/snippets to stories/epics" #### ⏱️ [32:20](https://youtu.be/lNnbJQ6l2B4?t=1940) — [H1] Bottleneck Shift - Key idea: Bottleneck moves from writing/typing to guidance and review - Evidence: "Agents make advanced mistakes now - differences of opinion rather than silly errors" #### ⏱️ [38:45](https://youtu.be/lNnbJQ6l2B4?t=2325) — [H2] Agent Loop Process - Key idea: Give objective → inspect context → edit/plan → run/revise cycle - Evidence: "Agent will inspect environment, see DDEV, spin up environment autonomously" #### ⏱️ [44:10](https://youtu.be/lNnbJQ6l2B4?t=2650) — [H3] Practical Requirements Setup - Key idea: Need API keys, local dev environment, safety habits with Git - Evidence: "Always work in branches - Git is your undo button for agent changes" #### ⏱️ [48:55](https://youtu.be/lNnbJQ6l2B4?t=2935) — [H1] Tool Recommendations - Key idea: Claude Code, Codex CLI, Open Code as terminal-based agent harnesses - Evidence: "Each works best with their own models but can be configured for others" #### ⏱️ [52:30](https://youtu.be/lNnbJQ6l2B4?t=3150) — [H2] Live Demo: Drupal Issue Debug - Key idea: Agent pulls issue from drupal.org, reproduces bug, creates targeted patch - Evidence: "Found issue reproduces with HTTP submission but not drush entity save" #### ⏱️ [68:15](https://youtu.be/lNnbJQ6l2B4?t=4095) — [H3] Context Management with agents.md - Key idea: Skills are procedural knowledge, agents.md provides consistent context - Evidence: "Your job is giving agent right context - like onboarding developer README" ## 📚 References - [Drupal AI Best Practices](https://github.com/drupal-ai/drupal-ai-best-practices) - [Claude Code on GitHub](https://github.com/anthropics/claude-code) - [Kubernetes AI Guidelines](https://kubernetes.io/docs/) - [Linux Foundation AI Guidelines](https://www.linuxfoundation.org/) - [Joomla AI Guidelines](https://docs.joomla.org/) ## 🗣️ Paraphrase - AI agents represent evolution from typing assistance to autonomous task execution that can debug, test, and patch code - The fundamental shift is from human-driven autocomplete to AI-driven loops that inspect, plan, execute and revise - Safety requires treating agents like junior developers with proper sandboxing, version control, and human review ## 📰 Newsletter 🤖 **From Autocomplete to Autopilot: Drupal Developers Embrace AI Agents** Scott Falconer from Acquia demonstrated how AI agents can autonomously debug Drupal issues, moving beyond simple code completion to full development workflows. In a live demo, an agent successfully reproduced a node-saving bug from the Drupal issue queue and generated a targeted patch - all while Scott managed his three children at home. **Key Takeaway**: The bottleneck has shifted from writing code to providing architectural guidance and reviewing AI output. Agents can now handle the mechanical aspects of development while humans focus on strategy and quality assurance. **Getting Started**: Begin with one annoying local task using tools like Claude Code or Codex CLI. Set up DDEV for Drupal development and work within Git branches for safety. **Resources**: [Drupal AI Best Practices](https://github.com/drupal-ai/drupal-ai-best-practices), [Learn WordPress](https://learn.wordpress.org/), [Developer Docs](https://developer.wordpress.org/), [WP-CLI](https://wp-cli.org/) ## ✅ Action Items #### DevRel - Developer advocacy: Create "AI Agents for Drupal" workshop series with hands-on DDEV setup - Education: Develop curriculum bridging autocomplete to agent workflows for different skill levels - Developer experience: Document agent best practices for Drupal contribution guidelines - Developer marketing: Share success stories of agent-assisted contributions to core and contrib - Community management: Facilitate discussions on AI contribution ethics and review standards #### OSPO - Policy & Governance: Review and adapt AI contribution guidelines from Kubernetes, Linux, Joomla - License Compliance: Establish protocols for AI-generated code review and attribution - Security Management: Assess risks of agents accessing local development environments - Software Inventory: Track AI tools and models used in development workflows - Developer Support: Provide training on safe agent usage and proper human oversight - Community Engagement: Fund development of Drupal-specific agent skills and context files - Strategic Alignment: Measure productivity gains and quality impacts from agent adoption #### Community - Maintainers: Develop review checklists for AI-assisted contributions and patches - Contributors: Create mentorship programs pairing experienced developers with AI-curious newcomers - Sponsors: Support development of open-source agent skills for Drupal ecosystem - Ecosystem: Coordinate cross-project sharing of AI best practices and safety protocols #### Drupal - Core: Establish AI contribution disclosure requirements and review processes - Core Performance: Use agents for systematic performance testing and optimization - Core AI: Develop Drupal-specific agent skills for core development workflows - Core Program Team: Create guidelines for AI-assisted issue triage and patch creation - Community: Integrate AI agent training into DrupalCon and camp programming - Training: Develop certification paths for AI-assisted Drupal development - Documentation: Create comprehensive guides for agent setup and Drupal-specific contexts - Meta: Build agent skills for drupal.org interaction and automated issue management ## 🧪 Hypotheses - AI agents will shift Drupal development from code writing to architectural guidance and quality review - Developers using agent loops will contribute more patches but require enhanced review processes - Agent-assisted development will democratize complex Drupal contributions for less experienced developers - Local agent environments will become standard development tooling within 18 months ### 📊 Methodology - Study design: Track contribution volume and quality before/after agent adoption across Drupal contributors - Data sources: Drupal.org issue queues, commit histories, patch review cycles, developer surveys - Experiments: A/B testing of agent-assisted vs traditional development workflows on controlled tasks ### ❓ Questions - How will AI agents change the skills required for effective Drupal development and contribution? - What review processes can adequately assess AI-generated code while maintaining development velocity? - How can the community ensure equitable access to AI tools that enhance developer productivity? ### 🔮 Future Research - Impact assessment of agent adoption on Drupal contribution patterns and community dynamics - Development of standardized agent skills library for common Drupal development tasks - Investigation of collaborative human-AI workflows for complex architectural decisions ### 💡 Implications - Practice: Development workflows shift toward higher-level task planning and code review - Policy: Need for clear AI disclosure requirements and contribution guidelines - Security: Enhanced focus on sandbox environments and code validation processes - Education: Training programs must evolve to include agent management and oversight skills - Funding: Investment needed in AI-specific tooling and infrastructure for open source development ### 🧠 Learning Styles Assessment Reading level estimate: 12th grade #### 👀 Visual learners - Present: Live terminal demonstrations, agent workflow diagrams - Improve: Architecture diagrams showing agent decision trees, visual process flows #### 🔊 Aural learners - Present: Live narrated demonstration, conversational explanations - Improve: Audio-only summary of key concepts, podcast format follow-up #### 🗣️ Verbal learners - Present: Clear definitions (agent vs model), analogies (agent as junior developer) - Improve: More concrete examples, step-by-step verbal walkthroughs #### 🧩 Kinesthetic learners - Present: Live terminal interaction, hands-on debugging demonstration - Improve: Guided practice exercises, downloadable sandbox configurations #### 🧠 Logical learners - Present: Clear progression ladder, systematic tool comparisons - Improve: Decision trees for tool selection, ROI calculations for agent adoption #### 🧑‍🤝‍🧑 Social learners - Present: Community discussion format, Q&A interaction - Improve: Pair programming exercises, collaborative agent training sessions #### 🧘‍♀️ Solitary learners - Present: Individual practice recommendations, self-guided exploration - Improve: Personal reflection prompts, solo project ideas #### 🌿 Naturalistic learners - Present: Evolution metaphor for AI development progression - Improve: Ecosystem thinking about AI tools, sustainability considerations ### 📊 Accessibility Assessment #### Screen reader semantics - Present: Clear heading structure, descriptive link text - Improve: Alt text for terminal screenshots, ARIA labels for code examples #### Keyboard navigation - Present: Standard video controls accessible via keyboard - Improve: Skip links for long code demonstrations, keyboard shortcuts documentation #### Contrast and readability - Present: High contrast terminal display - Improve: Verify color choices meet WCAG standards, provide high contrast alternatives #### Text resizing and reflow - Present: Responsive video player - Improve: Scalable code examples, flexible layout for various zoom levels #### Media and captions - Present: Video demonstration with audio narration - Improve: Complete captions, downloadable transcript, audio descriptions #### Forms and interactions - Present: Simple chat interface for questions - Improve: Error handling for message submission, clear input labels ### 🌍 DEIB Assessment #### Cultural breadth - Present: English-language presentation with global Drupal community context - Improve: Multi-language resources, consideration of global AI access disparities #### Inclusive language - Present: Gender-neutral technical terms, avoids exclusionary jargon - Improve: Plain language summaries, cultural context for non-Western developers #### Neurodiversity support - Present: Structured presentation, clear topic transitions - Improve: Content warnings for rapid demonstrations, alternative pacing options #### Representation - Present: Technical focus with community-oriented approach - Improve: Diverse contributor examples, acknowledgment of varying technical backgrounds #### Psychological safety - Present: "No stupid questions" policy, beginner-friendly framing - Improve: Explicit discussion of economic barriers to AI tools, support resources ### ✅ Logical Validity #### Strong reasoning - Deductive strength: Clear progression from simple to complex AI tools with logical steps - Inductive strength: Personal anecdotes support broader claims about agent capabilities - Causal reasoning: Well-established link between agent loops and productivity improvements - Analogical strength: Effective comparison of agents to junior developers for risk management #### Fallacies - Appeal to novelty: Assumes newer AI tools are inherently better without sufficient comparison - Hasty generalization: Scott's weekend success extrapolated broadly without addressing failure cases - False dilemma: Presents agent adoption as inevitable without discussing legitimate alternatives #### Tone of voice - Overall tone: Enthusiastic but practical, balances excitement with realistic warnings about complexity and risks ### 🛡️ DARVO Analysis - Deny: No denial patterns observed - presenter acknowledges risks and limitations - Attack: No attacks on critics - respectfully addresses skepticism --- ## Full Transcript [00:00] Hello everybody and welcome to the uh uh [00:04] um April 14th edition of our Drupal AI [00:06] learners club. Today we have the topic [00:08] um from autocomplete to autopilot uh [00:11] where Scott Falconer has kind of prepped [00:13] some materials at our last uh meetup [00:15] which was just last week. So thank you [00:17] Scott for for offering to help so [00:18] quickly. It's nice to keep the momentum [00:20] up on these. Um what it was kind of [00:22] brought up is like that was great but [00:24] that was a lot of like very very senior [00:26] like core developery kind of people [00:28] talking about their workflows and uh [00:31] there's a bunch of folks in our [00:32] community who are like I would call them [00:34] like AI curious and they sort of were [00:36] like what about a session more like [00:38] beginner level that you know if I've [00:39] used chat GBT as like a a thought [00:42] partner or like to help me draft [00:44] something or whatever and then Scott's [00:46] over here like you know orchestrating [00:48] agents to write code for him that's [00:51] written by other agent like that kind of [00:52] like what how do you get from here to [00:54] there and so that's sort of what we're [00:56] uh talking about today or that's what I [00:57] think we're talking about Scott would [00:59] you like to div any more details about [01:00] what you have planned for today [01:02] >> I I think that's what we're going to [01:04] talk about today yeah so it what it will [01:06] kind of focus on is the assumption that [01:09] you've all used something like chat GPT [01:11] and the assumption that you've [01:12] potentially done some sort of [01:15] autocomplete style stuff and we're going [01:17] to kind of bridge that gap between that [01:20] and the using agent. So that's the kind [01:24] of framing we'll have. [01:26] >> Well, let's jump in. Um, but before we [01:28] jump in, we do this thing where we do [01:29] the ground rules. Ground rules are we [01:31] follow the Drupal code of conduct. AI is [01:33] a spicy topic and we can talk about the [01:36] pros and cons uh without getting nasty [01:38] with each other. So, let's do that. Uh, [01:40] there are no stupid questions. We're all [01:42] here to learn. Uh, anyone who says [01:44] they're an AI expert is lying to you. um [01:46] because this stuff changes way too often [01:48] for anyone to be an expert in it all the [01:50] time. And if you haven't started AI yet, [01:52] it's no big deal. Best time to start is [01:54] now. And uh you know, we'll we'll all [01:56] learn together. Uh we want to make sure [01:58] that when we present on here, we're [02:00] doing pragmatic real world advice, warts [02:02] and all, not not sales pitches. I know [02:04] everyone's really excited about their AI [02:06] stuff they're working on and there is a [02:08] place for the hype machine, but this [02:09] isn't really it. These are engineers uh [02:11] for the most part who want to talk [02:13] engineery talk. And finally, your [02:15] participation is key. So, uh, ple feel [02:17] free to ask questions in the chat, raise [02:19] your hand if if you know, if you want to [02:21] talk out on on a mic and, uh, and we'll [02:23] get through it. Uh, this week in AI, so [02:26] Amber made me this awesome prompt, which [02:28] I haven't used yet, so I just did this [02:30] the oldfashioned way, but a few kind of [02:31] AI headlines. Uh, Claude Mythos, we [02:35] touched on that last week with the [02:36] project Glass Wing thing. So, it's a [02:38] their new frontier model. it's only [02:40] released to like 12 different companies [02:42] and basically it's very very good at [02:44] fighting very obscure security [02:45] vulnerabilities and then chaining them [02:47] together. Um, which is not great. And so [02:50] there's a bunch of frantic funding by [02:52] like the Linux Foundation and stuff like [02:54] that to try to like get a bunch of [02:56] projects perhaps including Drupal I [02:58] don't know like up to speed on uh [03:01] getting in front of this before every [03:02] script kitty can can do uh this kind of [03:04] thing at scale. And you know some people [03:07] are saying oh this is like a marketing [03:08] thing from from anthropic and maybe but [03:10] it also found like some open BSD [03:12] vulnerability that was like 17 years [03:14] undetected. So like you know it seems [03:16] like it might be both basically. Um AI [03:20] is now used to predict how not bowl [03:22] bowel cancer. It's a totally different [03:24] kind of cancer patients are going to [03:25] respond. So that's some good AI news for [03:27] a change. I know there's a bunch of [03:29] people in this community that want to [03:30] know about local uh models that they [03:33] don't have to send stuff off to [03:34] Winthropics. So, Min Miniax um is a new [03:37] one. Uh it's from a China based lab that [03:39] uh falls under that uh bandit. And then [03:42] uh this interesting uh article about 84% [03:45] of developers use AI tools, but only 29% [03:47] trust what ships. So, the review gap [03:50] that we're seeing in the Drupal [03:52] community and other open source [03:53] communities is real. Um I I found a [03:56] bunch of just like what? So, like I [03:58] guess somebody threw a Molotov cocktail [04:00] at Sam Alman's house. That's not really [04:02] cool. I guess Mark Zuckerberg to scale [04:05] himself and created an AI version of [04:07] himself so the staff could talk to the [04:09] boss without him being involved. That's [04:11] interesting. And then finally, uh, Trump [04:13] generated some AI image of him as Jesus [04:16] and it made some people angry and he [04:17] took that down. So that was kind of a [04:19] thing. And then a new thing, Drupal AI [04:21] news. Uh, so this is um, uh, stuff that [04:26] that uh, Amber's prompt surfaced for me. [04:28] So it's like Drupal AI module hit 1.3.2. [04:31] That's exciting. uh little like shout [04:33] outs for that team. AI agents uh has an [04:36] RC released. There's a new AI budget [04:38] control module that's been submitted to [04:40] the AI ecosystem. And then one thing I [04:42] wanted to call on that that got some [04:44] traction is there's a bunch of other [04:45] open source projects just in the past [04:48] couple of weeks that have announced some [04:49] some of their AI contribution guidelines [04:51] because I know that's that's an open [04:52] issue in the in the Drupal community [04:53] right now. Uh so you can read [04:55] Kubernetes, Linux, and Joomla there. Um [04:59] uh and all of them kind of have the same [05:01] general vibe which is you're allowed to [05:04] use AI um but you need to disclose it [05:07] and um you're expected to basically [05:10] exactly what Dre said on the keynote [05:11] stage you're expected to understand all [05:13] the code that you submit right don't [05:14] submit code you don't understand so uh [05:17] and if you're wondering how to do that [05:18] with aentic yada yada uh Mike Hershel's [05:21] session last week on how he uses AI for [05:23] core development was a really good uh he [05:26] gave a really good overview on how to [05:28] break up a a change set so that it can [05:30] actually be reviewed and it's not just [05:31] some like seven megabytes worth of code [05:33] to to hit at once. Uh, also in in [05:37] Amber's little thing is there's some [05:38] upcoming and notable Drupal and AI [05:40] events. So, Drupal Dev Days is not an AI [05:42] event, but for example, the AI people [05:44] will be there uh from the AI initiative [05:46] giving an update on on uh what's [05:48] happening there. I don't know anything [05:50] about Stanford webcam. Amber, I think [05:52] you threw that one in there. [05:54] >> Yeah, I did. And there's quite a few uh [05:57] sessions on AI. So, and one of the um [06:00] one of the dates, April 30th on Thursday [06:03] is all online. So, it's a free camp. [06:05] >> That's awesome. [06:06] >> Um so, yeah, you can join April 30th if [06:08] you can't make it to uh the West Coast [06:10] on May 1st. [06:12] >> Amazing. I love that. Um and then if you [06:14] if you like going to places that exist, [06:17] then Drupal 87 New York City is is [06:19] happening uh on midMay. So, all kinds of [06:22] events. I love that though. That's [06:23] that's great to have online sessions [06:24] that are available to everyone. So [06:26] fantastic. Um any additions from the [06:29] crowd? News, events, releases? Maybe [06:31] time box this couple minutes because I [06:34] want to make sure that um that Scott [06:37] gets his full time available. [06:42] Uh there we go. That's what Zoom is. I [06:45] see some people posting some fun stuff [06:47] in the chat. [06:50] Midamp. Tell me about Mid Camp, Bob. [06:54] Oh, yeah. Aqua sponsored med to work on [06:56] the MCP mall. I saw that just this [06:57] morning. Yeah. [06:59] >> Yes. 12th to 14th of May. Register now. [07:04] >> Is there AI stuff happening? [07:06] >> Of course. [07:08] >> Sweet. [07:12] I will clean this up when I publish the [07:14] deck at the end of the session because [07:15] it's pretty rough. Uh but uh cool. [07:18] That's awesome. Um, [07:21] rumors of Anthropic 4.7 being released [07:23] soon. That's cool. Didn't know that. Um, [07:27] excellent. Okay. [07:31] Codeex now supports plugins like Clawude [07:33] Code. [07:36] That's handy. [07:39] Are they interoperable? I imagine not, [07:41] but that would be cool. [07:45] It's interesting because Codeex is kind [07:47] of becoming this whole um this whole [07:49] like secondary ecosystem. So they they [07:52] are not correlating tools, but they're [07:54] they're very interesting that they're [07:55] both trying to connect to a bunch of [07:56] things. And uh um Oh, you you folks are [08:00] just full of uh you are full of wisdom [08:04] and knowledge. All right. Microsoft will [08:05] release VS Code agents. I don't know [08:08] anything about that. um and works [08:11] similar to anti-gravity in aentic mode. [08:14] Uh Dino, did you want to come on mic and [08:15] say more about that? I don't know what [08:17] some of those words mean. [laughter] [08:20] >> Uh sure. I I just saw it pass by today. [08:23] So for those that tried anti-gravity, [08:25] it's the development tool from Google. [08:27] Uh oh, camera was trying to enable it. [08:30] Um and basically it works in agentic [08:32] mode where you give it tasks and you [08:34] have an inbox anti-gravity that gives [08:35] you the results. So it's more like you [08:37] have agents that you give tasks on [08:38] several projects and apparently VS Code [08:40] is doing it too. So one of the biggest [08:42] open- source IDEs is now releasing an [08:46] agentic version of it that really [08:47] doesn't work in having um [08:51] interactive tool to work with AI but as [08:53] an agentic tool that you give it tasks [08:55] and it works in the background. [08:57] >> So very curious to try it out. I tried [08:59] anti-gravity and I'm not ready to do it [09:01] in agent agent more than I noticed. So [09:03] >> yeah I don't know. My cat is making [09:05] noise. [09:06] >> Oh, that's mine. Do not ever apologize [09:08] for a cat. Cats are great. Um, neat. All [09:12] right. Well, I just learned two cool [09:14] things in that one sentence. Thank you. [09:16] That's That's great. Um, awesome. Uh, [09:21] cool. All right. I'm going to check the [09:23] chat one more time. I'm sorry. Like, [09:25] Amber, you're totally on the chat. Like, [09:27] I'm I'm used to running these solo. So, [09:30] um, [09:32] great. All right. Um, let's go ahead and [09:35] jump back to it then. Thank you so much [09:37] for your extra additions and and uh [09:39] things of that nature. So, uh, showand [09:41] tell time and I'm going to kick it off [09:43] to Scott to talk about from autocomplete [09:45] to autopilot, a practical guide to AI [09:48] assisted coding or at least that's what [09:49] Claude made the session called when he [09:51] described it to me as a primer of styles [09:54] uh on styles of AI assisted coding and [09:56] moving from IDE copilot style to agent [09:58] libs. So, [09:59] >> Claude also wrote the session. So, like [10:01] if it's wrong, it's Claude's fault. [10:02] >> Yeah. So if it's wrong, you can blame [10:04] AI. We always blame AI for everything. [10:05] It's fine. So yeah. Um, so I got to [10:08] figure out I'm so sorry. Zoom. Zoom is [10:10] acting really bizarrely to me. There we [10:12] go. All right. I have stopped the screen [10:15] share, which I think means you can now [10:18] screen share. [10:19] >> Yes. [10:25] >> Okay. Are you seeing my screen? [10:29] >> Yes. [10:29] >> Perfect. So uh yeah. So I am Scott [10:32] Thoffener. I am an AI expert and a liar. [10:34] So it's a perfect combination. No, I [10:37] lead um AI applied AI at Aqua. So [10:40] applied AI if you're not familiar is [10:42] just using AI, not developing AI, not [10:45] making the tools. So [clears throat] as [10:47] we go through this today, this can be an [10:49] open discussion. You don't have to wait [10:51] to the end. I prefer with some back and [10:53] forth. Um really what I wanted to kind [10:55] of focus on was you know many of you [10:59] have heard like there are people out [11:00] here doing stuff with AI related to [11:02] coding but that can mean a lot of things [11:05] right and so I'm assuming that most of [11:07] you have gone into chat GPT Gemini [11:10] claude and asked it to help debug some [11:12] stuff potentially spun up something like [11:15] co-pilot or cursor but what we're going [11:18] to start from there is we're going to [11:19] take the next step to [11:22] what it means to developing with an [11:24] agent and specifically what are called [11:26] agent loops. And so, you know, it's kind [11:28] of a balance here. So, jump in there. It [11:30] may be easy for me to miss things uh [11:33] that [11:34] are seem clear. So, feel free to jump [11:38] in, but I want to start with a uh story [11:41] of uh what I did this weekend. So, um [11:44] we were working on some functionality at [11:47] Aqua that I kind of jumped in and got [11:49] wind of on Friday. Um, and this is, you [11:53] know, it's a product thing that I was [11:55] familiar with, but hadn't really kind of [11:57] touched the code before. And it's all [12:00] written in Python, more or less, some [12:01] systems I didn't know. Um, [12:04] in the context of this weekend, I was [12:07] home alone with my three children. Um, [12:09] so it was chaotic and still managed to [12:12] guide an agent into doing some debugging [12:15] of the system. It filed 28 Jurro tickets [12:19] and did 25 pull requests. Um, since [12:21] then, many of them are already in review [12:23] and there's been a couple small bugs [12:25] found, but nothing that you would expect [12:26] to be that big. And why I want to bring [12:28] this up is because it is working with an [12:32] agent opens some fundamental changes. [12:34] Like it it's relatively conceptually [12:36] easy once you see it, but the [12:38] implications of it are huge. And it [12:40] really was like instead of having three [12:42] kids, I had four kids. That's what [12:44] agents are like. You kind of give them [12:45] some guidance. They're going to do [12:47] ridiculous things. You get them to where [12:50] you want to go and they learn and you [12:51] move forward. So that's the end state [12:53] that you want to get to. This idea that [12:55] like even right now while we're on this [12:57] call, I have multiple agents doing work [12:59] for me. So I will go back and look at [13:02] this at the end and and see what they've [13:04] done during this time. So it completely [13:07] changes the amount of things that you're [13:09] able to get done. So I want to start [13:12] with a couple of these ground rules. [13:13] Don't overthink this, right? It's very [13:15] easy to overthink it, but at the end of [13:17] the day, it's relatively basic. Uh [13:19] you're not behind. At some point, you [13:22] may fall behind, but dabbling counts. [13:24] You know, as Angie said earlier that, [13:26] you know, nobody's really an expert in [13:27] this yet. We're all kind of faking it [13:29] and figuring it out. So, just start [13:31] doing things. Don't focus on the perfect [13:33] setup. What I'd like you to do is start [13:35] with one annoying task. So when we're [13:38] talking about using these agents for [13:40] development, the ones I'm going to show [13:42] today do have access to your local [13:43] machine. So some fun tasks are even [13:46] things like I need to organize these [13:48] files. I have something I need to put [13:50] together. I have these CSVs that I need [13:52] to merge something local and simple that [13:54] you need to get done. I have found that [13:56] through the course of using these for [13:57] development, I use these local agents [13:59] for all sorts of just random tasks that [14:02] I wouldn't have expected to do before. [14:05] And then let me with my experience point [14:08] say this let the loop teach you right so [14:11] leave things on the table and one rule [14:13] or one really interesting thing about AI [14:15] it's one of the first tools we've ever [14:17] have access to that you can ask it how [14:19] to use it so if you're ever confused [14:22] just ask the AI how to use it so once [14:24] you have these systems set up it can [14:27] teach you how to best use it now you [14:29] still have to give it some guidance but [14:31] that's how most of us learn by just [14:33] pushing these things now I also add to [14:36] Don't rabbit hole on tools. You can very [14:39] very easily do this. As we started [14:40] talking about codecs and cloud code and [14:43] all of these, those are some of the base [14:45] fundamental tools. You can add skills, [14:47] you can add plugins, you can add all [14:48] this structure around it. Don't until [14:52] you realize that you need to um because [14:54] you can spend all of your time going [14:57] down building the tools because they are [14:59] super fun. There's some really cool [15:01] stuff. I mean, it gets wild as soon as [15:02] you start running agents. People [15:04] eventually be like, "Oh, we need an [15:06] interface for this agent that looks like [15:07] Sim City because I'm running 200 [15:09] agents." Very cool stuff. But if your [15:11] focus is on like getting things done, [15:13] try not to get tempted because it's very [15:15] easy in this ecosystem. [15:17] >> So, two definitions. [15:18] >> Can I pause you? Can I pause you for one [15:20] second because that seems like a good [15:21] scoop. Yeah. So, uh first, uh you know, [15:25] meeting people where they are. I think a [15:27] lot of folks who have only interfaced [15:28] with AI through a chatbot kind of thing [15:31] where it's in a web browser and you're [15:33] going back and forth and maybe you've [15:34] pasted some code but the code remains [15:37] over here. I copy and paste it on on [15:39] into my code myself or I'm using an IDE [15:43] and it's got a a chatbot that is AI [15:45] enabled in sidebar and I'm sort of [15:47] talking to it but at the end of the day [15:49] like I'm looking at this code to what [15:51] you're describing for for like local [15:54] agents. they have access to things on my [15:56] computer, that transition I think is is [15:59] tricky for people who haven't done it [16:01] before. So, when you mentioned like just [16:03] pick one small task and stuff like that, [16:05] I guess kind of kind of a two-part [16:06] question. One, [16:08] >> what what guard rails, if any, do you [16:10] put around these things to help people [16:11] when like when you first started doing [16:13] this, right? You know, you probably were [16:15] also a little bit like skeptical or [16:18] you've read horror stories or you know, [16:20] that kind of thing. And so, that would [16:21] be like when when you say like start [16:23] with one thing. I agree with you like [16:24] start with one real thing, not a not a [16:26] demo, not a toy, like an actual task you [16:28] have to do, but maybe maybe maybe you [16:31] could call out a couple of you you call [16:32] it like the file example because I think [16:34] people are a little scared to just like [16:36] turn it loose and maybe you could help [16:38] calibrate that anxiety a little bit. [16:41] >> Yeah, let me walk through that because [16:42] I'm going to touch on those things and [16:43] the risk shut. [16:45] >> No, it's good. It's good. It's good. [16:47] It's a perfect segue to the next slide [16:50] here. So kind of want to start with just [16:52] some foundations as I'm going through [16:54] that. When you hear the word model or [16:56] when you hear or say model that is just [16:59] the core upstring language systems that [17:01] basically predicts plausible next words [17:03] from patterns that's what they do. So [17:05] when you're talking about you know chat [17:08] GPT4 or five or um opus that is the [17:12] model. Now when we're talking about [17:14] agent an agent that is effectively that [17:17] model that works inside a loop. So [17:20] there's some structure around it that [17:22] has to do with working towards an [17:24] objective has memory which basically [17:27] just means writing files somewhere or [17:28] writing information down. It can use [17:31] tools, right? And so this is where the [17:33] scary part comes in because this is [17:35] running on your machine. It can [17:37] potentially delete directories, move [17:39] things around, take actions um and it [17:42] will follow multiple steps. So that's [17:44] what we're going to talk about in this [17:45] context. And that's probably the big [17:47] fundamental shift of using a model to [17:50] help you do autocomplete or using a [17:53] model directly in like chatgp.com [17:56] versus an agent that actually starts to [17:58] take these actions. And that is starting [18:00] to blur a little bit in these systems. [18:03] But when we're saying agent here, just [18:04] think of it like an actor in the system. [18:06] Like I tend to think of an agent as just [18:09] another user that can do things. It has [18:12] its permissions, its roles, its goals [18:14] and all the implications of that. So if [18:17] we think about the risk of that like it [18:20] is the same as handing your laptop to [18:22] somebody else, right? And I would be [18:24] very weary to hand my laptop to my [18:26] child. Not you know the agent probably [18:28] isn't going to pour orange juice on it, [18:29] but they will cause some sort of chaos. [18:32] So that's the model to think of it. Just [18:34] think of it as another actor in the [18:35] system and what that means. Now, the [18:37] balance in reality and you'll see I do [18:39] stuff very fast and loose and uh because [18:42] I want to move fast and hasn't really [18:43] bit me yet. It will bite me at some [18:45] point and I'll kind of talk about what I [18:48] do to deal with that. So, the latter of [18:51] how this works and those of you that [18:54] have dabbled or relatively new probably [18:56] started with browser chat. So again went [19:00] to chatgp.com pasting something in got [19:02] an answer back copy paste back and forth [19:05] you're having a conversation [19:08] adjacent to that when some came earlier [19:10] for others or after was the idea of [19:12] autocomplete. So early versions of [19:14] Copilot or Cursor, you actually had an [19:18] IDE open, you would have a file there [19:20] and it could help you autocomplete. And [19:22] this was a step above the autocomplete [19:25] we'd been using for decades, right? It [19:27] had some AI type abilities. It had some [19:30] knowledge and sometimes it would write [19:32] full functions and things like that, but [19:34] it was still kind of in line. Um, this [19:36] was also good for honing in on a line [19:39] and saying what is this piece of code [19:41] doing that was then rapidly superseded [19:44] by this idea of repoaware chat. So [19:48] cursor and co-pilot again were [19:51] applications that you would install that [19:53] still looked much like an IDE [19:56] but in addition to the code you were [19:58] looking at there they could look at the [20:00] entirety of the code in the codebase and [20:02] the repo, right? And early on this was [20:04] still a bit of a challenge because when [20:07] we talk about models and context and [20:09] context just meaning how much [20:10] information they can keep in their heads [20:12] at one point there was a limit to this [20:15] but it was nice that you could ask a [20:17] question in a repo aware chat it would [20:19] be on your kind of local machine and it [20:21] would be able to look at the other files [20:23] in the system so it got much better at [20:26] predicting and implementing. [20:29] So the next stage of that though and [20:30] what we're going to talk about today is [20:32] this concept of agents and agents loop. [20:35] And so this is where it has shifted from [20:39] just being an assistant that kind of [20:41] answers questions and looks at your code [20:43] to one that can answer question and look [20:46] at your code and take action and execute [20:48] multiple steps and multiple systems. So [20:51] everywhere you move up the ladder there [20:53] here changes the unit of work you're [20:55] doing. Now, I tend to still use most of [20:59] these back and forth, and if we have [21:01] time, we can get into this, but [21:03] sometimes I'll be having the agent [21:05] running locally to help gather [21:07] information, gather logs, and things [21:09] like that. And I will still jump into my [21:12] browser and ask specific questions [21:14] because there are strengths to these [21:17] like some of the most powerful models [21:18] may be available to you via the browser [21:21] or you have a higher quota you can use [21:23] and things like that. So, it's not [21:24] necessarily either either or on this. [21:27] Um, autocomplete is still useful if you [21:29] bring up a bit of code and you're like, [21:30] "Help explain this to me. I need to [21:32] understand what's going on here." It's [21:34] really good to be able to do that in [21:36] line. Um, and again, if things come up [21:39] in the chat, feel free to stop me. But [21:42] the big jump here, so the autocomplete [21:45] versus the agent loop in the [21:47] autocomplete, the co-pilot style, your [21:49] unit of work was a function or a [21:50] snippet. was you being the pilot, the [21:53] driver there and the assistant helping [21:56] you with whatever was open in the editor [21:59] and the bottleneck that that really [22:02] lifted was writing and typing, right? It [22:05] would actually help you write the code. [22:06] And to me, you know, it was just a [22:08] replacement for I would go search on [22:10] Stack Overflow and copy and paste [22:11] something. That's what I did for [22:12] decades, right? Um it it helped with [22:15] that step. So that was great. Uh killed [22:17] Stack Overflow, but it was efficient. [22:20] Now agents and agent loops work in a [22:22] different unit of work which is looking [22:25] more at stories, epics, slices of [22:29] information and then it can pull in tons [22:32] and tons of context. So we can talk a [22:34] bit about this but one of the ways that [22:35] I tend to work in this is I will [22:37] actually point an agent at Jira or I'll [22:40] show an example here in a minute where [22:42] we point it at an issue in the Drupal [22:44] issue queue. I don't have to type all [22:46] that context in. I can just say go look [22:48] at this. it's up to date. It's showing [22:50] the real status. It can pull in all of [22:52] that information together and then start [22:55] taking action on that. So, it changes [22:57] the bottleneck where it is no longer in [22:59] the bottleneck of writing and typing it. [23:01] It is you guiding what the agent is [23:04] doing much like a manager of the agent [23:07] um around ambiguity, you know, [23:10] architecture. We're starting to see this [23:12] a lot where older AI would make really [23:15] silly mistakes, right? And now they're [23:19] making really advanced mistakes, I guess [23:21] I would say. And it's not even so much [23:23] mistakes, but they are differences of [23:25] opinion. You know, you'll you'll see [23:28] this in modern models and agents right [23:31] now that sometimes they will write way [23:33] too many tests, tests that aren't [23:35] needed, tests that test things that [23:36] aren't ever possibly going to happen. [23:39] And that is in part because early [23:41] models, they wouldn't write tests and we [23:43] would say, "Write test, write test, [23:44] write test." And it's just like a junior [23:45] developer that's now scared. They're [23:47] like, "Oh, we got to write tests for [23:48] everything." Your judgment is still [23:50] really important in in that, right? Um [23:52] the same that you'll see when an you go [23:56] to chat GPT or one of these tools and [23:57] you ask it to write a report to you, [23:59] it's very verbose, right? It's very [24:01] long. It says it takes 10 times as long [24:03] to say what it needs to do. It does this [24:06] with code sometimes, too. So your the [24:09] bottleneck is your review, those [24:11] constraints, and the guidance in that [24:13] regard. [24:15] So this is where you know if you've been [24:17] using the browserbased ones like great [24:21] for explaining a snippet drafting test [24:24] but what they really can't do is [24:27] execute. Now this isn't a perfect [24:29] delineation. They kind of run stuff [24:31] sometimes, but if you go into chat GPT [24:34] and you type something in, what it's not [24:35] going to do is set up DDEV, [24:38] pull down Drupal core, execute the code, [24:41] run the test suite in part because [24:43] there's some CPU cost constraints and [24:45] things like that. So, you're getting a [24:48] smaller viewpoint into what's happening [24:50] there. Um, and that leads to a hidden [24:52] cost, which is you're copying and [24:55] pasting. You have some sale context, [24:56] you're going back and forth. It really [24:59] you're spending a lot of time doing [25:01] something in the loop that [25:04] needs to be done, but it there's no [25:06] value being added to it by you as a [25:08] human doing it. Where the value is [25:10] coming in from you is your guidance and [25:12] things like that. So, this interesting [25:13] jump is when the model can actually run [25:16] that work. And so before we kind of show [25:19] this in practice, what an agent loop [25:21] actually is is I mean it's relatively [25:24] simple. It is you give the agent an [25:26] objective and you say go do this thing. [25:29] You say I have this file of pictures [25:32] here. Um they've all got the same name. [25:35] I'd like you to actually look at the [25:37] metadata in there, extract it, and then [25:40] reabel these with something coherent in [25:43] that. And this is the difference. Like [25:45] you're not looking at the code. you are [25:47] just giving the agent a directive that [25:49] you want. And one of the best things I [25:51] found for this is like our normal agile [25:54] methodology works really well for this. [25:56] Handing a user story, acceptance [25:59] criteria and implementation details, you [26:02] get that to an agent and they're most [26:03] likely going to do exactly what you [26:05] want. And this is one of the shifts that [26:07] we're seeing where executing and writing [26:10] the code, there's not necessarily a [26:12] whole lot of value in that anymore, [26:14] right? The execution part of that, the [26:16] typing of the specific functions [26:20] is relatively easy. It is the thought [26:22] that goes into it before on what you're [26:24] actually trying to accomplish and how [26:26] you're trying to accomplish it and [26:27] describing it correctly, which is [26:29] granted, it's always been a challenge in [26:31] development for us, right? Any of us [26:32] that have gotten tickets as a developer [26:34] and said like, well, this doesn't [26:35] actually describe the color you want [26:37] this button to be, right? You have to [26:39] put those things in there. But with that [26:41] information, the agent will both inspect [26:44] the context and its environment. It can [26:46] look and see things. You'll see this [26:48] when I show a demo here in a minute. [26:50] Whenever I tell my local agent to do [26:53] something um in Drupal, the first thing [26:56] it'll do is try to run it just locally [26:58] on my machine and I think I have like [27:00] PHP 5.2 installed there and it's like I [27:02] can't do that. And then it will jump and [27:05] say, oh, I see DDS here. I can spin up [27:07] an environment and do that. So the [27:10] inspection and the ability for this [27:12] agent to be able to respond to its [27:13] environment is absolutely critical. This [27:16] goes into an edit and plan kind of loop [27:18] where it's, you know, sometimes it plans [27:21] first, sometimes it edits first, and [27:22] there's different strategies there, but [27:24] it says, here's what I want to try to [27:26] accomplish, right? This is a good place [27:28] for you as a human to kind of jump in [27:30] editing. It'll actually write the code. [27:33] It will then, and this is the most [27:34] important part here, not most important, [27:37] but the thing that unblocked this, run [27:39] and revise. So, it actually executes the [27:42] code. And this is where it's both scary [27:45] in that it is executing code, but it's [27:47] also incredibly important because this [27:50] is the difference from [27:53] something like chat GPT writing [27:55] something that looked plausible but [27:56] didn't actually execute versus something [27:59] that truly runs and executes. Now, you [28:03] still have to [28:06] review this. So in the example I used of [28:08] those 25 pull requests when it was in [28:11] review, we noticed that one of the tests [28:12] it was testing actually had hardcoded a [28:15] path for my local machine in that test [28:17] of just like a random file. Um and these [28:20] are the kind of things this is still [28:21] classic works on my machine, right? So a [28:23] lot of our development principles are [28:24] still the same here. But this loop of [28:27] the agent writing the code, running the [28:28] test, seeing the failure, fixing the [28:30] failure without you typing is [28:33] fundamentally different. And when we're [28:35] talking about these loops, they can go [28:36] on and on and on. So when I was doing [28:38] some of the test of asking an agent to [28:41] port um Umami to Drupal CMS, at one [28:45] point the loop ran for 48 hours in part [28:48] because I didn't really care how it got [28:49] there. I just wanted to see that it got [28:50] there. But it can run these loops for a [28:52] very very long time. So before we start [28:55] and get into the demo, some of the [28:57] practical stuff that you will need to do [29:01] this. So generally um an API key is [29:06] required for one of the services. So [29:08] cost is a real thing. Now you can run [29:10] some local models. You can get all the [29:12] set up. If you're starting out on this, [29:15] just use one of the established ones to [29:18] start with. Uh I threw here if anyone [29:20] wants to copy this down. And I have [29:21] three free passes for a week of Claude. [29:24] So Claude code you can use now. [29:28] try to get one of these Pro or Max plans [29:30] if you're using this every day because [29:32] you know if you're using these and the [29:34] agent is running in one of those loops [29:36] and it goes off for 48 hours like that [29:38] could be hundreds if not thousands of [29:40] dollars of cost. We are in a really [29:42] interesting window right now where these [29:44] are heavily subsidized. So when you hear [29:46] people talking about OpenAI Pro or Cloud [29:48] Max and these are the ones that go from [29:50] 20 to $100 to $200, like if you can get [29:55] one of those now because they're not [29:57] always going to be at that price and [29:59] we're already starting to see that shift [30:00] and this is a golden window for you to [30:02] learn while it's subsidized by these [30:04] companies. Now one caveat here, the code [30:07] does leave your machine with an asterisk [30:09] here. If you are using some of those [30:10] local models, there are ways you can get [30:12] around that. Now, generally the push [30:14] back of they're going to train on your [30:16] data. If you're on one of your paid [30:17] plans, that's not going to happen. There [30:19] are carveouts for that within as far as [30:22] much as you want to trust Sam Alman and [30:25] the people behind this. But check your [30:27] organization's policy and just [30:28] understand it's going to leave your [30:30] machine. And then what's important and [30:33] critical for this is um that they these [30:37] are going to be running on your local [30:38] machine. They need the tools available [30:40] to it. You can run these in a sandbox [30:43] like having it set up something like [30:44] DDAB or Lando for Drupal development is [30:47] essential. A lot of the same fundamental [30:48] things that you would do. And then you [30:51] you need a local site. This is the same [30:53] as doing local development. So all of [30:56] those things that we would do to set up [30:57] a development so you could execute [30:59] locally are important. Um some safety [31:02] habits on this um as you're going [31:05] through always work within a branch. So [31:07] this is just the safety of your code, [31:09] but you know classic developer stuff [31:12] like git is your undo button. They will [31:14] often warn you now, but there's a lot of [31:16] changes that are going to happen in your [31:18] codebase here and git and related tools [31:21] are um important. So the agent asks [31:24] before running command you approve each [31:26] step. Yes, in practice that's what you [31:28] should do. No, in reality this is not [31:30] what I do. I just put it into yolo mode [31:32] and I let it go loose because if you're [31:35] really wanting to embrace the power of [31:37] this thing, it's going to be running in [31:39] a loop and you're going eventually get [31:40] to the point where you have multiple [31:42] agents going at one point and you don't [31:44] want to feel like that little hamster [31:45] hitting the button over and over. You're [31:47] going to stop reading it and stop [31:49] thinking about it. The better ways to do [31:52] this are to [31:54] limit the access that your agent has, [31:56] right? So like I have agents that are [31:59] connected to information you can pull [32:01] from drupal.org but I do not give them [32:03] the ability to post. And this is not me [32:05] asking the agent nicely not to post [32:07] because they will. This is they do not [32:10] have the access to do this. Same thing [32:12] you can set up machines and sandboxes. [32:14] Some people get unique machines. Um [32:17] setting up things like your agents file. [32:19] These are just readmes on how the agent [32:22] should behave and really your [32:24] conventions in there. So um the tools [32:28] you'll actually want to install if you [32:30] haven't yet and these are the ones you [32:32] hear about a lot. So cloud code, codec [32:35] cli or open code and these are basically [32:38] just terminalbased repoaware loop [32:42] capable kind of harnesses around the [32:45] agent. So each one of these cloud code [32:48] is on GitHub. It is not technically open [32:52] source although they accidentally opened [32:53] their source uh a week or so ago. Codex [32:56] CLI yeah is actually open source. Now [33:00] they are still they want you to use [33:02] their models and then open code is truly [33:04] open source. Now each one of these tends [33:07] to work best with the model that they [33:10] provide and to some extent if you're [33:13] using a subscription plan on cloud you [33:16] need to use cloud code with it. they've [33:18] kind of locked it down. Open AAI and [33:20] Codeex is much more comfortable with [33:23] people using their models and other [33:24] ones. Um, now each one of these can also [33:28] use the other models. You can configure [33:30] these. You can use local models. So [33:31] within Aqua, we use cloud code, but it [33:34] runs against models, the actual [33:36] anthropic models, but ones that are [33:38] hosted within our AWS ecosystem. So the [33:41] code doesn't actually leave a perimeter. [33:44] So there are ways that you can kind of [33:45] set these up. [33:48] So, I'm going to show an agent loop in [33:50] action here. And I grabbed a ticket. Um, [33:54] this is just one that Kristoff of AI [33:57] initiative had filed where he has an [33:58] error when saving a note. Right? So, I [34:01] will do this and I will work a lot like [34:03] this where I'm just looking for things [34:05] that are in the AI initiative sprint and [34:08] I'll say, "Hey, I can probably help with [34:09] this." And this is the kind of thing [34:11] that's like maybe I'd be interested in [34:13] doing this normally probably if it bit [34:16] me, right? if I ran into it. But let's [34:18] see if we can help Kristoff out. So, [34:20] here's how I would approach this. Now, [34:22] in this demo, I use OpenAI codecs more [34:26] than Cloud Code for kind of like [34:28] personal work. Um, I find that the the [34:33] cutting edge codeex models like spend [34:34] more time, but I tend to use both. And I [34:37] will bounce back and forth. I'll get a [34:38] second opinion of each of these, but [34:41] here's how I would take a stab at [34:43] solving this. So, [34:46] let me go ahead and change [34:50] my screen. Um, let's see which one ran [34:54] here. [34:57] I see my bash. Okay, so I'm just great [34:59] terminal here. I'm going to open codeex. [35:02] This is where I do the yolo. Just do [35:04] whatever you want. [35:06] Do what you wish there. If you don't [35:08] have that, it's going to ask you every [35:09] step of the way what you want to do. um [35:12] which I did during the demo. [35:15] Okay, so now I'm in and this is [35:17] basically the the the base codeex CLI. [35:21] If you're in cloud code, it's going to [35:22] look the same. You know, you have some [35:24] commands in here. You can change the [35:26] models, permissions, skills, all that [35:29] stuff doesn't necessarily matter. At [35:31] this point, I can ask it questions like, [35:35] and it's probably going to tell me in [35:36] this, you know, I'm in this directory [35:39] where there's stuff about Drupal. So, I [35:42] have a folder here where I just do all [35:43] my Drupal contrib. Um, it's kind of [35:46] telling me what's in here. I can ask it [35:48] what it is. But to the point, what I [35:51] would do is jump in on this ticket and [35:54] say, um, [snorts] here is a ticket. [35:59] Let me [36:01] Copy the right thing. [36:22] So, I'm going to say here's the ticket [36:23] right here. Right. I gave it the URL. [36:25] Now, because it has access to all of [36:27] these tools and everything else on my [36:29] system, and I do have a little bit of [36:31] like scripts and stuff around here, it's [36:33] actually going to look and pull down [36:34] from drupal.org and figure out what is [36:37] going on with the issue. So, let me [36:39] actually switch [36:41] screens right here. So, you can actually [36:43] see one where I ran this earlier because [36:46] it's going to take some time to do that, [36:48] right? So, where I came in here, are you [36:51] What are you guys seeing here? of no I [36:55] think [37:02] oh that's the other one too many windows [37:10] I'll get it one day [37:12] here we go so this was about an hour ago [37:14] this has the whole conversation here but [37:16] I can walk through it in kind of faster [37:19] time and explain what he's doing I said [37:20] take a look at this right so I said, [37:22] "Okay, I'm going to pull this through." [37:24] It loaded the skill. It searched the [37:26] web. Now, this is where you need to be [37:29] polite with these upstream systems, [37:30] right? This is actually only against [37:32] drupal.org. So, don't go in here and [37:34] say, "Pull down every issue ever because [37:36] it will do some stuff like that." But it [37:38] found it said, "Okay, I have the issue [37:40] context. Now, it understands the [37:43] symptom." Now, again, take your judgment [37:44] to this, but you know, reading this [37:48] says, "Here's what it's trying to do. [37:49] It's going to check the local code if [37:51] there's an extra fix. It's actually [37:53] looking through here. So [37:56] now it goes through a little bit more [37:59] information. This is just more it [38:01] pulling the ticket. Um it is now at this [38:04] point as I since I asked it to debug, [38:06] it's pulling down the recipe project [38:08] itself. It's cloning the source down [38:11] locally. So it is taking action. So [38:13] already this would be, you know, 15 [38:15] steps of me copying and paste back and [38:17] forth or looking into this. It is [38:19] starting to [38:22] explain what's going on here. And this [38:24] is one of the things the whole thing [38:25] from Drupal kind of like don't commit [38:28] code or contribute code that you don't [38:29] understand. It's fine if you don't [38:31] understand it beforehand. You can use [38:33] these tools to help you understand it [38:34] and learn. This is how we all learn. So [38:37] this part is fine, right? But it's going [38:39] through understand that sometimes this [38:41] can lead you down paths that you [38:43] wouldn't have gotten to yourself. So if [38:45] you're completely unfamiliar, like if [38:47] you don't know what a node is in Drupal, [38:49] this is probably not where you want to [38:50] be. But eventually kind of step through [38:52] these things. But it's running the [38:55] tradeoffs here, right? It is finding, [38:58] you know, a couple different design [39:01] problems that it's going through here. [39:04] And you'll see as it keeps going [39:05] through, here's the assessment that it [39:08] shows me, right? So, here's the issue. [39:10] It's active. There's no patch, so [39:12] nothing's really there. Um, here's what [39:14] it thinks happening, and here's the [39:16] plan. This is generally the first place [39:18] that I come in and say, okay, this [39:21] actually seems correct. [39:25] Here's what it wants to do. So, I kind [39:27] of give it a a once over on like, is [39:29] this right? And this is the first place [39:31] to see that like, hey, you're off the [39:33] mark here, right? like you're planning [39:34] to fix this, you know, in the wrong way [39:37] and models are better now, but this is [39:40] the equivalent of a a junior dev coming [39:42] to you and saying, uh, this view wasn't [39:44] working, so I made the list in the twig [39:46] file, right? Plausible. Sure, that will [39:50] work. But with enough experience, you'll [39:52] be like, uh, some problems here. There's [39:54] probably some other ways and some [39:55] guidance. So, this is where you [39:56] generally provide that first human [39:58] expert guidance on that. Now, the expert [40:00] guidance doesn't have to be Drupal [40:02] expert guidance, right? Say you ran into [40:05] this issue because it affected something [40:06] at your company. Your expert guidance [40:08] here could be like, well, here's [40:10] actually the implication of this issue. [40:12] This is what we're running into as an [40:14] organization. So, maybe there's [40:15] something else we need to do. Use this [40:17] as this discussion that is much more [40:19] grounded than the actual things. So, we [40:21] look at the plan here and I said, "Okay, [40:24] great. Will you actually test this and [40:26] see if you can recreate the issue?" Cuz [40:29] at this point, one of the things I [40:30] noticed was it gave me an answer, but I [40:32] was like, "You didn't actually run this. [40:34] You didn't actually test this." So, it [40:37] runs it. This is the thing where you'll [40:38] see it first tries to run it on PHP 5 [40:42] for whatever reason, and then it's like, [40:43] "Let's use DD." Sets up DD, runs it [40:47] through, [40:48] and I keep letting it go for a while. [40:50] It's building the site. And this, you [40:52] know, was probably 20 minutes here [40:54] setting up DDA that I was, you know, [40:56] doing something else. Um, [40:59] this part's not really important to me [41:01] on like necessarily how it was set up, [41:03] but you can see it's testing. It's [41:04] validating through here. Uh, let's [41:07] actually get down to [41:10] where it did. So then it tells me yes, I [41:12] was able to reproduce it locally. Um, [41:17] I can reproduce it when I submit the [41:19] node over HTTP, but it does not [41:21] reproduce with a raw drush entity save. [41:23] So some really important context here [41:26] that you know again you would never get [41:29] if you weren't using an agent that can [41:31] actually execute these things like and [41:33] this is all going like it ran the stack [41:35] trace it understand what's going on and [41:37] it's the implication of that is [41:39] incredibly important here right so if [41:41] you just did that first plan or the [41:43] first thing it generated [41:46] yes maybe plausibly it would have fixed [41:47] but could have had some not fix it [41:49] completely or miss the fundamental thing [41:52] here So then it's like, okay, this is [41:57] real. Um, do you want me to fix it? And [42:00] then because I have some stuff in my [42:02] agent file that's like, disclose AI [42:04] responsibly, it's going to warn me over [42:06] and over. And I very gruffly say yes, [42:08] just go ahead and fix that. Um, so it [42:11] fixes it. Here you can see it went down [42:15] a path that didn't work. And again, I'm [42:18] not at the laptop while it's doing this. [42:20] It's exploring. It's trying stuff. It's [42:21] the kind of debugging that we would [42:23] spend all day doing. So eventually it [42:26] comes back and says, "Hey, I did the [42:28] fix. [42:30] Here's the smallest candidate patch." [42:32] And this is some guidance that I've [42:33] given it over time. So again, you'll [42:36] notice like it does this big superfluous [42:38] things and lots of code and I've told it [42:40] over and over like just give me the [42:42] smallest most targeted patch that would [42:43] actually fix this thing. You'll see here [42:47] it is a tiny diff right here. So I will [42:51] go through and here is its [42:52] recommendation right it says here is [42:57] what we think we should do here's the [42:59] actual diff which is a relatively simple [43:02] thing now at this point this is where it [43:06] ended now what you don't want to do is [43:08] say okay ship it right so at this point [43:11] I'm going to look into that code and be [43:13] like do I actually understand the [43:16] implications of that if I don't [43:17] understand the implications of that [43:19] change right now. I'm not going to ship [43:21] it. I'm also going to say, "Show me, [43:25] spin up a site for me so that I can [43:26] actually go test this." And again, this [43:28] is where you don't have to be an expert. [43:30] You don't have to understand what this [43:32] means. You can now take the time to say, [43:34] "Teach me what this means. What is the [43:36] implication? Cop this over to chat GPT. [43:38] Ask it questions." And then lo and [43:40] behold, at the end of this, you've [43:41] helped fix it and you understand the [43:43] implications of this change. Now, what I [43:45] would do at that point is I would say, [43:48] "Okay, I think this looks good. Can you [43:50] help me um write the notes for this?" [43:53] Which you kind of did here. [43:56] You can then say, "Please do the merge [43:59] request on drupal.org for me, like save [44:01] some overhead there. Monitor the test." [44:03] A lot of times what'll happen is you'll [44:05] do your merge request, the test will [44:07] run, the test will come back with some [44:08] errors. as opposed to you having to [44:09] spend time debugging it, you can ask the [44:11] agent, hey, can you pull that [44:12] information down? [44:14] Can you move it forward? So, we're kind [44:17] of getting towards the 10-minute mark. [44:19] Um, there's some couple directions here [44:20] I want to go, but questions on this. I [44:22] want to make sure this is if you're new [44:24] to this, is this helpful? Are there gaps [44:26] in what would get you started in using [44:28] these kind of tools? [44:30] >> Yeah, why don't we that's a great [44:32] question. So, if if what has happened so [44:34] far is kind of like, oh, raise your hand [44:37] now. Let's get you taken care of. In the [44:39] meantime, I'm going to um I'm going to [44:42] grab a couple questions from the chat. [44:44] Um [44:46] one is uh uh there's kind of like two of [44:49] these that are related. So, one question [44:51] is, is the DDEV environment spin up a [44:54] skill that you're using? And then a [44:55] related question for me is like, can you [44:57] show us the skills as in agent skills [44:59] that you're using, not your innate Scott [45:02] Falconer skills? Um because I I think a [45:05] lot of a lot of people who try these [45:07] tools run into the situation where [45:08] because Drupal does things really weird. [45:11] >> For example, requiring a tool called [45:12] DDev, right? Where normally you npm [45:15] install blah blah and you're done, [45:16] right? You know what I mean? That kind [45:17] of thing. It's like AI agents that [45:20] haven't been trained on like Drupalisms [45:23] are very poor at just like vanilla out [45:26] of the box things which kind of creates [45:28] this circular problem where Drupal [45:30] people dismiss them as useless and then [45:31] they keep getting better and then we [45:33] don't try them. And so that would be one [45:35] question is like how you mentioned like [45:38] I have some scripts and stuff like that. [45:39] Are you able to show I know you have [45:41] like a repo and stuff like are you able [45:43] to point us at what you're using and [45:45] kind of you know if that's the thing or [45:48] just like a general like how would you [45:49] recommend somebody get started like what [45:51] skills would be part of that package. [45:54] >> Yes. Um it is not so much that is a [45:57] question it is that that is an hour [45:58] presentation. [45:59] >> Yeah. Okay. Maybe that can be a future. [46:01] So, so but I I I did at one point I put [46:04] this into [46:06] one minute of what these are and it it [46:09] is not ultimately a question of skills. [46:11] The way to think about it, it's a [46:13] question of context. Your job is to give [46:16] the agent that is working the right [46:18] context to do the job. So, when we're [46:20] talking about things like agent skills [46:23] or agents MD or the prompt that you're [46:25] giving it, that is ultimately just [46:27] giving it context, right? [46:30] So the way to kind of break this down [46:33] and this is why you should start using [46:34] these tools because they are going to [46:36] make silly mistakes and then you're [46:37] going to give it context. And when you [46:39] start realizing I've told you this four [46:41] times, you're going to eventually put [46:44] something into agents which is like a [46:46] readme. It is the same as telling a new [46:47] de dev that's onboarding. So you [46:50] probably already have onboarding [46:52] developer readme docs. Probably they're [46:54] probably out of date because you [46:55] probably set them up when the project [46:56] started eight years ago, but go update [46:58] those, right? You can point the [46:59] [clears throat] agent at that and say [47:01] use this. This is how we do things in [47:03] this project. So again it's ultimately [47:05] it is just context. Now how you manage [47:08] that context starts to get important. So [47:11] something like agents MD it's just a [47:13] markdown file. It's just a readme file [47:15] and the agent knows to read that with an [47:19] asteris that is a standard but claude [47:22] still looks for one called cloud. MD put [47:24] it in a readme put it somewhere pointed [47:25] at it. This is something that an agent [47:29] is going to read every single time it [47:31] executes. Right? So it's not foolproof, [47:34] but if you want to have a big sign on [47:36] the wall that says use dub, use ddub, [47:39] use cdub, put it in your agents. Right? [47:41] Now, where skills come in, skills are [47:44] ondemand procedural knowledge. So this [47:47] tells them how to use these systems. [47:50] Right? I might have a ddev one in there. [47:54] it might just be in my agents file in [47:56] part I just end up working in one [47:58] directory that this knowledge starts to [48:00] kind of build on itself. Now, what I do [48:03] have for agent skills are things about [48:06] how to deploy to Aquia or how to [48:10] correctly pull logs from our systems. [48:13] Because if you don't, first and [48:15] foremost, like if I don't have anything [48:17] in my agency or whatever that says I'm [48:18] on on Aqua or even working with Drupal, [48:21] it'll just do every other thing in the [48:23] world. You say build a site and it will [48:24] do whatever, right? Or if I say go [48:27] deploy the site, um it doesn't it may [48:30] pick somewhere to deploy it. it may try [48:32] to figure out something. I eventually, [48:34] you know, when I'm working will say, I'm [48:35] going to be putting this on Aquia. [48:37] Here's what I mean. And then I have a [48:38] skill that basically says follow these [48:40] best practices like use Drush, use Aqua [48:44] CLI, use these tools. So, you can think [48:47] of it like that of the skills or just [48:50] that procedural knowledge, but again, [48:54] don't necessarily overthink it. Like, [48:56] get in there, tell it to use DD. Once [48:58] you tell it to do something twice, write [49:01] a playbook like you would for any kind [49:03] of junior dev in there. So [49:06] >> great [49:08] is the agents MD file you use for Drupal [49:10] core development something you can [49:12] share. [49:13] >> Absolutely. Um this is a perfect plug [49:16] for the what do you do? Do you want to [49:18] talk about the uh project you started [49:20] which was the Drupal? [49:21] >> Yeah. Well, I I can plug that, but it's [49:23] still very recent. But uh yeah, so [49:25] there's a project uh called Drupal A [49:28] Drupal AI best practices, which is [49:30] linked in the chat there. Um and uh it [49:33] is not much yet, but what it is uh yeah, [49:36] I guess I could I could demo that real [49:38] quick. I don't want to take your your [49:39] your awesome time away, but um it would [49:43] be especially helpful if I could figure [49:46] out what tab that's in. There we go. Uh [49:49] so this project was something we spun up [49:50] at Drupalcon be uh to try to make agents [49:53] less stupid basically about Drupal. So [49:57] um and it's it's uh there's not much [49:59] there but if you pop into the source [50:01] code here we kind of have a couple of [50:02] different uh things. So this is like [50:04] general agents guidance which we [50:07] centralize in agents.mmd but then have [50:09] um you know claude gemini and all them [50:12] basically saying don't look here look at [50:13] as to try to like centralize all that in [50:15] one place and then what we try to do [50:18] because this files loc loaded for every [50:20] it you know we want to keep this pretty [50:22] crisp is we point it off to individual [50:24] directories which are called skills so [50:26] there's one on how to write [50:27] documentation one on writing automated [50:29] tests we obviously need a lot more of [50:30] these around passing core gates and [50:32] using ddev and all that kind kind of [50:34] stuff. But my favorite thing about this, [50:36] which was done by I want to make sure I [50:39] get the person's name correct. Um, [50:44] uh, George Castanis, who as far as I can [50:47] tell is a freaking genius. But anyway, [50:49] we have these, uh, notion of eval. And [50:51] that means that we can, uh, specify, [50:55] um, that this is what it means to write [50:57] good documentation, right? you know I [50:59] expect the output that um an agent [51:02] identifies the any IP fixes the source [51:04] code whatever I haven't read all these [51:05] in detail um we expect that it [51:08] identifies these things and then we can [51:10] make updates [clears throat] to the [51:12] content of the skills files uh to add [51:15] you know better words here or whatever [51:18] um in order and then we can tell if it [51:20] made things better or worse um both on a [51:22] on a per uh skill basis on a per model [51:25] basis that kind of thing so um so yeah [51:28] we don't have a lot here yet. There's [51:29] still much more to do. And I think what [51:31] we probably need to do is just like pull [51:32] in a bunch of the existing skills that [51:34] exist. Like there's so many. If you go [51:37] here and you search for Drupal, it's [51:38] like there's a million of these. Um, but [51:41] what we're trying to do is kind of [51:42] create a framework for like establishing [51:45] which of these are like the best of the [51:46] best for a getting started thing so that [51:48] everybody who's trying to get started [51:50] tools basically is like download this [51:52] run command to load it and and you're [51:54] good to go. Um, so yeah, contributors [51:56] welcome for sure. [51:58] Um, [52:01] let's see. There's more things in here. [52:03] I, uh, while I'm catching up on the [52:05] chat, or maybe Amber, if you could catch [52:07] up on the chat, uh, I had one question, [52:09] which is going back to like paranoid. I [52:11] don't trust this stuff. I think it's [52:13] awful and I don't ever want to use it, [52:14] but I'm forced to use it because my [52:16] whole industry is moving in this [52:17] direction. I won't be left behind. Like [52:18] that persona. And I saw you type into [52:21] the CLI, dangerously skip all any and [52:24] all approvals. D, right? And I I'm with [52:26] you because it's like you you'll start [52:28] and be like, "Yes, I approve you to get [52:31] clone. Yes, I approve you to uh read [52:34] that file from Drupal." And eventually [52:36] you're just like, YOU KNOW WHAT? YOU [52:37] KNOW, IT'S JUST LIKE DO THE THING. SO, [52:39] what would you recommend for that [52:41] persona who's like, I've never played [52:43] with agents before. I know I should, but [52:44] it's kind of scary and I don't really [52:46] want to turn it loose, especially on my [52:48] work computer because who knows, right? [52:49] That kind of thing. what would be like [52:52] your recommendation of a safe, you know, [52:54] like harness or something like that to [52:56] set them up with like, you know, you [52:58] mentioned you give them read only access [53:00] to certain things or whatever like can [53:01] you share a little bit about that as far [53:03] as like how do you box it in or put [53:04] guard rails in so it doesn't go nuts and [53:07] reformat your hard drive? [53:09] >> Yeah, actually two questions with that [53:10] one. Um, this is why I'm working from [53:12] the bar, right? So, you know, just just [53:14] do it. [laughter] Um, no. So it when you [53:18] first start these, they they do a good [53:20] job in cloud code and codeex and I [53:23] presume open code does this where it's [53:26] going to be in a sandboxed very locked [53:29] down mode where you're going to have to [53:31] improve approve everything. So don't [53:34] necessarily be scared about that. Now [53:37] you have to have the the understanding [53:40] of like what a command line tool is, [53:41] what these commands cuz it's going to [53:43] say like I want to run ls. Is that okay? [53:46] I want to rep this file. Is that okay? I [53:49] want to write this Python. Is it okay? [53:51] And so to start that's fine. And and [53:55] there's different levels of comfort [53:56] people have and there's different ways [53:58] people will do that. Sometimes you can [53:59] do a sandbox. Now that's the interesting [54:02] part about the new mythos thing. They [54:04] put it in the sandbox and they said, [54:05] "Please escape the sandbox." And it was [54:06] like, "I'm out." And it like slacked the [54:08] developer or something. It was like, [54:09] "I'm here. I got out." Um, so there are [54:13] some concerns with that. Um, sandboxing, [54:17] there's when you kind of use these by [54:20] default, it can't really execute things. [54:23] It can't leave the directory it's in. It [54:26] can't do network connections. But that's [54:29] going to be your balance. Now if you are [54:32] really concerned with that there are [54:35] cloud versions of this. So Jules from [54:38] Google um codeex does have a codeex [54:41] cloud [54:43] I think claude also has cloud cloud. The [54:47] problem is they are probably not going [54:49] to be sufficient to run Drupal on. [54:52] Right? So if you want to play with this [54:53] and just do some Python stuff do it that [54:55] way. Right? you could set up a Docker [54:57] instance and run the stuff in there, you [55:00] know, run a machine like that. So, just [55:03] start like it it'll it'll ask you um and [55:06] let let me actually show you. I'll ask [55:08] the same [55:10] question. So, let me [55:15] I'm in the wrong one. Right. [55:19] Okay. [55:21] Just going to copy this so we can do the [55:22] exact [clears throat] same one. [55:30] Just going to go into my one. So now I [55:32] am not in YOLO mode, right? Um I'm going [55:37] to ask it to look at this and it's [55:38] probably going to ask me what it wants [55:40] to do. Now I may have some stuff in here [55:42] already that has preapproved certain [55:44] things. [55:51] You see it's giving me the plan and I [55:52] think I do by default. I give it network [55:56] connection where the first time you run [55:57] this it may not even have that. Um yeah, [56:01] this may be a bad example because I [56:02] probably already pre-approved a lot of [56:03] these. So as you're going through these [56:05] like there's an option that's like [56:06] approve this once or approve this every [56:08] time but eventually it's going to run [56:10] into something here and it's going to [56:12] say do you want to deny this action, [56:14] approve this action or give it another [56:16] direction on how to work. So just kind [56:18] of start there. Um, by using the [56:21] default, you're going to be safeish in [56:23] what it's doing. [56:26] >> By the way, we're at time or a little [56:28] over time. Are you cool to stick on and [56:30] answer? [56:30] >> I can stay. Yeah, I can absolutely stay, [56:32] but if anybody else wants to leave, feel [56:34] free. Um, yeah, and as always, reach out [56:36] to me in Slack, you know, especially if [56:39] you're working on like a Drupal SQ, [56:40] like, hey, I want to test this out here. [56:42] I'm happy to jump on a call and help you [56:44] all with those things. So, yeah, I have [56:45] some time if there's more questions. [56:47] All right, there were uh I'm trying to [56:49] find them all. We helpfully tagged them [56:51] with questions and now I'm skimming to [56:52] because red hearts are also red and so [56:54] I'm like wait where are the questions? [56:56] [laughter] Did have a a question about [56:59] documentation. Has anyone had any [57:01] experience using AI tools to improve [57:06] Drupal's documentation? [57:08] Uh any luck with that? [57:11] >> Yes. [57:12] >> Uh yeah. [57:13] >> Yes. Scott says [laughter] yes. [57:14] >> Yeah. So um this is one when we're [57:18] talking about like task and especially [57:19] things that you feel comfortable with [57:21] have it try to work on things like that. [57:23] So we use this significantly with an [57:25] Aqua for Aqua's docs like to the extent [57:28] that we have if you've used Aqua's like [57:30] support chatbot that agent if it can't [57:33] answer a question will flag to us and [57:35] say I tried to answer this but that [57:36] documentation wasn't there. How can I do [57:38] it? It's really great to take an agent [57:40] like this, point it at a piece of [57:42] documentation, point it at the code and [57:45] say, "Where is this incorrect?" Or watch [57:47] this video of me using the site and [57:49] relabeled us where the labels have [57:50] drifted and changed over the years. So, [57:52] lots of really good stuff you can do [57:53] like that. [57:55] >> Specifically for drupal.org or though [57:58] with editing nodes and I mean I [58:02] >> so as of as of um as of so I haven't I [58:06] haven't turned it loose that way as like [58:07] editing nodes but I will say as of I [58:09] want to say one week ago two weeks ago [58:12] something like that um I got annoyed [58:14] enough about the fact that it kept [58:16] drupal.org kept blocking uh Claude code. [58:19] So I we couldn't do what I wanted to do [58:21] and I asked Claude code why it was [58:23] failing and it said something about [58:25] perimeter X and then I found this issue [58:28] um and anyway drum fixed it. So now [58:29] cloud code does not automatically get [58:31] blocked and it will allow you to like [58:34] retrieve nodes and and do that kind of [58:36] stuff. So I suspect it could work. Um [58:38] but I will say like uh I work now at a [58:41] company called Temporal and I lead a [58:42] Devril team developer relations and for [58:45] our off-site um we had the one [58:49] documentation person from Anthropic [58:51] there. they have one documentation [58:52] person for all of or for cloud code I [58:54] should say and she walked us through her [58:56] workflow and it was fascinating because [58:58] it was basically like uh she was sort of [59:01] like cosplaying is like let me roleplay [59:04] as though I'm joining your team your [59:06] docs team for the first time and I've [59:07] never seen your product and I've never [59:08] done anything. So like she first uh [59:11] points it at the repo right the docs [59:13] repo and says download all of this stuff [59:15] uh locally as context. Also um I want [59:19] you to review the most recent say you [59:23] know five weeks of issues in GitHub [59:27] right that have been closed and pushed [59:28] back and whatever and based on that [59:30] generate a guidelines MD file of like [59:33] what you feel like what are the things [59:35] that we've traditionally po post back on [59:37] is like this isn't acceptable because of [59:38] reasons and then it generated some [59:40] guidelines out of that so using actual [59:42] data to do that and then basically she [59:45] does not write any docs herself at all. [59:48] What she does is, you know, a new [59:50] feature will get shipped because Claude [59:51] is shipping things like constantly, [59:53] which you know, she knows nothing about [59:55] because it's like, well, this is going [59:57] now. You know what I mean? And then just [59:58] kind of like has sort of a like a [60:01] template prompt kind of thing that's [60:02] like, okay, every time a new feature [60:04] ships, we want to uh grab all the [60:06] details out of the commit log so we can [60:08] generate a change log. We want to um [60:11] create like an overview that is written [60:12] to this sort of target audience's level [60:14] of understanding. we want to [60:16] autogenerate an example application that [60:18] they can copy and paste and get working [60:19] blah blah. And then where she comes in [60:22] is the the judgment, the guidance, this [60:25] kind of thing because um because Claude [60:28] a lot of times is too smart and so it [60:30] will write things at way like you [60:32] already understand what all this stuff [60:33] is, right? And so that's an area where [60:35] she has to express her own judgment. And [60:37] then what's interesting is just like [60:38] Scott was saying, it's like if if she's [60:41] having to correct it on the same thing [60:42] like multiple times. It's like I already [60:44] told you we don't use that word here or [60:46] whatever, then um it gets added to [60:49] either a skill or an agents.mmd file, [60:52] something like that that is now now [60:53] directing the agent that from now on [60:55] don't make that problem anymore, right? [60:57] And so what she's been doing is [60:58] basically building these skills files up [61:00] over time. So now it is it still [61:02] requires manual work, but it's like she [61:04] can trust most of it most of the time. [61:06] But you know, again, it's it's one of [61:08] those things where a lot of times people [61:09] try one thing with AI and it's garbage [61:12] and then they're like, "Well, this is [61:13] stupid." You know what I mean? And a and [61:15] a compelling thing about Scott's demo is [61:18] or in your preamble, you know, Scott, [61:20] you were talking about like this ladder, [61:22] right, where we were in chat bots or [61:24] auto assisted or autocomplete. So it's [61:26] like autocomplete, then we were in chat [61:27] bots, then we were in like an IDE [61:29] situation, and then we were in agents [61:31] and all of that happened in like two [61:33] years. It's just like, so like if you [61:36] tried it back when it was screwing up [61:38] the number of ours in the word [61:39] strawberry and threw it in the garbage [61:40] at that point, you have like all of this [61:43] stuff of it running things on your local [61:44] computer. And so it's just like that. It [61:46] sounds like sci-fi stuff, but that's how [61:47] a lot of developers are developing [61:49] today. So [61:49] >> Well, this is my favorite thing in the [61:51] context of your [61:52] >> question. Well, no, but here here's a [61:54] quiz for that one. Uh, if you've been [61:56] asked this, don't answer it, but [61:58] >> when was the term vibe coding invented? [62:00] When did that first come into [62:01] prevalence? [62:05] >> I know the answer to that, but I'm [62:07] curious what people [62:17] >> It was last February. Yeah. So, a year [62:19] and two months, right? Feels like 10 [62:22] years. [62:24] Especially [laughter] [62:25] if you're on the other end of that. Yes. [62:28] >> Yeah. Yeah. So, yes. To your point, like [62:30] these things, like if you were spelling [62:32] strawberry wrong or every hilarious joke [62:33] that's happened in the p, you know, it's [62:37] it's fascinating. [62:38] >> I know there was one other question [62:40] here. Okay. So, uh this is from Andrew [62:43] Olsen. And if you're on Andrew, feel [62:45] free to go off uh mute and go on mic. [62:47] But it says, "If you give it all the [62:49] context, you know, you were talking [62:50] about you need to give it context so it [62:52] does things correctly. Does that cost [62:54] tokens each time it has to pass through [62:56] this? Uh, you know, to not um trying not [62:59] to penny pinch but trying to understand [63:02] if I give it all the Drupal things, is [63:03] that worth the questions I ask? [63:06] >> Yes, it is. Um, but there's two con [63:09] couple considerations there. One, if you [63:11] can swing it, get on one of those all [63:13] you can eat plans right now because then [63:15] you don't have to worry about it. like [63:17] token cost management stuff like yes you [63:19] do have to worry about that in some [63:20] instances but you're able to use one of [63:23] those and there are ways to apply for [63:25] like the cloud max account for open [63:27] source contributors and stuff like that [63:28] so that's your best way to deal with the [63:31] cost implication of that so you can [63:33] think about the more important part of [63:36] the implications of passing a lot of [63:37] context right so it is the same as [63:41] running up to a human and info dumping [63:45] on them right they're going to remember [63:46] the first thing you said, the last thing [63:47] you said, and maybe where you messed up [63:49] in the middle. Models are going to do [63:52] this. So, it is the right context at the [63:53] right time. Now, that's where the [63:55] strategy of like progressive exposure, [63:57] the skills come in. You don't want to [63:59] dump information into the model about [64:01] how to use views until it's going to use [64:03] views. So, if you have a skill that says [64:05] read this when you want to use a Drupal [64:07] view, that is useful. Um, so [64:09] >> you just pulled together a lot of things [64:11] that I've been um and a lot of the [64:13] things that were in the chat, too. Like, [64:15] oh, by the way, you're like, I use DDEV. [64:17] Uh, I'm I'm on Pantheon and it [64:19] immediately is like, oh yeah. So, I do [64:22] that already, but it was just that's [64:23] very helpful to understand how you point [64:25] it to the skills when you talk about a [64:26] thing and that just by loading up your [64:29] skills and everything. So really great [64:31] advice is like I could just spend all [64:32] day on my setup and like trying to find [64:34] all the skills and updating the skills [64:35] and going here or I could just actually [64:37] get stuff done and when it trips on [64:39] itself a few times add it. So really [64:41] appreciate the advice. Thanks [64:43] >> and and don't overoptimize like lean [64:46] towards context. Like one of the best [64:48] points of context that you can do is [64:49] like I record pretty much every single [64:51] Zoom meeting like get the transcript as [64:54] we were talking about documentation and [64:55] things like that. like get on a call, [64:57] talk through it with people, throw all [64:59] that context at the model. They can deal [65:00] with books worth of stuff very quickly [65:02] and say, "Pull out the important [65:03] relevant things." Like we could take [65:05] this transcript here and write [65:06] documentation on how to do this. [65:09] >> Yeah, I got really depressed at Fox [65:10] Valley Drupal. I took notes the whole [65:12] time and I thought I was doing really [65:13] great and then Bob Snider emailed me the [65:16] notes and I was really depressed because [65:18] it did a better job than me and I was [65:19] like, I was paying attention. So, I [65:21] can't agree completely of how well it [65:24] does. So, thanks y'all. Perfect. [65:29] >> Are there other questions people want to [65:30] ask on mic or in the chat? [65:33] Scott, thank you so much for being so [65:35] generous with your time. I really [65:36] appreciate it. [65:36] >> Yeah, no worries. I mean, the agents are [65:38] working in the background. See, that's [65:40] the thing. If you do this right, you can [65:41] screw around during your day job and [65:43] just let the agents work in the [65:44] background. [65:46] >> Yeah, it's funny. My my first time doing [65:48] this was was around February. I guess it [65:50] was April of last year when Vibe Coding [65:52] and I wanted to play around with it. So, [65:54] um, I had like a long-standing gripe, [65:56] which is like, um, conference CFPs are a [65:58] thing my my team needs to know about and [66:00] they're all over the place and there's [66:02] no, you know, I was like, I want to [66:03] write an agent that like goes out and [66:05] finds all these, filters it by our [66:07] target audiences, then posts them to [66:08] Slack, you know. So, but it was myself [66:10] and and uh Tara King if if folks [66:12] remember them from the uh Drupal um uh [66:15] diversity initiative and and other [66:17] things, but like we so we were in a [66:18] group together and it was really it it [66:20] honestly was like here this is what I [66:22] want you to go go and this is back in [66:24] like the cursor IDE days and then we [66:26] would just like chat like it's like and [66:28] we're like yeah how's your day going? [66:30] Oh, my day is great. He's like oh wait [66:32] it's ready for us to look. Oh it says it [66:34] wrote a Flask application. Cool. I don't [66:36] know anything about that. Uh, all right. [66:38] Tell me how to test it. Oh, go to this [66:40] URL. That URL is broken. Oops, you're so [66:43] right. Chunka just, [laughter] you know [66:45] what I mean? So, it was like screwing [66:47] things up constantly, but it was just [66:49] like, give it back to the agent to fix [66:51] it then. You know what I mean? And we're [66:52] just like chatting. [66:53] >> Um, [66:55] >> yeah, absolutely. Exactly. That reminds [66:57] me another question that uh Mike Hershel [66:59] asked when before he dropped which is [67:01] you mentioned way earlier in your [67:03] presentation that you used this agentic [67:06] loop thing that you're talking about in [67:08] order to do some kind of a major like [67:12] exorcism or whatever I don't remember [67:13] was like porting something to something [67:15] else. [67:16] >> Uh what was it again? What was the use [67:17] case? Yeah, I I I took the the Umami [67:20] theme um and ported it to Canvas and [67:24] Drupal CMS and it ran for 48 hours and [67:27] so his question there was how much did [67:28] that cost? [67:29] >> Yeah, that was his question. [67:31] >> So um [laughter] [67:33] what cost to us personally to the earth [67:36] to the financially because it used about [67:39] a billion tokens, right? It ran for 48 [67:41] hours. Um, now then this was on one of [67:44] my all you can eat plans. So financial [67:46] cost to me zero besides electricity. I [67:49] paid for that to happen. I ran the [67:52] numbers. Had I ran that through a just [67:55] the API stuff and paid for it probably [67:57] would have been $10 to $15,000 [68:00] um [68:00] >> to do that. So that's why. So be careful [68:02] with some of these if you're paying for [68:04] the credits like get on one of these [68:05] plans. Um now that again because I am on [68:09] that all you can eat model just I don't [68:11] worry about using a lesser model. I just [68:13] highest thinking highest model it [68:15] doesn't matter let it go. So [68:20] >> for reference though like when we fix a [68:22] typo in Drupal core right that's going [68:25] to be someone to have found the typo [68:28] someone to have written the issue about [68:30] the typo. Someone to have uploaded a [68:32] patch about the typo and market needs [68:34] review. someone else to look at the [68:35] typoix and say, "Yep, that looks good in [68:37] market RTBC." And then a fifth core [68:39] committer to go in. Right. So it's like [68:41] these are not free either. They just [68:44] have different costs associated with [68:46] them. [68:46] >> Yep. Exactly. And then like within a [68:48] enterprise like the manager to manage [68:50] those people like there's just so much [68:53] depth in this that we spend. Um and but [68:56] this is also leads to a thing like I I [68:58] don't think the result of this is going [68:59] to be like less jobs. we're going to be [69:01] busier because more stuff gets done, [69:04] right? And so like we're already seeing [69:05] this like it's just like the amount of [69:07] work we're going to end up doing. So [69:08] we're in the fun era where it's like oh [69:10] it's fun to do agents. It's not going to [69:11] be fun in a bit when it's just like wow [69:13] this is just the expectation that we [69:14] move this fast now. Um so enjoy it at [69:18] the moment. [69:21] Well, on that happy topic, [69:24] [laughter] [69:25] >> it's still fun, but the real work is [69:27] coming. [69:28] >> I will say like, you know, there are um [69:31] yeah, there there that is a very good [69:33] point, der. Yeah, it's it's um I I see [69:35] the same problem, which is uh you know, [69:38] the the senior engineers like the folks [69:40] in the Drupal community, right, who are [69:42] who've been in this forever. Um, I see [69:46] it when when you really embrace these [69:48] tools, you end up a lot like Scott where [69:49] it's like, ah, okay, I've got my agent [69:52] loop thinging. I know when it's t when [69:53] it's lying to me. I know like I I can [69:56] trust it for some things, trust it for [69:57] you because you have the judgment honed [69:58] and developed over time. But the only [70:01] way you get ju like senior Scott [70:04] Falconer kind of people is because [70:05] Scott, what was your first job in tech? [70:09] >> You want to know what my first job was? [70:11] >> Yeah, [70:12] >> Aqua. Aqua is my first REAL JOB. IS IT [70:14] REALLY? Oh my god, that's hilarious. [70:16] [laughter] [70:17] >> I've done three L. I mean, honestly, I [70:19] have a psychology degree where I barely [70:21] got a 2.0 and I took seven and a half [70:22] years to get it with no minor. Like, [70:24] that's why I'm less concerned about this [70:26] junior thing because it's like I learned [70:29] in Drupal, right? I don't understand [70:31] some of the fundamentals of computer [70:32] science that I didn't need below PHP [70:34] because they didn't matter to me. So, [70:37] yes, the new kids coming up aren't going [70:39] to understand the importance of making [70:40] sure a semicolon is there. it doesn't [70:42] matter anymore, right? So, I am not [70:44] worried about this problem because [70:46] they're going to be focused on the [70:47] bigger issues. They don't have to worry [70:49] about the things. And one of the [70:50] examples I heard about this was, you [70:51] know, someone had vibe coded something, [70:53] someone else spoke to the code and said, [70:54] "This is a mess. I can't read it." And [70:56] they said, "Who cares? Nobody's ever [70:57] going to read it." Right? It's a [70:59] fundamentally different way to think [71:01] through it. So, I'm not concerned with [71:02] the juniors. I think they'll they'll be [71:03] fine. The kids will be great. [71:05] >> I would love to put you in a room with [71:06] our engineers. [laughter] [71:08] Yeah. [71:08] >> Do it. Let's do it. Roterdam. [71:12] >> Perfect. I'm gonna make I'm literally [71:14] gonna make all of them come over. [71:16] >> Okay, sounds good. [71:17] >> That's [clears throat] great. I love [71:19] making connections happen. [laughter] [71:22] Um, all right. Amber just posted again. [71:24] We have a feedback form. I'd love so [71:26] much if people could take the time to [71:27] fill that out so we know how we can do [71:29] uh what other topics you're interested [71:31] in, how we can do this better. Scott, [71:32] thank you. This was so great. uh really [71:35] appreciate you taking the time to not [71:37] only uh dedicate time with the session, [71:39] stay over to answer more questions and [71:40] and the way you've framed this up for [71:42] folks, I think was was a great way to [71:44] kind of show them like a a progressive [71:46] way to get to, you know, your your your [71:48] standard of coding or whatever. Um and [71:51] yeah, [laughter] I love [71:52] >> whatever standard that is. Yeah. [71:54] >> Yeah. Well, you know, yellow. Uh no. Um, [71:57] but no, just thank you very much and [71:58] thank you everyone for your thoughtful [72:00] questions and thank you to uh to Amber [72:02] for for for um personing the chat. I [72:05] really appreciate that too. [72:06] >> Yeah, thank you all. So, [72:09] >> all right, have a good rest of your day [72:11] everyone. Uh, I'll get the recording [72:13] uploaded sometime, hopefully tonight. [72:16] Uh, we'll we'll see how that goes. So, [72:18] um, okay. Bye, everybody.  📄 **Full text:** ![[Full Document Contents/Articles/Drupal AI Learners Club From Autocomplete to Autopilot.md|Drupal AI Learners Club: From Autocomplete to Autopilot by YouTube]] ## Highlights - Hello everybody and welcome to the uh uh um April 14th edition of our Drupal AI learners club. Today we have the topic um from autocomplete to autopilot uh where Scott Falconer has kind of prepped some materials at our last uh meetup which was just last week. So thank you Scott for for offering to help so quickly. It's nice to keep the momentum up on these. Um what it was kind of brought up is like that was great but that was a lot of like very very senior like core developery kind of people talking about their workflows and uh ([View Highlight](https://read.readwise.io/read/01kp8jmmf4gvm3ffkcvanz6rv2)) - Tags: 👻 ai highlighted - what you have planned for today >> I I think that's what we're going to talk about today yeah so it what it will kind of focus on is the assumption that you've all used something like chat GPT and the assumption that you've potentially done some sort of autocomplete style stuff and we're going to kind of bridge that gap between that and the using agent. So that's the kind of framing we'll have. >> Well, let's jump in. Um, but before we jump in, we do this thing where we do the ground rules. Ground rules are we ([View Highlight](https://read.readwise.io/read/01kp8jmmfdc0zj8py8jn6mjkd1)) - Tags: 👻 ai highlighted - um, great. All right. Um, let's go ahead and jump back to it then. Thank you so much for your extra additions and and uh things of that nature. So, uh, showand tell time and I'm going to kick it off to Scott to talk about from autocomplete to autopilot, a practical guide to AI assisted coding or at least that's what Claude made the session called when he described it to me as a primer of styles uh on styles of AI assisted coding and moving from IDE copilot style to agent libs. So, >> Claude also wrote the session. So, like ([View Highlight](https://read.readwise.io/read/01kp8jmmgyb6bkwsnww3hnhwwh)) - Tags: 👻 ai highlighted - here doing stuff with AI related to coding but that can mean a lot of things right and so I'm assuming that most of you have gone into chat GPT Gemini claude and asked it to help debug some stuff potentially spun up something like co-pilot or cursor but what we're going to start from there is we're going to take the next step to what it means to developing with an agent and specifically what are called agent loops. And so, you know, it's kind of a balance here. So, jump in there. It ([View Highlight](https://read.readwise.io/read/01kp8jmmjevpaapeey38fxkwdn)) - Tags: 👻 ai highlighted - stuff. I mean, it gets wild as soon as you start running agents. People eventually be like, "Oh, we need an interface for this agent that looks like Sim City because I'm running 200 agents." Very cool stuff. But if your focus is on like getting things done, try not to get tempted because it's very easy in this ecosystem. >> So, two definitions. >> Can I pause you? Can I pause you for one second because that seems like a good scoop. Yeah. So, uh first, uh you know, meeting people where they are. I think a lot of folks who have only interfaced with AI through a chatbot kind of thing ([View Highlight](https://read.readwise.io/read/01kp8jmmmrjwz6md5y3nycv7cb)) - Tags: 👻 ai highlighted - tools, right? And so this is where the scary part comes in because this is running on your machine. It can potentially delete directories, move things around, take actions um and it will follow multiple steps. So that's what we're going to talk about in this context. And that's probably the big fundamental shift of using a model to help you do autocomplete or using a model directly in like chatgp.com versus an agent that actually starts to take these actions. And that is starting ([View Highlight](https://read.readwise.io/read/01kp8jmmrq5gp3jfhvs69kr7vc)) - Tags: 👻 ai highlighted - what we're going to talk about today is this concept of agents and agents loop. And so this is where it has shifted from just being an assistant that kind of answers questions and looks at your code to one that can answer question and look at your code and take action and execute multiple steps and multiple systems. So everywhere you move up the ladder there here changes the unit of work you're doing. Now, I tend to still use most of these back and forth, and if we have ([View Highlight](https://read.readwise.io/read/01kp8jmmv4t053hv1n5my2y6ed)) - Tags: 👻 ai highlighted - a playbook like you would for any kind of junior dev in there. So >> great is the agents MD file you use for Drupal core development something you can share. >> Absolutely. Um this is a perfect plug for the what do you do? Do you want to talk about the uh project you started which was the Drupal? >> Yeah. Well, I I can plug that, but it's still very recent. But uh yeah, so there's a project uh called Drupal A Drupal AI best practices, which is ([View Highlight](https://read.readwise.io/read/01kp8jmn9kvjc1r0ajgkk7mc6c)) - Tags: 👻 ai highlighted - linked in the chat there. Um and uh it is not much yet, but what it is uh yeah, I guess I could I could demo that real quick. I don't want to take your your your awesome time away, but um it would be especially helpful if I could figure out what tab that's in. There we go. Uh so this project was something we spun up at Drupalcon be uh to try to make agents less stupid basically about Drupal. So um and it's it's uh there's not much there but if you pop into the source ([View Highlight](https://read.readwise.io/read/01kp8jmnpt705cgxgh1a5wh0wz)) - Tags: 👻 ai highlighted - over time. So now it is it still requires manual work, but it's like she can trust most of it most of the time. But you know, again, it's it's one of those things where a lot of times people try one thing with AI and it's garbage and then they're like, "Well, this is stupid." You know what I mean? And a and a compelling thing about Scott's demo is or in your preamble, you know, Scott, you were talking about like this ladder, right, where we were in chat bots or auto assisted or autocomplete. So it's like autocomplete, then we were in chat bots, then we were in like an IDE situation, and then we were in agents ([View Highlight](https://read.readwise.io/read/01kp8jmpet8jv0d23d655cg8mk)) - Tags: 👻 ai highlighted