Research: Architecting Tools for AI Agents at Scale  🔗 **Source:** [Research: Architecting Tools for AI Agents at Scale](https://gziolo.pl/2026/04/09/research-architecting-tools-for-ai-agents-at-scale/) > [!note]+ Document Notes > tags: #ai-agents #tool-architecture #progressive-disclosure #wordpress #mcp #performance-optimization #developer-experience > author: Greg Ziółkowski > source: Grzegorz Ziółkowski Blog > reference: https://gziolo.pl/2026/04/09/research-architecting-tools-for-ai-agents-at-scale/ > date created: 2024-12-19 14:30 > > # 🏷️ Title: Research: Architecting Tools for AI Agents at Scale > <!-- deep-read:sections start --> ## 🧰 Action Inbox - [H1] Implement abilities filtering with category/namespace support — WordPress 7.1 roadmap #wordpress - [H2] Design tool consolidation patterns for WordPress ecosystem — reduce CRUD operation sprawl #api-design - [H3] Evaluate progressive disclosure for WordPress Abilities API — prevent context window overload #ai-integration ## 🧭 Table of Contents - [[#📝 Summary]] - [[#📖 Main Content]] - [[#🗒️ Notes]] - [[#📚 References]] - [[#🗣️ Paraphrase]] - [[#📰 Newsletter]] - [[#✅ Action Items]] - [[#🧪 Hypotheses]] - [[#📊 Methodology]] - [[#❓ Questions]] - [[#🔮 Future Research]] - [[#💡 Implications]] - [[#🧠 Learning Styles Assessment]] - [[#📊 Accessibility Assessment]] - [[#🌍 DEIB Assessment]] - [[#✅ Logical Validity]] - [[#🛡️ DARVO Analysis]] - [[#🎯 Conclusion]] ### 🔗 References - [Anthropic Advanced Tool Use Blog](https://www.anthropic.com/news/advanced-tool-use) - [WordPress Core AI Repository](https://github.com/WordPress/ai) - [MCP Protocol Documentation](https://modelcontextprotocol.io/) ## 📝 Summary - **What happened**: Research analysis of AI agent tool architecture at scale, focusing on progressive tool exposure patterns - **Why it matters**: Loading all tools into LLM context simultaneously causes dramatic accuracy degradation and token waste - **Key stakeholders**: WordPress Core AI team, plugin developers, AI integration teams - **Risks and tradeoffs**: Progressive disclosure adds latency but improves accuracy; consolidation saves context but can hurt parameter clarity - **Bottom line**: Tool architecture is a primary determinant of agent reliability at scale, requiring intentional design rather than endpoint mirroring ## 📖 Main Content The core problem is quantifiable: Anthropic reports that typical 5-server MCP setups consume ~55,000 tokens in tool definitions before any conversation begins, with internal observations reaching 134K tokens. At this scale, Claude Opus 4 achieved only 49% accuracy on tool selection. Three strategies emerge from production evidence: **Strategy 1: Design fewer, better tools** through consolidation patterns like STRAP (Single Tool Resource Action Pattern) and the Six-Tool Pattern, which can reduce 96 tools to 10 domain tools with 80% less context overhead. **Strategy 2: Progressive tool exposure** using static filtering, dynamic discovery, or code-as-tools approaches. Anthropic's Tool Search pattern shows 85% token reduction (77K → 8.7K) and accuracy improvements from 49% to 74%. **Strategy 3: Infrastructure-level filtering** through gateways and routing systems for multi-server environments. For WordPress specifically, the research maps to current Core AI work: abilities filtering (Trac #64990), orientation abilities for WordPress 7.1, and consolidation guidance for the plugin ecosystem. ## 🗒️ Notes #### ⏱️ [ARTICLE] — [H1] Tool overload causes measurable accuracy degradation - Key idea: Individual tool accuracy can be 96% but drops to under 15% in large-toolset scenarios - Evidence: "Anthropic observed tool definitions consuming 134K tokens before optimization. At that scale, Opus 4 achieved only 49% accuracy on tool selection" #### ⏱️ [ARTICLE] — [H2] Three error categories increase with tool count - Key idea: Tool interference, parameter hallucination, and incorrect selection all worsen with scale - Evidence: "Three categories of errors increase with tool count: incorrect tool selection, parameter hallucination, and tool interference" #### ⏱️ [ARTICLE] — [H1] STRAP pattern reduces tool sprawl through consolidation - Key idea: Single tool per domain with resource/action parameters instead of one-tool-per-CRUD-operation - Evidence: "Applied to the Outlet email platform, this reportedly reduced 96 tools to 10 domain tools with ~80% less context overhead" #### ⏱️ [ARTICLE] — [H2] Progressive disclosure shows dramatic improvements - Key idea: Anthropic's Tool Search enables 85% token reduction with significant accuracy gains - Evidence: "Results: 85% token reduction (77K → 8.7K) and accuracy improving from 49% to 74% (Opus 4) and 79.5% to 88.1% (Opus 4.5)" #### ⏱️ [ARTICLE] — [H1] WordPress ecosystem faces scaling challenges - Key idea: Plugin ecosystem growth creates tool explosion without individual wrongdoing - Evidence: "WooCommerce adds 30 more, Jetpack adds 20, a forms plugin adds 10, a SEO plugin adds 10, and suddenly a single site has 100+ abilities" #### ⏱️ [ARTICLE] — [H2] Cloudflare's radical approach eliminates tool definitions - Key idea: Two tools (search and execute) accepting JavaScript code achieve 99.9% token reduction - Evidence: "Their API has 2,594 endpoints...Token reduction: 99.9% (from 1,170,523 to 1,069 tokens)" #### ⏱️ [ARTICLE] — [H3] Tool description quality has outsized impact - Key idea: Improving tool descriptions alone can dramatically improve accuracy - Evidence: "Anthropic's internal testing confirms the payoff: adding tool use examples improved accuracy from 72% to 90%" #### ⏱️ [ARTICLE] — [H1] Production servers avoid full API exposure - Key idea: Major platforms use consolidation and filtering rather than exposing all endpoints - Evidence: "Among the servers surveyed, none exposes its full API surface as individual MCP tools" #### ⏱️ [ARTICLE] — [H2] WordPress 7.1 orientation abilities enable progressive disclosure - Key idea: Always-loaded foundation tools let agents discover task-specific abilities on demand - Evidence: "Orientation abilities...are the always-loaded foundation that makes progressive disclosure work" #### ⏱️ [ARTICLE] — [H3] Practical thresholds guide architectural decisions - Key idea: Clear tool count thresholds indicate when intervention is needed - Evidence: "1-5 tools is generally safe...10+ tools is where measurable degradation starts requiring architectural intervention" ## 📚 References - [Anthropic Advanced Tool Use Blog](https://www.anthropic.com/news/advanced-tool-use) - [TaskBench Paper (NeurIPS 2024)](https://neurips.cc/virtual/2024/poster/93508) - [WordPress Core AI Repository](https://github.com/WordPress/ai) - [MCP Protocol Specification](https://modelcontextprotocol.io/) ## 🗣️ Paraphrase - Loading all available tools into AI agent context simultaneously creates a scaling crisis where accuracy degrades dramatically as tool count increases, with measurable drops starting around 10+ tools - Three complementary strategies address this: designing fewer but more powerful consolidated tools, progressively exposing tools based on context and need, and implementing infrastructure-level filtering for multi-server environments - WordPress Core AI work should prioritize abilities filtering, orientation abilities for progressive disclosure, and ecosystem guidance to prevent the natural plugin sprawl from creating unusable tool surfaces ## 📰 Newsletter **AI Agent Tool Architecture: Why Progressive Disclosure Matters for WordPress** If you're building AI integrations, here's a critical insight: loading all your tools into the LLM context at once isn't just inefficient—it's actively harmful to accuracy. Research shows that tool selection accuracy can drop from 96% with individual tools to under 15% in large-toolset scenarios. The solution isn't bigger context windows. It's progressive tool exposure: showing agents only the tools relevant to each interaction. WordPress Core AI is already implementing this through abilities filtering and orientation abilities in WordPress 7.1. **Key takeaway**: As the plugin ecosystem grows, WordPress sites could easily have 100+ AI abilities without any individual plugin doing anything wrong. The architecture needs to stay ahead of that curve. **Practical next steps**: Design consolidated abilities where the data model supports it (like core/create-content with post_type parameter instead of separate abilities per post type), implement static filtering based on user context, and invest heavily in tool description quality. Resources: [WordPress Core AI](https://github.com/WordPress/ai), [MCP Protocol](https://modelcontextprotocol.io/), [Anthropic Tool Use Guide](https://www.anthropic.com/news/advanced-tool-use) ## ✅ Action Items #### WordPress - Core AI: Implement abilities filtering with category/namespace/meta support (Trac #64990) - Core AI: Design orientation abilities for WordPress 7.1 (core/get-site-info consolidation) - Core AI: Create consolidation guidance for plugin ecosystem to prevent tool sprawl - Core AI: Build evaluation suites for tool selection accuracy across WordPress workflows - Core AI: Implement JSON Schema normalization (Trac #64955) for agent reliability - Core AI: Design execution lifecycle filters (Trac #64989) for progressive disclosure safety #### OSPO - Developer Support: Create tool architecture training for AI integration teams - Community Engagement: Document progressive disclosure patterns for WordPress ecosystem - Strategic Alignment: Track tool count and context consumption metrics across WordPress AI implementations #### DevRel - Education: Workshop on AI tool consolidation patterns for plugin developers - Developer Experience: Create documentation for STRAP and Six-Tool patterns in WordPress context - Developer Marketing: Content series on scaling AI agent architectures ## 🧪 Hypotheses - WordPress sites with 10+ active plugins will exceed optimal tool count thresholds for AI agent accuracy without architectural intervention - Consolidating WordPress CRUD operations by post type will improve parameter accuracy more than the increased schema complexity hurts it - Progressive tool disclosure will reduce average context consumption by 60-80% in typical WordPress AI workflows while maintaining or improving task completion rates - Tool description optimization will have higher ROI than architectural changes for WordPress abilities with fewer than 15 tools ### 📊 Methodology - A/B testing of consolidated vs. individual abilities across representative WordPress workflows - Longitudinal analysis of plugin ecosystem tool registration patterns to predict scaling bottlenecks - Benchmark comparison of WordPress AI task completion rates with and without progressive disclosure - User study of developer comprehension and accuracy with different tool description formats ### ❓ Questions - How should WordPress Core AI balance consolidation benefits against schema complexity for mixed-shape operations like settings vs. content management? - What's the optimal threshold for transitioning from static filtering to dynamic tool discovery in WordPress environments? - How can the WordPress plugin ecosystem coordinate on tool architecture standards without central control? ### 🔮 Future Research - Long-term tracking of WordPress plugin AI ability registration patterns and ecosystem tool count growth - Cross-platform comparison of tool architecture approaches between WordPress, Shopify, and other extensible platforms - Development of automated tool consolidation recommendations based on usage patterns and schema similarity ### 💡 Implications - **Practice**: WordPress AI integrators need progressive disclosure architecture from the start, not as a retrofit - **Policy**: WordPress Core AI should establish tool count guidelines and consolidation patterns for the ecosystem - **Security**: Dynamic tool loading requires robust validation at the execution lifecycle filter layer - **Education**: Plugin developers need training on AI-friendly tool design patterns beyond REST endpoint mirroring - **Funding**: Investment in tool architecture evaluation infrastructure will determine long-term WordPress AI ecosystem health ### 🧠 Learning Styles Assessment Reading level estimate: 12th grade (technical content with specialized terminology) #### 👀 Visual learners - Present: Data tables comparing tool counts across platforms, token consumption charts - Improve: Architecture diagrams showing progressive disclosure flows, tool consolidation before/after visuals #### 🔊 Aural learners - Present: None identified - Improve: 15-minute podcast summary of key findings, recorded presentation on WordPress implications #### 🗣️ Verbal learners - Present: Clear definitions of STRAP, Six-Tool Pattern, MCP terminology - Improve: Analogies comparing tool overload to cognitive load, storytelling around production challenges #### 🧩 Kinesthetic learners - Present: Code examples from GitHub MCP server, Cloudflare implementation - Improve: Hands-on lab building progressive disclosure for WordPress abilities, interactive tool count calculator #### 🧠 Logical learners - Present: Decision framework, practical thresholds (1-5 tools safe, 10+ requires intervention) - Improve: Flowchart for choosing consolidation vs. progressive disclosure, ROI calculations for different approaches #### 🧑‍🤝‍🧑 Social learners - Present: Community examples from MCP ecosystem, production server analysis - Improve: Discussion forum for WordPress AI implementers, peer review of tool architecture decisions #### 🧘‍♀️ Solitary learners - Present: Self-assessment questions about tool count thresholds - Improve: Reflection prompts on current AI integration approach, personal action planning worksheet #### 🌿 Naturalistic learners - Present: Ecosystem growth patterns, natural consolidation boundaries - Improve: Metaphors from biological systems about information processing limits ### 📊 Accessibility Assessment #### Screen reader semantics - Present: Good heading structure, clear section organization - Improve: Add ARIA labels for complex data tables, ensure all charts have text alternatives #### Keyboard navigation - Present: Standard web navigation available - Improve: Skip links to major sections, keyboard shortcuts for technical reference sections #### Contrast and readability - Present: Standard text contrast appears adequate - Improve: Verify all code examples meet 4.5:1 contrast, check table cell contrast #### Text resizing and reflow - Present: Responsive layout supports text scaling - Improve: Test complex tables at 200% zoom for horizontal scroll issues #### Media and captions - Present: No audio/video content identified - Improve: Future podcast recommendations need full transcripts #### Forms and interactions - Present: Basic comment form only - Improve: Ensure any interactive elements have proper labels and error handling ### 🌍 DEIB Assessment #### Cultural breadth - Present: Global examples from Anthropic, Cloudflare, GitHub - Improve: Include perspectives from non-Western AI research, multilingual tool description examples #### Inclusive language - Present: Technical but generally accessible terminology - Improve: Define more jargon in-context, provide plain language summary section #### Neurodiversity support - Present: Well-structured with clear headings and logical flow - Improve: Add summary boxes for key takeaways, break up longest paragraphs #### Representation - Present: Technical focus limits representation opportunities - Improve: Include diverse voices in future case studies, quote practitioners from varied backgrounds #### Psychological safety - Present: Professional, non-judgmental tone about technical decisions - Improve: Acknowledge that current approaches aren't wrong, just need evolution for scale ### ✅ Logical Validity #### Strong reasoning - **Deductive strength**: Strong - clearly links empirical evidence (Anthropic data) to general principles about tool architecture - **Inductive strength**: Good - generalizes from multiple production examples (GitHub, Cloudflare, Shopify) to broader patterns - **Causal reasoning**: Well-supported - establishes mechanism (context window limits) linking tool count to accuracy degradation - **Analogical strength**: Appropriate - WordPress/plugin ecosystem analogies to other platform scaling challenges #### Fallacies - **Appeal to authority**: Minimal risk - cites specific data rather than just expert opinion, though some community examples lack peer review - **Hasty generalization**: Low risk - uses multiple sources and acknowledges limitations of practitioner case studies vs. peer-reviewed research #### Tone of voice - **Overall tone**: Authoritative but measured, technical but accessible, acknowledges uncertainties while making clear recommendations ### 🛡️ DARVO Analysis - **Deny**: No denial patterns detected - acknowledges current approaches and challenges directly - **Attack**: No attacking behavior - focuses on technical solutions rather than blaming individuals or teams - **Reverse victim/offender**: Not applicable to technical content - **Emotional cues and rhetoric**: Professional and solution-focused throughout ## 🎯 Conclusion Progressive tool exposure is not optional at scale but a primary determinant of AI agent reliability. The research establishes that successful AI integrations invest heavily in tool architecture rather than simply exposing API endpoints. For WordPress, this means implementing abilities filtering, designing orientation abilities for progressive disclosure, and establishing consolidation guidance for the ecosystem before plugin tool sprawl creates unusable AI surfaces. The most actionable insight is Anthropic's "3-5 always loaded, defer the rest" heur <!-- deep-read:sections end --> 📄 **Full text:** ![[Full Document Contents/Articles/Research Architecting Tools for AI Agents at Scale|Research: Architecting Tools for AI Agents at Scale, gziolo.pl]] ## Highlights - The core issue is not theoretical. Anthropic’s [Advanced Tool Use](https://www.anthropic.com/engineering/advanced-tool-use) blog (November 2025) provided concrete numbers: a typical 5-server MCP setup consumes **~55,000 tokens in tool definitions alone** before any conversation begins. Internally, Anthropic observed tool definitions consuming **134K tokens** before optimization. At that scale, Opus 4 achieved only **49% accuracy** on tool selection in Anthropic’s internal evaluation. ([View Highlight](https://read.readwise.io/read/01knwvd6jatnezrtwzt2ews62a)) - Tags: 👻 ai highlighted - **Consolidation guidance for the ecosystem** is a gap. Core can lead by example with content-type-based abilities (`core/create-content` with `post_type`), but plugin authors need clear patterns for when to consolidate and when to register separate abilities. The path of least resistance is mirroring REST API endpoints as abilities, the natural thinking we already see in the plugin ecosystem, and exactly how tool definitions balloon beyond what any context window can sustain. The right defaults and documentation can steer the ecosystem toward fewer, well-scoped abilities rather than thin wrappers around every endpoint. ([View Highlight](https://read.readwise.io/read/01knwvd6mfbr7vn83t3b2hptx5)) - Tags: 👻 ai highlighted