How Agentgateway Solves Agentic AI’s Connectivity Challenges  ## Metadata - Author: [[The New Stack Podcast]] - Full Title: How Agentgateway Solves Agentic AI’s Connectivity Challenges - Category: #podcasts - URL: https://share.snipd.com/episode/7887c8db-811c-41de-be87-135c485f0d81 ## Document Notes ## Highlights - **Agentgateway Bridges Agentic Connectivity** - Agentgateway provides security, governance, and observability for agent-agent, agent-tool, and agent-model traffic. - It natively understands MCP and A2A while also supporting traditional microservices and Kubernetes Gateway API. Transcript: Lin Sun The new project we launched, Asian Gateway, which we recently, yesterday, I think at nine o'clock, we donated to the Linux Foundation. It's accepted by the Linux Foundation. It's the first and only Asian Gateway project, open source project on the market. So Asian Gateway essentially is applying security, provides security, governance, and observability to agent-to agent-to tools, agent-to model communication. So we believe as part of everybody's agentic journey, everybody starts with either MCP tools or AI agents, right? But these agents need to communicate to each other. The MCP tools needs to be called by some agents, so multiple agents. And the agents also need to communicate to large language model. And we also believe nobody's going to run agentic AI without any traditional microservices. So it's going to be a mixed environment. Not only you have agents, you have a large language model, you have MCP server tools, you have traditional classical microservices. That's why Agent Gateway was born, to help solving connectivity challenges that can natively understand MCP and A2A, some of the leading agentic AI protocols, but can also understand Traditional microservices, can support Kubernetes Gateway API, which is the future of networking API. In Kubernetes, it also supports inference gateway. In fact, we just passed not only the Kubernetes gateway API conformance test, but also the inference gateway extension conformance test for agent gateway. ([Time 0:02:40](https://share.snipd.com/snip/d9eecc33-835a-4ba1-973d-ef3c4a72b94c)) - **Control Tool Exposure Through A Gateway** - Use a gateway to enforce fine-grained policies and resilience between agents and MCP servers instead of exposing everything directly. - Limit which MCP tools each agent can access to expose only tested or approved capabilities. Transcript: Lin Sun Got it. Frédéric Lardinois Got it. Now, going back to the agent gateway, I guess the question is, why do I need it? Right? Because I have A2A, I have MCP. Why have a middleman? Lin Sun Yeah, that's a good question. So you're absolutely right. A2A provides the standard of communication protocol between agents to agent, right? The MCP tool is like a USB-C driver or adapter for the tools, right? But there's no governance in between agents to agents. There's no security policy enforcement between your agents to MCP server tools. There's no, like balance or resilience. Some of the things we needed such as to provide communication media, the communication between microservices, we believe we're going to need the same thing plus more, right? Because not only we need traditional security policy, observability, and governance and compliance between these agents-to and agents-to server tool, we also need the gateway To be able to natively understand MCP server protocol, understanding A2A protocol. Let me give you an example, right? Traditionally, when we put a proxy between microservices, we use that proxy to mediate the traffic, like route to different versions of the microservice to do low balance, retry, Resilience magic, to be able to apply security policy to say, hey, this microservice can only reach out to maybe a GET request on this particular path, right? With MCP Server 2, you are no longer dealing, necessarily dealing with like a slash, a particular path or GET or post method, right? Because if you look at an MCP Server 2, typically, for example, the GitHub MCP server tool I'm playing with, it has like 20 plus, I think somewhere between 20 to 50 tools available. But you don't necessarily want to expose all the tools available to your agents. Some of the tools you may not want to expose to them because you haven't tested yourself enough to be able to release to them or some of the tools you don't you just don't want them to use It at all. Sure sure. So having the gateway in between your agents to your MCP server you can provide the policy and governance control to say hey maybe out of these 20 plus tools, I'm going to expose only five Of them to my AI agents. And also I could potentially configure this, maybe for this particular AI agent, I'm going to only expose one out of the 20 plus, but for this particular AI agent, I'm going to expose Maybe five. ([Time 0:05:20](https://share.snipd.com/snip/dc55e2f6-71f8-430c-bfc3-9b8b001440ff)) - **Agent Protocols Need More Than API Gateways** - MCP and A2A protocols are context-sensitive and differ from typical HTTP API patterns like path-based routing. - Agentgateway combines API-gateway features with agent-aware protocol handling to address those differences. Transcript: Lin Sun Yeah, I think you're absolutely right. It does sounds like the API gateway, but I feel like there's more to that, right? Because the MCP protocol is, I believe, is JSON-RPC based, and it's also very context sensitive, the communication between agents and MCP protocols, MCP And you can also have multiplex, One agent gateway connecting to multiple MCP server and tools. So it does have unique characteristic that's different from the traditional workloads behind API gateway. That's why I think a way to think about it is the agent gateway. You can think about it as traditional API gateway plus the AI agents components, right? Which is also why the agent gateway not only implements support for MCP and A2A, but also implements the support for traditional microservices. Like we implemented HTTP route, GRPC route, TCP route in agent gateway implementation in KGateway. ([Time 0:09:02](https://share.snipd.com/snip/bf253c9d-e2a8-47e4-b148-3fbef080a773)) - **Agentgateway Was Born From A Production Pain** - Solo.io built Agentgateway from internal needs while deploying Kagent to scale internal expertise into AI agents. - Running agents in production revealed gaps in security, compliance, and governance that Agentgateway addresses. Transcript: Lin Sun K-Agent, for those of you who are not know, it's a project we were inspired from Hurricane Helena last year and took one of our customers down in production, an insurance company, not Too surprising. And we were sitting on the car to help troubleshooting production outages. It's not a fun story, but what we learned in the hard way is we have very limited expertise in the company and we want to be able to scale out fast some of our expertise within the company To have our best engineer to not actually help our support engineer, but having their knowledge baked into AI agents to help them. So as we were developing agents, running these agents internally, we find out we need security, we need compliance, we need governance between agents to MCP server and agents to agents. So we are adopting this inside of Solo. We were also talking to some of the customers. In fact, I believe T-Mobile was one of them who giving us a quote. UBS was also one of them giving us a supporting quote when we were able to launch the agent gateway. So some of the users in the banking industry, you know, they are very interested in using identity AI to solve their challenges. Some of the challenges we see is how do we scale out platform engineers, right? With AI, right coding, you know, there's going to be more applications to be born, but how do you scale out your platform engineer to be able to accommodate more application, more workables? So that's real challenging, which is what K-agent is trying to solve. But one of the challenges with K-agents is you don't have a way to mediate that traffic between agents to agent and agent to MCP server tool. That's why Agent Gateway can come in. ([Time 0:10:51](https://share.snipd.com/snip/3136b4bf-59c8-43c8-8d47-8fca04f5fddc)) - **Broad Industry And Community Adoption** - Major cloud vendors and community projects are contributing to Agentgateway, showing cross-industry interest. - Contributions include Windows support and AWS Bedrock examples, and community initiatives plan to adopt Agentgateway for agent mediation. Transcript: Lin Sun But I think it's solving with like a medium-sized PR. I believe it's just got merged. AWS added some Bedrock examples for Asian Gateway. So they've been contributing to Asian Gateway. There's an initiative called Cloud Native Operation Excellence, CMOE, which is the name of, it's kind of like a genetic AI for platform engineer it's a initiative just found by Cisco AWS and a couple of others in the community and they are looking adopting agent gateway to help mediate that communication challenge between multiple agents and agents to MCP server Tools yeah so we've seen quite a lot of interesting contributions from different community members. We're super excited. ([Time 0:14:37](https://share.snipd.com/snip/98d678f8-cd8c-4101-bfe9-66ec49f86882)) - **Choose Modern Tools To Avoid Rewrites** - Build new agentic proxies in languages suited for modern protocols rather than retrofitting older proxies. - Choosing Rust avoided lengthy re-architecting and enabled faster development for Agentgateway. Transcript: Lin Sun I mean, it was created almost 10 years ago and before A2A or MCP was born. So you would have to re-architect Envoy. And we had a painful lesson from the Istio Ambient community when we were based on one of the key components on Envoy. And we wasted six months on that project and we had to rewrite using Rust. So this time we decided let's not waste that six months again. And we just started Asian Gateway over South. We also using Rust. Frédéric Lardinois Always starts with Rust, I think, as part of the message. That's the credit of Envoy said. Lin Sun I think he quoted him a few years ago. He said, if I were to start Envoy today, I wouldn't start with C++. I would choose Rust, right? ([Time 0:16:30](https://share.snipd.com/snip/334074a4-56e8-4f07-b1cf-4fc118ab2962)) - **Agentic AI Raises Mesh Importance** - Agentic AI will shift how teams view service mesh, increasing demands for auditability and security controls. - Organizations will require stronger observability and human-in-the-loop oversight as agents interact with critical systems. Transcript: Lin Sun It's actually interconnected because as people looking at cloud native, they're not just looking at traditional microservices now. With agentic AI, people are looking at how to bring AI agents fitting into their cloud native architecture, right? So they're looking at what agent frameworks should I adopt and how am I going to run my agentic infrastructure? Is Kubernetes the right way? I believe so. I mean, that's why we invest the past 10 years on infrastructure-wise. So I think people are looking not just a service mesh, they are also looking at can we make the mesh not only the gateway agentic but can we also make it the mesh agentic, right? So traditionally we haven't, I mean we've seen a lot of adoption with service mesh but not as high as we had hoped, right? Because a lot of users were happy with maybe their team owns the entire Kubernetes cluster, or maybe they're just happy to have plain text. And they don't need discovery and features like that. Totally cool. But I think with Argentic AI, the game might change as the perspective for the mesh, right? Especially with the service mesh without sidecar, we've seen people more willing to adopt the service mesh once we get rid of the complexity of the sidecar. But with Argentic, there's an additional challenge. They would have to solve like the security policy control we discussed early on, right? That's a digital challenge they didn't have before. The agent-to communication, they can't trust the agents more than the microservices they wrote, right? So I do think that auditability, you know, it's going to be way more important than microservice today. They are the owner. They feel more confident so they can trust. ([Time 0:17:52](https://share.snipd.com/snip/a0f8a27d-e653-42d3-8c4c-22c62a9e68c5))