Google Cloud Next Wrap-Up  ## Metadata - Author: [[The New Stack Podcast]] - Full Title: Google Cloud Next Wrap-Up - Category: #podcasts - URL: https://share.snipd.com/episode/f072297a-ff77-4be6-b26f-b8ad21905933 ## Document Notes ## Highlights - **First-Party Agentic Frameworks Rise** - AI agent frameworks have shifted from third-party tools to first-party solutions by major model providers. - Google launching its own Agent Development Kit (ADK) marks a key milestone in this progression. Transcript: Alex Williams So one of the predictions for 2025 for me was first-party agentic frameworks. Janakiram MSV Last year, we have seen Crew AI, Autogen, Lama Index, LangChain, but these were all third parties. They didn't own the models, they didn't own the platform, they only provided the framework. But this year, we're actually seeing first-party agentic frameworks. First, it was OpenAI that launched an agent's SDK a couple of weeks ago. And now it is the turn of Google to launch something native, which is their agent SDK. So that is a big deal. And Microsoft is also following the suite with OpenAI-based agent SDK. And I'm expecting a lot from Build, which is next month. So it's a big news and a splash that Google is launching their first-party agentic framework. ([Time 0:01:35](https://share.snipd.com/snip/a086a2d2-92e4-4d05-be3f-8db5252e369f)) - **Agents Are Becoming Autonomous** - AI agents are evolving into autonomous entities capable of reasoning and action, beyond mere single-function microservices. - This represents a next step toward more autonomous AI capabilities in workflows. Transcript: Frédéric Lardinois I mean, a lot of people are still figuring out really what are agents, right? I think for a lot of people, they're thinking about agents as almost little microservices that just perform one function and not much more than that. I think what we're really getting to now with all of these frameworks and MCP on top of that is really agents that can perform actions on behalf of somebody, do some reasoning and then Perform actions based on that. I think that's really what we're getting at. Next step being some way more autonomous functions, I would think. When you're thinking about these ([Time 0:04:14](https://share.snipd.com/snip/d1da3b6d-8f5c-4b8e-8e51-a318b01995fe)) - **Google's Comprehensive Agent Ecosystem** - Google treats agents as first-class citizens with a full-stack ecosystem like model gardens and runtimes. - Their agent stack includes an agent garden, runtime, engine, SDK, and no-code environment for enterprises, showing a cohesive AI agent strategy. Transcript: Janakiram MSV If we go back to last year, Google Cloud Next, models were the big thing. The models were the first-class citizens of the Gen.AI ecosystem. What's very interesting is Google has started to treat agents as the first-class citizens of their ecosystem. Just like we had model garden for Vertex AI and then the model runtime, the search and the tools and the entire stack. Now they have an end-to stack for agents. Now they have an agent garden where they have published first party agents that you can explore and consume. And I expect third parties and customers to populate that with more and more agents. So that is like an agent garden, a registry of agents. And then there is an agent runtime, which tracks observability, model evaluation, agent evaluation, hallucination index, and all of that. That is very, very important. And then there is an agent engine, which is meant for developers to actually build end-to agents. And it is complemented by an SDK. And at the top of the stack is agent space, which is a no-code, low-code environment for enterprises. Now, if you actually look at this entire stack, it's almost parallel to the Vertex AI model ecosystem. Now, this is becoming a first-class citizen in the Vertex AI and the overall Gen AI ecosystem. And there is clarity in how Google is articulating this entire stack. There is something for enterprises, there is something for developers, there is a runtime, and there is a marketplace or a registry of multiple agents. And that is coming out as a very cohesive, very well-articulated story. ([Time 0:05:00](https://share.snipd.com/snip/26372d44-6875-4e67-b220-d449ac15f813)) - **AI Agent Tools Still Maturing** - Google's recent ADK and related tools appear rushed with limited documentation, signaling more work is needed. - The AI agent field is still maturing, especially with challenges like security and operational clarity. Transcript: Frédéric Lardinois No, I was just surprised that they didn't provide any documentation ahead of time. That also made me feel that it was rather rushed when they made the announcement because they didn't provide us with any information at the time. I think the other topic, we're talking about Vertex and all of that, it always reminds me of the early days of the cloud-native Kubernetes ecosystem. And we're just figuring out how all of these pieces fit together. And one thing that Alex and I talked to Gabe Monroy, now at Google, at the a few days ago, and he brought up security, which I think is another topic that still always feels like it's last, And people should think about that a little bit earlier. ([Time 0:07:06](https://share.snipd.com/snip/088ad464-2125-46d6-ad83-d0bccace8159)) - **Gemini AI Enhances Observability** - Google integrates Gemini AI to improve observability and debugging with tools like App Hub and Cloud Assist. - This helps enterprises zoom into relevant telemetry and perform root cause analysis effectively. Transcript: Janakiram MSV Every service of their cloud platform. This may be slightly a detour from what we are talking about, but Google has created an entire new ecosystem of applications. They launched something called ADC, an application design center, where you can interactively draw your building blocks of your cloud infrastructure and push a button to deploy. Behind the scenes, it applies a Terraform template to whatever you have drawn on the canvas and pushes that into GKE or Cloud Run, which is very interesting. And then they augmented a service called App Hub, which is like drawing a boundary to various services that are fragmented over multiple organizations, multiple folders, multiple Projects. It brings application context. That's something that they launched. And that brings us to something called application-centric observability. Because observability today is like a data lake. Anything and everything will feed into the observability data lake. And it's like finding the needle in a haystack. Right. So now what they have done is that they have drawn a boundary, a dotted line within observability data lake, which will give you like a zoomed in perspective of observability that matters To the application that you have defined in App Hub. So it is almost like zooming 2x or 3x into observability telemetry and going straight into what matters to your application. Earlier it was like all over, events from Google, platform, Control Play, and everything. But now, you can apply filters, and you can zoom into really what matters. On top of that, they brought Cloud Assist, which is Gemini Cloud Assist. Now, when you bring the logs and the observability as context to Gemini, it can help you perform root cause analysis, triage issues, and basically help an SRE get to the debugging and Remediation pretty quick. ([Time 0:08:30](https://share.snipd.com/snip/2a5ef037-44de-40d0-89bf-71b34c6ef6f7)) - **Kubernetes Adapts for AI Inference** - Kubernetes is adapting to AI workloads with new LLM-aware tools and gateways for inference at scale. - Kubernetes remains critical for running fine-tuned models and AI-driven services in production environments. Transcript: Janakiram MSV So where Kubernetes matters is inference at scale. So when you're deploying models on Kubernetes, you need to have inference. But models are not just monolithic models. So if you look at foundation models, there are two parts to any model. One is the foundation model published by someone like Mistral or Lama or Gemma by Google. And then you have fine-tuned adopters that are specific to that model. Now, serving a fine-tuned model is far more complex than serving a monolithic foundation model. What I'm actually seeing as a trend is the cloud-native ecosystem is gearing up to support the nuances of hosting, deploying, and performing inference on these fine-tuned models. For example, the API gateway project from cloud-native ecosystem is now augmented to support the LLM-aware routing and LLM-aware inference. So that's a good sign. Solo.io and others have partnered together to launch an Envoy AI gateway. That's another good sign. So the innovation that I'm seeing in the cloud-native ecosystem is centered around AI inference and making sure that Kubernetes is LLM-aware. ([Time 0:12:08](https://share.snipd.com/snip/de36272f-8186-45de-80ba-c1eb10f21c8b)) - **Kubernetes Enables Autonomous AI Agents** - Kubernetes can orchestrate AI agents as stateful microservices and enable autonomous cluster self-healing. - SRE agents close to the Kubernetes control plane can predict issues and ensure cluster health automatically. Transcript: Janakiram MSV So Kubernetes has two dimensions when it comes to AI. Number one, Kubernetes has the runtime and orchestrator for agents, which we discussed in our podcast a couple of weeks ago. Kubernetes has the ability to orchestrate anything that is granular and resource-driven. So agents are not completely new. They are microservices, which are stateful. So Kubernetes is gearing up and it is getting ready to orchestrate agents. That is one dimension. The other dimension is when you bring agents closer to Kubernetes control plane and operation side of it, Kubernetes can become self-healing and autonomous. This is applying agents to ops. And Kubernetes operations are pretty complex. Alex Williams And I get a little curious about that because LLMs are not exactly deterministic. Frédéric Lardinois That's where you need the guardrails at that point, too. Janakiram MSV But you know, the AI agentic environment or the world now has SRE agents, which are fine-tuned and which are aligned with a typical day job of an SRE. Now, when you bring one of those SRE agents to Kubernetes and you put close to the control plane, it has all eyes and ears to the telemetry data coming out of the control plane. It can predict, it can remediate, it can ensure that the desired configuration state of the cluster is always green, and that's going to make Kubernetes self-reliant, self-healing, And almost autonomous. ([Time 0:14:19](https://share.snipd.com/snip/cf09ceff-ba97-45f7-a494-8971433d233a)) - **AI Agents Create New Ecosystem Roles** - AI agents introduce new roles like agent developers, administrators, and SREs, forming a parallel ecosystem to traditional software dev. - The agent lifecycle and operational needs mirror but expand on existing software lifecycle disciplines. Transcript: Janakiram MSV Agent developers, agent operators. It's a whole new ecosystem and it has a parallel to the software lifecycle, software design environment. So I see this as a parallel new universe, which is going to create agent developers, agent administrators, agent SREs, agent ops. So we are creating a new whole ecosystem. Alex Williams Who are the people that you've been talking to that were surprising to you? Or just was it the same kind of crowd you've seen year after year? Frédéric Lardinois It's very much the same crowd that I've seen before here. I've yet to meet an agent developer who described himself or herself as that. But I haven't seen a big difference here in who is here. It's the same developer type. Obviously everybody wants to talk about AI now, not just agents. I don't think everybody's ready for agents yet. But a lot of people that I've talked to are still figuring out how to really bring AI into their companies. For a lot of companies that's still so early. We're really far ahead by talking about agents to some degree. Janakiram MSV What I mentioned was a very forward looking view. Currently developers are still learning. It's almost like cloud developers to cloud native developers. We have seen a transition. How do you learn microservices? ([Time 0:16:10](https://share.snipd.com/snip/3d74134d-2fdf-4ae7-b3b7-39339a5c6b1c))