Avoiding Cost Drift in the Age of AI - CourtneyR.dev's 2nd Brain

Avoiding Cost Drift in the Age of AI ![rw-book-cover](https://img.youtube.com/vi/Eq4NRgIkBuU/maxresdefault.jpg) 🔗 **Source:** [Avoiding Cost Drift in the Age of AI](https://www.youtube.com/watch?v=Eq4NRgIkBuU) > [!note]+ Document Notes > tags: #finops #artificial-intelligence #cloud-cost #engineering-governance #cost-optimization > author: Lead Dev > source: YouTube > reference: https://www.youtube.com/watch?v=Eq4NRgIkBuU > date created: 2025-01-16 19:30 > > # 🏷️ Title: Avoiding Cost Drift in the Age of AI >  ## 🧰 Action Inbox - [H1] Implement cost awareness as non-functional requirement — pre-production review process #cost-governance - [H2] Establish AI request policy engine with throttling and routing — runtime cost control #ai-architecture - [H3] Create FinOps scope for bounded AI spending segments — business priority alignment #enterprise-strategy ## 🧭 Table of Contents - [[#📝 Summary]] - [[#📖 Main Content]] - [[#🗒️ Notes]] - [[#📚 References]] - [[#🗣️ Paraphrase]] - [[#📰 Newsletter]] - [[#✅ Action Items]] - [[#🧪 Hypotheses]] - [[#📊 Methodology]] - [[#❓ Questions]] - [[#🔮 Future Research]] - [[#💡 Implications]] - [[#🧠 Learning Styles Assessment]] - [[#📊 Accessibility Assessment]] - [[#🌍 DEIB Assessment]] - [[#✅ Logical Validity]] - [[#🛡️ DARVO Analysis]] - [[#🎯 Conclusion]] ### 🔗 References - [State of FinOps 2026 Report](https://data.finops.org) - [FinOps Foundation](https://finops.org) - [Lead Dev Slack Community](https://leaddev.com/slack) - [FinOps Jobs Board](https://jobs.finops.org) ## 📝 Summary - FinOps has evolved beyond cloud cost management to enterprise technology value discipline covering AI, SaaS, and infrastructure decisions - 98% of FinOps teams now manage AI costs, with organizations reinvesting cloud savings into AI programs - Shift-left approach embeds cost governance in CI/CD pipelines and workload templates before deployment - AI workloads require different optimization strategies due to spiky usage patterns, GPU intensity, and experimentation-driven waste - Every AI request becomes a financial transaction requiring real-time policy enforcement and telemetry feedback loops ## 📖 Main Content FinOps has transformed from reactive cloud cost management to proactive technology value optimization that spans AI, SaaS, and infrastructure spending. The discipline now operates through "shift-left" governance embedding cost awareness into development workflows and "shift-up" strategic decision-making at executive levels. AI workloads present unique challenges with their spiky, GPU-intensive patterns and experimentation-driven waste. Organizations are implementing policy engines with real-time controls including token throttling, budget enforcement, model routing, and kill switches for runaway costs. The complexity stems from AI spending spanning multiple infrastructure types with immature billing practices from providers. The 2026 State of FinOps report shows 78% of FinOps leadership reporting to CIO offices, indicating strategic elevation beyond finance. Teams operate through thin central functions with spoke-based champion networks across engineering, product, and business units where "FinOps is everybody's job." ## 🗒️ Notes #### ⏱️ [00:00](https://youtu.be/Eq4NRgIkBuU?t=0) — [H1] Introduction and Panel Setup - Key idea: FinOps evolution discussion with industry experts from EY UK, FinOps Foundation, and Harness - Evidence: "Avoiding cost drift in the age of AI" with 45-minute panel format #### ⏱️ [03:45](https://youtu.be/Eq4NRgIkBuU?t=225) — [H2] FinOps Definitions from Practitioners - Key idea: FinOps now focuses on value where "every workload must prove its value" - Evidence: Ben Linares: "every skew must prove its value" and Abby Singh: "cost aware AI by design and not just by audit" #### ⏱️ [08:30](https://youtu.be/Eq4NRgIkBuU?t=510) — [H1] FinOps Foundation Overview and State of FinOps 2026 - Key idea: 96,000 global participants with cultural practice for technology value management - Evidence: "98% of FinOps teams are being asked to manage AI costs" and significant numbers managing SaaS #### ⏱️ [12:15](https://youtu.be/Eq4NRgIkBuU?t=735) — [H1] Shift-Left Governance Strategy - Key idea: Proactive workload-level governance with embedded policies and guardrails - Evidence: "true shift left begins at the workload level" with templates, auto-provisioning, and optimization #### ⏱️ [16:20](https://youtu.be/Eq4NRgIkBuU?t=980) — [H2] AI Workload Characteristics and Challenges - Key idea: AI workloads are "spiky and bursty, GPU heavy and expensive, experimentation driven" - Evidence: "by design it's waste" requiring different optimization approaches than traditional infrastructure #### ⏱️ [18:45](https://youtu.be/Eq4NRgIkBuU?t=1125) — [H2] Data-Driven Governance Implementation - Key idea: Data bridges control and speed through metrics like cost per PR and inference cost per transaction - Evidence: Teams using "cost delta per PR" and "AI inference cost per transaction" as differentiators #### ⏱️ [21:30](https://youtu.be/Eq4NRgIkBuU?t=1290) — [H1] Runtime AI Cost Management Patterns - Key idea: Every AI request becomes financial decision requiring request-policy-decision-execution-telemetry pattern - Evidence: "every AI request becomes a financial decision" with runtime enforcement and feedback loops #### ⏱️ [25:10](https://youtu.be/Eq4NRgIkBuU?t=1510) — [H3] Cost as Non-Functional Requirement - Key idea: Cost joins security, reliability, and performance as mandatory architecture review criteria - Evidence: "if you won't ship code which has not been tested why would you ship something which has not been priced" #### ⏱️ [28:45](https://youtu.be/Eq4NRgIkBuU?t=1725) — [H2] FinOps Organizational Placement - Key idea: 78% of FinOps leadership reports to CIO office, often adjacent to portfolio teams - Evidence: Starbucks example moving from infrastructure to CTO office for pre-project involvement #### ⏱️ [32:20](https://youtu.be/Eq4NRgIkBuU?t=1940) — [H1] Advanced AI Runtime Controls - Key idea: Sophisticated real-time controls including confidence-based escalations and kill switches - Evidence: "token throttling, budget enforcement, model routing, rate limiting, value-based gating" #### ⏱️ [35:50](https://youtu.be/Eq4NRgIkBuU?t=2150) — [H2] COO for CTO Emergence - Key idea: FinOps-enabled executives unifying cloud, SaaS, data center, and AI spending decisions - Evidence: Fortune 100 interviews showing "emergence of effectively a COO for the CTO" #### ⏱️ [38:15](https://youtu.be/Eq4NRgIkBuU?t=2295) — [H3] AI Investment and Business Economics - Key idea: AI spending as strategic capital allocation with CXO board representation - Evidence: "AI spending is not just a cost center anymore, it's strategic capital allocation" #### ⏱️ [41:30](https://youtu.be/Eq4NRgIkBuU?t=2490) — [H2] FinOps Scope Strategy for AI Complexity - Key idea: Creating bounded segments of AI spending aligned to business priorities rather than managing entire complexity - Evidence: "FinOps scope allows you to zoom in on things that are important" for decision-making #### ⏱️ [43:45](https://youtu.be/Eq4NRgIkBuU?t=2625) — [H1] Cost of Intelligence Framework - Key idea: AI becomes expensive when unmanaged, requiring upfront governance and policy conversations - Evidence: "cost of intelligence - how much are we paying for each incremental improvement in output quality" ## 📚 References - [State of FinOps 2026 Report](https://data.finops.org) - [FinOps Foundation](https://finops.org) - [FinOps Maturity Assessment](https://harness.io/finops-assessment) - [Lead Dev Community](https://leaddev.com) ## 🗣️ Paraphrase - FinOps has grown from reactive cloud bill analysis to proactive technology value management embedding cost awareness throughout development lifecycles - AI workloads require fundamentally different optimization approaches due to their experimental nature, resource intensity, and unpredictable usage patterns - Organizations are implementing sophisticated runtime controls and policy engines to manage AI costs in real-time rather than discovering overruns after the fact ## 📰 Newsletter The evolution of FinOps from cloud cost management to comprehensive technology value discipline is accelerating with AI adoption. With 98% of FinOps teams now managing AI costs, the focus has shifted to proactive governance embedded in development workflows. Key developments include shift-left practices that embed cost policies in CI/CD pipelines, treating cost as a non-functional requirement alongside security and performance. AI workloads demand sophisticated runtime controls with token throttling, budget enforcement, and kill switches for cost management. The strategic elevation of FinOps to CIO-level reporting (78% of organizations) reflects its role in enabling rapid technology decisions while maintaining financial discipline. Resources: [State of FinOps Report](https://data.finops.org), [FinOps Foundation](https://finops.org), [Maturity Assessment](https://harness.io/finops-assessment) ## ✅ Action Items #### DevRel - Developer advocacy: Create AI cost optimization workshops for engineering teams - Education: Develop curriculum on cost-aware AI architecture patterns - Developer experience: Build cost visibility into development tools and dashboards - Developer marketing: Promote shift-left cost governance success stories - Community management: Facilitate FinOps practitioner knowledge sharing #### OSPO - Policy & Governance: Establish AI spending governance frameworks and approval processes - Software Inventory: Track AI model usage and licensing across organization - Developer Support: Train teams on cost-aware AI development practices - Community Engagement: Participate in FinOps Foundation working groups - Strategic Alignment: Align AI cost management with business value metrics #### Community - Maintainers: Provide cost impact visibility for infrastructure changes - Contributors: Educate on cost implications of feature contributions - Sponsors: Demonstrate ROI and cost optimization achievements - Ecosystem: Coordinate cross-project cost optimization initiatives #### WordPress - Core: Implement cost awareness for AI features and infrastructure scaling - Core Performance: Monitor and optimize hosting cost impacts of performance changes - Core AI: Establish governance for AI model usage and token consumption - Community: Provide cost-effective hosting guidance for WordCamp organizers - Training: Develop lessons on cost-effective WordPress hosting and optimization - Marketing: Create campaigns highlighting WordPress cost advantages - Hosting: Collaborate on cross-host cost optimization benchmarks - Meta: Optimize .org infrastructure costs while maintaining service levels ## 🧪 Hypotheses - Organizations with embedded cost governance in CI/CD pipelines achieve 30-50% better AI cost predictability than reactive approaches - FinOps teams reporting to CIO offices demonstrate faster AI adoption with better cost control than those under CFO or CTO - Real-time AI cost controls with automatic throttling reduce cost overruns by 60-80% compared to post-hoc optimization - Companies treating cost as non-functional requirement show 40% better alignment between AI investment and business value ### 📊 Methodology - Comparative analysis of cost variance between proactive vs reactive FinOps approaches - Survey FinOps organizational placement correlation with AI adoption speed and cost control - A/B testing real-time controls vs traditional cost management on AI workloads - Business value correlation analysis for organizations with embedded cost governance ### ❓ Questions - How can organizations quantify the ROI of shift-left FinOps practices beyond simple cost savings? - What governance frameworks best balance AI experimentation needs with cost control requirements? - How should FinOps scope strategies evolve as AI infrastructure billing practices mature? ### 🔮 Future Research - Development of standardized AI cost attribution models across cloud providers - Integration patterns between FinOps tools and AI development platforms - Cost optimization strategies for emerging AI workload patterns and architectures ### 💡 Implications - Practice: Cost awareness must become core competency for all engineering roles, not just FinOps specialists - Policy: Organizations need governance frameworks that enable AI experimentation while preventing runaway costs - Security: Cost anomaly detection becomes security indicator for potential AI model abuse or attacks - Education: Engineering curricula must include cost-aware architecture and AI resource optimization - Funding: AI project budgeting requires dynamic allocation models rather than traditional fixed budgets ### 🧠 Learning Styles Assessment Reading level estimate: 12th Grade (complex technical concepts with industry jargon) #### 👀 Visual learners - Present: No visual diagrams or charts in video format - Improve: Cost optimization workflow diagrams, AI spending dashboards, FinOps maturity visualizations #### 🔊 Aural learners - Present: 45-minute panel discussion with expert insights - Improve: Shorter audio summaries, key takeaway recordings #### 🗣️ Verbal learners - Present: Clear definitions, industry terminology, practical examples - Improve: Glossary of FinOps terms, AI cost management analogies #### 🧩 Kinesthetic learners - Present: References to hands-on tools and implementation - Improve: Interactive cost calculation tools, policy template workshops #### 🧠 Logical learners - Present: Framework-based approach, maturity models, systematic optimization - Improve: Decision trees for AI cost management, ROI calculation templates #### 🧑‍🤝‍🧑 Social learners - Present: Panel discussion format, community references - Improve: FinOps practitioner forums, peer review processes #### 🧘‍♀️ Solitary learners - Present: Comprehensive resource links for individual research - Improve: Self-assessment checklists, independent learning paths #### 🌿 Naturalistic learners - Present: Limited environmental connections - Improve: Link cost optimization to environmental sustainability, green computing practices ### 📊 Accessibility Assessment #### Screen reader semantics - Present: Video format without transcript provided - Improve: Full transcript with speaker identification, heading structure #### Keyboard navigation - Present: Standard video player controls - Improve: Chapter navigation, searchable transcript #### Contrast and readability - Present: Video format with varying visual quality - Improve: High contrast slides, clear speaker identification #### Text resizing and reflow - Present: Fixed video format - Improve: Responsive transcript layout, adjustable text size #### Media and captions - Present: Audio content without visible captions - Improve: Professional captions, downloadable transcript, audio descriptions #### Standards reference - Video accessibility improvements needed per WCAG 2.2 AA requirements ### 🌍 DEIB Assessment #### Cultural breadth - Present: Global FinOps Foundation perspective, international examples - Improve: More diverse regional case studies, emerging market perspectives #### Inclusive language - Present: Technical jargon defined in context, professional tone - Improve: Plain language summaries, avoid financial industry assumptions #### Neurodiversity support - Present: Structured panel format with clear speakers - Improve: Written summary sections, consistent information chunking #### Representation - Present: Mixed gender panel, varied professional backgrounds - Improve: More diverse industry representation, accessibility considerations #### Psychological safety - Present: Professional, respectful discussion tone - Improve: Content warnings for financial stress topics, supportive resources ### ✅ Logical Validity #### Strong reasoning - Deductive strength: Clear progression from FinOps evolution to AI-specific challenges - Inductive strength: Examples from Starbucks and other organizations support general principles - Causal reasoning: Links between proactive governance and cost control well established - Analogical strength: Comparisons between traditional and AI workloads effectively illustrate differences #### Fallacies - Appeal to authority: Relies heavily on expert opinions without independent verification - --- ## Full Transcript [00:12] Hello everyone and a huge thank you for [00:15] joining us today for a discussion on [00:17] accounting or accountability avoiding [00:20] cost drift in the age of AI. This is a [00:22] lead dev online panel discussion created [00:25] in partnership with harness. It's going [00:27] to last about 45 minutes after which [00:29] both myself and the palace will head [00:31] over to the lead dev slack to answer [00:33] some of your questions in the hashtag do [00:36] we say hashtag panel Q&A channel I've [00:39] shared in the resources you have an [00:41] invite to the lead dev slack community [00:43] if you haven't joined yet it's a really [00:44] positive community of engineering [00:46] leaders that share a lot in common and [00:50] are you know working together to solve [00:52] these [00:54] changing times we'll call But AI [00:58] uh we may also have time to answer [01:00] questions live. So please do submit [01:03] those in the Q&A feature on Zoom if [01:05] you're joining us live. Thank you for [01:07] being a participatory audience and we'll [01:10] either get to them during or maybe if we [01:13] don't have time in the end we will [01:15] continue there. Yeah. So we'll see what [01:18] happens and we'll continue in Slack. So [01:20] let's get started with introductions. My [01:21] name is Jennifer Riggins. I'm a tech [01:23] journalist and host and I'll be your [01:25] tour guide today. I am joined by manager [01:29] of AI and data EYU [01:33] Abby Singh director of the PHOPS [01:35] foundation Dean Oliver and har har har [01:37] har har har har har har har har har har [01:37] har har har har har har har har har har [01:38] har har har har har har har har har har [01:38] har har har har har har har harness'es d [01:38] d d d d d d d d d d d d d d d d d d d d [01:38] d d d d d d d d d d d d d d d d d d d of [01:39] phops advisory Ben Linares to share [01:42] their experiences insight. I'm going to [01:44] invite each of you to introduce [01:46] yourselves and in a few words if [01:48] possible define what PHOPS means to you [01:50] in 2026. the shortest definition, not [01:53] the official one, please. So, Ben, why [01:56] don't you join first? Why don't you go? [01:58] >> Thank you so much, Jennifer. My name is [02:00] Ben Lenares. Uh, I am the, uh, director [02:03] of a Phops advisory at Harness. Prior to [02:05] that, I was a practitioner at Starbucks [02:07] and I've had about 28 years in [02:09] technology all around infrastructure and [02:12] cloud. Um, for me, Spinops really does [02:15] nowadays speak of value. And if I had to [02:18] really boil it down to a sentence, I [02:19] would say that every workload, every [02:21] skew must prove its value. That's how I [02:23] see it today. [02:25] >> Okay. What are you gonna add to that, [02:27] Abby? And please introduce yourself. [02:29] >> Yeah, sure. Hi everyone. I'm Abby Singh. [02:31] Um I'm AI and data manager at EYUK. Um I [02:35] specialize in designing and building [02:37] data platforms, managing data teams, AI [02:40] and data workloads. Um I've got [02:42] experience um working for a bank, one of [02:44] the biggest banks in the UK leading data [02:46] engineering teams building solutions, [02:48] products, assets, working with cross [02:50] functional teams. I also have decent [02:53] experience in migration from the good [02:56] old onrem to cloud world. So I've seen [02:58] quite a bit of um how the cloud became [03:01] the number one thing and then now we're [03:03] talking about AI workloads and PHOPS [03:05] through the modern architectures. Um for [03:08] me phenopsis is not just finance it's [03:10] it's an engineering discipline. Um and [03:12] now it's the time to build a cost aware [03:15] AI by design and not just by audit and [03:19] uh I'd be happy to talk through some of [03:20] the architectural aspects and and some [03:23] of the economic decisions around it. [03:26] >> I look forward to joining the leader [03:27] panel [03:29] >> and Dean I'm going to need your not your [03:32] like I'm going to need your definition [03:33] not the official FinOps line for this. [03:36] >> Yeah. Yeah. Shira. So hello everyone. [03:38] I'm Dean Oliver, a director here at the [03:41] Phops Foundation. [03:43] And I'm responsible for community across [03:46] the Europe, Middle East, and Africa. And [03:48] also content where I work withme [03:51] practitioners to produce a lot of the [03:53] open-source papers that support [03:55] practitioners globally with all of their [03:58] challenges. Um, prior to the PHOPS [04:01] Foundation, I was a principal director [04:04] at a tier one strategy and consulting [04:06] firm and I set up there the global PHOPS [04:09] business and I also worked with [04:11] organizations across all industries on [04:14] many of their complex PHOPS challenges. [04:17] And for me, Finnops is a broad [04:21] technology value discipline and a [04:24] strategic enabler for executives to [04:28] harness PhOPS to support them with um [04:32] technology decision-m but also for [04:35] engineering teams to use to support them [04:38] with architectural decisions and [04:39] architectural trade-offs. It's not just [04:42] a cloud cost discipline. [04:45] >> Okay. Yeah, it's not just, but certainly [04:47] that is involved as well. [04:52] So, as we get started, I wanted to kick [04:55] off with a little poll to help us get an [04:58] idea of where you are on your FHOPS [05:00] journey to get the pulse for our [05:03] listeners live because well, we are [05:06] serving both you and those in the [05:09] recording. It's based on the FHOPS [05:11] Foundation's PHOPS maturity model that [05:13] you can find our resources. because we [05:15] have a lot of resources to download in [05:17] this and it basically goes through crawl [05:19] maturity level we can imagine walk [05:22] maturity level and run maturity level [05:25] alla a human being so yeah while I [05:29] invite you all to answer that the [05:31] onslaught let's talk about the onslaught [05:33] of AI has finops shifting left and up [05:36] covering a whole lot more than cloud [05:38] cost though cloud cost is still a thing [05:41] I think this is great because like [05:43] personally I'd much rather other [05:44] organizations look to optimize their [05:46] cross organizational technology spend [05:48] before they look up look at having [05:51] layoffs. So, by the way, I just wanted [05:54] to make sure you know because we were [05:55] talking about jobs there. Fination has [05:58] an global open source jobs board for all [06:00] FinOps jobs. So, check it out. It's [06:03] jobs.phinops.org. It's also in our [06:05] resources. We always want to make sure [06:08] that as a community both in the lead dev [06:10] slack and community in technology [06:13] especially in these times that we're [06:15] promoting jobs and we're also like [06:18] finding jobs and helping people find it. [06:22] Yeah. So in this age of AI there's just [06:24] so much more to manage. There's just so [06:26] much more of everything in AI. Today [06:29] we're here to talk about that and its [06:31] impact on PHOPS and then FHOP's impact [06:34] on AI choices. SAS choices, etc. So, [06:38] it's pretty cool that this just happened [06:41] to be happening or maybe that was [06:43] coincidence that the state of PHOPS 2026 [06:47] report came out last week and we're [06:50] really going to get into it today [06:52] because it's the only thing that's [06:55] certain is that FHOPS is really evolving [06:57] and changing and I think this is it's [07:01] not even at precipice. We are at a point [07:02] where things are changing and FHOPS [07:05] chain PHOps teams are moving up in their [07:08] influence within organizations and [07:10] taking on more and more responsibility. [07:13] So let's jump into how PHOPS is shaping [07:15] technology decisions before money is [07:18] even spent. And again that report's in [07:21] the chat but it's data.finops.org. [07:25] So yeah uh should we check those poll [07:28] results? [07:30] Yeah, [07:32] >> I don't know. [07:33] >> It would be interesting to see because I [07:34] I I I myself meet with a lot of [07:36] customers and it's always interesting to [07:38] see where people are at in their [07:39] journey. Uh it's fascinating. You can [07:41] see large enterprise companies, Fortune [07:44] 5 companies who are having some common [07:46] challenges that you would see companies [07:47] who are not quite in that level of of [07:49] business. And it's interesting to see [07:51] that it [07:53] anywhere across a company's enterprise, [07:55] you can see there's opportunities. And [07:57] that's kind of the neat thing with the [07:58] foundation is that it really helps you [08:00] understand and guide that kind of thing. [08:01] So I'm interested to see what these [08:03] results show. [08:04] >> Yeah. And it's also even more [08:05] interesting time because definition has [08:07] evolved. It's not just about cloud cost [08:09] control. You're talking about the [08:10] enterprise value management. So there we [08:13] go. [08:14] >> I'm going to do [08:16] for it's so true. Yeah. As we said, [08:19] value design, strategy, discipline, [08:21] these are all part of it in your [08:23] definitions. So because for whatever [08:25] reason the panel the webinar of Zoom [08:29] does not show the results to non-live [08:31] listeners. I just want to read out loud [08:33] what is your FOAP's maturity. Uh [08:38] what is that like 60% of people are said [08:41] they're still at the crawling stage. [08:43] They're little reporting their [08:45] measurement. They're early on their [08:46] journey but they're getting there. And [08:49] then about a third are walking their [08:52] capability understood followed [08:55] everyone's kind of understanding it [08:56] within the organization. They know what [08:59] their edge cases are though those of [09:01] course are evolving but maybe they [09:04] haven't they're just picking and [09:05] choosing and only doing the most urgent [09:07] ones. No one on our call is at a running [09:11] status yet. And yeah, I just think to [09:14] note that maybe it's so different [09:16] enterprises because they inherently are [09:18] more complex and PHOPS is a pretty [09:21] complex [09:23] situation. Uh, and that was clearly a [09:27] very quick like pulse setting for us and [09:30] for you. But among the many resources in [09:33] our Zoom, you can find the free a free [09:36] 30 minute FNAPS maturity assessment you [09:39] can book through harness. So definitely [09:41] do that. So, thank you for that. And now [09:44] I have to say, Dean, let's kick off with [09:46] the state of it all. Please explain what [09:48] the FHOPS Foundation is, what PHOps is [09:51] by their definition, and then let's talk [09:54] about the results of this year's report, [09:56] please. [09:57] >> Yeah, of course. Of course. So, the [09:59] Phops Foundation is a Linux Foundation [10:03] program and it's a nonprofit [10:05] membership-based trade association and [10:08] has 96,000 participants globally and [10:12] growing right now very very quickly. [10:15] And the mission is advancing the people [10:20] and most importantly people here who [10:23] manage the value of technology. And the [10:26] foundation has the people at its core. [10:29] And the main aim of the state of Phops [10:31] is supporting what their priorities and [10:35] challenges are by everything the [10:37] foundation does through best practices, [10:39] standards, assets, everything opensource [10:43] to help them on that journey and value [10:47] is the overall goal. So maximizing the [10:49] value that the organizations get from [10:52] their technology investments. [10:54] Now, FHOPS itself is more of a cultural [10:58] practice at its core and it's a way for [11:02] teams all across the organization to [11:05] manage and optimize technology value [11:09] where everyone takes ownership of their [11:11] cost and usage. And that's really [11:13] important from today's call especially [11:15] for an engineering community because [11:18] PHOPS and it's a term that we use a lot. [11:21] PHOPS is everybody's job. It's not just [11:24] a central team in terms of owning [11:27] everything to do with PHOPS. A central [11:29] team just drives it. And we're actually [11:32] seeing from the state of PHOPS data that [11:36] PHOPS teams on average are quite thin. [11:40] And then it's a sporkbased champions [11:44] network across engineering teams, [11:47] product teams, the business units and [11:49] the product owners who are having that [11:51] cost accountability and intersecting [11:53] with what the PHOPS team is driving [11:55] centrally. [11:57] So the state of PHOPS is the sixth [12:00] annual survey and it has a wide range of [12:03] organizations of all sizes and the data [12:06] this year on data.finops Phops um is [12:09] actually um clickable so it's [12:11] interactive where you can see what the [12:14] data shows for an organization of your [12:16] size or your industry on certain key um [12:19] insights [12:20] and AI obviously is dominating the [12:24] agenda. So AI skills, the future [12:28] priorities of organizations, [12:31] the investment and managing the [12:33] investment and also the PHOPS open cost [12:37] and usage specification which is [12:39] supporting normalized data [12:41] specifications from technology vendors [12:44] and requirements for that specification. [12:48] Now, as I mentioned before, it's then [12:50] showing with data that PHOps has [12:53] expanded across technology outside of [12:56] its public cloud origins and 98% of [13:01] PHOPS teams for example are being now [13:04] asked to manage the cost of AI and a [13:07] significant number in the upper 60s to [13:10] 70% are being man asked to manage SAS [13:14] and a lot of this is driving is being [13:16] driven through the complexity of the [13:19] pricing models. It's no longer seatbased [13:22] licenses even in SAS. It's [13:24] consumptionbased licenses and then on AI [13:28] you've got application costs, token [13:31] costs, model operating costs, model [13:33] training costs and it's all driving the [13:37] complexity of why AI is dominating the [13:39] agenda there. [13:41] Optimization itself again for the [13:43] engineering community is becoming more [13:45] table stakes. A lot of this is being [13:48] automated by what we're seeing as AI for [13:51] Pinpops [13:52] but it's very much then in the hands of [13:54] PHOPS teams working with engineering [13:56] teams on how they can automate a lot of [14:00] the policies or put the guard rails in [14:02] place for that um AI cost and usage as [14:06] an example. [14:08] Shift left then is happening and it's a [14:11] highly desired feature in tooling [14:13] particularly but then challenges are [14:15] remaining in how can engineering teams [14:19] take credit and demonstrate cost [14:21] avoidance like PHOPS teams were doing on [14:24] cost saves when they were turning off [14:26] wasted spend. [14:28] shift up is then something we're seeing [14:30] and and determine in enabling higher [14:34] levels of the organization [14:36] to make technology tradeoffs and [14:39] decisions. But in a very fast-paced [14:41] world, technologies moving significantly [14:44] faster now. like back in 2013 days um [14:48] when public cloud was emerging we're [14:50] backing that fastpaced [14:53] um environment [14:55] and FHOPS's superpower or what we see as [14:58] the superpower is the intersection and [15:01] the collaboration between disciplines [15:03] and this is starting to converge with [15:06] ITAM and consumptionbased spend again um [15:11] being being quite apparent and then IT [15:14] financial management ment and PHOps. So [15:16] a lot of the larger enterprises are [15:18] working out new ways to collaborate with [15:21] this fast pace of technology [15:23] particularly the disruption of AI. [15:27] >> Absolutely. Yeah. And we've seen like [15:30] this new priority emerging like the cost [15:32] of infrastructure and architecture which [15:34] is something we've just kind of taken [15:36] for table stakes before for granted and [15:38] I think that's very interesting. Ben, [15:41] this shift left. Can you talk more about [15:44] what that is? Because that maybe that [15:46] feels like a DevOps term that maybe [15:48] FinOps folk or financial people that [15:51] it's not only people from the DevOps [15:53] side that are becoming Pinpops, it might [15:54] be financial people learning more of the [15:57] pipeline management in the cloud. So, [15:59] what does that mean and what are you [16:00] seeing with your clients? [16:02] >> So, that that's a great question. [16:03] Thanks, Jennifer and Dean. Thank you for [16:05] that. I mean again Finaps Foundation uh [16:07] just taking a real quick step back [16:09] that's how I got planted is I started [16:11] doing things and then I started looking [16:12] around and found the foundation who [16:14] actually gave me the framework to be [16:16] successful and I think anybody who's [16:17] been in FinOps and who's leaned on the [16:19] foundation can have that can attest to [16:21] the same thing. This helped us do things [16:24] better. Now that being said we started [16:27] off with shift right which was reactive. [16:29] Hey, we got the bill. There's a spike. [16:31] Let's go talk to this team. Or finance [16:33] got the bill and they're freaking out [16:35] because it's a huge spike and they come [16:36] to, you know, engineering and they're [16:38] like, what's going on? Um, FOPS became [16:40] that group that could be the [16:41] intermediary, right? They're going to [16:42] work between finance and engineering and [16:44] leadership and they're taking [16:46] accountability initially and then [16:48] finding where that accountability should [16:50] be. So reactive is always been a painful [16:53] thing because you're not finding out [16:54] until you receive your bill. shift left [16:57] is a bit more proactive and and you're [16:59] right in the ideology of around DevOps [17:02] because true shift left begins at the [17:04] workload level and that's one of the [17:06] things that attracted me when I first [17:07] saw harness a few years ago. I I [17:09] conducted RFIs for Starbucks and one of [17:11] the things I kept thinking about is I [17:14] keep seeing products that help give me [17:16] visibility or they help me structure [17:18] cost and tie it back to a team but how [17:21] do I become preventative in this? like [17:23] how do I get these teams to pick the [17:26] right skew? How do I get them to uh [17:28] acknowledge that they may not understand [17:31] what actual right sizing looks like? So, [17:35] shift left takes governance policy and [17:38] embeds it at the workload level. Now you [17:42] can go further and the trifecta that I [17:44] see with this being done is you embed [17:47] guard rails and policy at that level [17:49] right at the workload level prior as [17:51] you're getting ready to release but then [17:53] you also have autoprovisioning. You rely [17:55] heavily on automation to help provision [17:58] the assets that are required to run the [18:01] application. And you're doing so by [18:03] again a templatized method that ensures [18:06] that nobody's going to overprovision or [18:08] go after machines that they don't need. [18:11] Therefore, you now have that capability [18:13] to say, "Hey, listen, you're only going [18:15] to use what has been authorized. And if [18:17] it needs another level of approval of [18:19] EP, then you can get that. But these [18:21] standardized templates are within [18:23] everybody's realm of being able to run [18:25] their workloads. And then finally, you [18:27] optimize these things. And you do that [18:29] through things like autostopping. If [18:31] it's the weekend or it's the evening and [18:32] your machine doesn't need to run, then [18:34] guess what? Shut it down. You can also [18:36] do things like Kubernetes orchestration, [18:38] being able to optimize at the Kubernetes [18:40] level within the clusters and the pods [18:42] themselves to understand what that looks [18:44] like. And then you also have the [18:45] capability to do commitment [18:47] orchestrations. If you have many, many [18:49] commitments, it can be really [18:50] challenging to manually get the highest [18:53] value out of those investments. So, you [18:55] use a tool that does it for you. You set [18:57] a threshold saying that I want 90% [18:59] coverage on my commitments and the AI [19:02] will do it for you. Between these three [19:04] levels, you can get into a shift left [19:06] state. The challenge is most PHOPS [19:09] departments that I've seen and been part [19:11] of are ground up. That means that we had [19:14] an idea, we wanted to do something and [19:16] we went and did something. However, [19:18] leadership for probably the first year [19:20] kept asking what is Finops? So getting [19:22] to that shift left space can be a [19:24] challenge. And if you get leadership buy [19:26] on from the top down, it certainly [19:28] becomes much more adoptable because you [19:30] now know what your leader expectations [19:32] are. That's the challenge. But the thing [19:34] about AI is this is not like your [19:37] traditional workloads, right? Spiky and [19:40] bursty for training jobs and inference [19:42] spikes, GPU heavy and expensive, [19:45] experimentation driven. So you're, you [19:47] know, by design it's waste. So you have [19:50] to take a different approach to [19:52] optimizing AI because it does come at [19:55] you a bit differently. [19:56] >> Yeah. And building on on top of that I I [20:00] can clearly see governance as a core not [20:03] as as committees. Uh to the point that [20:05] you touched it's it's pretty much uh [20:08] embedded within the engineering teams [20:10] functions and cultures and even if [20:12] organizations who've not progressed to [20:14] that state where they have this within [20:15] their engineering function they are [20:17] moving towards that state. um from my [20:19] own um awareness and and working on the [20:22] ground within the regulators, financial [20:23] industries. Um I think the the [20:25] governance is is very much needed. So [20:28] it's it's not just about shifting left, [20:31] it's also about how you shift left [20:33] without losing the control and which is [20:34] where so the tools how you codify you [20:37] basically codify the rules and then it [20:39] it takes care of it. could be through [20:41] policy is core. It could be through [20:42] building guardrails within your CI/CD [20:44] pipelines. It could be um automated [20:46] tagging enforcements um all whether it's [20:49] the standard AI and data platforms [20:50] whether it's snowflake um data bricks or [20:53] of that sort you you don't wait for [20:55] finance to find the drift um after [20:58] something has occurred rather you build [21:00] in within the core through [21:01] infrastructure as core templates or [21:03] through platform blueprints and [21:05] deployment pipelines so and so forth. Um [21:07] so yeah that's that's clearly a shift [21:09] and and very much um required um [21:13] especially for the AI workloads. Um the [21:16] the other part that I see is um there is [21:19] data is is really the bridge between um [21:22] your control and speed. Um by that what [21:25] I mean is um there's clearly a [21:27] distinction between the teams who look [21:30] at some of those data around uh whether [21:32] it's cost of delta per PR or um run rate [21:36] which impacts on their scaling change or [21:39] AI inference cost per transaction. [21:42] They're clearly the differentiators and [21:43] and and shifts the focus and ultimately [21:47] governance is is not to block them. and [21:50] try to enable them and and data is is [21:52] absolutely guide them. [21:55] >> Yeah, I I think I think from a from a [21:57] Phops Foundation perspective, we're [21:59] seeing it the same and across two clear [22:02] areas. So you have shift left in terms [22:06] of architecting and workload placement [22:09] where you've got phops practi [22:11] practitioners engineering and product [22:14] and they're making cost and value [22:18] awareness on the architectural options [22:21] and trade-offs and linking that to [22:24] business unit metrics. or you might want [22:27] to pay more for increased performance if [22:30] it gives you a jump on your competitors, [22:32] but without the business context and tie [22:35] that into the pace that technologies [22:39] moving right now and engineers needing [22:41] to make decisions much more quickly. Fin [22:44] is complementing with the speed they [22:46] work at the speed the engineering teams [22:48] work to help them to make those [22:50] tradeoffs. And then exactly to Ben and [22:54] Aby's point on the usage optimization [22:57] side traditionally or in initial setting [23:01] out it was very much reactive usage [23:04] optimization once the overspend had [23:06] occurred catching that. What we're now [23:09] seeing is phops in deployment. So cost [23:12] and value constraints into pull requests [23:14] and delivery workflows. So budget [23:17] thresholds require approval before a [23:19] merge. Phops at runtime particularly [23:22] with AI and real-time enforcement [23:25] decisions. So enforcing spend and policy [23:28] controls at the action points throttling [23:30] tokens concurrency tool integrations per [23:33] tenant and then finally more continuous [23:37] state controls. So again to that point [23:39] of tier and delete data use and [23:42] retention policies but automatically [23:44] don't fix that and say you've saved a [23:46] million. just don't do it in the first [23:48] place and automate that by understanding [23:50] the policy and and having that in place. [23:53] That's a big partnership model with [23:55] PinOps and engineering. [23:57] >> Yeah. You know what the way I start [23:58] seeing this is that every AI request [24:00] becomes a financial decision. And that's [24:03] where you from a runtime architecture [24:04] perspective have to think about the [24:06] pattern of request policy engine [24:08] decision execution and then telemetry. [24:10] And a lot of times people miss that [24:12] telemetry piece because if you're just [24:14] observing this and you've got the tools [24:15] but you're not doing anything feeding it [24:17] into a loop to help your infrastructure [24:19] learn and heal and become better then [24:21] you're missing a key element to how you [24:23] optimize and how you ensure that this [24:25] pattern is running correctly. But spot [24:27] on Dean. [24:28] >> Yeah. And I think adding on to the [24:30] Philops culture, it's it's ensuring [24:32] architecture side, they they basically [24:34] evaluate what's the cost per transaction [24:36] whether it's engineers, then they [24:38] understand what's the cost per feature [24:40] and if it's platform teams, they [24:41] understand what's the cost per [24:43] environment. So it's it's definitely a a [24:45] shift which we need whether you you wipe [24:47] code a solution or you build a custom [24:49] tool for a large institution. [24:52] Definitely that's the correlation is [24:54] that it AI isn't special. [24:58] What makes it special is having the [25:00] engineering best practices that we've [25:02] all been beating our heads against the [25:03] wall to get people to adopt anyway. So, [25:06] there's a word I haven't heard here that [25:09] I will mention that it feels like would [25:11] be the conduit for it, but nobody's [25:13] really talking about how to do PHOPS or [25:15] where uh one of the results from the [25:17] state of Phops report was that 78% of [25:19] PHOPS leadership report to the CIO [25:21] office, which kind of surprised me [25:23] because I thought it would have been [25:24] either CFO or CTO. Um, it seems to me [25:29] just like success with platforms in a [25:33] month we're going to talk on March 25th [25:35] about how plat like AI necessitates a [25:39] platform model. It feels like platform [25:41] engineering and PHOPS. Granted, I know [25:44] platform engineering is getting a lot [25:45] right now, especially in the age of AI. [25:47] It's getting a lot more responsibility [25:49] and not growing its teams as fast. So I [25:52] don't want to put pressure, but where [25:53] are the PHOPS team sitting and how is [25:57] FinOps actually deployed? I guess at EI [26:01] I'd be curious first and then we'll talk [26:04] more in general. Where does your FinOps [26:06] sit, Abby? [26:08] >> So PHOPS from my perspective is is very [26:11] much embedded into the engineering teams [26:13] and and the architecture. Um when when I [26:16] look at phops it's basically um letting [26:20] or enabling developers within the guard [26:22] guardrails you've established um it [26:24] could be as an example something like um [26:26] how do you see the cost of impact on a [26:29] pull request as simple as that you you [26:30] ship the code you you've changed let's [26:32] say part of the UI how do you see the [26:34] impact developer when it submits a ps it [26:37] sees the instance types and scaling [26:39] rules and maybe it adds an AI inference [26:42] uh workload now when they do [26:44] through the tools they they see what's [26:46] the roughly projected monthly delta cost [26:49] what could be the transactional uh [26:52] impact are there any any trends based on [26:54] on the consumptions and and this this [26:57] helps in seeing whether this uh [27:00] performance gain is is worth the spend [27:02] let's say everybody wants to have some [27:04] form of AI in in the solutions or tools [27:07] they're building but is that really um [27:10] is that performance gain really worth [27:11] the spend um or is it do Do I just [27:14] optimize some of the things that I have [27:15] built or should this be um let's say do [27:19] you really need something to be real [27:20] time do you make it asyning so all those [27:22] insights u one of the place could be [27:24] when the developer submits the p the [27:27] other one could be around the the [27:30] feature level accountability now this is [27:33] where you see a mix of product [27:35] engineering and architecture so um it's [27:38] no more about the cloud cost has gone up [27:41] by let's say 10% 15% [27:44] um a team can can see the cost per [27:47] feature whether it's um cost per API [27:49] endpoints or the experimentation on the [27:51] eye that they're running now product [27:53] engineering architecture all three of [27:55] them together they can um decide whether [27:58] do you have to optimize this uh piece do [28:01] you have to sunset this this new feature [28:02] do you maybe price it differently um so [28:05] that's that's where I see it and where [28:08] it's done so you you do have [28:10] pre-approved architecture patterns some [28:12] configuration ations and blueprints [28:14] which guys a theme. Now of course [28:15] there's a finance element of it and but [28:18] now we talk about shift left which is [28:21] very much an engineering um thing and [28:25] and hence the finance rather the rather [28:27] than the other way. [28:30] >> Okay. Is that ring true to the other to [28:33] your other your different customers Ben? [28:37] So yeah, when when I was at Starbucks, [28:39] we um we were a groundup. We were [28:41] actually a products and services [28:43] department. Um once I landed around the [28:45] PHOPS Foundation web page, I went to my [28:47] leader and said, "I I think we're a [28:48] Finops department." And he said, "No, [28:50] no, we're not." So I said, "Hey, come to [28:52] me. Come come come with me to this [28:54] conference." And uh he uh we were in the [28:56] middle of listening to Tim O'Brien from [28:58] Walmart speak. And we're sitting in San [29:01] Diego and he leans over, he goes, [29:02] "You're right. We got to change [29:03] everything." So we we really had to go [29:06] from a different route and convince [29:07] people of what we were doing and [29:10] initially it was a hard cell because it [29:12] didn't support what we were doing and we [29:14] were under infrastructure and cloud. So [29:16] we were directly supporting [29:17] infrastructure and cloud which is a very [29:19] large team but even more so an [29:21] incredibly large network. Um you think [29:23] about Starbucks 44,000 stores um all [29:26] connected. It's a lot. So when you think [29:29] about what we're doing and we're wanting [29:31] to change it, it was it was not [29:33] necessarily a hard sell. It was more of [29:35] an educational thing. People don't [29:36] understand what it is. Eventually, [29:39] people caught fire. Our VP really [29:41] understood it and got closer to it. We [29:44] really evangelized the work that we were [29:45] doing. And a big part of that was again [29:47] my leader, David Rogan, who was just an [29:49] amazing guy. Um, you know, no experience [29:52] in this space, has the vision and sees [29:54] clearly what can be done. and we ended [29:57] up getting moved under the office of the [29:59] CTO because they saw the relevancy to [30:01] what we were doing and how they wanted [30:03] us to scale this overall of technology. [30:06] The uh benefit of that too is that we [30:08] were adjacent to the portfolio team, [30:09] meaning that we can now get involved in [30:11] projects before they're released and we [30:13] can start doing things like pricing out [30:15] what this is going to look like for [30:17] budgeting. And one of the great things [30:19] that we achieved at Starbucks is we got [30:20] to an 8 figureure annual savings and we [30:22] got to a v uh forecasting variable um [30:26] finance was between 35 and 45% and we [30:28] got it down between 1 to one and a [30:29] half%. So we became very accurate and [30:32] that helped earn a lot of trust within [30:34] that trust that allowed us to scale what [30:36] we were doing and actually grow our [30:38] department and take on more [30:39] responsibility. But the key successes [30:42] there were building the accountability [30:44] with the teams and we had teams that [30:46] really didn't lean into us. they really [30:47] wanted to be part of it. We went from, [30:50] you know, I used to I said this at the [30:52] uh last year at the Finops conference. [30:54] Working with engineers was like asking a [30:56] two-year-old what's in your mouth. [30:58] >> Run, you know, they're just like out of [31:00] there. They don't want to talk about [31:01] cost. They don't want to talk about [31:03] optimization. However, once we showed [31:05] them how it could be done and what could [31:07] be done and how effective it could be [31:08] effectively it could be applied in their [31:10] team specifically, uh we saw a great uh [31:14] everybody started coming to the table. [31:16] Everybody wanted to work with us. [31:18] Everybody wanted to get involved. They [31:19] wanted to play their part. And we had a [31:21] decentralized organization. So imagine [31:22] dozens of teams managing thousands of [31:24] apps. That was one of the larger and [31:27] bigger things to overcome. But as far as [31:29] how it sat, we really once we started to [31:32] catch on and we started to show this [31:34] greater savings, we were able to really [31:37] show the value of the work that we were [31:39] doing and leaders got behind that. And [31:41] that was a really big win for us because [31:44] that's what we needed. We needed now [31:45] that we've worked from the bottom up to [31:47] evangelize and help teams uh within the [31:49] organization and we needed leadership to [31:51] look at this and help help us gain more [31:53] traction and and we did and that was [31:55] part of our storytelling. We would tell [31:58] the story of how we were saving money [31:59] and optimizing but we also wanted to [32:02] change it as we started to look at AI [32:03] because AI started to take a different [32:05] turn. So we had to change of had to [32:08] change and look at our policy engine as [32:09] the brain of all of this. So things like [32:12] token throttling, things like budget [32:14] enforcement, model routing, rate [32:16] limiting, valuebased gating. So there's [32:18] a lot of those things that are not part [32:20] of the normal conversation as far far as [32:22] your normal infrastructure. It changes [32:24] the conversation which means you need to [32:26] change the way you think. And that's [32:28] where we found, you know, a lot of [32:30] gratitude in the fact that we started to [32:32] gain that support and we can now start [32:34] getting that conversation elevated to [32:36] the right levels and start having [32:38] decision outcomes. the real power of [32:40] what you're doing, right? You know, do [32:42] you modify? How do you route? What does [32:44] throttling look like? When do you [32:46] reject? You know, these are all things [32:48] that you you need to build in prior to [32:50] launching. And the challenge is as [32:51] everybody's playing in AI, they really [32:54] don't have a firm foundation of what [32:56] that looks like. Therefore, you do start [32:58] to see that drift. And Abby, I'm sure [33:00] you can probably talk about that, but [33:01] what was that like for you guys as you [33:03] started getting into that? How did you [33:06] address the potential drift that could [33:08] happen? [33:09] So I think one of the the thing that we [33:12] or I have seen in in my my recent um [33:15] engagement as well as the last one is um [33:19] there there's a strong focus from [33:21] architecture on on the side. So for for [33:23] any any application you build you go [33:25] through a set of approvals and and [33:28] reviews and cost has become one of the [33:31] non-functional requirements now just [33:33] like you have security reliability and [33:36] and performance. So the core idea is if [33:39] you if you don't if you won't shift ship [33:41] the code which has not been tested why [33:44] would you ship something which has not [33:46] been priced pretty much that. So um that [33:50] that's that's a thing and and that what [33:52] that needs us to do is um do some kind [33:55] of of estimations well before something [33:57] hits to production. U you you do all [34:00] sorts of consumptions and and uh whether [34:03] it's the credit credits or the dollar or [34:05] the pound value. Um that that's one way [34:07] of it. The other part is the cost [34:09] observability. Um you talked about [34:11] telemetry and some of those aspects. Um [34:14] they are really key not just setting up [34:17] but also helping the team understand how [34:20] to read and in for meanings out of it [34:22] and and then thereby the whole PHOPS [34:26] culture and how we drive it comes into [34:28] the picture. [34:30] >> Oh that's great. You know what? That's [34:32] that's a lot of the conversations I'm [34:34] having now with a lot of our enterprise [34:36] customers as we start looking at things [34:37] like advanced runtime controls. You [34:39] know, we talked about AI at runtime. you [34:41] know, con confidence-based escalations, [34:44] you know, semantic caching, real-time [34:46] anomaly detection, kill switches, you [34:48] know, per team, per feature, you know, [34:50] instant shutdowns if you have a runaway [34:52] cost, you know, and then telemetry [34:54] closing the loop, you know, when you [34:56] track cost per request, token per [34:58] request, model usage, distribution, [35:00] throttle rate, you're now able to [35:02] capture, like I said, the very data that [35:04] you should be feeding back in to your [35:06] system. Because when you feed this back [35:08] into budgets, policies, and optimization [35:10] loops, you're now taking valuable data, [35:12] and you're optimizing in a way that most [35:14] people haven't even thought of yet [35:16] because they're still trying to get [35:17] their hands around what does it look [35:18] like to understand this. And you know, [35:20] I'm going to release something on [35:21] LinkedIn a little bit later. It's, like [35:22] I said, it's a seven layer approach that [35:24] I have to optimizing AI. And I want to [35:26] share it with everybody, all my [35:28] colleagues and constituents. And if it [35:29] helps, that's great. If there's anything [35:31] to add to it, um, I would love to see [35:33] people build off of it, too. But I want [35:34] to put that out there because I I have [35:36] seen there's a lot of I don't want to [35:38] call it struggle but challenges. You're [35:40] already doing something and now you've [35:42] got something else that has a different [35:43] a little bit different language still [35:45] within your infrastructure that you now [35:47] have to manage and it can be really [35:49] really challenging. [35:51] >> I think it's join it's joining that up [35:53] as well in terms of those decisions for [35:56] that tooling that is focusing on the [35:59] cost control, the observability, the [36:01] telemetry. Um, we ran interviews with [36:05] dozens of Fortune 100 company leaders [36:08] and what we're seeing is the emergence [36:11] of effectively a COO for the CTO to [36:15] Jennifer's original point and it's [36:17] really a PHOPS enabled executive and [36:20] what they're doing is unifying PHOPS [36:23] across cloud SAS data centers AI [36:27] spending under a single practice to make [36:30] effectively those decisions. isions in a [36:32] more centralized controlled way. And [36:36] that enables them to focus cloud cost [36:39] optimization and efficiency into a [36:43] strategic infrastructure value [36:44] capability that is controlled through [36:47] the lens of what the business needs to [36:50] get the right value from technology. But [36:53] that then gives engineering teams also [36:55] one clear direction on what they need to [36:59] do in order to have that level of [37:00] controls, policies, automation in place [37:04] that is operating in line with what the [37:06] business need. [37:08] >> Yep, you nailed it. Every AI request is [37:10] a financial transaction. Every [37:12] transaction has to be governed before it [37:14] executes. [37:15] >> Especially in heavily regulated [37:16] industries as well, it's even more so. [37:19] >> And in the EU in general now, because [37:21] the EU is [37:23] requiring auditability of your software [37:26] bill of materials. [37:26] >> Visibility and audability. Absolutely. [37:29] >> Yeah. [37:30] >> Absolutely. So, as we wind down, so I [37:33] know we're not focusing on cloud cost as [37:35] much anymore, but it's still the biggest [37:37] driver and AI will just drive up cloud [37:39] cost. So, we do see that one of the [37:42] results from the state of PHOPS report [37:43] was that 98% of companies were directly [37:47] the they weren't reinvesting in anything [37:49] else as much as AI. they were [37:50] reinvesting what they saved in cloud [37:52] drift into their AI program. So that's [37:56] obviously like one win because [37:59] numerical tying to business metrics and [38:02] money is always going to be one of them [38:05] uh is always going to be important. So I [38:10] want to as we're closing out talk about [38:12] more about tying that AI spend and FHOPS [38:16] purview in general to business economics [38:19] uh because just like PHOPS is a new [38:23] department at most of these companies [38:27] and last hard first fired still rules. [38:30] So they still have to just like platform [38:32] engineering teams and other teams have [38:34] to be able to justify their existence [38:36] and make this argument that maybe both [38:39] financially not very literate and [38:42] technologically not very literate [38:44] leadership may need to understand more [38:47] under the business perspective. So I'd [38:49] love to hear the opinion on that. Dean [38:51] if you want to kick off of what you're [38:53] seeing in organizations around that. [38:56] >> Yeah. So AI is new to everyone. [38:59] Engineering, management, fin teams. Not [39:03] many have seen this speed, volatility or [39:06] uncertainty since 2013 as I said before. [39:10] And AI is super complex in the nature [39:13] that the spending can sit across public [39:15] cloud, data center, SAS, Color, um the [39:20] NeoCloud providers just depending on [39:22] your AI infrastructure strategy. And [39:25] that's before you start getting the [39:27] different variations of bills come in. [39:29] The actual infrastructure providers [39:32] themselves haven't even normalized on [39:35] being able to give you the level of [39:37] granularity that you need from the bill [39:40] in order to make the decisions that [39:42] you're currently making on public cloud [39:44] right now in terms of where they are on [39:47] their billing maturity. [39:49] And what we're seeing is PHOPS [39:52] practitioners are creating what we've [39:55] called a scope. And the point of a scope [39:58] is it allows you to zoom in on the [40:00] things that are important instead of [40:02] trying to bite off the whole thing [40:03] straight away, which is very complex. [40:06] And the FOP scope is really a defined [40:09] segment of spending across those [40:12] technology categories. And it's aligned [40:14] to a business priority. So you might [40:17] have products cost centers or a specific [40:19] environment and that's guiding the [40:22] application of PinOps into bounded [40:25] efforts right now and PHOPS teams [40:29] engineering teams all the teams are [40:31] really needing to ask what's the real [40:34] why when the business are asking for [40:36] certain insights or certain visibility [40:39] of cost and usage so that you can really [40:42] provide them the insight needed to make [40:44] the decision that they want And that [40:46] decision is usually around value and [40:48] profitability right now. But until you [40:51] actually know how you can ring fence [40:53] that segment of spend and usage and use [40:56] the PHOPS capabilities and the [40:58] intersection with engineering and [41:00] automation and so on, [41:04] you're really just operating on a very [41:06] very complex landscape that you're not [41:08] going to provide much use unless you [41:10] zone in. [41:13] >> Yeah. The Go ahead, Abby. Um the other [41:16] thing I a trend I see is um almost all [41:19] the large financial institutions they've [41:21] got AI representation at at CXO board. [41:25] Um and and that does tell a lot about [41:27] how serious the organizations are and um [41:30] and that also drives the governance [41:32] mechanism not just for the I adoption [41:34] but also PHOPS around AI or or AI for [41:38] fops as well. Um there's a lot about um [41:41] a lot um especially with these agent [41:43] tech stuff going on plenty of things [41:46] where you can actually use AI for ops as [41:48] well. Um and AI spending is is not just [41:52] a cost center anymore. It's pretty much [41:55] a strategic capital allocation. you you [41:58] do get um budgets allocated and it's [42:01] it's not just for the experimentation [42:03] but um there's a strong focus um yeah [42:07] and the other trend I I also see is um [42:10] when it comes to the SAS world we we not [42:13] talking much about probably the software [42:15] as assisted service platforms um they [42:19] they are exploding especially after the [42:21] vibe coding and and and number of ways [42:24] you can create how quickly you can [42:26] create application I think we are also [42:27] seeing [42:28] uh through the PHOPS culture some sort [42:30] of push for platform consolidation you [42:34] you're standardizing the overall tooling [42:36] and simplifying your platform estate um [42:39] and having a a shared AI infrastructure [42:42] rather than building something team by [42:44] team so more more centralized uh in in [42:47] in many ways and um this is less about I [42:51] think cost cutting but more about [42:53] eliminating duplication and and removing [42:57] the chaos around your architecture. [43:00] >> That's a good point. You know, the the [43:02] way I see it is the cost of [43:03] intelligence, right? How much are we [43:05] paying for each incremental improvement [43:06] in output quality? And when I think [43:08] about this, it's not that AI is [43:10] expensive. It's that it becomes [43:12] expensive because it's unmanaged. And [43:14] that becomes the biggest telltale sign [43:16] is that people are worried about it. [43:18] Well, they should be if it's unmanaged. [43:19] If they're not doing anything [43:20] preemptively, if they're allowing people [43:22] just to experiment and they haven't had [43:24] conversations around governance and [43:25] policy uh limitations, thresholds, it [43:29] will get expensive and in fact it'll get [43:30] out of control. And that's the bigger [43:32] problem. It's that it it has that [43:34] capability to take off if it's not being [43:38] managed properly. And there is a lot of [43:40] upfront things that should happen and [43:42] occur before you start playing around [43:44] with it to ensure that you're getting [43:46] the highest value out of those dollars [43:47] invested. [43:50] Absolutely. I I mean, we could go on for [43:53] several more podcasts or webinars like [43:57] this. So, thank you so much, Ben, Dean, [43:59] and Abby for all the good insights on [44:02] today's panel. And thank you to our [44:03] listeners both live and then joining us. [44:06] We will continue the conversation the [44:08] lead dev Slack in the panel Q&A channel. [44:11] And also we'll be sure to share the [44:15] sevenstep idea that Ben wants to gain [44:18] some feedback on so we can continue that [44:20] conversation as well. It feels like the [44:24] state of PHOPS has changed and much [44:27] broader but it's still at the beginning. [44:29] So I think that is something we're going [44:33] to keep talking about. We're going to [44:34] hear more and more about in the upcoming [44:38] months and years. And then we're going [44:40] to continue the conversation in a way on [44:42] March 12th when we'll be taking the [44:46] route from PHOPS to this we haven't [44:48] really touched on it but cloud cost is [44:51] one of the greatest ways to measure [44:54] environmental cost which is something [44:56] certainly in the EU is very important [44:58] and uh so we're going to be talking [45:01] about green ops on March 12th which we [45:05] think is a great way and how to build [45:07] greener software cuts your bottom line [45:09] and how to measure all of that in that. [45:12] So definitely look for that on the lead [45:14] dev events page and thank you everyone [45:17] and I hope you have a great day and good [45:20] luck on your PHOPS journey. [45:23] >> Take care everyone. [45:24] >> Cheers. [45:25] >> Take care. [45:26] >> Bye.  📄 **Full text:** ![[Full Document Contents/Articles/Avoiding Cost Drift in the Age of AI|Avoiding Cost Drift in the Age of AI, YouTube]] ## Highlights - the number one thing and then now we're talking about AI workloads and PHOPS through the modern architectures. Um for me phenopsis is not just finance it's it's an engineering discipline. Um and now it's the time to build a cost aware AI by design and not just by audit and uh I'd be happy to talk through some of the architectural aspects and and some of the economic decisions around it. >> I look forward to joining the leader panel >> and Dean I'm going to need your not your ([View Highlight](https://read.readwise.io/read/01km99dp7x0qy4er7bee1cfd8e)) - Tags: 👻 ai highlighted - PHOPS teams for example are being now asked to manage the cost of AI and a significant number in the upper 60s to 70% are being man asked to manage SAS and a lot of this is driving is being driven through the complexity of the pricing models. It's no longer seatbased licenses even in SAS. It's consumptionbased licenses and then on AI you've got application costs, token ([View Highlight](https://read.readwise.io/read/01km99dp8v3x2sevh7yt8sny9n)) - Tags: 👻 ai highlighted - costs, model operating costs, model training costs and it's all driving the complexity of why AI is dominating the agenda there. Optimization itself again for the engineering community is becoming more table stakes. A lot of this is being automated by what we're seeing as AI for Pinpops but it's very much then in the hands of PHOPS teams working with engineering teams on how they can automate a lot of ([View Highlight](https://read.readwise.io/read/01km99dp9z3b909ch4sbaj2v3c)) - Tags: 👻 ai highlighted - the collaboration between disciplines and this is starting to converge with ITAM and consumptionbased spend again um being being quite apparent and then IT financial management ment and PHOps. So a lot of the larger enterprises are working out new ways to collaborate with this fast pace of technology particularly the disruption of AI. >> Absolutely. Yeah. And we've seen like ([View Highlight](https://read.readwise.io/read/01km99dpc1ktzb8kmnzb41tqdb)) - Tags: 👻 ai highlighted - this new priority emerging like the cost of infrastructure and architecture which is something we've just kind of taken for table stakes before for granted and I think that's very interesting. Ben, this shift left. Can you talk more about what that is? Because that maybe that feels like a DevOps term that maybe FinOps folk or financial people that it's not only people from the DevOps side that are becoming Pinpops, it might be financial people learning more of the pipeline management in the cloud. So, what does that mean and what are you ([View Highlight](https://read.readwise.io/read/01km99dpea2006m9179fdj44ws)) - Tags: 👻 ai highlighted - at some of those data around uh whether it's cost of delta per PR or um run rate which impacts on their scaling change or AI inference cost per transaction. They're clearly the differentiators and and and shifts the focus and ultimately governance is is not to block them. and try to enable them and and data is is absolutely guide them. >> Yeah, I I think I think from a from a Phops Foundation perspective, we're seeing it the same and across two clear ([View Highlight](https://read.readwise.io/read/01km99dpja3ex06dg5b859jgjx)) - Tags: 👻 ai highlighted - Philops culture, it's it's ensuring architecture side, they they basically evaluate what's the cost per transaction whether it's engineers, then they understand what's the cost per feature and if it's platform teams, they understand what's the cost per environment. So it's it's definitely a a shift which we need whether you you wipe code a solution or you build a custom tool for a large institution. Definitely that's the correlation is that it AI isn't special. What makes it special is having the ([View Highlight](https://read.readwise.io/read/01km99dpm9ak56samvtrkmjnpx)) - Tags: 👻 ai highlighted - and that also drives the governance mechanism not just for the I adoption but also PHOPS around AI or or AI for fops as well. Um there's a lot about um a lot um especially with these agent tech stuff going on plenty of things where you can actually use AI for ops as well. Um and AI spending is is not just a cost center anymore. It's pretty much a strategic capital allocation. you you do get um budgets allocated and it's ([View Highlight](https://read.readwise.io/read/01km99dpy9nezj9sb3jhhyj9s2)) - Tags: 👻 ai highlighted