Why Kubernetes is becoming the backbone of modern AI development 🔗 **Source:** [Why Kubernetes is becoming the backbone of modern AI development](https://www.youtube.com/watch?v=T98tarVzr9U) > [!note]+ Document Notes > tags: [ > #kubernetes, > #ai, > #llm, > #finetuning, > #declarativeml, > #gpu, > #mlops, > #opensource, > #devrel, > #allthingsopen > ] > Author: Jason (host, All Things Open) interviewing “Shevii” (guest; DevRel engineer — company named “Couchspace” in transcript; attribution uncertain) > Source: All Things Open / YouTube > Reference: https://www.youtube.com/watch?v=T98tarVzr9U > date created: 2025-09-06 12:00 > > # 🎬 Title: Why Kubernetes is becoming the backbone of modern AI development — ATO interview with “Shevii” > <!-- deep-read:sections start --> ## 🧭 Table of Contents: - [📚 References (up-front)](#-references-include-a-list-of-sources-consulted-and-cited-in-the-article) - [📝 Summary](#-summary) - [📖 Main Content](#-main-content) - [🗒️ Notes (deep, timestamped)](#️-notes) - [🔗 References (full list)](#-references-1) - [🗣️ Paraphrase](#-paraphrase) - [📰 Newsletter](#-newsletter) - [✅ Action Items](#-action-items) - [🧪 Hypotheses](#-hypotheses) - [📊 Methodology](#-methodology) - [❓ Questions](#-questions) - [🔮 Future Research](#-future-research) - [💡 Implications](#-implications) - [🧠 Learning Styles Assessment](#-learning-styles-assessment) - [♿ Accessibility Assessment](#-accessibility-assessment) - [🌍 DEIB Assessment](#-deib-diversity-equity-inclusion-and-belonging-assessment) - [🧩 Logical validity (tone + fallacies)](#-logical-validity) - [🔍 SEO Evaluation](#-seo-evaluation) - [🏁 Conclusion](#-conclusion) ### 📚 References: Include a list of sources consulted and cited in the article. - 🎥 Video: *Why Kubernetes is becoming the backbone of modern AI development* — https://www.youtube.com/watch?v=T98tarVzr9U - 🧑‍🤝‍🧑 Event: All Things Open — https://2024.allthingsopen.org/ *(event link included for context; interview is ATO media content)* ## 📝 Summary: - 🔧 **Fine-tuning, not full retrain**: Enterprises need model adaptation to proprietary data. Fine-tuning adjusts some weights to improve accuracy without the cost of training from scratch ([1:01–2:24](https://www.youtube.com/watch?v=T98tarVzr9U&t=61)). - 🧾 **Declarative ML**: Specify compute per stage (prep/train/validate) so GPUs are used where needed, saving cost and improving throughput ([3:11–4:36](https://www.youtube.com/watch?v=T98tarVzr9U&t=191)). - 🐳 **Kubernetes for AI**: Mature orchestration, GPU plugins, and scaling make K8s a strong fit for training and inference; techniques like GPU time-slicing/sharding appear in the workflow ([5:05–6:17](https://www.youtube.com/watch?v=T98tarVzr9U&t=305)). - 🧑‍💻 **Developers vs AI**: AI accelerates boilerplate and testing, yet architects remain vital for system design and production readiness ([6:28–7:31](https://www.youtube.com/watch?v=T98tarVzr9U&t=388)). - 🧠 **Productivity stack**: Local LMs + “recall” style timelines can stitch Slack/Jira/GitHub/IDE context via local OCR and vectors for private, searchable memory ([7:37–8:58](https://www.youtube.com/watch?v=T98tarVzr9U&t=457)). - 🗣️ **Community note**: ATO is pitched as a community-run conference; CFPs invite builders and maintainers to share work ([9:19–10:03](https://www.youtube.com/watch?v=T98tarVzr9U&t=559)). ## 📖 Main Content: - 🧩 **Enterprise LLM adoption**: Out-of-the-box models often miss org-specific nuance. Fine-tuning with curated tickets, chats, and manuals can raise accuracy for tasks like support bots and RAG-assisted agents. - 🧮 **Declarative resource planning**: Treat the ML pipeline as stages with explicit resource classes. CPUs for ETL; GPUs for training; small GPU/CPU for validation. This avoids “all-GPU, all-the-time” waste. - ☸️ **Why K8s**: It brings schedulers, quotas, autoscaling, node pools, and device plugins into one ops surface. That lets teams run multi-tenant AI, isolate noisy neighbors, and burst to cloud. - 🧑‍🔧 **Human in the loop**: Code generators speed scaffolding. However, architecture, performance profiling, and prod hardening still need human judgment. - 🧰 **Context unification**: A dev’s day spans Slack ⟷ Jira ⟷ GitHub ⟷ IDE. Local pipelines that OCR screenshots and index them as vectors can rebuild “what I was doing” without sending data off-box. - 🧑‍🤝‍🧑 **Open source & events**: Conferences like ATO seed connections among maintainers, vendors, and users; that social graph drives project momentum as much as code. --- ### 🗒️ Notes > Each subheading links to a precise moment in the interview. Bullets include key claims, constraints, and actionable ideas. #### 🔗 [00:08 — Intro: who, what, why](https://www.youtube.com/watch?v=T98tarVzr9U&t=8) - 🎤 **Host/guest**: Jason (ATO) with *Shevii* (DevRel engineer; works with gen-AI demos, agents, vector search). - 🧭 **Focus**: LLMs at scale, K8s, open source contributions (Kubernetes, WebAssembly) (names/projects as stated). #### 🔗 [01:01 — Why fine-tune LLMs](https://www.youtube.com/watch?v=T98tarVzr9U&t=61) - 🧩 **Problem**: Enterprise data is proprietary; base models lack org-specific context. - 💸 **Cost reality**: Full retrain = months + many GPUs; fine-tune = narrower data + selected weights. - 🎯 **Outcome**: Better task accuracy with manageable spend. #### 🔗 [02:19 — What changes during fine-tuning](https://www.youtube.com/watch?v=T98tarVzr9U&t=139) - 🧮 **Mechanics**: Deep nets store patterns in weights; fine-tuning nudges a subset with a small, labeled dataset. - 🧪 **Scope**: Choose domain-specific corpora (e.g., support threads) to bias outputs toward valid answers. #### 🔗 [2:46–3:06 — Example: support chatbot](https://www.youtube.com/watch?v=T98tarVzr9U&t=166) - 🤖 **Tactic**: Train on historic tickets to answer similar queries with fewer escalations. - 🔁 **Guardrails**: Add validation against knowledge base; log-feedback loop to spot drift. #### 🔗 [03:11 — Declarative ML: compute as code](https://www.youtube.com/watch?v=T98tarVzr9U&t=191) - 🧾 **Idea**: For each pipeline step, declare CPU/GPU needs and limits. - 🧱 **Stages**: Data prep (CPU), training (N GPUs), validation (small CPU/GPU). - 💡 **Win**: Cost control, queue predictability, less idle GPU burn. #### 🔗 [05:05 — Kubernetes for AI workloads](https://www.youtube.com/watch?v=T98tarVzr9U&t=305) - ☸️ **Why K8s**: Distributed by design; device plugins expose GPUs to pods. - 🧩 **Techniques**: Mentions GPU sharding/time-slicing as utilization tools. - 🏗️ **Topology**: On-prem clusters or cloud; same control plane patterns. #### 🔗 [06:28 — Will AI replace developers?](https://www.youtube.com/watch?v=T98tarVzr9U&t=388) - 🧠 **Position**: AI will write boilerplate and basic tests; humans still own system design and prod-grade optimization. - 🧭 **Shift**: Devs move “up” toward architecture, perf, and constraints management. #### 🔗 [07:37 — Productivity: local AI + “recall”](https://www.youtube.com/watch?v=T98tarVzr9U&t=457) - 🧰 **Tooling mention**: Pieces for Developers; local small LMs. - 🗂️ **Pattern**: Periodic screenshots → local OCR → vector DB → cross-tool memory (Slack/Jira/GitHub/IDE). - 🔒 **Privacy**: Kept on device; avoids sending workstream data to third-party clouds. #### 🔗 [09:19 — ATO community plug + CFP](https://www.youtube.com/watch?v=T98tarVzr9U&t=559) - 📣 **Message**: ATO is community-run; attend and submit talks. - 🌱 **Audience**: Builders, maintainers, and open source contributors. --- ## 📖 References: - 🎥 Interview video: https://www.youtube.com/watch?v=T98tarVzr9U - 🧑‍🤝‍🧑 All Things Open: https://2024.allthingsopen.org/ ## 🗣️ Paraphrase: - ✂️ **Fine-tuning vs. retraining** (from [1:01–2:24](https://www.youtube.com/watch?v=T98tarVzr9U&t=61)): *Original idea*: Big models don’t match proprietary data; full retraining is slow and costly; tweak a subset of weights with a small, targeted dataset to fit the business case. *Paraphrase*: Instead of rebuilding a model from zero, bring a trained model closer to your domain by updating only select parameters using curated, company-specific examples. It’s cheaper, faster, and often accurate enough for production. - ✂️ **Declarative ML** (from [3:11–4:36](https://www.youtube.com/watch?v=T98tarVzr9U&t=191)): *Original idea*: Define compute needs per pipeline step to avoid wasting GPUs where CPUs suffice. *Paraphrase*: Treat each stage of the ML workflow like an infrastructure recipe. Assign just-right resources to prep, training, and validation, so GPUs work only where they’re essential. ## 📰 Newsletter: **Subject**: Kubernetes isn’t just for microservices anymore — it’s your AI runway **Byline**: This week in WordPress & open source - ⚡ **Learn dev**: LLMs need domain fit. Fine-tune on your docs and tickets; don’t jump to full retrain. - ☸️ **Stack pick**: Kubernetes + declarative ML lets you map GPUs to only the hot paths. Budgets breathe again. - 🛡️ **Privacy win**: Local LMs and “recall” timelines can unify Slack/Jira/GitHub context without shipping data off-box. - 🧑‍🤝‍🧑 **Community**: ATO stays community-run. If you build or maintain OSS, consider a talk. - 🧭 **Links**: Watch the ATO interview: https://www.youtube.com/watch?v=T98tarVzr9U ## ✅ Action Items: - 🧭 **Reusable review prompt tailored to DevRel/OSPO goals** > **Purpose**: For each article/interview you process, map impact on open-source advocacy, contributor onboarding, and education within WordPress and adjacent infra (K8s/AI). > > **Prompt**: > 1) **Summary in 6 bullets**: What the piece asserts; target audience; claimed benefits; costs/risks; OSS ties; education hooks. > 2) **Alignment**: Explain how this advances: (a) contributor engagement (first issues, docs edits, mentorship), (b) sustainable funding (co-marketing, grants, sponsorships), (c) education (courses, workshops, curricula). > 3) **WordPress lens**: Identify intersections with WP core, Hosting, Plugin/Theme ecosystem, LearnWP content, or Make teams. > 4) **OSPO path**: Extract policies/processes an OSPO would formalize (licensing, compliance, contribution guidelines, internal enablement). > 5) **Experiments**: Propose 30/60/90-day tests with KPIs (contributors retained, PR cycle time, docs completion, course enrollments). > 6) **Assets to ship**: Tutorials, sample repos, slide decks, office hours, contributor journey maps, sponsor one-pagers. > 7) **Partners**: Name likely co-owners (Make teams, hosts, tooling vendors, education orgs). > 8) **Risks & ethics**: List privacy, security, licensing, and bias concerns; add mitigation steps. > 9) **Decision**: Recommend proceed/pilot/park with rationale. --- ## 🧪 Hypotheses: - 🧪 **H1**: Declarative resource profiles on K8s reduce per-epoch training cost ≥25% vs static GPU allocation for similar workloads. - 🧪 **H2**: Fine-tuned support models cut human escalations ≥15% compared to untuned base models on the same ticket set. - 🧪 **H3**: Local “recall” context + on-device LMs reduce dev context-switch time ≥20% in multi-tool workflows. ### 📊 Methodology: - 🧪 **Cost A/B**: Train identical tasks with (A) static GPU pods vs (B) stage-scoped requests/limits; track GPU-hours, idle %, wall time. - 📈 **Support bot trial**: Randomly route tickets to base vs fine-tuned bot; measure resolution rates, CSAT, time-to-answer. - 🧰 **Productivity RCT**: Cohorts with/without local recall; log task switches/hour and PR throughput; run 4-week study. ### ❓ Questions: - ❓ Which pipeline stages benefit most from GPU time-slicing versus exclusive allocation? - ❓ What governance is needed to keep fine-tune datasets privacy-safe and license-clean? - ❓ How do we teach “declarative ML” to ops teams that don’t speak ML yet? ### 🔮 Future Research: - 🔮 Scheduling policies for multi-tenant GPU clusters under spiky inference loads. - 🔮 Standard formats for fine-tune datasets, including provenance and revocation. - 🔮 Teaching patterns that bridge WordPress devs into K8s/ML ops. ### 💡 Implications: - 📉 Lower training spend expands who can participate in AI. - 🧑‍🏫 Clear curricula grow contributor pools across docs, data, and infra. - 🛡️ Privacy-first local tooling keeps enterprise data off third-party LLMs. --- ### 🧠 Learning Styles Assessment: - 📚 **Readability**: Conversational interview; approximate **Grade 11–13** reading level. 1) 🖼️ **Visual** - ✅ Present: Video format. - ➕ Improve: Add diagrams for pipeline stages and K8s GPU topology. 2) 🎧 **Aural** - ✅ Present: Spoken content. - ➕ Improve: Provide a clean audio-only feed + 5-min summary. 3) 📝 **Verbal** - ⚠️ Present: High-level terms; few definitions. - ➕ Improve: Glossary (fine-tune, sharding, time-slicing). 4) ✋ **Kinesthetic** - ❌ Present: None. - ➕ Improve: Include a hands-on lab (deploy a fine-tune job on K8s). 5) 🧠 **Logical** - ✅ Present: Stepwise case for declarative ML. - ➕ Improve: Show numbers (GPU-hours saved, throughput gains). 6) 👥 **Social** - ✅ Present: Interview format. - ➕ Improve: Link to forums/Discord for follow-ups. 7) 🧍 **Solitary** - ⚠️ Present: Limited. - ➕ Improve: Self-paced workbook with checkpoints. 8) 🌿 **Naturalistic** - ❌ Present: None. - ➕ Improve: Use energy-efficiency framing (carbon per epoch). ### 📊 Accessibility Assessment: - 🗣️ **Screen reader**: Ensure accurate captions and a cleaned transcript with speaker labels. - ⌨️ **Keyboard**: Player controls must be tabbable; visible focus rings. - 🌓 **Contrast**: High-contrast lower-thirds and subtitles. - 🔍 **Text resizing**: Transcript page should reflow at 200% without loss. - 🖼️ **Alt text**: Thumbnails and any on-screen diagrams need descriptive alt text. ### 🌍 DEIB (Diversity, Equity, Inclusion, and Belonging) Assessment: - 🌎 **Cultural relevance**: Global OSS angles present; specific regional examples absent → add. - 🗣️ **Inclusive language**: Generally neutral; define jargon to include newcomers. - 🧠 **Neurodiversity**: Break concepts into chunks; provide summaries and checklists. - 👥 **Representation**: One guest + one host; broaden future panels by role, region, and background. - 🛟 **Psychological safety**: Invite beginner questions; share “getting started” resources prominently. --- ### 🧩 Logical validity: **Tone**: Practical, optimistic, vendor-neutral in framing; occasional speculative claims. **Strong reasoning (with timestamps):** 1) ✅ **Causal reasoning** — Fine-tuning yields domain fit at lower cost ([1:01–2:24](https://www.youtube.com/watch?v=T98tarVzr9U&t=61)). - 🔧 *Strength*: Connects proprietary data gap → method (fine-tune) → outcome (accuracy). - 🛠️ *Improve*: Provide quantitative deltas (accuracy, cost, latency). 2) ✅ **Process logic** — Declarative ML assigns fit-for-purpose resources ([3:11–4:36](https://www.youtube.com/watch?v=T98tarVzr9U&t=191)). - 🔧 *Strength*: Stage-by-stage mapping is clear and generalizable. - 🛠️ *Improve*: Add sample manifests and SLOs. 3) ✅ **Balanced view** — AI augments devs; architects remain key ([6:28–7:31](https://www.youtube.com/watch?v=T98tarVzr9U&t=388)). - 🔧 *Strength*: Avoids extremes; matches current practice. - 🛠️ *Improve*: Cite studies on code-gen productivity and defect rates. **Potential fallacies / weak spots:** 1) ⚠️ **Appeal to authority / Uncited claim** — “OpenAI said they’re still losing money…” ([1:35–1:45](https://www.youtube.com/watch?v=T98tarVzr9U&t=95)). - ❌ *Why weak*: Lacks a specific source/timeframe; finances change. - ✅ *Fix*: Cite a dated report or remove. 2) ⚠️ **Hasty generalization / Uncited** — “OpenAI training clusters run on K8s…” ([5:33–5:50](https://www.youtube.com/watch?v=T98tarVzr9U&t=333)). - ❌ *Why weak*: Broad, company-wide claim without reference. - ✅ *Fix*: Provide a public engineering post or rephrase as “many orgs…” 3) ⚠️ **Ambiguity** — Tool names/projects (“Couchspace”, “recall”) are mentioned without clear identifiers ([0:20](https://www.youtube.com/watch?v=T98tarVzr9U&t=20), [8:02–8:58](https://www.youtube.com/watch?v=T98tarVzr9U&t=482)). - ❌ *Why weak*: Hard to verify; names may be misspoken. - ✅ *Fix*: Link to canonical repos and privacy models. 4) ⚠️ **Overprecision** — “Thousands of GPUs” for GPT-level models ([3:36–3:47](https://www.youtube.com/watch?v=T98tarVzr9U&t=216)). - ❌ *Why weak*: Plausible, but needs scope (model size, training length). - ✅ *Fix*: Give ranges or cite benchmarks. **Bias watch:** - 🧑‍💼 **Role bias**: Guest is DevRel for a database vendor; may lean toward developer productivity narratives and K8s alignment. - 🔐 **Privacy framing**: Local “recall” is promoted; risks and legal considerations (screen capture at work) aren’t discussed. --- 🔍 **DARVO Analysis Prompt (Applied)** > **Context**: This is a technical interview; no allegations or interpersonal conflicts. 1) **Deny** — None detected. No refutation of wrongdoing or minimization behavior. 2) **Attack** — None detected. No personal attacks on critics or competitors. 3) **Reverse Victim/Offender** — None detected. No inversion narratives. **Evidence**: Content focuses on architectures, costs, and workflows ([1:01–6:17](https://www.youtube.com/watch?v=T98tarVzr9U&t=61), [7:37–9:15](https://www.youtube.com/watch?v=T98tarVzr9U&t=457)). **Rhetorical notes**: Anecdotal support and practitioner framing; no DARVO cues. --- ### 🔍 SEO Evaluation: 1) 🔑 **Keywords**: *Kubernetes for AI*, *LLM fine-tuning*, *declarative ML*, *GPU time-slicing*, *MLOps on Kubernetes*. 2) 🧾 **Meta** (≤160 chars): “LLM fine-tuning on Kubernetes: declarative ML, smarter GPU use, and why devs become architects. Notes from an ATO interview.” 3) 🏷️ **Title** (≤60 chars): “Kubernetes for AI: Fine-Tuning, Declarative ML, GPUs” 4) 🧱 **Headers**: Use H2s for Fine-Tuning / Declarative ML / K8s GPUs / Devs & AI / Privacy-first tooling. 5) 📏 **Length**: Target 1,500–2,000 words with code blocks and manifests. 6) 🔗 **Links**: Internal to prior K8s and AI posts; external to the interview and official K8s docs (device plugins, scheduling). 7) 🔗 **URL**: `/blog/kubernetes-ai-fine-tuning-declarative-ml` 8) 🖼️ **Images**: Alt text like “K8s GPU scheduling diagram”; compress images. 9) 📱 **Mobile**: Lazy-load embeds; avoid heavy GIFs. 10) 👋 **Engagement**: Add a “Try this lab” CTA; offer a sample repo. 11) 🧩 **Schema**: `Article` + `VideoObject` with clip timestamps. 12) 🗣️ **Social**: OG image with K8s + AI motif; Twitter Card summary_large_image. ## 🎯 Conclusion: - Kubernetes plus declarative ML gives teams a practical way to scale AI without melting budgets. - Fine-tuning meets enterprise needs faster than full retraining. - Developers won’t vanish; they’ll steer architecture and reliability while AI drafts code. - Local, privacy-first context tools can cut friction in daily dev work. - Community hubs like ATO keep the ecosystem healthy by connecting builders, maintainers, and learners. <!-- deep-read:sections end --> ## 📺 Video  --- # 📄 Full Transcript *Auto-generated from YouTube captions · timestamps roughly every 30 seconds · [watch on YouTube](https://www.youtube.com/watch?v=T98tarVzr9U)* **[00:08]** I'm Jason with the All Things Open team. I'm here with Shiovi. We're going to talk about large language models, achieving AI at scale with Kubernetes and much more. Shiovi, please introduce yourself to the all things open community. >> Yeah, hi. So, I'm Shiovi. I currently work at Couchspace. It's a NoSQL database company as a developer evangelist or you can call it a developer relations engineer. So most of my day-to-day work is building things really cool demos and integrations with the help of generative AI such as AI agents or vector searchs. So those are the things that I'm pretty much playing **[00:39]** around on a day-to-day basis. But apart from that I'm also a proactive contributor to open source projects like Kubernetes web assembly a project called so. So I like to diversify my interests in AI web assembly. >> Sounds great. Want to talk more about large language models. You're passionate about fine-tuning LLMs. what does this mean to you and why is it important? >> Yeah, so I think one of the core fundamentals of LLMs is that while they have been trained on vast amount of data, uh when you look at enterprises and how they adopt LLMs, a lot of times **[01:09]** the native module or the native model will not just work out of the box for them because they are dealing with a lot of proprietary data on which these LLMs have not been trained upon. Now there are three different ways in which you can address this problem. One is like completely retraining a model from scratch. But that would be too expensive because as we know that these models are so heavy that take probably like months and tons of GPUs to just run these. We have seen this uh where OpenAI has said that they're still losing money because they train these models and the amount **[01:41]** that they're charging from people it's very less right. So the other alternative is that you don't retrain the entire model but you just adjust certain weights of the model and because the idea is that these large models are still based on the core principle of deep learning models which have these weights and biases. So the weights are what basically store the metadata of the original training data set. So idea is that you don't just go ahead and retrain everything from scratch but you slightly adjust these some of these weights in **[02:12]** your model which are uh done on the basis of a set of new data set that you have just prepared just for the finetuning. So it's a lot cheaper as compared to doing like full retraining but it still gives you the performance benefits of being able to optimize these LLMs for a business use case. So any company who is trying to leverage an LLM I think it's for a business critical use case you don't want to spend too much money on retraining then finetuning is your best option because it's cheaper and it brings more uh accuracy for the **[02:43]** models when you're trying to deploy them in a use case like a customer chatbot. So for example like you're building a support chatbot and uh you know rather than giving you like a generic answer you fine-tune it based on the previous threads or all of your uh support tickets that have been asked earlier so that if there's a similar kind of query that has been asked by new user you can uh then reply them with you know a supporting answer. >> Thanks for sharing that information. Want to get some more of your knowledge. What's a declarative ML? Yeah. So the idea here is that as you know that **[03:14]** whether it's retraining or fine-tuning these models, it's still very expensive because you need GPUs right now. If you look at how DevOps teams have generally worked, they don't come necessarily with a lot of experience deploying machine learning models. So SR teams, they might be well equipped with Kubernetes, Docker, but when you talk about software versus machine learning, they there are some critical differences because machine learning is a lot more compute intensive, right? So just taking an example of a GPT 40 model, it will probably require you to have thousands **[03:44]** of GPUs to train that model and it can be very expensive. So the idea here is then rather than just simply throwing some GPUs and doing the training which will be very expensive. What you do is that you define because as you know that within machine learning there are multiple steps. There is the data preparation step. Then there is the training of the model and then there's running validation. So across each and every different part of your machine learning workflow, you declare what is the amount of compute resources that you want to basically give to that particular step. So something like data **[04:15]** cleaning will not require a lot of compute. So you can just assign CPU over there. But for training which requires the most amount of compute, you will say that I need four GPUs for this workload. Validation similarly might not require as much amount of GPUs. So this way you selectively pick and choose your infrastructure for specific parts of your machine learning workload and that makes it a lot more uh resource uh in I mean a lot more resource friendly and also cost effective. >> How are people using Kubernetes to achieve AI at scale? >> Yeah, I think like you know Kubernetes **[04:45]** it's no secret last year we celebrated 10 years of Kubernetes. It is uh an open source technology that has really impacted the cloud space as we know right there's no formal introduction needed for Kubernetes as such but we are seeing a significant adoption of Kubernetes for also running AI workloads. So as I mentioned that whether we talk about fine-tuning whether we talk about inference of these LLMs or even just you know regular AI models it requires you to have uh technology that can scale up because you **[05:16]** know when you have multiple people coming in with requests trying to use your model it it is very resource and compute inensive. So you want a technology that has the scalability that can really you know perform well. So just taking an example of OpenAI, they have released this very famous blog post where all of their training clusters for all of their state-of-the-art models are completely trained on GPUs that are running on top of Kubernetes because as we know that Kubernetes is a truly distributed technology and it has native **[05:47]** support for being able to then add GPUs to your pods and you can do things like GPU sharding uh or time slicing of your GPUs. Again, I'll not probably go too deep into them, but these are techniques where you can basically run the fine-tuning and training of your models in a much more effective manner. And you get the scalability of Kubernetes where you can deploy it on a on-prem cluster or perhaps if you have a cloud-based hosting, you get a lot of flexibility when it comes to using Kubernetes as a technology. So, you know, we have just **[06:17]** seen that it has very seamlessly transitioned from core software to also then working very well with AI workloads. Kubernetes 10 years can't believe it. >> Yeah, >> there's a lot of discussion about AI's role in development. Do you think AI will replace developers? >> I don't think so. Uh there will be certain aspects where it will like probably like if you talk about the baseline software testing you know or probably like writing very basic level code. Um we are already seeing technologies like v 0ero from versell **[06:48]** like it can create fullfledged frontends and now there are technologies like github copilot agents which can generate full stack apps for you but I think fundamentally there is still a challenge that they will not be still you know like production ready you still need someone who understands the code from base level and then they can make that those level of optimizations. So when it comes to like proper system design where you're trying to design scalable systems and you want to have proper highle design and low-level design, you still need software architects. So I think **[07:19]** what us as developers will do is that we'll wear that hat of a software architect where we no longer concerned about writing that HTML code but we are actually now perfecting it and we are optimizing it which AI might also become better but we'll still be the guiding force for that AI. >> Sounds good. What's a new tool that you've been using lately to boost your productivity? >> Yeah, so I have been using this tool called pieces for developers. Now, it is not technically an open source project, but you can actually run it. You can run small language models which are **[07:49]** completely open source directly on your system. And the idea here is that when we talk about productivity, right, as developers, it's not just the code that we write in our ID. We use a multitude of different tools, right? So we constantly we uh transition between slack for messaging or pro perhaps like you know we are on GitHub answering a query or we are reviewing a PR and then we are going back to our code. A lot of times there might be use cases where all of these are involved like let's say a manager gives you a task on Jira. So you **[08:20]** go through that entire Jira task and you start to implement it. You have discussion with manager on slack. Now there's no specific tool that can give you context of everything and how all of these are linked with each other. it will only have context of your ID, right? So, we are looking at open source tools like recall and there are in fact like open source projects written in rust that also do this. So, they will basically take a screen grab of your entire screen from time to time and store these screenshots locally. It will run an OCR again locally in your system and save these in a vector database and **[08:52]** then with the help of that now it's not only aware of what you did like let's say 10 minutes ago but then it will be able to understand the context from your slack and how is it related to your ID. So that's I think like the best productivity booster where you don't have to worry about your uh time getting split across multiple tools and you still get relevant contexts interlin between different tools. >> That's really cool. Yeah. What's one tip, best practice or piece of advice you'd like to share with the ATO community? Yeah, I think the as you know that ATO as a brand for open source it's **[09:24]** there right there's no formal introduction needed but uh one feedback that I'll probably give to everyone is that definitely if you ever get a chance do come and attend because it is the best opensource conference that I've been part of and trust me like I go to a lot of conferences right a lot of them are vendor focused this is pure community run by a community and of course ATO has its call for proposals open right now. I'm not sure when this podcast get published, but you can always apply for the next year because **[09:55]** it's always looking for people who are building good and maintaining good open source projects, right? So, I'll I'll I'll say that just come and apply. >> Sounds great. Thanks for sharing and thanks for your time today. I really appreciate it. >> Thank you so much. >> All right.