devrelshow episode 43 - Brenna Buuck from MinIO

devrelshow episode 43 - Brenna Buuck from MinIO 🔗 **Source:** [devrelshow episode 43 - Brenna Buuck from MinIO](https://www.youtube.com/watch?v=Uq5NNE-f7ts) > [!note]+ Document Notes > tags: [developer-relations, devrel, data-engineering, object-storage, minio] > author: DevRel Show (Fred) > source: YouTube > reference: https://www.youtube.com/watch?v=Uq5NNE-f7ts > date created: 2025-01-15 10:30 > > # 🏷️ Title: devrelshow episode 43 - Brenna Buuck from MinIO >  ## 🧰 Action Inbox - [H1] Explore MinIO's AI Store Tables for data lakehouse architecture — 00:30:15 #data-architecture - [H2] Consider open source alternatives to AWS S3 Tables for on-premise solutions — 00:32:45 #cloud-alternative - [H3] Investigate PyIceberg for Python-based data lake operations — 00:36:20 #python #data-engineering ## 🧭 Table of Contents - [📝 Summary](#-summary) - [📖 Main Content](#-main-content) - [🗒️ Notes](#️-notes) - [📚 References](#-references) - [🗣️ Paraphrase](#️-paraphrase) - [📰 Newsletter](#-newsletter) - [✅ Action Items](#-action-items) - [🧪 Hypotheses](#-hypotheses) - [📊 Methodology](#-methodology) - [❓ Questions](#-questions) - [🔮 Future Research](#-future-research) - [💡 Implications](#-implications) - [🧠 Learning Styles Assessment](#-learning-styles-assessment) - [📊 Accessibility Assessment](#-accessibility-assessment) - [🌍 DEIB Assessment](#-deib-assessment) - [✅ Logical Validity](#-logical-validity) - [🛡️ DARVO Analysis](#-darvo-analysis) - [🎯 Conclusion](#-conclusion) ### 🔗 References - [MinIO](https://min.io) - [Apache Iceberg](https://iceberg.apache.org) - [PyIceberg](https://py.iceberg.apache.org) ## 📝 Summary - Developer advocate Brenna Buuck discusses her transition from data engineering to developer relations at MinIO - MinIO provides on-premise S3-compatible object storage with over 2 billion Docker pulls worldwide - The company introduced AI Store Tables, embedding Apache Iceberg catalog API directly into object storage - This eliminates the need for separate metadata databases traditionally required for data lakehouses - Key benefits include cost savings (especially beyond 10 petabytes), simplified architecture, and unified structured/unstructured data access ## 📖 Main Content Brenna Buuck serves as a Subject Matter Expert on Databases and Modern Data Links at MinIO, though she describes her role as developer advocacy work. She transitioned from traditional data engineering to developer relations, motivated by her passion for teaching and helping others learn about data technologies. MinIO provides on-premise, S3-compatible object storage that serves as an alternative to cloud solutions like AWS S3 or Azure Blob Storage. The company has achieved significant adoption with over 2 billion Docker pulls worldwide and maintains both open-source (AGPL licensed) and enterprise versions. A key innovation discussed is AI Store Tables, which embeds the Apache Iceberg catalog API directly into MinIO's object storage. This architectural approach eliminates the traditional requirement for separate metadata databases (like MySQL or PostgreSQL) that external Iceberg catalogs typically need. The result is a simplified data lakehouse architecture that reduces operational complexity, especially for on-premise deployments. The technology enables unified access to both structured data (through Iceberg tables) and unstructured data (PDFs, videos, images) stored in the same object storage system. This convergence is particularly valuable for AI initiatives and analytics workflows that require both data types. ## 🗒️ Notes #### ⏱️ [00:00](https://youtu.be/Uq5NNE-f7ts?t=0) — [H1] New Year Introduction and Show Context - Key idea: First episode of 2025 with continuing guest lineup - Evidence: "First episode of the year, episode 43. We're continuing to talk about DevRel" #### ⏱️ [02:30](https://youtu.be/Uq5NNE-f7ts?t=150) — [H2] Brenna's Introduction and Role at MinIO - Key idea: Subject Matter Expert title encompasses developer advocacy responsibilities - Evidence: "I work as a developer evangelist basically at MinIO, which is object storage software company" #### ⏱️ [04:45](https://youtu.be/Uq5NNE-f7ts?t=285) — [H3] MinIO Product Overview - Key idea: MinIO provides on-premise S3-compatible object storage - Evidence: "MinIO is basically on-prem S3. It's truly S3 native. There's no map data gateway" #### ⏱️ [06:20](https://youtu.be/Uq5NNE-f7ts?t=380) — [H1] Developer Relations Title Discussion - Key idea: Title flexibility reflects both expertise and practical communication needs - Evidence: "I say developer advocate because it gives people a better sense of what I do day-to-day, but it's shorter on a name tag" #### ⏱️ [09:15](https://youtu.be/Uq5NNE-f7ts?t=555) — [H2] Technical Audience Understanding - Key idea: Technical software buyers have technical backgrounds regardless of current job titles - Evidence: "Everybody that buys technical software has a degree of technical aptitude" #### ⏱️ [11:30](https://youtu.be/Uq5NNE-f7ts?t=690) — [H3] Previous Role at StreamSets - Key idea: StreamSets provided visual ETL tools with data drift detection capabilities - Evidence: "StreamSets had this ability to detect and remediate data drift" #### ⏱️ [16:45](https://youtu.be/Uq5NNE-f7ts?t=1005) — [H1] Career Transition Motivation - Key idea: Teaching and knowledge sharing drove the move from engineering to developer relations - Evidence: "I was most fulfilled by teaching undergrads political science" #### ⏱️ [19:20](https://youtu.be/Uq5NNE-f7ts?t=1160) — [H2] Benefits of Developer Relations Role - Key idea: More human interaction and positive feedback compared to traditional data engineering - Evidence: "Nobody ever talks to you unless something's broken in data engineering" #### ⏱️ [25:30](https://youtu.be/Uq5NNE-f7ts?t=1530) — [H1] Data Lakehouse Architecture Introduction - Key idea: Apache Iceberg enables data lakehouse functionality through metadata management - Evidence: "Iceberg metadata is what gives you really cool things like time travel and schema evolution" #### ⏱️ [28:45](https://youtu.be/Uq5NNE-f7ts?t=1725) — [H2] Cost Effectiveness of On-Premise Storage - Key idea: On-premise becomes more cost-effective than cloud at approximately 10 petabytes - Evidence: "About 10 petabytes of data where it becomes much more cost effective to store your data on-prem" #### ⏱️ [30:15](https://youtu.be/Uq5NNE-f7ts?t=1815) — [H1] AI Store Tables Innovation - Key idea: Embedded Iceberg catalog eliminates need for separate metadata databases - Evidence: "Our version is just your object storage with your built-in catalog service" #### ⏱️ [36:20](https://youtu.be/Uq5NNE-f7ts?t=2180) — [H2] PyIceberg Demo Setup - Key idea: PyIceberg provides Python access to Iceberg without JVM requirements - Evidence: "I like PyIceberg because I have been victimized personally by Spark and PyIceberg doesn't require JVM" #### ⏱️ [42:10](https://youtu.be/Uq5NNE-f7ts?t=2530) — [H1] Unified Data Access Benefits - Key idea: Structured and unstructured data can be accessed together in same location - Evidence: "Your structured data and unstructured data can come together in a really fast and important way" #### ⏱️ [45:30](https://youtu.be/Uq5NNE-f7ts?t=2730) — [H2] Data Lakehouses as AI Foundation - Key idea: Data lakehouses provide the clean data foundation necessary for AI initiatives - Evidence: "Clean data, vast amounts of clean data, is really critical for AI initiatives to work" ## 📚 References - [MinIO](https://min.io/) - [Apache Iceberg](https://iceberg.apache.org/) - [PyIceberg Documentation](https://py.iceberg.apache.org/) - [StreamSets](https://streamsets.com/) ## 🗣️ Paraphrase - MinIO offers on-premise object storage that's fully compatible with Amazon S3 APIs, serving as an alternative for organizations wanting to avoid cloud vendor lock-in - The new AI Store Tables feature simplifies data lakehouse architecture by embedding the Iceberg catalog directly into the storage layer, eliminating operational complexity - Transitioning from data engineering to developer relations provides more opportunities for teaching and human interaction while still maintaining technical depth ## 📰 Newsletter 🎯 **New Year, New Data Architecture: MinIO's AI Store Tables Revolution** Starting 2025 strong with insights from Brenna Buuck at MinIO! If you're building data-intensive applications, this episode reveals a game-changing approach to data lakehouses. **Key Takeaway:** MinIO's AI Store Tables embed Apache Iceberg catalog API directly into object storage, eliminating the traditional need for separate metadata databases. This means simpler architecture, lower operational overhead, and unified access to both structured and unstructured data. **For WordPress Developers:** While this might seem outside our usual scope, consider how media-heavy WordPress sites could benefit from object storage solutions, especially for large-scale content management systems. **Career Insight:** Brenna's transition from data engineering to developer relations highlights how technical expertise combined with teaching passion creates powerful developer advocacy roles. Resources: [Learn WordPress](https://learn.wordpress.org/), [Developer Docs](https://developer.wordpress.org/), [WP-CLI](https://wp-cli.org/) ## ✅ Action Items #### DevRel - Developer advocacy: Create technical content bridging structured/unstructured data use cases, conference talks on data lakehouse architecture - Education: Develop workshops on Iceberg catalog implementation, office hours for data engineering questions - Developer experience: Improve PyIceberg documentation, create getting-started tutorials - Developer marketing: Content strategy around on-premise vs cloud cost comparisons - Community management: Moderate data engineering forums, gather feedback on AI Store Tables #### Community - Maintainers: Support Apache Iceberg project governance, contribute to PyIceberg roadmap - Contributors: Onboarding programs for data engineering newcomers, first-issue programs for MinIO - Sponsors: Transparent budgets for open source contributions, impact reports on community engagement #### Data Engineering - Architecture: Evaluate data lakehouse implementations, cost analysis for on-premise storage - Performance: Benchmark Iceberg catalog performance, optimize query engines - Integration: API compatibility testing with S3 ecosystem, multi-cloud data strategies - Security: Data governance frameworks, access control for unified data access ## 🧪 Hypotheses - Organizations with >10 petabytes of data will find on-premise object storage more cost-effective than cloud solutions - Embedded Iceberg catalogs will reduce operational complexity compared to external catalog architectures - Unified structured/unstructured data access will accelerate AI initiative development - Developer advocates with deep technical backgrounds will be more effective at engaging technical audiences ### 📊 Methodology - Cost analysis comparing cloud storage tiers vs on-premise infrastructure at various data scales - Performance benchmarking of embedded vs external Iceberg catalog implementations - Survey data engineers about operational complexity factors in data lakehouse deployments ### ❓ Questions - How does the embedded Iceberg catalog handle high-availability and disaster recovery scenarios? - What are the performance implications of embedding catalog API within object storage? - How does this architecture scale compared to traditional external catalog approaches? ### 🔮 Future Research - Multi-cloud data lakehouse architectures with hybrid catalog strategies - Integration patterns between AI Store Tables and major query engines - Developer experience improvements for data lakehouse operations ### 💡 Implications - Practice: Simplified data lakehouse deployments with reduced operational overhead - Policy: Potential shift in enterprise data architecture standards toward embedded catalogs - Security: Unified access patterns may require new governance frameworks - Education: Need for training materials on modern data lakehouse architectures - Funding: Cost savings from reduced cloud storage bills could fund additional on-premise infrastructure ### 🧠 Learning Styles Assessment Reading level estimate: College level (technical content with domain-specific terminology) #### 👀 Visual learners - Present: Architecture diagrams, cost comparison charts - Improve: Interactive data flow visualizations, storage tier comparisons #### 🔊 Aural learners - Present: Video interview format, spoken explanations - Improve: Podcast version, technical deep-dive audio supplements #### 🗣️ Verbal learners - Present: Clear definitions of data lakehouse, Iceberg, object storage - Improve: Glossary of terms, analogy-based explanations #### 🧩 Kinesthetic learners - Present: Live code demonstration, hands-on PyIceberg examples - Improve: Step-by-step setup guides, interactive tutorials #### 🧠 Logical learners - Present: Cost analysis charts, architectural decision trees - Improve: Performance benchmarks, capacity planning calculators #### 🧑‍🤝‍🧑 Social learners - Present: Interview format, community discussion - Improve: Forum discussion prompts, collaborative exercises #### 🧘‍♀️ Solitary learners - Present: Self-contained explanations, individual reflection time - Improve: Self-assessment questions, independent lab exercises #### 🌿 Naturalistic learners - Present: Data ecosystem analogies - Improve: Environmental impact considerations of on-premise vs cloud ### 📊 Accessibility Assessment #### Screen reader semantics - Present: Clear heading structure, descriptive content - Improve: Alt text for technical diagrams, ARIA labels for demo elements #### Keyboard navigation - Present: Standard video controls - Improve: Keyboard shortcuts for demo sections, transcript navigation #### Contrast and readability - Present: Clear audio, readable slide content - Improve: High contrast mode for technical diagrams #### Text resizing and reflow - Present: Responsive video player - Improve: Scalable transcript formatting #### Media and captions - Present: Full transcript available - Improve: Timestamped captions, technical term definitions #### Forms and interactions - Present: Basic YouTube interface - Improve: Accessible demo interaction elements #### Standards reference - Video content meets WCAG 2.2 AA for multimedia content ### 🌍 DEIB Assessment #### Cultural breadth - Present: Discussion of global technology adoption - Improve: Multi-region data sovereignty examples, international case studies #### Inclusive language - Present: Gender-neutral technical terms, defined jargon - Improve: Plain-language summaries for complex concepts #### Neurodiversity support - Present: Clear structure, consistent pacing - Improve: Content warnings for complex technical sections #### Representation - Present: Female technologist in leadership role - Improve: Diverse technical background representation #### Psychological safety - Present: Respectful interview format, acknowledgment of learning curve - Improve: Beginner-friendly content warnings, multiple learning paths ### ✅ Logical Validity #### Strong reasoning - Deductive strength: Cost analysis based on storage pricing models - Inductive strength: Industry trends toward data lakehouse architecture - Causal reasoning: Simplified architecture leads to reduced operational overhead - Analogical strength: Comparison between cloud and on-premise trade-offs #### Fallacies - Appeal to popularity: "2 billion Docker pulls" as quality indicator - success metric, not inherent quality proof - False dichotomy: On-premise vs cloud presented without hybrid options - address multi-cloud strategies #### Tone of voice - Overall tone: Technical but conversational, expertise balanced with accessibility ### 🛡️ DARVO Analysis - Deny: No defensive patterns observed - Attack: No attacks on competitors or critics noted - Reverse victim/offender: No victim/offender reversal present - Emotional cues and rhetoric: Professional, educational tone throughout ## 🎯 Conclusion This episode provides valuable insights --- ## Full Transcript [00:32] Hello. Hello, lovely people. I mean, I [00:35] guess I have to do that. It's the first [00:36] week of the year. So happy new year [00:39] folks. Uh you know I don't know people [00:42] wish some stuff to other people. I will [00:44] wish you health and you know having a [00:47] beautiful year. I don't know. I think [00:48] for 2025 uh sucks for mo many people [00:52] from what I've read on social media what [00:54] I've seen during uh the holidays. So [00:56] hopefully this year is going to be [00:57] better. Uh do what makes sense for you [01:00] but yeah I always wish health uh to [01:02] people. It seems boring but it's uh it's [01:04] kind of like so important. So, with that [01:07] said, uh, and we're close to, you know, [01:10] I I think we need to stop saying happy [01:12] New Year's after the first week. Uh, so [01:14] we're still good. But, uh, yeah, first [01:16] episode of the year. I'm really happy. [01:19] I'm continuing to doing the Devrell [01:21] show. As I told you, I have so many [01:23] guests lining up. So, that's going to be [01:25] an interesting year. So, it will just [01:27] depend on my traveling schedule. But, [01:29] first episode of the year, it's uh, [01:31] episode 43. So, uh, we're continuing to [01:36] talk about Deell, having amazing guests, [01:38] and today is no different. I have Brena [01:41] with me. How is life? [01:44] >> Oh, it's great. Um, it's uh finally [01:48] cooled down in Las Vegas where I live, [01:50] and I'm just enjoying our our winter [01:53] wonderland [01:55] here. [01:58] >> I I hear you. I mean, we don't have the [02:00] same kind of winter. I'm in Canada in [02:02] the east side and Montreal. So, uh yeah, [02:05] we have like snow and and it's freaking [02:07] cold right now. So, um I don't really [02:10] enjoy the cool down that we're having in [02:12] Canada, but I understand from your point [02:14] of view. Uh it really makes sense. It [02:17] really makes sense. Uh did you did you [02:19] have time to uh disconnect a little bit [02:21] uh during the holiday? [02:22] >> Um to to sorry to what [02:25] >> disconnect or just like take some time [02:27] off? [02:28] >> Yeah, it was great. Um uh I'm really [02:30] grateful for like my company's like [02:33] respect for um like family time. So like [02:37] I had a really nice uh relaxing break. I [02:40] got to work on some personal dev [02:43] projects and um and hang out with my [02:46] family and that was it's always always [02:49] fun. [02:50] >> That's pretty cool. So first question [02:52] usually and now I had like multiple [02:54] question for you before my usual first [02:56] question which is who are you? Tell us [02:58] who you are. [03:00] >> Uh, wow. Um, I feel like I'm in a [03:03] therapist office. Uh, [03:06] well, my name is, you already, you [03:08] already know, my name is Bren. Um, I I [03:11] work as a developer evangelist basically [03:14] at Min.io, which is a object storage [03:17] software company, [03:19] mainly onrem object storage. So an [03:22] alternative to Amazon's S3 or um you [03:26] know Azure blob storage on prem. Um I [03:30] before this role I um was a basically a [03:35] real data engineer. I um I worked uh [03:39] full-time um you know building pipeline [03:42] and um delivering [03:44] clean data. [03:46] And so I transitioned into this role [03:49] because um I really enjoy uh teaching [03:53] and helping people learn and uh data is [03:56] one of my u favorite subjects. So I'm [03:58] really um like keyed well to this role [04:03] because it's it's a couple of the things [04:06] that I um care about the most and have a [04:08] lot to say about. [04:10] >> Yeah. No. Uh love that. And um we met if [04:15] I'm not if I'm not wrong I think it was [04:17] like some cube con non America 2024 uh [04:21] you were at the minio booth and I was [04:24] looking for guests for primary it was [04:26] for the cube first live which was uh the [04:28] live stream that I was doing at my [04:30] previous company construct uh [04:32] unfortunately uh that didn't happen but [04:35] I was like hey you know you do developer [04:36] relation uh you should come on the dev [04:39] show and I really want to thank you for [04:40] your patience because uh it was like a [04:43] year and a half ago. You were scheduled [04:45] six months ago, but I had like a last [04:47] like I got a new job. I had a last [04:49] minute trip. I had to reschedule you. [04:51] And I did the quote unquote mistake of [04:54] like trying to schedule all my guests [04:56] for the full year, which I learned the [04:57] hard way. It's not the way to go [04:59] because, you know, live and schedule and [05:01] stuff. So, I was not about to reschedule [05:03] you up to like January 2026. But, uh [05:07] thanks for being patient with me and [05:09] thanks for being on the show. Uh, I [05:10] really appreciate that. [05:12] >> Yeah, I'm happy to. Um, you know, um, [05:14] we're all busy and, um, you know, in [05:18] some ways it's nice to have an idea of [05:21] what where you're going to be far in [05:23] advance. You know, I travel a lot for [05:24] the role that I, uh, do. So, you know, [05:28] I'm also um, a developer at heart, so [05:31] change is part, you know, the of my [05:34] life. It's in the fabric of what I do. [05:36] So, I'm totally I just roll with [05:38] punches. It is what it is. No, that that [05:40] makes sense. I like that. Uh so one of [05:43] the first thing I wanted to ask you [05:44] because when you introduce yourself, you [05:46] say like hey I'm a developer advocate [05:48] and when you look at your LinkedIn uh [05:50] the title is different. You're a subject [05:53] matter expert on databases and modern uh [05:56] data links and um I know when you read [05:59] the description you understand that like [06:00] yeah okay you're [06:03] a matter expert but most of the things [06:05] you do are developer relations related. [06:08] So can you tell us a little more about [06:10] like you know maybe the title itself [06:12] like why that title and and your role [06:15] actually like what is your role? [06:17] >> Well to be frank I I didn't pick it. I [06:19] just um applied for it but it is um uh [06:24] if when I was looking for [06:27] developer advocate roles it's really um [06:30] matched a lot of the [06:33] um things I actually want to do. Um, [06:35] actually I don't mind that we're calling [06:37] subject matter expert. I say developer [06:39] advocate a lot of times because it gives [06:42] people a better sense of what I do [06:44] dayto-day, but also um [06:47] it's shorter on a name tag [06:51] um often, you know, of data links and [06:54] modern um data or whatever it is, it's [06:57] too long to fit on a on a placard. So um [07:01] it's shorter. It's more like explan [07:04] explan explanation [07:07] I don't know um but um but it it I you [07:11] know I work under product marketing um [07:14] I'm a I'm a a real data engineer um and [07:18] I spend my time uh you know writing [07:21] blogs making um decks speaking having [07:24] going to speaking engagements uh manning [07:27] the booth like where you met me talking [07:30] questions about um our product technical [07:33] questions for people that walk by. You [07:36] know, often you need to pair either like [07:39] a person like me or like a um an FA like [07:44] a like a sales engineer or an FE, [07:47] however you whatever you guys call um [07:49] those people that do the sales [07:51] engineering, you have to pair uh like a [07:53] saleserson and a technical person with [07:56] in the booth. And I often stand in the [07:58] place of uh sales engineers because I [08:01] write code. Um uh like you know I I make [08:05] demos [08:06] um I I I package uh technical content [08:11] for the marketing team. So product but I [08:14] work under product marketing. So like [08:17] you know anything that comes around in [08:19] marketing around data bases or data lake [08:23] um data lake houses I typically take a [08:25] look at and check for technical accuracy [08:28] especially now in the time of AI where a [08:31] lot of things moving really fast you [08:33] know like sometimes content gets [08:35] generated that's um [08:40] uh you know um [08:43] trying to nice word I don't know like [08:45] not uh complete I guess or inaccurate or [08:50] just you know like hallucinated like [08:52] that happens a lot with I think well and [08:56] probably with all content but I [08:57] especially deal with uh technical [08:59] content so sometimes chatbt or claude [09:01] will envision a future or a reality that [09:04] doesn't actually exist [09:05] >> um and so I have to like go through [09:08] stuff and make sure that it makes sense [09:11] um and so for everyone like that anyone [09:13] else writes so I write but I also a [09:15] review. Uh I don't know. Does that does [09:18] that answer your question? Like so I do [09:20] a role of a developer advocate. I have a [09:23] different title. I'm not mad at it. I [09:24] mean it is does a subject matter expert. [09:27] Also describes like what my expertise [09:30] is. Um [09:32] but it's harder people for people to [09:34] visualize. So you know what I do [09:36] day-to-day unless I say developer [09:37] advocate. So they're they're fine with [09:39] us putting what we want on our [09:41] >> No, makes sense. It make total sense. [09:44] And I I'm asking that just for the sake [09:46] of like having a discussion about it and [09:48] and like for people listening. But um [09:50] also um I think u anyway like a [09:54] developer advocate should be a subject [09:57] matter expert about like their own [09:59] technology, their own product. So like [10:01] that's kind of like that works. But um [10:04] when I've seen like people with that [10:06] title, usually they were like you know [10:08] engineers like like you know principal [10:11] engineers or or I forgot what's the even [10:14] the other title after that but like you [10:15] know the main resources in the [10:18] engineering team about like a specific [10:20] topic or a specific technology. So it's [10:22] why I was curious about like oh are you [10:25] like like really like just like the the [10:28] the the top expert in your company about [10:31] that specific topic or is it like [10:33] another way or which could be true also [10:35] but like is it also another way to like [10:37] say oh you're a developer advocate too [10:39] but like you just have a different [10:40] title. So your explanation was was [10:42] pretty good about that. I was just uh [10:44] curious about that and I assume uh maybe [10:47] people joining us live or watching your [10:48] recording may had also the same uh [10:51] answer because on the thumbnail of the [10:53] video I put like subject matter expert [10:55] and the full title uh that was written [10:59] on on LinkedIn actually. So [11:01] >> yeah, no um I you know our minio company [11:04] I work for has a history of being [11:07] engineering le and so I do see the that [11:12] history written in the title that I [11:14] have. I'm the only well um prior we [11:17] hired a an internal data engineer um [11:20] like a year ago but I was prior to that [11:22] was the only data engineer at the [11:24] company. So um I do [11:27] uh I do claim to have subject matter [11:31] expertise in in data engineering and so [11:35] and you know specifically for data links [11:37] and databases. So yes and also a [11:41] developer advocate if it's easier for [11:42] people to visualize what I do dayto-day [11:45] than than just to say that I that I know [11:48] stuff which is [11:51] >> No, but I like that. And it's great that [11:53] you're saying that too because uh when I [11:56] started in Devro like nobody known about [11:58] that. Nobody known about a developer [12:00] advocate. Actually we were not even [12:02] called that. We're called technical [12:03] evangelist and people didn't know what [12:06] it was. It was like people were asking [12:07] me like are you going to church or [12:08] something? And I'm like no I work in [12:10] tech in tech. And and now that you're [12:13] saying that like oh when I say to people [12:15] that like I use the title developer [12:17] advocate uh people recognize and [12:20] understand what I do it's a great sign [12:22] that like the industry understand a lot [12:25] more developer relation and see also [12:27] hopefully those role as something [12:29] important for the business. So yeah, [12:32] heard that. I I also recognize that [12:35] change um and that shift uh between um [12:39] making developer advocates or developer [12:41] evangelists or whatever you want to call [12:43] them more important uh like prominent [12:47] roles for business because especially [12:50] for software [12:52] um people are recognizing that you have [12:55] to [12:56] speak to developers first like you know [12:59] we don't sell teddy bears or like you [13:01] know widgets [13:02] We're not we're not going to um ever not [13:05] be technical. We hear that a lot [13:06] especially in marketing like oh is this [13:08] too technical? Is this going to like you [13:10] know um make people sit down too long [13:12] and read it or is it like too in depth? [13:14] Like everybody that buys technical [13:17] software has a degree of technical [13:19] aptitude. They may not be in their job [13:22] title today, but you I have sat in as a [13:26] data engineer on um buying decisions and [13:30] everybody in that room from like chief [13:32] data officer to um you know the business [13:36] analytics folks have [13:39] a technical background. So to to argue [13:42] that we need to dumb things down or um [13:44] make things salesy when we're talking [13:47] about selling [13:49] technical software I feel like misses [13:51] the mark on who our audience is uh for [13:55] who's who's actually buying our product. [13:57] Right. [13:58] >> Yeah. [13:59] >> That's what I think. I think it's so [14:00] important. And I think it and we're [14:01] getting um you know you don't want to [14:04] alienate your um [14:08] your your buyers and the reality is that [14:12] you know I am a developer so I know what [14:16] that like person personality type is I [14:20] guess and I can tell you that if you it [14:23] doesn't take me long to figure out if [14:25] you don't know what you're talking [14:26] about. You know what I mean? So, like if [14:27] I read an article or I see a video or I [14:30] look at docs and it I I can instantly [14:34] tell if it's not, you know, if it's just [14:37] fluff or if it doesn't make any sense or [14:40] if it's um just sales speak, you know, I [14:45] want to I want to get and I'll skip over [14:48] that to find what it matters to me as a [14:51] developer at like which is actual which [14:54] is actually code or you know in depth [14:57] talks or like an explanation of what I [14:59] exactly I'm trying to do you know so I [15:01] think that this role is really important [15:03] to find that deeper level of involvement [15:07] with um software customer base that's [15:11] what I think [15:12] >> yeah no and and yeah I agree I agree [15:15] totally u I always say that like if you [15:18] want to like like just talking about [15:20] developer relation itself if you want to [15:22] be success successful as a developer [15:24] advocate no matter your title if your [15:26] job is to talk to a developer audience. [15:29] Uh you need to be credible. You need to [15:31] know what you're talking about. So [15:33] exactly as you said like I I think it [15:35] makes sense. You don't have to dumb down [15:36] down things. And it may depends on the [15:38] product though. I mean like minio is is [15:41] for technical people [15:44] basically only from from my knowledge of [15:46] the product. I mean uh it may be a [15:49] little bit different when I'm thinking [15:50] about my own product uh tiny MC right [15:53] now which is like yes it's for [15:55] developers uh because they're going to [15:57] integrate a rich text editor wizawig [15:59] within their application they're [16:00] building uh so it's a a product for [16:03] developers but in my case once developer [16:06] implement the the product tiny MC within [16:09] their own product uh it's the end users [16:11] that may not be technical who's going to [16:13] use it because our product is targeted [16:15] to developer to implement it But the [16:17] usage of the product at the end once [16:19] it's implemented it's for anybody who [16:22] need to create rich text content. So in [16:24] that case uh the approach is a little [16:26] bit different but not for me as a [16:28] developer advocate because I'm not [16:29] talking necessarily to the buyer or the [16:32] end users. I'm talking to the developer [16:35] who implement my solution. So I totally [16:37] agree about that. And speaking about uh [16:40] minio uh would you like to just talk a [16:42] little bit about like you know the [16:43] product and the company uh because this [16:45] is where I know you from from my past [16:47] life in like the cloud native and [16:49] kubernetes uh at cube first we were [16:51] using um and I offer a local uh [16:54] installation local flavor. So if you [16:56] would like to just talk a little bit [16:58] about that for people that do not know [16:59] the company or the product. [17:01] >> Sure. And like I said when I did my [17:03] little intro, um it's kind of a [17:05] rehashing, but minio is basically onrem [17:08] S3. So it's uh truly S3 native. There's [17:11] no map data gateway. Um there's no other [17:15] protocols. We don't do fly on block. We [17:17] don't do um uh anything else but S3 [17:23] onrem. And so we have more [17:27] uh than two billion darker poles [17:30] worldwide. Um they like to say that [17:33] Candy Crush has five billion the B. So [17:36] we're not that far off of of uh Candy [17:39] Crush. Um [17:40] >> that's crazy though. [17:42] >> Uh yeah, we're really proud of like the [17:44] amount of developer support that we've [17:46] gotten um uh from the beginning. So [17:50] um it the origin of minio is an [17:55] open-source um uh codebase but in the [18:00] last few years we've uh implemented and [18:04] developed an enterprise version of the [18:06] software. So we'll always have the [18:07] community edition available for anyone. [18:10] it's um distributed under um a GDPL uh [18:16] license but the enterprise version has [18:20] different and extra features [18:23] um like console for example we also [18:26] introduced something that's really cool [18:29] which I'll hopefully have time to demo [18:30] today which is we've embedded the [18:33] iceberg catalog API within our object [18:36] storage and a couple a lot of a couple [18:39] other cool stuff coming down the line. [18:42] And we also have stuff for um in the [18:44] enterprise version for enter uh managing [18:46] enterprise software like um like a a a a [18:52] key manager [18:54] um [18:56] extra like stuff like encryption. So [18:59] hopefully uh everyone on the call know [19:04] you know knows who we are. If you're in [19:05] the data space, you'll probably know who [19:08] uh and you have onrem or local [19:09] installations, you've heard of us. Um [19:12] but um you know that's what that's what [19:14] Mio is. [19:16] >> Ah sounds good. And um prior to your [19:19] you've been at Minio for a little more [19:21] than two years now. Uh prior to that you [19:23] were a developer evangelist at uh Stream [19:26] Sets. Um yeah, would you like to talk a [19:29] little bit about that role? [19:30] >> Yeah, so Stream Set is really cool. It [19:33] is um or was um a [19:38] software company that created [19:41] um like an ETL tool, a visual ETL tool, [19:45] play similar to uh like a no code um low [19:49] code ETL tool similar to Taland or you [19:52] know other um ETL tools around there. It [19:55] had um this ability to differentiate it [20:01] had this ability to do uh something [20:04] called um data drift. So it detect and [20:09] remediate data drift. So that's what [20:10] happens when like a like say you have um [20:14] like a this happens a lot in data [20:16] engineering where you have a destination [20:19] say at a a a database that has it's it's [20:23] uh field type is a date or something and [20:26] something happens at the um you know [20:30] input level that changes that. So maybe [20:33] instead of a date they start they type [20:35] out in words February you know 21st [20:40] 2006 [20:42] 26 and then um you know in full full [20:45] letter English letters you know and so [20:47] when it when it it it'll that'll break a [20:49] pipeline right if the database the end [20:52] state is uh expecting a date and you [20:55] send [20:56] >> a var or a string that can often um just [21:00] kill a pipeline and it's really annoying [21:01] because um it happens a lot and um um it [21:07] seems minor like and then people don't [21:08] downstream your customers don't always [21:10] your data customers don't always know [21:12] why things like that don't work. So um [21:16] stream size is cool because you could [21:17] detect that and uh you know either skip [21:20] that set up whatever kind of uh [21:23] remediation you wanted. You could skip [21:24] it. You could have some kind of cleaning [21:27] step that kicked off and that kind of um [21:31] weird string was detected to fix it and [21:34] send it off as a um a date. You could um [21:38] uh kill a pipeline completely if you [21:40] wanted to. There was a lot of like [21:41] remediation steps that you could [21:43] automate in your pipelines that made [21:44] them more durable. But it was really [21:46] cool. It was a really cool company. I [21:48] really liked working there. Um but they [21:50] were acquired by um IBM. Um [21:54] >> Oh, okay. [21:55] >> Was the tail end of when I was working [21:56] there and so [21:59] you know like [22:02] it happens you know. [22:03] >> Yeah. [22:05] >> Yeah. There's always a lot of surprise [22:06] with acquisition. So uh yeah. No. And [22:10] there was um [22:12] you know as you know like I said change [22:15] is inevitable and especially when you're [22:17] working in a startup that kind of change [22:18] is like you know always on the table and [22:21] it's like one of the best case scenarios [22:23] right you either acquire or you IPO and [22:26] so you know it's fine um I wasn't let go [22:29] or anything they but [22:32] it's all kind of like you know like when [22:34] you can when the train's heading [22:37] somewhere and you are on the train and [22:39] you like you notice an impending cliff [22:42] or something and you're like, I better [22:46] take this exit. It was that kind of [22:47] thing. So, I was just like, you know, I [22:48] don't I don't know. I already I I see [22:51] that they have already have a marketing [22:53] team and they already have developer [22:54] advocates and the roles I do are [22:56] covered. So, I think maybe um [23:02] >> Yeah. So, but I really enjoyed working [23:03] there and um like everyone was super [23:06] nice and I learned so much because prior [23:09] to that I was a real data engineer and [23:10] so coming into marketing I was learning [23:12] all kinds of words you know like MQLs [23:16] and um you know I don't know uh B2B I [23:21] you know all all the stuff I don't I [23:23] didn't really wasn't familiar with [23:24] before. It's a definitely a new world [23:26] for me and that role really prepared me [23:28] for the career that I have now and I [23:31] will always be grateful for the people [23:34] that took a risk in hiring I feel like [23:36] took a risk in hiring me for a real and [23:38] data engineer for a marketing role that [23:40] I never done before. So, um, especially, [23:43] um, my my boss there was, uh, Sean [23:45] Anderson. He's like an incredible, uh, [23:48] product manager, and I just feel like he [23:50] took me under his mentorship wing, and I [23:53] just learned [23:53] >> Oh, nice. [23:54] >> like a ton. He was amazing, um, or is [23:56] amazing. Um, so, um, [23:59] >> that's great. That's great. [24:01] >> It was really, really foundational for [24:03] me and I learned a lot and I loved the [24:05] product. So like I I feel like and [24:08] everybody like especially low code no [24:10] code low code no code tools help um [24:16] bring in uh people that aren't you know [24:19] I'm thinking also the the company that [24:23] you're working for I guess because these [24:25] types of tools bring in you know the [24:27] democratized data engineering which is [24:30] >> so important for bringing memes like [24:33] myself I guess into actually building um [24:36] pipeline, it's often the people that [24:38] know the most about the data. And so [24:40] having them be able to be like in [24:43] instrumental in actually moving it [24:45] around is like really key to making um [24:48] data pipeline that don't break, right? [24:52] If you know if you know the if you know [24:54] the data the best, you should be the one [24:57] actively, you know, making the pipeline. [25:00] you shouldn't have to send it off to [25:01] like people that don't that have to ask [25:03] you questions about is this field [25:06] supposed to be you know whatever [25:07] whatever kind of questions that they [25:09] have um for for you about that. So I [25:12] really think those tools are cool and I [25:14] I loved working there and I love [25:15] learning about um developable developer [25:19] evangelism and or develop whatever you [25:22] want to call this type of role [25:26] >> like I and I you know I loved same same [25:31] um in my role now I just I still really [25:33] enjoy writing code and I love uh like [25:38] relaying information to people. So it [25:40] was that I really came into my own I [25:42] feel like in that role and I brought all [25:44] that experience to this one. So I'm just [25:46] like really stoked. I like seemed like a [25:49] really good decision. I was like very [25:51] nervous coming transitioning from real [25:54] engineering to this type of role. Um as [25:58] I'm sure [26:00] I'm sure a lot of your um listeners are [26:02] they had a developer advocate. So I'm [26:04] sure they have that similar they came [26:07] from engineering [26:09] because I don't know I just it's [26:11] different you know like it's [26:14] >> um and [26:17] I didn't know anything about marketing [26:19] so it's it's strange making that [26:21] transition. [26:22] >> So [26:23] >> how how would the uh question about that [26:25] then uh because yeah you were engineers [26:27] before and you moved to like a role in [26:30] developer relations. So um [26:33] >> how did that happen or or why did you [26:35] move from like a a more like like [26:39] engineer role to a more engineer centric [26:41] role to like now doing developer [26:43] advocacy or developer evangelism. Uh I [26:46] like to ask that question for people [26:48] that mo most of us transition like [26:50] rarely someone like your first job were [26:52] developer advocates or developer [26:54] relation. We all most of us like were [26:56] software developer engineers before and [26:58] we moved to developer relations. So I [27:00] really love to ask that question because [27:01] everybody has like a different story. So [27:04] what's what's your story there? [27:05] >> Yeah. Well, so like I said, I love um [27:09] data and I love teaching and so this [27:12] really fits um [27:15] like around the time that I started [27:17] looking for that stream sets role, that [27:20] first developer advocate role, I was [27:23] asked by I was working in engineering, [27:25] right? And I was asked by my boss to [27:28] make like a fiveyear plan. You know, [27:29] it's kind of like those kind of [27:32] things. Oh, I know. Can I swear? I'm [27:34] sorry. Those [27:35] >> Yeah, you can totally on that. It was [27:37] fine. [27:38] >> Um, you know, those silly um, you know, [27:41] career goal things, but I actually took [27:42] it seriously and one of the questions [27:45] was something like like where do you [27:48] like what in your past like career so [27:53] far have you found the most fulfilling? [27:56] like what about or what role specific [27:58] role or specific [28:00] um [28:02] you know um um project or whatever in [28:07] your former roles you find most [28:08] fulfilling and I really thought about it [28:10] and I I feel like I was most fulfilled [28:13] in the at that time by my experience um [28:18] I actually I went to grad school for [28:20] political science [28:22] >> I know I don't have a te traditional [28:24] technical background I taught myself how [28:25] to code which one of the reasons why I [28:28] love no code low code solutions because [28:30] it helps pe people like me get into [28:32] technology where they you know ease them [28:35] in into coding which is really super [28:37] fun. Um but anyways um [28:41] I taught myself wait so I I found most [28:44] fulfilling teaching undergrads [28:47] um political science which is what I [28:49] majored in in grad school and in [28:50] undergrad. So I was like what can I do [28:54] in in my career to like I have all this [28:57] experience in data like I don't want to [28:59] like I love solve the the puzzle solving [29:01] element of data engineering I love like [29:04] learning new things all the time I don't [29:06] want to just discard all this experience [29:10] um but how can I bring in more of this [29:13] like what I found fulfilling which was [29:15] imparting knowledge to [29:18] >> to other others like being of service [29:21] And I just like Googled that exact [29:23] question [29:25] and like it came up with like a oh you [29:27] could like be a mentor, you can like be [29:29] in um engineering leadership and I was [29:32] like and then it was like or you can be [29:34] a developer advocate. So I was like wow [29:36] I should see what that's like. I applied [29:38] for one job. Sean Anderson, the boss [29:41] that um I um just mentioned, uh called [29:46] me the next day for an interview and I [29:48] got the job the next week and then just [29:49] like from there like it was I feel like [29:52] it was like I don't want to like throw [29:54] it around too much, but I think it was [29:56] like a little bit of fate, you know what [29:57] I mean? Like I was like, [29:59] >> uh, what [30:01] like I went from day one googling like [30:04] what how do I bring an education and [30:06] engineering together um to like having a [30:09] job doing exactly that and it being like [30:11] the perfect role for me. Do you know [30:13] what I mean? Like like I love this [30:15] stuff. Like I love I love data and I [30:17] love teaching people things. And so far [30:20] all of this has just been it's been like [30:22] fun every day, you know? I don't I'm not [30:25] on pager duty anymore. No one's upset. [30:28] So like the thing about data engineering [30:30] especially like a real as a like a [30:33] enterprise data engineer like a real one [30:36] is that nobody ever talks to you unless [30:37] something's broken you know like no one [30:40] ever was like hey you're doing a great [30:42] job Brena the best thing I got was like [30:44] sometimes my PRs would be approved no no [30:47] comments and I'd be like yes I'm doing [30:49] like somebody thinks I'm doing a good [30:53] job [30:54] but like nobody ever like I don't get [30:58] talked to. I like filled out tickets and [31:00] that was kind of like, you know, kind of [31:03] what I what I did dayto- day whenever I [31:06] tried to like reach out to like ask [31:08] business users like, "Hey, what are you [31:09] going to use this uh report for? Like [31:13] what what do you really like what do you [31:15] really need? Like I saw what you asked [31:16] for, but like can you like [31:18] >> Yeah, I would like kind of got shut down [31:21] or like I don't have time for that. Just [31:22] give me what I asked, you know? Like and [31:24] I just felt [31:26] >> now I get to talk to people like you're [31:28] you and everyone that comes to the booth [31:31] and people tell me in marketing [31:33] especially they'll tell you you're doing [31:34] a good job all the time like like you [31:36] know I get to I feel like I'm a free [31:39] elf, you know, like I'm um I'm out into [31:42] the world like I'm I do do some travel [31:46] like I get to meet I get to talk to [31:47] people face to face like [31:50] >> Yep. Um, how cool is that? Like they let [31:54] me out into the world. So, uh, it's way [31:58] better than engineering. It's like for a [32:01] certain kind of a person, right? You [32:02] have to be kind of outgoing. [32:04] >> Um, but I do [32:07] love it a lot and I still write code [32:10] and it if it breaks, it's not a big [32:13] deal. I'm like, oh, I'll fix that. You [32:14] know, they make changes or like, you [32:16] know, like there's a new library or [32:18] stuff gets unsupported. Like I just it's [32:20] like I just I'll I'll do that later. [32:22] Like I'll add it to my backlog and I'll [32:24] get to it in a couple weeks. I'm not No [32:27] one's like waking me up at three in the [32:28] morning. [32:28] >> Yeah. There's no production code getting [32:31] on fire. [32:32] >> Yeah. I do not miss that part of like uh [32:35] working on product. uh like at at one [32:38] company we had a product that people uh [32:41] were using strictly between 2 and 4:00 [32:43] a.m. So um yeah I was uh the only and it [32:48] was a really small company that like we [32:50] created the product. So uh and I was the [32:52] most senior one which mean that like I [32:54] was the one always on call for those [32:56] things and we had so many bucks that [32:58] like they were always waking me up at [33:00] like 2 3 a.m. [33:02] >> I really do not miss that at all. [33:06] that'll burn you out and you know and [33:08] it's um it's not fun. So I I don't I [33:12] don't I don't is that at all like people [33:14] are nice to me now you know [33:19] >> I mean let me tell you the story it [33:21] depends where you work and and and how [33:23] people like the company you work for the [33:25] product you work for because my first [33:27] job um as a developer advocate uh was at [33:30] Microsoft and it it was during the [33:32] Balmer era if you watch this show you [33:34] already heard me tell that story a [33:36] couple of time but uh there was during [33:38] the like like whether you like Microsoft [33:40] or not right now they're so much better [33:42] and so much more friendlier than uh they [33:45] used to be and you may disagree with [33:47] that but like uh in the past when I [33:49] started to work there at first when they [33:51] approached me I said no one or two time [33:52] because I didn't want to go there but [33:54] actually they were like no Fred I want [33:56] someone like you like who who talk open [33:58] source who is in the open source [33:59] community who is not like a typical [34:01] Microsoft person so my audience were [34:04] open source developers but I was working [34:06] for Microsoft during a time where [34:08] Microsoft was not playing well at all [34:09] with open source So people didn't like [34:12] me. Some people like me, but there are [34:13] some people that didn't like me. There's [34:14] like two people that refused to shake my [34:16] hand because I was working at Microsoft. [34:18] So just a small warning. Most of the [34:21] time people like you, but it depends on [34:24] so many factors. Uh it's not always the [34:26] case. But in my case, I miss coding [34:28] though. Uh I still code part of the [34:30] role, but I miss like being a developer [34:32] full-time. But when I put put this in a [34:35] scale of like oh do I prefer being a [34:38] developer advocate or being like an [34:40] engineer or software developer full-time [34:42] I still prefer doing dev. It's still [34:44] technical. I still code but like also [34:47] miss a little bit like like coding eight [34:49] hours a day which I cannot do anymore [34:51] because like I have so many other things [34:52] to do with that role but I still love [34:55] like being a dev. So [34:57] >> I like um I miss that. I I mean I I [35:01] don't miss and I do miss it because of [35:03] that focus that you can like that flow [35:06] state that you could get into. Um you [35:09] know like you I miss that element. I [35:12] miss problem solving like something [35:14] really deep like working on something [35:16] for a long time it break it's breaking [35:18] breaking breaking breaking and then like [35:19] you go take a shower or something you're [35:21] like oh that's what it like I [35:22] that's what it is like that's that's [35:24] gonna fix it and then like you know [35:26] that's kind of fun. I and I don't work [35:28] on I do still code but it's like it's [35:32] demos you know I can make the data [35:34] whatever I want it to be and I usually [35:36] make it easy so it's not you know like [35:41] >> Makes sense. Yeah speaking of Oh sorry. [35:43] >> Oh no go ahead. Yeah, I was going to say [35:46] uh speaking of of of Demo, I think I [35:49] think it would be a great time now to [35:51] because you know, we could talk forever [35:53] uh because we're being paid to talk, but [35:56] um that could be a good time to move to [35:58] the demo portion of the show. Uh which [36:01] I'm really looking forward to it because [36:03] uh you're going to talk about data, [36:05] which is something I do not know about [36:08] at all. I know nothing about it. So, I'm [36:11] looking forward to uh what you're going [36:13] to talk about. [36:14] All right, I'm gonna try to do I'm gonna [36:17] give you the We don't have like that [36:19] much time, so I'm going to give you like [36:20] the condensed version of the slide deck [36:25] and then I'll show you some code. How [36:26] about that? [36:27] >> Wonderful. Sounds good to me. [36:28] >> All right. So, let me just share my [36:31] screen. [36:34] Okay. So, I want to just let's see just [36:38] this one. [36:42] move this over so you guys [36:45] don't [36:48] really Okay, hold on. [36:56] You know what? Maybe I'll just [37:01] Can you see what do you do you see my [37:04] deck? [37:04] >> Not anymore. You just I I used to see [37:06] your deck, but you removed the screen [37:08] sharing, [37:08] >> right? Sorry. [37:10] >> That's That's fine. Technology is hard. [37:13] >> What I say? [37:15] >> Yeah. Okay. How about that? Is that [37:16] better? Can you see now? [37:19] >> Yeah. Still seeing the slide now. [37:21] >> Okay, cool. So, I'm just going to give [37:24] you the compressed version of it. I'm [37:25] not going to go through all these stuff, [37:26] but um the main point about this is that [37:31] um the most important data that you have [37:36] or organizations have is [37:39] not just structured data like you see uh [37:44] XML, JSON or or even the new open table [37:48] formats like iceberg and delta lake. um [37:51] often it's unstructured or like videos, [37:55] images, things like that. And so there's [37:58] a really need to bridge this gap between [38:01] these different types of of formats of [38:04] data and the actual needs for the uh [38:08] workflows that people are actually [38:10] actually doing, right? So uh we really [38:13] as an organization well actually I feel [38:15] like as a data community has really [38:17] rallied around iceberg as the open table [38:19] format. This is a data lake house table [38:22] format. Um it's um essentially so I'm [38:26] like I'm used to talking to data lake [38:28] folks so this is like an interesting um [38:30] but iceberg Apache iceberg is an Apache [38:33] uh project where um the parquet files [38:37] are stored alongside metadata [38:40] that um turn a uh typical or traditional [38:44] data lake into a data lake house. So the [38:49] iceberg metadata is actually what gives [38:51] you the really cool things like time [38:53] travel and schema evolution. So remember [38:56] I was talking about how stream sets [38:58] could detect data drift like so if your [39:00] data types were different. Iceberg has [39:02] this type of um intelligent design in [39:05] its foundation. So um for example um if [39:11] your uh [39:14] if your data coming in um changes schema [39:19] so if it's it's an int it was a formerly [39:23] an int and now it's a varchar it will [39:25] just um add a new column and u move on [39:28] so you don't have broken pipelines you [39:31] have um like the this is actually the [39:35] the the I believe the future of data [39:38] because it makes our jobs as data [39:41] engineers much much easier and and as a [39:44] whole that's been that's an accepted [39:46] view in data engineering. [39:48] >> Okay, [39:49] >> this is a just a description about how [39:51] unstructured and structured work [39:52] together. This is why people want to [39:54] build data lake houses on prem because [39:56] obviously we're minio is an on-rem [39:59] uh uh S3 compatible object storage which [40:02] is in of itself just your data lake but [40:06] I always like this it's right here this [40:08] is like the money shot I guess where you [40:11] see where [40:13] >> it's about 10 pabytes of data where it [40:15] becomes much more cost effective to [40:18] store your data on prem than it is in [40:20] the cloud and this is super hot data so [40:21] this data that gets a lot transactions [40:23] real data lakehouse [40:26] transactions. [40:28] I want to talk about the thing. This is [40:30] a data lakehouse diagram where you have [40:32] the query engines then an iceberg [40:35] catalog which is necessary to manage [40:37] that metadata that I was discussing when [40:39] I described iceberg to you and then we [40:42] have all the stuff in object storage. So [40:45] this is what a typical lakehouse [40:48] architecture looks like from a thousand [40:49] ft up right you you have interoperable [40:52] querying engines you have to have a [40:54] catalog of some kind there's different [40:56] kinds of cataloges iceberg catalog um in [40:59] it of itself the rest catalog is also an [41:01] Apache project so this is open source [41:03] and available for everyone it's a rest [41:05] api so a lot of people have implemented [41:07] enterprise versions of this like um [41:09] polaris which is snowflake project um [41:12] that is open source now but um also So [41:15] project nessie which is um [41:19] um dramo's uh version of this but all [41:22] these are external cataloges that [41:23] require a separate um uh database like [41:28] so you have to have a the whole promise [41:30] of a data lakehouse is that you're [41:32] moving away from transactional or [41:34] traditional databases but all external [41:38] iceberg cataloges require you to have a [41:40] transactional database like my SQL or [41:42] Postgress attached to it for managing [41:45] the metadata. So it's like [41:46] >> okay okay [41:47] >> move into the future no more but except [41:51] for this one that you have to have um [41:54] and so I have like a deck on that like a [41:56] slide deck which is oh yeah so here's [41:58] like the tra traditional or like current [42:01] iceberg external catalog which where you [42:03] have your object storage layer whatever [42:04] it is it's S3 it's minio um then you [42:08] have uh your metadata database for your [42:10] iceberg catalog um and then you have the [42:13] catalog service whatever it is If it's [42:16] Hive, is it Polaris, is it Nessie, is [42:18] it, you know, um, Gravitino, whatever it [42:21] is, [42:22] >> it has to have a it has to have a a [42:24] separate database. But our version [42:27] >> is just [42:29] and uh your object storage with uh your [42:34] built-in catalog service. So these [42:35] compress down and then it's just that. [42:37] So it makes it uh easier to have a [42:41] lakehouse, right? That's the point. [42:43] That's why that's why we did this part. [42:45] This is a cool new feature that we've [42:46] introduced called AI store tables. Um S3 [42:49] has a version or sorry Amazon has a [42:51] version of it's called S3 tables but [42:54] obviously it's cloud-based. This is [42:57] >> vetted iceberg catalog on prem. We don't [43:00] charge you. So obviously every time you [43:02] have a special bucket in AWS you get [43:05] special bucket charges right there's no [43:07] extra or additional charge for using [43:09] this um API. [43:12] It's just the same um we charge for um [43:15] data under management. So [43:17] >> okay, [43:18] >> a standard for for any of the additional [43:21] like features we have. You don't don't [43:22] charge for seats. You don't charge for [43:23] API calls. Um and that's why you can we [43:26] can get to this type of graph where we [43:29] show how much cheaper it is over time, [43:31] right? Because if you got an expensive [43:33] tier bucket because you want to have a [43:35] simple [43:36] >> fast [43:37] >> what [43:37] >> that goes fast. Uh [43:39] >> yeah, exactly. your SP cloud spend gets [43:41] up in a way, right? All right, cool. So, [43:43] I'll show you my demo now that I've [43:45] explained hopefully what I'm trying to [43:48] show you. [43:49] >> That was good. That was perfect. Yeah, [43:51] thanks for like I'm for sure some people [43:53] listening didn't need that part, but [43:55] like that was good for me because Yeah, [43:57] I'm like I know what is a data lake, but [43:59] like never work with it, never used it, [44:01] so I I don't have that knowledge. So, [44:03] that was perfect intro. Love it. [44:05] >> Okay, good, good, good. All right. So, [44:07] I'm going to show you um a pi uh pi [44:10] iceberg script. I like pi iceberg [44:13] because [44:14] I have been uh victimized personally [44:17] victimized by spark in the past. And um [44:20] pi iceberg doesn't require you to have a [44:23] JVM. So, uh it it won't hurt your [44:27] feelings like Spark does, but you can [44:29] you can use this with Spark or Trino or [44:31] whatever. I have the other notebooks [44:33] here if that's your flavor of query [44:35] engine. But this is just um a a so this [44:40] is connecting to I'll just go into it. [44:42] So this is a demo of AI store tables [44:44] which is again the embedded iceberg [44:47] catalog API into object storage so that [44:50] you don't need a separate metadata [44:52] database to maintain your iceberg [44:54] catalog. You have to have an iceberg [44:56] catalog if you want to have an iceberg [44:58] um [45:00] you want to have an iceberg uh data lake [45:03] um mandatory. So the question is what [45:06] kind of iceberg catalog do you want? We [45:08] love all types of iceberg. You can [45:09] always use AI store for you know regular [45:13] regular data links uh where you you use [45:17] Polaris or Hive and you know have your [45:20] stack like that totally fine you know [45:22] it's it will work just fine but if um [45:26] especially building on prem the big [45:28] problem with having an additional [45:30] service like so if you're you have a [45:31] cloud stack um let me let the dog let me [45:35] just let the dog [45:38] >> wants to go out Ariel. [45:40] >> If you have a cloud stack, it's really [45:42] easy to um uh spin up a Postgress [45:45] warehouse like whatever. You just like [45:48] hit a button and you have a cloud-based [45:49] Postgress warehouse stood up. The [45:52] problem is when you build on prem then [45:54] you have to you know maybe you're hiring [45:55] a DBA and then you're like managing that [45:57] resource and those discs have to be [45:59] separated and then you have to think [46:00] about security and like what about [46:02] backups and restore for your on-rem [46:05] database. So you get complexity adds on [46:09] complexity right when you're uh actually [46:11] managing your infrastructure. So if you [46:13] can reduce it down to one thing that [46:15] you're having to manage then it's better [46:18] and AI store uh raisor tables actually [46:21] I'll just say that the catalog itself [46:23] inherits all of the durability of AI [46:25] store. So everything people come to love [46:28] about AI sort of like um strict [46:30] consistency and active active [46:32] replication all that applies to the [46:34] catalog in a way that has never happened [46:36] before right because catalog metadata is [46:39] usually a transactional database. [46:42] >> Think about replication in terms of [46:44] database talk instead of object storage [46:47] talk. [46:49] >> Does that make sense? I mean hopefully [46:50] that makes sense. Let me let me show you [46:53] what this is. So, I'm going to like open [46:55] I'm going to pip install um pi iceberg [46:57] which is you know one of my faves and [46:59] boto3. I'm going to import some [47:03] um utilities for this one. Um the rest [47:05] of these libraries so this should be [47:07] this type of configuration should be [47:09] sort of familiar with anyone who's [47:10] familiar with AI store. The only [47:12] difference is now you have a catalog URL [47:15] right um your host your warehouse. [47:21] So this one I guess is also new but all [47:23] this other stuff is just like regular [47:25] M.IO stuff. [47:28] Um we create a warehouse. So that's the [47:30] first step. Oh the other thing is uh we [47:33] made the architecture. So uh AWS tables [47:37] which is sort of like you know we always [47:39] strive to be as closely um [47:43] uh like lock step with AWS um because we [47:47] support fully support the S3 API. The [47:50] way they did this um what they did their [47:54] tables is a little bit different. We do [47:56] warehouse and then the architecture and [48:01] then name space and then tables whereas [48:06] they have a different um hierarchy but [48:10] it's the same. It does it's functionally [48:12] the same. We just feel like this is more [48:13] familiar to people that are more [48:15] familiar with traditional databases. So [48:17] if you're like a database engineer, [48:18] which or a data engineer, somebody who's [48:21] actually building or getting their [48:22] fingers in this code, it's a API. You're [48:24] more familiar with these terms. [48:26] >> Yeah. [48:26] >> Than the ones that AWS, but it's [48:29] >> it's the same function. Um so you make a [48:33] warehouse and then you initialize pi [48:35] iceberg [48:37] and then you make a namespace. This one [48:40] already has one. And I just want to show [48:42] you this is this one is listing all the [48:44] name spaces that I have. So after this [48:46] point really we're just dealing with the [48:49] iceberg rest catalog. So everything [48:51] under here is not ours. It's an open [48:54] source REST API. Um somewhere in there [48:57] that we support the S3 API. [49:00] Um this is an API for developers who are [49:03] developing the lakeouses. Right? So um [49:06] this is not ours. This is just the API, [49:10] right? So we'll create a name space [49:12] properties. properties are cool in your [49:13] lake houses because you get to say [49:17] things like uh this is only for [49:22] um uh sales to read this or this one is [49:27] read only for engineering or um uh you [49:31] can you can set like different security [49:34] um or user uh level um uh permissions in [49:39] your properties. It'll make a table [49:40] called employees. I'm just going to show [49:42] you um which is uh so I'm going to [49:46] hardcode. So these are examples of these [49:48] strings. I'm going to hardcode these [49:50] hardc code these S3 location. So this is [49:54] a whole S3 URL. In real life um again I [49:58] make the demo so I can make them however [49:59] I want to. But in real life you probably [50:02] will have this not be hardcoded but [50:04] actually be a vector or something with [50:06] the S3 location. So this S3 location, [50:09] let's say it points to a bucket in [50:11] min.io and let's say in production, it's [50:14] actually a vector, right? Where you have [50:17] uh something there for your AI model or [50:20] something. So it's a it's pointing to an [50:22] actual location in AI store, but it's [50:26] not hardcoded in because that's [50:27] obviously ridiculous, right? You use [50:29] vector database to find images or [50:32] pictures or videos or something like [50:33] that. Um or in this case, PDFs and and [50:36] things of that nature. So list all the [50:38] tables. I made a bunch of tables. Um and [50:41] then again this is again this is [50:42] hardcoded but like go imagine the use of [50:45] vectors actually. Okay. So [50:47] >> it's a demo. That's fine. [50:48] >> Yes. Okay. So I want to make made five [50:51] employee records. I'm going to have [50:52] dupes but um this is show you how easy [50:55] it is to add more um uh records to your [50:59] table. This again this is not ours. This [51:01] is just the rest API but it's cool. like [51:04] the part that we did is like up here [51:06] where you can connect to it and it's all [51:07] right there. Everything's there. So the [51:09] point of showing the hard-coded um S3 [51:12] locations is to say that you could have [51:15] let me just show you example of query. [51:18] You could have a table [51:20] that has um a structured table an [51:24] iceberg table that has access to [51:27] unstructured [51:28] S3 locations. So your unstructured data [51:31] and your structured data can um actually [51:33] be findable in the same location, right? [51:36] Because your unstructured iceword tables [51:38] are in the same place as your contracts, [51:40] as your resumes, as your videos, as your [51:43] whatever whatever you have in object [51:45] storage. Object stores in itself, [51:47] anything can be an object, right? But [51:49] now with the catalog embedded, your [51:51] structured data and your unstructured [51:52] data can come together in like a really [51:55] um fast um and um important way, [51:59] especially for agentic AI or human AI. [52:02] It could be this could be an agent. This [52:04] could be a person that's running a [52:07] model, but everything's at their [52:09] fingertips. Everything is all in one [52:10] spot. Does that make sense? [52:12] >> No, makes sense. [52:13] >> I don't know how much time I have, but [52:15] this is this is different patterns of [52:16] the same thing. So, this is another [52:18] >> you have as much or as little time as [52:20] you want. As as long as you're still [52:21] having fun demoing, you're good to go. I [52:24] told you it's a really laid-back show. [52:26] If you're like, you know, Fred, I'm [52:27] tired of talking about this and we're [52:29] good. We're done. So, it's for you. Um [52:32] basically this is just the rest of this [52:33] is just different um in like uh flavors [52:37] of the same thing where you have [52:38] structured data which is iceberg which [52:40] is parquet plus metadata and then you [52:43] have locations to unstructured data [52:46] PDFs, music files, whatever whatever [52:49] objects you want. you can finally pull [52:51] it all together so that you could say [52:53] you could give this to Dana and she [52:54] could click on this link maybe or this [52:56] vector or whatever it is that you've [52:59] embedded into this um into your iceberg [53:03] catalog or sorry your iceberg table and [53:06] they'll they'll have it all at their [53:07] fingertips. So imagine this is being [53:08] served up as like a query in a internal [53:12] um HR like um tool or something. So they [53:18] can see all their records in one spot. [53:20] Um or it's an agent and Dana is trying [53:23] to figure out if she did her training [53:26] this year or whatever. It's all [53:28] available to you. All your internal [53:31] documents in one spot, organized in the [53:33] way that makes that's most sensible. So [53:37] anyways, that's that's tables. That's as [53:41] in general. [53:43] Um I guess that's all I have to show. Do [53:45] you have any questions? [53:47] >> No. I know that that was pretty clear. [53:48] Uh every time I have a guest to talk [53:51] about data, I kind of miss like playing [53:54] more with data. You know, when I when I [53:56] was a when I was building software [53:58] full-time, uh I it was always in smaller [54:01] start smaller company. It was not even [54:04] called startup at the time on that whole [54:05] but like it was in smaller companies and [54:07] we didn't have DBA. Uh it was like like [54:09] developer were doing everything. So I [54:12] was usually the one like like building [54:13] the database, thinking about the schema [54:16] and doing that part. I kind of miss [54:17] that. Uh so every time I have someone [54:19] talking about like data on the show, I'm [54:21] like, "Oh, I I miss like being able to [54:23] think about like the the architecture of [54:26] this and like the the the schema of like [54:28] how I'm going to like, you know, store [54:29] that data and how it's going to be [54:31] efficient and going be useful for the [54:33] developer, but also useful for like the [54:35] people's going to consume the data. So [54:37] uh no, great. uh and and always a fan [54:40] like I'm a I'm an open source person so [54:42] I love also when uh demo is like part of [54:45] like oh we have that product there's a [54:47] part that is like like like hey charge [54:50] for us there's another part that is free [54:51] there's a part of open source like it's [54:53] it's a great mix so no great demo that [54:55] was super useful I will not consider [54:58] myself a data lake expert after demo [55:00] though but it's good enough for me to [55:02] know a little more uh what people are [55:04] talking about uh when they talk about [55:06] data links and um having a great preview [55:08] you also about like Heisert and uh the [55:10] new uh Mino uh product. So uh thanks a [55:14] lot for uh for for the demo. [55:16] >> Yeah, I'm always happy to talk about [55:18] this stuff. I think data links are so [55:20] cool. Like I said, I think I really [55:22] think that they just as a data engineer [55:24] perspective, like I'll take off my NIO [55:27] hat, that they're the future of [55:29] architecture, like the largest data lake [55:32] houses in the world are running on the [55:35] coolest AI stuff that you've heard of [55:37] like um [55:40] >> you know this is just that's just facts [55:42] like so it's really exciting and I think [55:45] iceberg especially has made this process [55:48] much easier for an deep anal analytics [55:50] and people forget that like AI is first [55:52] and we don't need analytics or like you [55:54] know moving around columns and like [55:56] cleaning up data but that clean data [55:59] vast amounts of clean data aster clean [56:02] data really critical for AI initiatives [56:05] to work so all this is like fits [56:06] together like a puzzle piece I think [56:08] it's just so cool [56:11] >> love it so that bring us to the end of [56:14] the show uh I actually I love your demo [56:17] but my cat was not impressed I [56:20] the old time. Uh, [56:23] >> I'll work on it. I, you know, I have a [56:25] um a dog, so I I feel like I've I've [56:30] demoed to the wrong audience. I got the [56:33] dog. [56:33] >> No, he's just he's just a slacker. [56:35] Sometimes he's going to join some [56:36] meeting. Sometimes you just don't want [56:38] to. But, uh, he's often in the [56:40] background of the show or my meetings. [56:42] Um, so yeah. No. Uh, thanks a lot for [56:45] Sorry. Go ahead. [56:46] >> Well, I was just wondering what his name [56:48] was or her name. It's tells [56:50] >> Tails. [56:51] >> Yeah. And and the story be I got him uh [56:54] it's a it's a male. I got him from a [56:56] shelter and unfortunately when he [56:58] arrived he he had a beginning of a rough [57:00] life. So uh his tail were uh like all [57:04] screwed up. Like he got on the fight or [57:06] got stuck somewhere. So uh when the [57:08] shelter took him in um they went to the [57:11] vet and they had to cut part of the [57:13] tail. So now he's a cat but like his [57:15] tail is like like this. Uh, so I guess [57:18] he it looks like he had multiple tails. [57:20] Uh, it's it's kind of like sad, but at [57:22] the same timeish story because this is [57:24] where his name is coming from and I just [57:26] kept the name. Uh, but he is he's [57:28] lovely. He's he's a little bit a little [57:30] bit like a dog like always happy to see [57:32] me following me around a little bit the [57:34] house. Uh, you know, he has his his cat [57:36] moments where he's independent, but he's [57:39] not like the usual independent cat who [57:41] does not care that much about you. uh [57:44] this this one is just codly and they [57:45] like to spend time with me but uh yeah [57:48] >> is part is part of the the show uh quite [57:51] often. So yeah, thanks uh thanks for [57:54] being a guest and thanks for your time. [57:56] Uh happy to have a discussion with you. [57:58] That was really interesting. Your demo [57:59] was great and um also want to thank [58:02] everybody u that were listening live but [58:05] also people who's going to watch the [58:07] recording after and um we will see you [58:11] in I will see you actually in two weeks [58:13] for the next episode of the Devrell [58:16] show. [58:17] >> Okay, cool. Bye everybody.  ## 📺 Video ![](https://www.youtube.com/watch?v=Uq5NNE-f7ts) --- # 📝 Full Notes (Deep-Read Analysis) # 🏷️ Title: devrelshow episode 43 - Brenna Buuck from MinIO ## 🧰 Action Inbox - [H1] Explore MinIO's AI Store Tables for data lakehouse architecture — 00:30:15 #data-architecture - [H2] Consider open source alternatives to AWS S3 Tables for on-premise solutions — 00:32:45 #cloud-alternative - [H3] Investigate PyIceberg for Python-based data lake operations — 00:36:20 #python #data-engineering ## 🧭 Table of Contents - [📝 Summary](#-summary) - [📖 Main Content](#-main-content) - [🗒️ Notes](#️-notes) - [📚 References](#-references) - [🗣️ Paraphrase](#️-paraphrase) - [📰 Newsletter](#-newsletter) - [✅ Action Items](#-action-items) - [🧪 Hypotheses](#-hypotheses) - [📊 Methodology](#-methodology) - [❓ Questions](#-questions) - [🔮 Future Research](#-future-research) - [💡 Implications](#-implications) - [🧠 Learning Styles Assessment](#-learning-styles-assessment) - [📊 Accessibility Assessment](#-accessibility-assessment) - [🌍 DEIB Assessment](#-deib-assessment) - [✅ Logical Validity](#-logical-validity) - [🛡️ DARVO Analysis](#-darvo-analysis) - [🎯 Conclusion](#-conclusion) ### 🔗 References - [MinIO](https://min.io) - [Apache Iceberg](https://iceberg.apache.org) - [PyIceberg](https://py.iceberg.apache.org) ## 📝 Summary - Developer advocate Brenna Buuck discusses her transition from data engineering to developer relations at MinIO - MinIO provides on-premise S3-compatible object storage with over 2 billion Docker pulls worldwide - The company introduced AI Store Tables, embedding Apache Iceberg catalog API directly into object storage - This eliminates the need for separate metadata databases traditionally required for data lakehouses - Key benefits include cost savings (especially beyond 10 petabytes), simplified architecture, and unified structured/unstructured data access ## 📖 Main Content Brenna Buuck serves as a Subject Matter Expert on Databases and Modern Data Links at MinIO, though she describes her role as developer advocacy work. She transitioned from traditional data engineering to developer relations, motivated by her passion for teaching and helping others learn about data technologies. MinIO provides on-premise, S3-compatible object storage that serves as an alternative to cloud solutions like AWS S3 or Azure Blob Storage. The company has achieved significant adoption with over 2 billion Docker pulls worldwide and maintains both open-source (AGPL licensed) and enterprise versions. A key innovation discussed is AI Store Tables, which embeds the Apache Iceberg catalog API directly into MinIO's object storage. This architectural approach eliminates the traditional requirement for separate metadata databases (like MySQL or PostgreSQL) that external Iceberg catalogs typically need. The result is a simplified data lakehouse architecture that reduces operational complexity, especially for on-premise deployments. The technology enables unified access to both structured data (through Iceberg tables) and unstructured data (PDFs, videos, images) stored in the same object storage system. This convergence is particularly valuable for AI initiatives and analytics workflows that require both data types. ## 🗒️ Notes #### ⏱️ [00:00](https://youtu.be/Uq5NNE-f7ts?t=0) — [H1] New Year Introduction and Show Context - Key idea: First episode of 2025 with continuing guest lineup - Evidence: "First episode of the year, episode 43. We're continuing to talk about DevRel" #### ⏱️ [02:30](https://youtu.be/Uq5NNE-f7ts?t=150) — [H2] Brenna's Introduction and Role at MinIO - Key idea: Subject Matter Expert title encompasses developer advocacy responsibilities - Evidence: "I work as a developer evangelist basically at MinIO, which is object storage software company" #### ⏱️ [04:45](https://youtu.be/Uq5NNE-f7ts?t=285) — [H3] MinIO Product Overview - Key idea: MinIO provides on-premise S3-compatible object storage - Evidence: "MinIO is basically on-prem S3. It's truly S3 native. There's no map data gateway" #### ⏱️ [06:20](https://youtu.be/Uq5NNE-f7ts?t=380) — [H1] Developer Relations Title Discussion - Key idea: Title flexibility reflects both expertise and practical communication needs - Evidence: "I say developer advocate because it gives people a better sense of what I do day-to-day, but it's shorter on a name tag" #### ⏱️ [09:15](https://youtu.be/Uq5NNE-f7ts?t=555) — [H2] Technical Audience Understanding - Key idea: Technical software buyers have technical backgrounds regardless of current job titles - Evidence: "Everybody that buys technical software has a degree of technical aptitude" #### ⏱️ [11:30](https://youtu.be/Uq5NNE-f7ts?t=690) — [H3] Previous Role at StreamSets - Key idea: StreamSets provided visual ETL tools with data drift detection capabilities - Evidence: "StreamSets had this ability to detect and remediate data drift" #### ⏱️ [16:45](https://youtu.be/Uq5NNE-f7ts?t=1005) — [H1] Career Transition Motivation - Key idea: Teaching and knowledge sharing drove the move from engineering to developer relations - Evidence: "I was most fulfilled by teaching undergrads political science" #### ⏱️ [19:20](https://youtu.be/Uq5NNE-f7ts?t=1160) — [H2] Benefits of Developer Relations Role - Key idea: More human interaction and positive feedback compared to traditional data engineering - Evidence: "Nobody ever talks to you unless something's broken in data engineering" #### ⏱️ [25:30](https://youtu.be/Uq5NNE-f7ts?t=1530) — [H1] Data Lakehouse Architecture Introduction - Key idea: Apache Iceberg enables data lakehouse functionality through metadata management - Evidence: "Iceberg metadata is what gives you really cool things like time travel and schema evolution" #### ⏱️ [28:45](https://youtu.be/Uq5NNE-f7ts?t=1725) — [H2] Cost Effectiveness of On-Premise Storage - Key idea: On-premise becomes more cost-effective than cloud at approximately 10 petabytes - Evidence: "About 10 petabytes of data where it becomes much more cost effective to store your data on-prem" #### ⏱️ [30:15](https://youtu.be/Uq5NNE-f7ts?t=1815) — [H1] AI Store Tables Innovation - Key idea: Embedded Iceberg catalog eliminates need for separate metadata databases - Evidence: "Our version is just your object storage with your built-in catalog service" #### ⏱️ [36:20](https://youtu.be/Uq5NNE-f7ts?t=2180) — [H2] PyIceberg Demo Setup - Key idea: PyIceberg provides Python access to Iceberg without JVM requirements - Evidence: "I like PyIceberg because I have been victimized personally by Spark and PyIceberg doesn't require JVM" #### ⏱️ [42:10](https://youtu.be/Uq5NNE-f7ts?t=2530) — [H1] Unified Data Access Benefits - Key idea: Structured and unstructured data can be accessed together in same location - Evidence: "Your structured data and unstructured data can come together in a really fast and important way" #### ⏱️ [45:30](https://youtu.be/Uq5NNE-f7ts?t=2730) — [H2] Data Lakehouses as AI Foundation - Key idea: Data lakehouses provide the clean data foundation necessary for AI initiatives - Evidence: "Clean data, vast amounts of clean data, is really critical for AI initiatives to work" ## 📚 References - [MinIO](https://min.io/) - [Apache Iceberg](https://iceberg.apache.org/) - [PyIceberg Documentation](https://py.iceberg.apache.org/) - [StreamSets](https://streamsets.com/) ## 🗣️ Paraphrase - MinIO offers on-premise object storage that's fully compatible with Amazon S3 APIs, serving as an alternative for organizations wanting to avoid cloud vendor lock-in - The new AI Store Tables feature simplifies data lakehouse architecture by embedding the Iceberg catalog directly into the storage layer, eliminating operational complexity - Transitioning from data engineering to developer relations provides more opportunities for teaching and human interaction while still maintaining technical depth ## 📰 Newsletter 🎯 **New Year, New Data Architecture: MinIO's AI Store Tables Revolution** Starting 2025 strong with insights from Brenna Buuck at MinIO! If you're building data-intensive applications, this episode reveals a game-changing approach to data lakehouses. **Key Takeaway:** MinIO's AI Store Tables embed Apache Iceberg catalog API directly into object storage, eliminating the traditional need for separate metadata databases. This means simpler architecture, lower operational overhead, and unified access to both structured and unstructured data. **For WordPress Developers:** While this might seem outside our usual scope, consider how media-heavy WordPress sites could benefit from object storage solutions, especially for large-scale content management systems. **Career Insight:** Brenna's transition from data engineering to developer relations highlights how technical expertise combined with teaching passion creates powerful developer advocacy roles. Resources: [Learn WordPress](https://learn.wordpress.org/), [Developer Docs](https://developer.wordpress.org/), [WP-CLI](https://wp-cli.org/) ## ✅ Action Items #### DevRel - Developer advocacy: Create technical content bridging structured/unstructured data use cases, conference talks on data lakehouse architecture - Education: Develop workshops on Iceberg catalog implementation, office hours for data engineering questions - Developer experience: Improve PyIceberg documentation, create getting-started tutorials - Developer marketing: Content strategy around on-premise vs cloud cost comparisons - Community management: Moderate data engineering forums, gather feedback on AI Store Tables #### Community - Maintainers: Support Apache Iceberg project governance, contribute to PyIceberg roadmap - Contributors: Onboarding programs for data engineering newcomers, first-issue programs for MinIO - Sponsors: Transparent budgets for open source contributions, impact reports on community engagement #### Data Engineering - Architecture: Evaluate data lakehouse implementations, cost analysis for on-premise storage - Performance: Benchmark Iceberg catalog performance, optimize query engines - Integration: API compatibility testing with S3 ecosystem, multi-cloud data strategies - Security: Data governance frameworks, access control for unified data access ## 🧪 Hypotheses - Organizations with >10 petabytes of data will find on-premise object storage more cost-effective than cloud solutions - Embedded Iceberg catalogs will reduce operational complexity compared to external catalog architectures - Unified structured/unstructured data access will accelerate AI initiative development - Developer advocates with deep technical backgrounds will be more effective at engaging technical audiences ### 📊 Methodology - Cost analysis comparing cloud storage tiers vs on-premise infrastructure at various data scales - Performance benchmarking of embedded vs external Iceberg catalog implementations - Survey data engineers about operational complexity factors in data lakehouse deployments ### ❓ Questions - How does the embedded Iceberg catalog handle high-availability and disaster recovery scenarios? - What are the performance implications of embedding catalog API within object storage? - How does this architecture scale compared to traditional external catalog approaches? ### 🔮 Future Research - Multi-cloud data lakehouse architectures with hybrid catalog strategies - Integration patterns between AI Store Tables and major query engines - Developer experience improvements for data lakehouse operations ### 💡 Implications - Practice: Simplified data lakehouse deployments with reduced operational overhead - Policy: Potential shift in enterprise data architecture standards toward embedded catalogs - Security: Unified access patterns may require new governance frameworks - Education: Need for training materials on modern data lakehouse architectures - Funding: Cost savings from reduced cloud storage bills could fund additional on-premise infrastructure ### 🧠 Learning Styles Assessment Reading level estimate: College level (technical content with domain-specific terminology) #### 👀 Visual learners - Present: Architecture diagrams, cost comparison charts - Improve: Interactive data flow visualizations, storage tier comparisons #### 🔊 Aural learners - Present: Video interview format, spoken explanations - Improve: Podcast version, technical deep-dive audio supplements #### 🗣️ Verbal learners - Present: Clear definitions of data lakehouse, Iceberg, object storage - Improve: Glossary of terms, analogy-based explanations #### 🧩 Kinesthetic learners - Present: Live code demonstration, hands-on PyIceberg examples - Improve: Step-by-step setup guides, interactive tutorials #### 🧠 Logical learners - Present: Cost analysis charts, architectural decision trees - Improve: Performance benchmarks, capacity planning calculators #### 🧑‍🤝‍🧑 Social learners - Present: Interview format, community discussion - Improve: Forum discussion prompts, collaborative exercises #### 🧘‍♀️ Solitary learners - Present: Self-contained explanations, individual reflection time - Improve: Self-assessment questions, independent lab exercises #### 🌿 Naturalistic learners - Present: Data ecosystem analogies - Improve: Environmental impact considerations of on-premise vs cloud ### 📊 Accessibility Assessment #### Screen reader semantics - Present: Clear heading structure, descriptive content - Improve: Alt text for technical diagrams, ARIA labels for demo elements #### Keyboard navigation - Present: Standard video controls - Improve: Keyboard shortcuts for demo sections, transcript navigation #### Contrast and readability - Present: Clear audio, readable slide content - Improve: High contrast mode for technical diagrams #### Text resizing and reflow - Present: Responsive video player - Improve: Scalable transcript formatting #### Media and captions - Present: Full transcript available - Improve: Timestamped captions, technical term definitions #### Forms and interactions - Present: Basic YouTube interface - Improve: Accessible demo interaction elements #### Standards reference - Video content meets WCAG 2.2 AA for multimedia content ### 🌍 DEIB Assessment #### Cultural breadth - Present: Discussion of global technology adoption - Improve: Multi-region data sovereignty examples, international case studies #### Inclusive language - Present: Gender-neutral technical terms, defined jargon - Improve: Plain-language summaries for complex concepts #### Neurodiversity support - Present: Clear structure, consistent pacing - Improve: Content warnings for complex technical sections #### Representation - Present: Female technologist in leadership role - Improve: Diverse technical background representation #### Psychological safety - Present: Respectful interview format, acknowledgment of learning curve - Improve: Beginner-friendly content warnings, multiple learning paths ### ✅ Logical Validity #### Strong reasoning - Deductive strength: Cost analysis based on storage pricing models - Inductive strength: Industry trends toward data lakehouse architecture - Causal reasoning: Simplified architecture leads to reduced operational overhead - Analogical strength: Comparison between cloud and on-premise trade-offs #### Fallacies - Appeal to popularity: "2 billion Docker pulls" as quality indicator - success metric, not inherent quality proof - False dichotomy: On-premise vs cloud presented without hybrid options - address multi-cloud strategies #### Tone of voice - Overall tone: Technical but conversational, expertise balanced with accessibility ### 🛡️ DARVO Analysis - Deny: No defensive patterns observed - Attack: No attacks on competitors or critics noted - Reverse victim/offender: No victim/offender reversal present - Emotional cues and rhetoric: Professional, educational tone throughout ## 🎯 Conclusion This episode provides valuable insights --- # 📄 Full Transcript *Auto-generated from YouTube captions · timestamps roughly every 30 seconds · [watch on YouTube](https://www.youtube.com/watch?v=Uq5NNE-f7ts)* **[00:01]** [music] **[00:32]** Hello. Hello, lovely people. I mean, I guess I have to do that. It's the first week of the year. So happy new year folks. Uh you know I don't know people wish some stuff to other people. I will wish you health and you know having a beautiful year. I don't know. I think for 2025 uh sucks for mo many people from what I've read on social media what I've seen during uh the holidays. So hopefully this year is going to be better. Uh do what makes sense for you but yeah I always wish health uh to **[01:02]** people. It seems boring but it's uh it's kind of like so important. So, with that said, uh, and we're close to, you know, I I think we need to stop saying happy New Year's after the first week. Uh, so we're still good. But, uh, yeah, first episode of the year. I'm really happy. I'm continuing to doing the Devrell show. As I told you, I have so many guests lining up. So, that's going to be an interesting year. So, it will just depend on my traveling schedule. But, first episode of the year, it's uh, episode 43. So, uh, we're continuing to **[01:36]** talk about Deell, having amazing guests, and today is no different. I have Brena with me. How is life? >> Oh, it's great. Um, it's uh finally cooled down in Las Vegas where I live, and I'm just enjoying our our winter wonderland here. >> I I hear you. I mean, we don't have the same kind of winter. I'm in Canada in the east side and Montreal. So, uh yeah, we have like snow and and it's freaking **[02:07]** cold right now. So, um I don't really enjoy the cool down that we're having in Canada, but I understand from your point of view. Uh it really makes sense. It really makes sense. Uh did you did you have time to uh disconnect a little bit uh during the holiday? >> Um to to sorry to what >> disconnect or just like take some time off? >> Yeah, it was great. Um uh I'm really grateful for like my company's like respect for um like family time. So like I had a really nice uh relaxing break. I **[02:40]** got to work on some personal dev projects and um and hang out with my family and that was it's always always fun. >> That's pretty cool. So first question usually and now I had like multiple question for you before my usual first question which is who are you? Tell us who you are. >> Uh, wow. Um, I feel like I'm in a therapist office. Uh, well, my name is, you already, you already know, my name is Bren. Um, I I **[03:11]** work as a developer evangelist basically at Min.io, which is a object storage software company, mainly onrem object storage. So an alternative to Amazon's S3 or um you know Azure blob storage on prem. Um I before this role I um was a basically a real data engineer. I um I worked uh full-time um you know building pipeline **[03:42]** and um delivering clean data. And so I transitioned into this role because um I really enjoy uh teaching and helping people learn and uh data is one of my u favorite subjects. So I'm really um like keyed well to this role because it's [laughter] it's a couple of the things that I um care about the most and have a lot to say about. >> Yeah. No. Uh love that. And um we met if **[04:15]** I'm not if I'm not wrong I think it was like some cube con non America 2024 uh you were at the minio booth and I was looking for guests for primary it was for the cube first live which was uh the live stream that I was doing at my previous company construct uh unfortunately uh that didn't happen but I was like hey you know you do developer relation uh you should come on the dev show and I really want to thank you for your patience because uh it was like a year and a half ago. You were scheduled **[04:45]** six months ago, but I had like a last like I got a new job. I had a last minute trip. I had to reschedule you. And I did the quote unquote mistake of like trying to schedule all my guests for the full year, which I learned the hard way. It's not the way to go because, you know, live and schedule and stuff. So, I was not about to reschedule you up to like January 2026. But, uh thanks for being patient with me and thanks for being on the show. Uh, I really appreciate that. >> Yeah, I'm happy to. Um, you know, um, we're all busy and, um, you know, in **[05:18]** some ways it's nice to have an idea of what where you're going to be far in advance. You know, I travel a lot for the role that I, uh, do. So, you know, I'm also um, a developer at heart, so change is part, you know, the of my life. It's in the fabric of what I do. So, I'm totally I just roll with punches. It is what it is. No, that that makes sense. I like that. Uh so one of the first thing I wanted to ask you because when you introduce yourself, you say like hey I'm a developer advocate **[05:48]** and when you look at your LinkedIn uh the title is different. You're a subject matter expert on databases and modern uh data links and um I know when you read the description you understand that like yeah okay you're a matter expert but most of the things you do are developer relations related. So can you tell us a little more about like you know maybe the title itself like why that title and and your role actually like what is your role? >> Well to be frank I I didn't pick it. I **[06:19]** just um applied for it but it is um uh if when I was looking for developer advocate roles it's really um matched a lot of the um things I actually want to do. Um, actually I don't mind that we're calling subject matter expert. I say developer advocate a lot of times because it gives people a better sense of what I do dayto-day, but also um it's shorter on a name tag **[06:51]** um often, you know, of data links and modern um data or whatever it is, it's too long to fit on a on a placard. So um it's shorter. It's more like explan explan explanation I don't know um but um but it it I you know I work under product marketing um I'm a I'm a a real data engineer um and I spend my time uh you know writing blogs making um decks speaking having **[07:24]** going to speaking engagements uh manning the booth like where you met me talking questions about um our product technical questions for people that walk by. You know, often you need to pair either like a person like me or like a um an FA like a like a sales engineer or an FE, however you whatever you guys call um those people that do the sales engineering, you have to pair uh like a saleserson and a technical person with **[07:56]** in the booth. And I often stand in the place of uh sales engineers because I write code. Um uh like you know I I make demos um I I I package uh technical content for the marketing team. So product but I work under product marketing. So like you know anything that comes around in marketing around data bases or data lake um data lake houses I typically take a look at and check for technical accuracy **[08:28]** especially now in the time of AI where a lot of things moving really fast you know like sometimes content gets generated that's um [laughter] uh you know um trying to nice word I don't know like not uh complete I guess or inaccurate or just you know like hallucinated [clears throat] like that happens a lot with I think well and probably with all content but I especially deal with uh technical **[08:59]** content so sometimes chatbt or claude will envision a future or a reality that doesn't actually exist >> um and so I have to like go through stuff and make sure that it makes sense um and so for everyone like that anyone else writes so I write but I also a review. Uh I don't know. Does that does that answer your question? Like so I do a role of a developer advocate. I have a different title. I'm not mad at it. I mean it is does a subject matter expert. Also describes like what my expertise **[09:30]** is. Um but it's harder people for people to visualize. So you know what I do day-to-day unless I say developer advocate. So they're they're fine with us putting what we want on our >> No, makes sense. It make total sense. And I I'm asking that just for the sake of like having a discussion about it and and like for people listening. But um also um I think u anyway like a developer advocate should be a subject matter expert about like their own technology, their own product. So like **[10:01]** that's kind of like that works. But um when I've seen like people with that title, usually they were like you know engineers like like you know principal engineers or or I forgot what's the even the other title after that but like you know the main resources in the engineering team about like a specific topic or a specific technology. So it's why I was curious about like oh are you like like really like just like the the the the top expert in your company about that specific topic or is it like **[10:33]** another way or which could be true also but like is it also another way to like say oh you're a developer advocate too but like you just have a different title. So your explanation was was pretty good about that. I was just uh curious about that and I assume uh maybe people joining us live or watching your recording may had also the same uh answer because on the thumbnail of the video I put like subject matter expert and the full title uh that was written on on LinkedIn actually. So >> yeah, no um I you know our minio company **[11:04]** I work for has a history of being engineering le and so I do [clears throat] see the that history written in the title that I have. I'm the only well um prior we hired a an internal data engineer um like a year ago but I was prior to that was the only data engineer at the company. So um I do uh I do claim to have subject matter expertise in in data engineering and so **[11:35]** and you know specifically for data links and databases. So yes and also a developer advocate if it's easier for people to visualize what I do dayto-day than than just to say that I that I know stuff which is [laughter] >> No, but I like that. And it's great that you're saying that too because uh when I started in Devro like nobody known about that. Nobody known about a developer advocate. Actually we were not even called that. We're called technical evangelist and people didn't know what **[12:06]** it was. It was like people were asking me like are you going to church or something? And I'm like no I work in tech in tech. And and now that you're saying that like oh when I say to people that like I use the title developer advocate uh people recognize and understand what I do it's a great sign that like the industry understand a lot more developer relation and see also hopefully those role as something important for the business. So yeah, heard that. I I also recognize that change um and that shift uh between um **[12:39]** making developer advocates or developer evangelists or whatever you want to call them more important uh like prominent roles for business because especially for software um people are recognizing that you have to speak to developers first like you know we don't sell teddy bears or like you know widgets We're not we're not going to um ever not be technical. We hear that a lot especially in marketing like oh is this too technical? Is this going to like you **[13:10]** know um make people sit down too long and read it or is it like too in depth? Like everybody that buys technical software has a degree of technical aptitude. They may not be in their job title today, but you I have sat in as a data engineer on um buying decisions and everybody in that room from like chief data officer to um you know the business analytics folks have a technical background. So to to argue **[13:42]** that we need to dumb things down or um make things salesy when we're talking about selling technical software I feel like misses the mark on who our audience is uh for who's who's actually buying our product. Right. >> Yeah. >> That's what I think. I think it's so important. And I think it and we're getting um you know you don't want to alienate your um your your buyers and the reality is that **[14:12]** you know I am a developer so I know what that like person personality type is I guess and I can tell you that if you it doesn't take me long to figure out if you don't know what you're talking about. You know what I mean? So, like if I read an article or I see a video or I look at docs and it I I can instantly tell if it's not, you know, if it's just fluff or if it doesn't make any sense or if it's um just sales speak, you know, I **[14:45]** want to I want to get and I'll skip over that to find what it matters to me as a developer at like which is actual which is actually code or you know in depth talks or like an explanation of what I exactly I'm trying to do you know so I think that this role is really important to find that deeper level of involvement with um software customer base that's what I think >> yeah no and and [laughter] yeah I agree I agree totally u I always say that like **[15:17]** if you want to like like just talking about developer relation itself if you want to be success successful as a developer advocate no matter your title if your job is to talk to a developer audience. Uh you need to be credible. You need to know what you're talking about. So exactly as you said like I I think it makes sense. You don't have to dumb down down things. And it may depends on the product though. I mean like minio is is for technical people basically only from from my knowledge of the product. I mean uh it may be a **[15:49]** little bit different when I'm thinking about my own product uh tiny MC right now which is like yes it's for developers uh because they're going to integrate a rich text editor wizawig within their application they're building uh so it's a a product for developers but in my case once developer implement the the product tiny MC within their own product uh it's the end users that may not be technical who's going to use it because our product is targeted to developer to implement it But the usage of the product at the end once **[16:19]** it's implemented it's for anybody who need to create rich text content. So in that case uh the approach is a little bit different but not for me as a developer advocate because I'm not talking necessarily to the buyer or the end users. I'm talking to the developer who implement my solution. So I totally agree about that. And speaking about uh minio uh would you like to just talk a little bit about like you know the product and the company uh because this is where I know you from from my past life in like the cloud native and kubernetes uh at cube first we were **[16:51]** using um and I offer a local uh installation local flavor. So if you would like to just talk a little bit about that for people that do not know the company or the product. >> Sure. And like I said when I did my little intro, um it's kind of a rehashing, but minio is basically onrem S3. So it's uh truly S3 native. There's no map data gateway. Um there's no other protocols. We don't do fly on block. We don't do um uh anything else but S3 **[17:23]** onrem. And so we have more uh than two billion darker poles worldwide. Um they like to say that Candy Crush has five billion the B. So we're not that far off of of uh Candy [laughter] Crush. Um >> that's crazy though. >> Uh yeah, we're really proud of like the amount of developer support that we've gotten um uh from the beginning. So um it the origin of minio is an **[17:55]** open-source um uh codebase but in the last few years we've uh implemented and developed an enterprise version of the software. So we'll always have the community edition available for anyone. it's um distributed under um a GDPL uh license but the enterprise version has different and extra features um like console for example we also **[18:26]** introduced something that's really cool which I'll hopefully have time to demo today which is we've embedded the iceberg catalog API within our object storage and a couple a lot of a couple other cool stuff coming down the line. And we also have stuff for um in the enterprise version for enter uh managing enterprise software like um like a a a a key manager um extra like stuff like encryption. So **[18:59]** hopefully uh everyone on the call know you know knows who we are. If you're in the data space, you'll probably know who uh and you have onrem or local installations, you've heard of us. Um but um you know that's what that's what Mio is. >> Ah sounds good. And um prior to your you've been at Minio for a little more than two years now. Uh prior to that you were a developer evangelist at uh Stream Sets. Um yeah, would you like to talk a little bit about that role? **[19:30]** >> Yeah, so Stream Set is really cool. It is um or was um a software company that created um like an ETL tool, a visual ETL tool, play similar to uh like a no code um low code ETL tool similar to Taland or you know other um ETL tools around there. It had um this ability to differentiate it **[20:01]** had this ability to do uh something called um data drift. So it detect and remediate data drift. So that's what happens when like a like say you have um like a this happens a lot in data engineering where you have a destination say at a a a database that has it's it's uh field type is a date or something and something happens at the um you know input level that changes that. So maybe **[20:33]** instead of a date they start they type out in words February you know 21st 2006 26 and then um you know in full full letter English letters you know and so when it when it it it'll that'll break a pipeline right if the database the end state is uh expecting a date and you send >> a var or a string that can often um just kill a pipeline and it's really annoying because um it happens a lot and um um it **[21:07]** seems minor like and then people don't downstream your customers don't always your data customers don't always know why things like that don't work. So um stream size is cool because you could detect that and uh you know either skip that set up whatever kind of uh remediation you wanted. You could skip it. You could have some kind of cleaning step that kicked off and that kind of um weird string was detected to fix it and send it off as a um a date. You could um **[21:38]** uh kill a pipeline completely if you wanted to. There was a lot of like remediation steps that you could automate in your pipelines that made them more durable. But it was really cool. It was a really cool company. I really liked working there. Um but they were acquired by um IBM. Um >> Oh, okay. >> Was the tail end of when I was working there and so you know like it happens you know. >> Yeah. [laughter] >> Yeah. There's always a lot of surprise with acquisition. So uh yeah. No. And **[22:10]** there was um you know as you know like I said change is inevitable and especially when you're working in a startup that kind of change is like you know always on the table and it's like one of the best case scenarios right you either acquire or you IPO and so you know it's fine um I wasn't let go or anything they but [laughter] it's all kind of like you know like when you can when the train's heading somewhere and you are on the train and you like you notice an impending cliff **[22:42]** or something and you're like, I better [laughter] take this exit. It was that kind of thing. So, I was just like, you know, I don't I don't know. I already I I see that they have already have a marketing team and they already have developer advocates and the roles I do are covered. So, I think maybe um >> Yeah. So, but I really enjoyed working there and um like everyone was super nice and I learned so much because prior to that I was a real data engineer and so coming into marketing I was learning **[23:12]** all kinds of words you know like MQLs and um you know I don't know uh B2B [laughter] I you know all all the stuff I don't I didn't really wasn't familiar with before. It's a definitely a new world for me and that role really prepared me for the career that I have now and I will always be grateful for the people that took a risk in hiring I feel like took a risk in hiring me for a real and data engineer for a marketing role that I never done before. So, um, especially, um, my my boss there was, **[23:45]** uh, Sean Anderson. He's like an incredible, uh, product manager, and I just feel like he took me under his mentorship wing, and I just learned >> Oh, nice. >> like a ton. He was amazing, um, or is amazing. Um, [laughter] so, um, >> that's great. That's great. >> It was really, really foundational for me and I learned a lot and I loved the product. So like I I feel like and everybody like especially low code no code low code no code tools help um **[24:16]** bring in uh people that aren't you know I'm thinking also the the company that you're working for I guess because these types of tools bring in you know the democratized data engineering which is >> so important for bringing memes like myself I guess into actually building um pipeline, it's often the people that know the most about the data. And so having them be able to be like in instrumental in actually moving it around is like really key to making um **[24:48]** data pipeline that don't break, right? If you know if you know the if you know the data the best, you should be the one actively, you know, making the pipeline. you shouldn't have to send it off to like people that don't that have to ask you questions about is this field supposed to be you know whatever whatever kind of questions that they have um for for you about that. So I really think those tools are cool and I I loved working there and I love learning about um developable developer **[25:19]** evangelism and or develop whatever you want to call this type of role >> like I and I you know I loved same same um in my role now I just I still really enjoy writing code and I love uh like relaying information to people. So it was that I really came into my own I feel like in that role and I brought all that experience to this one. So I'm just like really stoked. I like seemed like a really good decision. I was like very **[25:51]** nervous coming transitioning from real engineering to this type of role. Um as I'm sure [laughter] I'm sure a lot of your um listeners are they had a developer advocate. So I'm sure they have that similar they came from engineering [snorts] because I don't know I just it's different you know like it's >> um and I didn't know anything about marketing so it's it's strange making that transition. **[26:22]** >> So >> how how would the uh question about that then uh because yeah you were engineers before and you moved to like a role in developer relations. So um >> how did that happen or or why did you move from like a a more like like engineer role to a more engineer centric role to like now doing developer advocacy or developer evangelism. Uh I like to ask that question for people that mo most of us transition like rarely someone like your first job were **[26:52]** developer advocates or developer relation. We all most of us like were software developer engineers before and we moved to developer relations. So I really love to ask that question because everybody has like a different story. So what's what's your story there? >> Yeah. Well, so like I said, I love um data and I love teaching and so this really fits um like around the time that I started looking for that stream sets role, that first developer advocate role, I was **[27:23]** asked by I was working in engineering, right? And I was asked by my boss to make like a fiveyear plan. You know, it's kind of like those kind of things. Oh, I know. Can I swear? I'm sorry. Those >> Yeah, you can totally on that. It was fine. >> Um, you know, those silly um, you know, career goal things, but I actually took it seriously and one of the questions was something like like where do you like what in your past like career so **[27:53]** far have you found the most fulfilling? like what about or what role specific role or specific um you know um um project or whatever in your former roles you find most fulfilling and I really thought about it and I I feel like I was most fulfilled in the at that time by my experience um I actually I went to grad school for political science >> I know I don't have a te traditional **[28:24]** technical background I taught myself how to code which one of the reasons why I love no code low code solutions because it helps pe people like me get into technology where they you know ease them in into coding which is really super fun. Um but anyways um I taught myself wait so I I found most fulfilling teaching undergrads um political science which is what I majored in in grad school and in undergrad. So I was like what can I do **[28:54]** in in my career to like I have all this experience in data like I don't want to like I love solve the the puzzle solving element of data engineering I love like learning new things all the time I don't want to just discard all this experience um but how can I bring in more of this like what I found fulfilling which was imparting knowledge to >> to other others like being of service And I just like Googled that exact question [laughter] **[29:25]** and like it came up with like a oh you could like be a mentor, you can like be in um engineering leadership and I was like and then it was like or you can be a developer advocate. So I was like wow I should see what that's like. I applied for one job. Sean Anderson, the boss that um I um just mentioned, uh called me the next day for an interview and I got the job the next week and then just like from there like it was I feel like it was like I don't want to like throw it around too much, but I think it was **[29:56]** like a little bit of fate, you know what I mean? Like I was like, >> uh, what like I went from day one googling like what how do I bring an education and engineering together um to like having a job doing exactly that and it being like the perfect role for me. Do you know what I mean? Like like I love this stuff. Like I love I love data and I love teaching people things. And so far all of this has just been it's been like fun every day, you know? I don't [laughter] I'm not on pager duty anymore. No one's **[30:27]** upset. So like the thing about data engineering especially like a real as a like a enterprise data engineer like a real one is that nobody ever talks to you unless something's broken you know like no one ever was like hey you're doing a great job [laughter] Brena the best thing I got was like sometimes my PRs would be approved no no comments and I'd be like yes I'm doing like [laughter] somebody thinks I'm doing a good job but like nobody ever like I don't get **[30:58]** talked to. I like filled out tickets and that was kind of like, you know, kind of what I what I did dayto- day whenever I tried to like reach out to like ask business users like, "Hey, what are you going to use this uh report for? Like what what do you really like what do you really need? Like I saw what you asked for, but like can you like >> Yeah, I would like kind of got shut down or like I don't have time for that. Just give me what I asked, you know? Like and I just felt >> now I get to talk to people like you're **[31:28]** you and everyone that comes to the booth and people tell me in marketing especially they'll tell you you're doing a good job all the time like like you know I get to I feel like I'm a free elf, you know, like I'm [laughter] um I'm out into the world like I'm I do do some travel like I get to meet I get to talk to people face to face like >> Yep. Um, how cool is that? Like they let me out into the world. So, uh, it's way better than engineering. It's like **[32:00]** [laughter] for a certain kind of a person, right? You have to be kind of outgoing. >> Um, but I do love it a lot and I still write code and it if it breaks, [laughter] it's not a big deal. I'm like, oh, I'll fix that. You know, they make changes or like, you know, like there's a new library or stuff gets unsupported. Like I just it's like I just I'll I'll do that later. Like I'll add it to my backlog and I'll get to it in a couple weeks. I'm not No one's like waking me up at three in the morning. >> Yeah. There's no production code getting **[32:31]** on fire. >> Yeah. I do not miss that part of like uh working on product. uh like at at one company we had a product that people uh were using strictly between 2 and 4:00 a.m. So um yeah I was uh the only and it was a really small company that like we created the product. So uh and I was the most senior one which mean that like I was the one always on call for those things and we had so many bucks that like they were always waking me up at like 2 3 a.m. **[33:02]** >> I really do not miss that at all. that'll burn you out and you know and it's um it's not fun. So I I don't I don't I don't is that at all like people are nice to me now you know [laughter] >> I mean let me tell you the story it depends where you work and and and how people like the company you work for the product you work for because my first job um as a developer advocate uh was at Microsoft and it it was during the Balmer era if you watch this show you **[33:34]** already heard me tell that story a couple of time but uh there was during the like like whether you like Microsoft or not right now they're so much better and so much more friendlier than uh they used to be and you may disagree with that but like uh in the past when I started to work there at first when they approached me I said no one or two time because I didn't want to go there but actually they were like no Fred I want someone like you like who who talk open source who is in the open source community who is not like a typical Microsoft person so my audience were open source developers but I was working **[34:06]** for Microsoft during a time where Microsoft was not playing well at all with open source So people didn't like me. Some people like me, but there are some people that didn't like me. There's like two people that refused to shake my hand because I was working at Microsoft. So just a small warning. Most of the time people like you, but it depends on so many factors. Uh it's not always the case. But in my case, I miss coding though. Uh I still code part of the role, but I miss like being a developer full-time. But when I put put this in a scale of like oh do I prefer being a **[34:38]** developer advocate or being like an engineer or software developer full-time I still prefer doing dev. It's still technical. I still code but like also miss a little bit like like coding eight hours a day which I cannot do anymore because like I have so many other things to do with that role but I still love like being a dev. So >> I like um I miss that. I I mean I I don't miss and I do miss it because of that focus that you can like that flow state that you could get into. Um you **[35:09]** know like you I miss that element. I miss problem solving like something really deep like working on something for a long time it break it's breaking breaking breaking breaking and then like you go take a shower or something you're like oh that's what it like I that's what it is like that's that's gonna fix it and then like you know that's kind of fun. I and I don't work on I do still code but it's like it's demos you know I can make the data whatever I want it to be and I usually make it easy so it's not you know like **[35:41]** >> Makes sense. Yeah speaking of Oh sorry. >> Oh no go ahead. Yeah, I was going to say uh speaking of of of Demo, I think I think it would be a great time now to because you know, we could talk forever uh because we're being paid to talk, but um that could be a good time to move to the demo portion of the show. Uh which I'm really looking forward to it because uh you're going to talk about data, which is something I do not know about at all. I know nothing about it. So, I'm **[36:11]** looking forward to uh what you're going to talk about. All right, I'm gonna try to do I'm gonna give you the We don't have like that much time, so I'm going to give you like the condensed version of the slide deck and then I'll show you some code. How about that? >> Wonderful. Sounds good to me. >> All right. So, let me just share my screen. Okay. So, I want to just let's see just this one. **[36:42]** move this over so you guys don't really Okay, hold on. You know what? Maybe I'll just Can you see what do you do you see my deck? >> Not anymore. You just I I used to see your deck, but you removed the screen sharing, >> right? Sorry. >> That's That's fine. Technology is hard. **[37:13]** >> What I say? >> Yeah. Okay. How about that? Is that better? Can you see now? >> Yeah. Still seeing the slide now. >> Okay, cool. So, I'm just going to give you the compressed version of it. I'm not going to go through all these stuff, but um the main point about this is that um the most important data that you have or organizations have is not just structured data like you see uh **[37:44]** XML, JSON or or even the new open table formats like iceberg and delta lake. um often it's unstructured or like videos, images, things like that. And so there's a really need to bridge this gap between these different types of of formats of data and the actual needs for the uh workflows that people are actually actually doing, right? So uh we really as an organization well actually I feel **[38:15]** like as a data community has really rallied around iceberg as the open table format. This is a data lake house table format. Um it's um essentially so I'm like I'm used to talking to data lake folks so this is like an interesting um but iceberg Apache iceberg is an Apache uh project where um the parquet files are stored alongside metadata that um turn a uh typical or traditional data lake into a data lake house. So the **[38:49]** iceberg metadata is actually what gives you the really cool things like time travel and schema evolution. So remember I was talking about how stream sets could detect data drift like so if your data types were different. Iceberg has this type of um intelligent design in its foundation. So um for example um if your uh if your data coming in um changes schema **[39:19]** so if it's it's an int it was a formerly an int and now it's a varchar it will just um add a new column and u move on so you don't have broken pipelines you have um like the this is actually the the the I believe the future of data because it makes our jobs as data engineers much much easier and and as a whole that's been that's an accepted view in data engineering. >> Okay, **[39:49]** >> this is a just a description about how unstructured and structured work together. This is why people want to build data lake houses on prem because obviously we're minio is an on-rem uh uh S3 compatible object storage which is in of itself just your data lake but I always like this it's right here this is like the money shot I guess where you see where >> it's about 10 pabytes of data where it becomes much more cost effective to store your data on prem than it is in **[40:20]** the cloud and this is super hot data so this data that gets a lot transactions real data lakehouse transactions. I want to talk about the thing. This is a data lakehouse diagram where you have the query engines then an iceberg catalog which is necessary to manage that metadata that I was discussing when I described iceberg to you and then we have all the stuff in object storage. So this is what a typical lakehouse architecture looks like from a thousand ft up right you you have interoperable **[40:52]** querying engines you have to have a catalog of some kind there's different kinds of cataloges iceberg catalog um in it of itself the rest catalog is also an Apache project so this is open source and available for everyone it's a rest api so a lot of people have implemented enterprise versions of this like um polaris which is snowflake project um that is open source now but um also So project nessie which is um um dramo's uh version of this but all these are external cataloges that **[41:23]** require a separate um uh database like so you have to have a the whole promise of a data lakehouse is that you're moving away from transactional or traditional databases but all external iceberg cataloges require you to have a transactional database like my SQL or Postgress attached to it for managing the metadata. So it's like >> okay okay >> move into the future no more but except for this one that you have to [laughter] have um and so I have like a deck on **[41:55]** that like a slide deck which is oh yeah so here's like the tra traditional or like current iceberg external catalog which where you have your object storage layer whatever it is it's S3 it's minio um then you have uh your metadata database for your iceberg catalog um and then you have the catalog service whatever it is If it's Hive, is it Polaris, is it Nessie, is it, you know, um, Gravitino, whatever it is, >> it has to have a it has to have a a separate database. But our version **[42:27]** >> is just and [laughter] uh your object storage with uh your built-in catalog service. So these compress down and then it's just that. So it makes it uh easier to have a lakehouse, right? That's the point. That's why that's why we did this part. This is a cool new feature that we've introduced called AI store tables. Um S3 has a version or sorry Amazon has a version of it's called S3 tables but obviously it's cloud-based. This is >> vetted iceberg catalog on prem. We don't **[43:00]** charge you. So obviously every time you have a special bucket in AWS you get special bucket charges right there's no extra or additional charge for using this um API. It's just the same um we charge for um data under management. So >> okay, >> a standard for for any of the additional like features we have. You don't don't charge for seats. You don't charge for API calls. Um and that's why you can we can get to this type of graph where we show how much cheaper it is over time, **[43:31]** right? Because if you got an expensive tier bucket because you want to have a simple >> fast >> what >> that goes fast. Uh >> yeah, exactly. your SP cloud spend gets up in a way, right? All right, cool. So, I'll show you my demo now that I've explained hopefully what I'm trying to show you. >> That was good. That was perfect. Yeah, thanks for like I'm for sure some people listening didn't need that part, but like that was good for me because Yeah, I'm like I know what is a data lake, but like never work with it, never used it, **[44:01]** so I I don't have that knowledge. So, that was perfect intro. Love it. >> Okay, good, good, good. All right. So, I'm going to show you um a pi uh pi iceberg script. I like pi iceberg because I have been uh victimized personally victimized by spark in the past. And um pi iceberg doesn't require you to have a JVM. So, uh it it won't hurt your feelings like Spark does, but you can you can use this with Spark or Trino or **[44:31]** whatever. I have the other notebooks here if that's your flavor of query engine. But this is just um [laughter] a a so this is connecting to I'll just go into it. So this is a demo of AI store tables which is again the embedded iceberg catalog API into object storage so that you don't need a separate metadata database to maintain your iceberg catalog. You have to have an iceberg catalog if you want to have an iceberg um you want to have an iceberg uh data lake **[45:03]** um mandatory. So the question is what kind of iceberg catalog do you want? We love all types of iceberg. You can always use AI store for you know regular regular data links uh where you you use Polaris or Hive and you know have your stack like that totally fine you know it's it will work just fine but if um especially building on prem the big problem with having an additional service like so if you're you have a cloud stack um let me let the dog let me **[45:35]** just let the dog >> wants to go out Ariel. >> If you have a cloud stack, it's really easy to um uh spin up a Postgress warehouse like whatever. You just like hit a button and you have a cloud-based Postgress warehouse stood up. The problem is when you build on prem then you have to you know maybe you're hiring a DBA and then you're like managing that resource and those discs have to be separated and then you have to think about security and like what about backups and restore for your on-rem database. So you get complexity adds on **[46:09]** complexity right when you're uh actually managing your infrastructure. So if you can reduce it down to one thing that you're having to manage then it's better and AI store uh raisor tables actually I'll just say that the catalog itself inherits all of the durability of AI store. So everything people come to love about AI sort of like um strict consistency and active active replication all that applies to the catalog in a way that has never happened before right because catalog metadata is **[46:39]** usually a transactional database. >> Think about replication in terms of database talk instead of object storage talk. >> Does that make sense? I mean hopefully that makes sense. Let me [laughter] let me show you what this is. So, I'm going to like open I'm going to pip install um pi iceberg which is you know one of my faves and boto3. I'm going to import some um utilities for this one. Um the rest of these libraries so this should be this type of configuration should be sort of familiar with anyone who's **[47:10]** familiar with AI store. The only difference is now you have a catalog URL right um your host your warehouse. So this one I guess is also new but all this other stuff is just like regular M.IO stuff. Um we create a warehouse. So that's the first step. Oh the other thing is uh we made the architecture. So uh AWS tables which is sort of like you know we always strive to be as closely um **[47:43]** uh like lock step with AWS um because we support fully support the S3 API. The way they did this um what they did their tables is a little bit different. We do warehouse and then the architecture and then name space and then tables whereas they have a different um hierarchy but it's the same. It does it's functionally the same. We just feel like this is more **[48:13]** familiar to people that are more familiar with traditional databases. So if you're like a database engineer, which or a data engineer, somebody who's actually building or getting their fingers in this code, it's a API. You're more familiar with these terms. >> Yeah. >> Than the ones that AWS, but it's >> it's the same function. Um so you make a warehouse and then you initialize pi iceberg and then you make a namespace. This one already has one. And I just want to show you this is this one is listing all the **[48:44]** name spaces that I have. So after this point really we're just dealing with the iceberg rest catalog. So everything under here is not ours. It's an open source REST API. Um somewhere in there that we support the S3 API. Um this is an API for developers who are developing the lakeouses. Right? So um this is not ours. This [laughter] is just the API, right? So we'll create a name space properties. properties are cool in your lake houses because you get **[49:15]** to say things like uh this is only for um uh sales to read this or this one is read only for engineering or um uh you can you can set like different security um or user uh level um uh permissions in your properties. It'll make a table called employees. I'm just going to show you um which is uh so I'm going to **[49:46]** hardcode. So these are examples of these strings. I'm going to hardcode these hardc code these S3 location. So this is a whole S3 URL. In real life um again I make the demo so I can make them however I want to. But in real life you probably will have this not be hardcoded but actually be a vector or something with the S3 location. So this S3 location, let's say it points to a bucket in min.io and let's say in production, it's actually a vector, right? Where you have **[50:17]** uh something there for your AI model or something. So it's a it's pointing to an actual location in AI store, but it's not hardcoded in because that's obviously ridiculous, right? You use vector database to find images or pictures or videos or something like that. Um or in this case, PDFs and and things of that nature. So list all the tables. I made a bunch of tables. Um and then again this is again this is hardcoded but like go imagine the use of vectors actually. Okay. So >> it's a demo. That's fine. **[50:48]** >> Yes. [laughter] Okay. So I want to make made five employee records. I'm going to have dupes but um this is show you how easy it is to add more um uh records to your table. This again this is not ours. This is just the rest API but it's cool. like the part that we did is like up here where you can connect to it and it's all right there. Everything's there. So the point of showing the hard-coded um S3 locations is to say that you could have let me just show you example of query. You could have a table **[51:20]** that has um a structured table an iceberg table that has access to unstructured S3 locations. So your unstructured data and your structured data can um actually be findable in the same location, right? Because your unstructured iceword tables are in the same place as your contracts, as your resumes, as your videos, as your whatever whatever you have in object storage. Object stores in itself, anything can be an object, right? But now with the catalog embedded, your **[51:51]** structured data and your unstructured data can come together in like a really um fast um and um important way, especially for agentic AI or human AI. It could be this could be an agent. This could be a person that's running a model, but everything's at their fingertips. Everything is all in one spot. Does that make sense? >> No, makes sense. >> I don't know how much time I have, but this is this is different patterns of the same thing. So, this is another >> you have as much or as little time as you want. As as long as you're still **[52:21]** having fun demoing, you're good to go. I told you it's a really laid-back show. If you're like, you know, Fred, I'm tired of talking about this and we're good. We're done. So, it's for you. Um basically this is just the rest of this is just different um in like uh flavors of the same thing where you have structured data which is iceberg which is parquet plus metadata and then you have locations to unstructured data PDFs, music files, whatever whatever objects you want. you can finally pull it all together so that you could say **[52:53]** you could give this to Dana and she could click on this link maybe or this vector or whatever it is that you've embedded into this um into your iceberg catalog or sorry your iceberg table and they'll they'll have it all at their fingertips. So imagine this is being served up as like a query in a internal um HR like um tool or something. So they can see all their records in one spot. Um or it's an agent and Dana is trying **[53:23]** to figure out if she did her training this year or whatever. It's all available to you. All your internal documents in one spot, organized in the way that makes that's most sensible. So anyways, that's that's tables. That's as in general. Um I guess that's all I have to show. Do you have any questions? >> No. I know that that was pretty clear. Uh every time I have a guest to talk about data, I kind of miss like playing **[53:54]** more with data. You know, when I when I was a when I was building software full-time, uh I it was always in smaller start smaller company. It was not even called startup at the time on that whole but like it was in smaller companies and we didn't have DBA. Uh it was like like developer were doing everything. So I was usually the one like like building the database, thinking about the schema and doing that part. I kind of miss that. Uh so every time I have someone talking about like data on the show, I'm like, "Oh, I I miss like being able to think about like the the architecture of **[54:26]** this and like the the the schema of like how I'm going to like, you know, store that data and how it's going to be efficient and going be useful for the developer, but also useful for like the people's going to consume the data. So uh no, great. uh and and always a fan like I'm a I'm an open source person so I love also when uh demo is like part of like oh we have that product there's a part that is like like like hey charge for us there's another part that is free there's a part of open source like it's it's a great mix so no great demo that was super useful I will not consider **[54:58]** myself a data lake expert after demo though but it's good enough for me to know a little more uh what people are talking about uh when they talk about data links and um having a great preview you also about like Heisert and uh the new uh Mino uh product. So uh thanks a lot for uh for for the demo. [snorts] >> Yeah, I'm always happy to talk about this stuff. I think data links are so cool. Like I said, I think I really think that they just as a data engineer perspective, like I'll take off my NIO hat, that they're the future of **[55:29]** architecture, like the largest data lake houses in the world are running on the coolest AI stuff that you've heard of like um >> you know this is just that's just facts like so it's really exciting and I think iceberg especially has made this process much easier for an deep anal analytics and people forget that like AI is first and we don't need analytics or like you know moving around columns and like cleaning up data but that clean data vast amounts of clean data aster clean **[56:02]** data really critical for AI initiatives to work so all this is like fits together like a puzzle piece I think it's just so cool >> love it so that bring us to the end of the show uh I actually I love your demo but my cat was not impressed I the old time. Uh, >> I'll work on it. I, you know, I have a um a dog, so I I feel like I've I've demoed to the wrong audience. I got the **[56:32]** [laughter] dog. >> No, he's just he's just a slacker. Sometimes he's going to join some meeting. Sometimes you just don't want to. But, uh, he's often in the background of the show or my meetings. Um, so yeah. No. Uh, thanks a lot for Sorry. Go ahead. >> Well, I was just wondering what his name was or her name. It's tells >> Tails. >> Yeah. And and the story be I got him uh it's a it's a male. I got him from a shelter and unfortunately when he arrived he he had a beginning of a rough life. So uh his tail were uh like all **[57:04]** screwed up. Like he got on the fight or got stuck somewhere. So uh when the shelter took him in um they went to the vet and they had to cut part of the tail. So now he's a cat but like his tail is like like this. Uh, so I guess he it looks like he had multiple tails. Uh, it's it's kind of like sad, but at the same timeish story because this is where his name is coming from and I just kept the name. Uh, but he is he's lovely. He's he's a little bit a little bit like a dog like always happy to see me following me around a little bit the **[57:34]** house. Uh, you know, he has his his cat moments where he's independent, but he's not like the usual independent cat who does not care that much about you. uh this this one is just codly and they like to spend time with me but uh yeah >> is part is part of the the show uh quite often. So yeah, thanks uh thanks for being a guest and thanks for your time. Uh happy to have a discussion with you. That was really interesting. Your demo was great and um also want to thank everybody u that were listening live but **[58:05]** also people who's going to watch the recording after and um we will see you in I will see you actually in two weeks for the next episode of the Devrell show. >> Okay, cool. Bye everybody.