You Ask, I Answer: Replacing Fine Tuned Models With RAG?  🔗 **Source:** [You Ask, I Answer: Replacing Fine Tuned Models With RAG?](https://www.youtube.com/watch?v=zRhIZ_iAlwA#transcript) > [!note]+ Document Notes > tags: [rag, fine-tuning, enterprise-ai, knowledge-graphs, ai-automation, llm-strategy] > author: Christopher Penn > source: Christopher Penn (YouTube) > reference: https://www.youtube.com/watch?v=zRhIZ_iAlwA > date created: 2026-07-16 08:44 > > # 🏷️ Title: You Ask, I Answer: Replacing Fine Tuned Models With RAG? > <!-- deep-read:sections start --> ## 🧭 Table of Contents - [📝 Summary](#-summary) - [📖 Main Content](#-main-content) - [🗒️ Notes](#️-notes) - [📚 References](#-references) - [🗣️ Paraphrase](#️-paraphrase) - [📰 Newsletter](#-newsletter) - [✅ Action Items](#-action-items) - [🧪 Hypotheses](#-hypotheses) - [📊 Methodology](#-methodology) - [❓ Questions](#-questions) - [🔮 Future Research](#-future-research) - [💡 Implications](#-implications) - [🧠 Learning Styles Assessment](#-learning-styles-assessment) - [📊 Accessibility Assessment](#-accessibility-assessment) - [🌍 DEIB Assessment](#-deib-assessment) - [✅ Logical Validity](#-logical-validity) - [🛡️ DARVO Analysis](#-darvo-analysis) - [🎯 Conclusion](#-conclusion) ## 📝 Summary - Viewer Larry asks whether RAG (retrieval-augmented generation) can entirely replace fine-tuned models for enterprise knowledge-base use cases. Penn's short answer: yes. - Fine-tuning was the early-AI norm — retraining a base model (e.g., original Meta Llama) with new knowledge — but Penn calls it inefficient, expensive, and prone to going stale almost immediately. - Current enterprise best practice, per Penn, is a hybrid database combining three technologies: a vector database (RAG), a structured database (Postgres/BigQuery-style), and a knowledge graph (e.g., Microsoft Copilot's knowledge graph, Neo4j) layered on top for fast traversal. - Stakeholders: enterprise AI/IT leaders, knowledge-management teams, data governance/security teams, and vendors (Microsoft, Anthropic, OpenAI referenced by name). - Risk/tradeoff named explicitly: without a hybrid RAG + structured + graph setup, token budgets and AI costs balloon, and AI systems can end up with inappropriate access to sensitive data (Penn's example: payroll data) — a governance failure, not just a cost one. - Bottom line: for most enterprise applications, fine-tuning is no longer a good use of AI resources; a hybrid RAG/structured/knowledge-graph architecture paired with existing permission systems is the current recommended pattern. ## 📖 Main Content Penn frames the question as an "enterprise AI question" and answers directly before explaining. His core teaching device is a library analogy: an AI model is a public library; a RAG (vector) database is an addition built onto that library holding "all your stuff," which the AI is instructed to consult before falling back on its own trained knowledge. He contrasts this with the older fine-tuning approach — retraining a base model with company-specific data via techniques like reinforcement learning — which he likens to copying an entire public library and trying to insert your own books into it. He states this was expensive and went out of date quickly, which is why enterprises moved away from it. The current best practice, in his account, is a three-part hybrid database: 1. **Vector database / RAG** — semantic retrieval via embeddings, the "addition on the library." 2. **Structured database** (Postgres, BigQuery, etc.) — holds the same data but indexed in a machine-friendly way (his analogy: ISBN/Library-of-Congress numbers vs. human-friendly Dewey Decimal). 3. **Knowledge graph** (Microsoft's Copilot Premium knowledge graph, Neo4j, graph networks) — a fast traversal layer on top of the other two. He ties this architecture directly to governance: enterprises with large, messy document/email stores and existing access-control lists risk exploding token budgets, exploding costs, and — notably — AI systems gaining access to data "it maybe shouldn't have for everybody, like payroll data" if the hybrid system isn't in place. A properly built hybrid database inherits and obeys existing permission structures rather than requiring new ones. He also makes a vendor-differentiation point: most AI providers ask enterprises to bring their data to the provider for processing (he names OpenAI), whereas Anthropic's approach is to bring its models to the customer's infrastructure, which he characterizes as more privacy-friendly. No sourcing or citation is given for this claim — it is presented as background knowledge from "early days." He closes with a practical recommendation: pair the hybrid database (vectors + structured data + knowledge graph) with an existing human/machine dual-access system, using Microsoft SharePoint as his (self-deprecating) example of a system with both human-readable and machine-readable structure. ### 🗒️ Notes #### ⏱️ [00:00:00](https://www.youtube.com/watch?v=zRhIZ_iAlwA&t=0) Larry's question and the direct answer - Larry asks if RAG can entirely replace fine-tuned models for enterprise knowledge-base use. - Penn answers "yes" immediately, then defines RAG: vectorization + embedding used to index/prepare a database for retrieval. - Library analogy introduced: AI model = library; RAG/vector database = an addition holding "all your stuff," consulted before the model's internal knowledge. #### ⏱️ [00:01:08](https://www.youtube.com/watch?v=zRhIZ_iAlwA&t=68) Why fine-tuning fell out of favor - Early AI era: companies fine-tuned base models (e.g., original Meta Llama) with reinforcement learning to "learn new knowledge." - Analogy: copying a public library and inserting your own books, then trying to make the public library behave as your private company library. - Named drawbacks: "fairly inefficient," "very expensive," and it "went out of date very, very, very quickly." #### ⏱️ [00:01:44](https://www.youtube.com/watch?v=zRhIZ_iAlwA&t=104) The hybrid-database best practice - Current best practice = hybrid database combining three technologies, scoped explicitly to large enterprises with "very large company budgets." - Technology 1: RAG/vector database (semantic retrieval). - Technology 2: structured database (Postgres, BigQuery) — Dewey Decimal (human) vs. ISBN/Library-of-Congress (machine) analogy for how the same data gets indexed two ways. - Technology 3: knowledge graph (Microsoft Copilot Premium's knowledge graph, Neo4j, graph networks) as a fast traversal "meta-layer" on top. #### ⏱️ [00:04:00](https://www.youtube.com/watch?v=zRhIZ_iAlwA&t=240) Cost and governance risk without the hybrid stack - Without vector + structured + knowledge-graph systems, "it gets very hard to use your data with AI," with "exploding token budgets and exploding costs." - Root cause named: enterprises hold hundreds of thousands to millions of documents/emails with varied access controls, some dating to the 1990s. - Good knowledge management → AI works with it easily (Microsoft's knowledge graph cited as a complement). Poor knowledge management → high AI cost and difficulty. #### ⏱️ [00:05:14](https://www.youtube.com/watch?v=zRhIZ_iAlwA&t=314) Payroll-data example and AI governance - Named consequence of a poorly governed setup: AI systems have "gotten access to things that it maybe shouldn't have for everybody, like payroll data" at unnamed companies. - Claim: good AI governance + knowledge graphs + RAG as part of a hybrid database solves this because the system "adheres to and obeys your existing structure of permissions and access control." #### ⏱️ [00:05:44](https://www.youtube.com/watch?v=zRhIZ_iAlwA&t=344) Anthropic vs. other providers on data handling - Claim: most AI providers ("Open AI for example") ask customers to bring data to them for processing. - Contrast: "Anthropic says no, we'll bring our models to your infrastructure and work inside of them," framed as "a bit more privacy-friendly." - No citation or sourcing given for this vendor comparison; presented as established background ("even from early days"). #### ⏱️ [00:06:08](https://www.youtube.com/watch?v=zRhIZ_iAlwA&t=368) Conclusion: fine-tuning is no longer necessary - Direct claim: "You do not need to be fine-tuning models anymore" for most applications and places — not a good use of AI resource. - Recommended pattern: hybrid database (vectors + structured data + knowledge graph) combined with a dual human/machine-accessible system, using SharePoint as the example. ## 📚 References - [Christopher Penn's YouTube channel](https://www.youtube.com/watch?v=zRhIZ_iAlwA) — primary source (this video) - [csper.com](https://csper.com) — Penn's site, referenced in the video's closing AI-training disclosure block as his professional/speaking contact point - No external studies, papers, or third-party sources are cited in the video; all technical claims are presented from Penn's own stated expertise. ## 🗣️ Paraphrase - Penn argues that retrieval-augmented generation, layered with a structured database and a knowledge graph, has functionally replaced fine-tuning as the way enterprises should connect AI models to their proprietary knowledge — fine-tuning being framed as an outdated, costly, quickly-stale approach from AI's early days. - He connects the technical architecture directly to data governance: a properly built hybrid RAG system doesn't just retrieve information faster and cheaper, it also respects existing access controls, preventing scenarios where an AI model surfaces sensitive information (his example: payroll data) to people who shouldn't see it. ## 📰 Newsletter - Quick hit for anyone still wondering whether fine-tuning is worth the enterprise AI budget: Christopher Penn's answer is a flat no. His pitch is a "hybrid database" — vector search (RAG) plus a structured database plus a knowledge graph on top — as the pattern powering things like Microsoft Copilot Premium today. - The most useful nugget isn't the architecture diagram, it's the governance angle: skip the hybrid setup and you don't just pay more in tokens, you risk an AI system surfacing data (think payroll records) to people who were never supposed to see it. Worth a watch for anyone scoping enterprise knowledge-base AI work. - Note: this video is general enterprise-AI strategy content, not WordPress-specific — no WordPress angle to include here. ## ✅ Action Items #### DevRel - Not applicable to this source — no developer-advocacy, SDK, or docs content is discussed; this is a strategy explainer aimed at enterprise decision-makers, not developers. #### OSPO - Not applicable to this source — no open-source policy, licensing, or governance topics are addressed. #### Community - Not applicable to this source — no community, contributor, or sponsor topics are addressed. #### WordPress - Not applicable to this source — the video does not reference WordPress or its ecosystem. #### The WPCC - Not applicable to this source — no WPCC-related programs or funding topics are addressed. ### FAIR - Not applicable to this source — no FAIR governance, provenance, or funding-covenant topics are addressed. ## 🧪 Hypotheses - H1: Enterprises that pair RAG with a structured database and a knowledge graph will see lower per-query token costs than enterprises relying on RAG alone against a large, unstructured document set. - H2: Enterprises with mature 1990s-forward knowledge-management practices will report faster and cheaper AI knowledge-base deployment than enterprises without such practices, when both adopt the same hybrid RAG architecture. - H3: Hybrid RAG systems that inherit existing access-control lists will show measurably fewer instances of AI systems surfacing restricted data (e.g., payroll, HR records) than RAG systems built without permission inheritance. - H4: Fine-tuning a base model on proprietary enterprise knowledge produces a measurably higher total cost of ownership over 12 months than a hybrid RAG deployment covering the same knowledge base. ### 📊 Methodology - H1/H4 (cost claims): a controlled cost-comparison study — same enterprise document set, same query volume, run against (a) a fine-tuned model, (b) RAG-only, (c) hybrid RAG + structured DB + knowledge graph — tracking token spend, latency, and retraining/refresh cost over time. - H2 (knowledge-management maturity): a cross-sectional survey or case-study comparison of enterprises at different knowledge-management maturity levels, measuring time-to-deploy and query-accuracy for an equivalent hybrid RAG rollout. - H3 (governance/access control): a red-team-style audit — attempt to retrieve restricted-category documents (e.g., synthetic payroll records) via the AI system under (a) RAG without permission inheritance and (b) hybrid RAG with permission inheritance, measuring leak rate. ### ❓ Questions - What specific mechanism does the "hybrid database" use to inherit existing access-control lists — is permission-checking done at retrieval time, embedding time, or both? - How does Penn's framing account for smaller organizations that can't afford the "very large company budgets" he says this architecture requires — is there a lighter-weight equivalent? - What is the actual evidentiary basis for the Anthropic-vs-other-providers data-handling claim — is this specific to Anthropic's enterprise/on-prem offerings, or a general characterization of all Anthropic deployments? - How does a knowledge graph like Microsoft's Copilot Premium graph get built and maintained — is that a manual knowledge-engineering effort, or automated from the structured/vector layers? ### 🔮 Future Research - Independent, vendor-neutral cost/performance benchmarking of fine-tuning vs. hybrid-RAG architectures at varying enterprise data-set sizes. - Case studies on how smaller or mid-market organizations (outside the "very large company budgets" scope Penn names) achieve comparable governance and retrieval outcomes. - Deeper investigation into real-world incidents (referenced but unnamed by Penn) where AI systems accessed data like payroll records, to quantify how common this failure mode actually is. ### 💡 Implications - Practice: enterprise AI teams evaluating fine-tuning vs. RAG should weight governance/access-control inheritance as a first-class requirement, not just a retrieval-quality or cost consideration. - Policy: organizations with legacy (1990s-era) knowledge-management systems may need to prioritize knowledge-management modernization before AI deployment, since Penn frames poor KM as the direct cause of high AI cost and difficulty. - Security: the payroll-data example underscores that RAG architecture choices are a data-security control, not just a retrieval-accuracy one — security and IT governance teams should be stakeholders in RAG architecture decisions, not just AI/ML teams. - Funding models: Penn explicitly scopes this hybrid architecture to organizations with "very large company budgets," implying a cost barrier that isn't addressed for smaller organizations in this video. ### 🧠 Learning Styles Assessment - Reading level estimate: Flesch-Kincaid Grade Level ≈ 9.1 (computed from the ~700-word spoken-content transcript, excluding the closing AI-training disclosure block; conversational register, short-to-medium sentences). #### 👀 Visual learners - Present: none — no diagrams, charts, screenshots, or on-screen labeled visuals accompany the spoken explanation in this transcript-only capture. - Improve: an architecture diagram showing the three-layer hybrid database (vector / structured / knowledge graph) would substantially help viewers who don't retain the library analogy verbally. #### 🔊 Aural learners - Present: the entire video is a spoken, conversational Q&A response — strong native fit for aural learners. - Improve: a brief spoken recap of the three technologies at the end (beyond the single closing line) would reinforce retention. #### 🗣️ Verbal learners - Present: strong use of analogy throughout (library, library addition, Dewey Decimal vs. ISBN, card catalog) to make abstract database concepts concrete. - Improve: a one-line glossary of "RAG," "vector database," and "knowledge graph" would help viewers unfamiliar with the jargon before the analogies land. #### 🧩 Kinesthetic learners - Present: none — no hands-on steps, sandboxes, or reproducible tasks are offered. - Improve: a companion checklist ("how to assess whether your org needs hybrid RAG") would give this audience something actionable to do, not just watch. #### 🧠 Logical learners - Present: a clear causal chain is offered (poor knowledge management → high AI cost/difficulty → governance risk), and the three-technology breakdown is presented as a simple enumerated structure. - Improve: a comparison table (fine-tuning vs. RAG-only vs. hybrid, across cost/freshness/governance) would suit this audience better than the spoken enumeration. #### 🧑‍🤝‍🧑 Social learners - Present: the video itself is framed as answering a named community member's (Larry's) question — a light social/community framing. - Improve: pointing viewers to a discussion thread or comment section to share their own enterprise RAG setups would extend the social-learning value. #### 🧘‍♀️ Solitary learners - Present: content is dense enough (multiple technical concepts in under 8 minutes) to reward a self-paced rewatch. - Improve: timestamped chapter markers (which this note now effectively provides) would support self-paced review. #### 🌿 Naturalistic learners - Present: none — no ecological, sustainability, or systems-in-nature framing appears in the content. - Improve: not a natural fit for this topic; no meaningful improvement path here. ### 📊 Accessibility Assessment #### Screen reader semantics - Present: not assessable from a transcript-only capture — the source is a YouTube video page, and this note doesn't have visibility into the page's heading structure or landmarks. - Improve: verify the YouTube video page and description use proper heading/landmark structure if republished elsewhere (e.g., embedded on a blog). #### Keyboard navigation - Not applicable to this source — YouTube's native player keyboard controls are outside this content's scope to assess or improve. #### Contrast and readability - Not applicable to this source — no on-screen text/graphics are present to assess for contrast; the video appears to be talking-head format with no visible on-screen text elements per the transcript. #### Text resizing and reflow - Not applicable to this source — governed by the YouTube player, not this content. #### Media and captions - Present: auto-generated English captions exist (the source used for this transcript) and are downloadable via `yt-dlp`. - Improve: auto-captions contain minor artifacts (e.g., a duplicated "in inside of them" at ~05:58) — a manually-corrected caption track and a downloadable transcript on a companion page would improve accuracy and access. #### Forms and interactions - Not applicable to this source — no forms or interactive elements are part of this video content. #### Standards reference - Gap identified maps to WCAG 2.2 AA success criterion 1.2.2 (Captions, Prerecorded) — auto-captions technically satisfy the letter of this criterion but the transcription artifacts noted above fall short of best-practice accuracy. ### 🌍 DEIB Assessment #### Cultural breadth - Present: none — the video uses U.S.-centric examples (SharePoint, generic "enterprise," Meta Llama) with no international case studies or multilingual framing. - Improve: at least one non-U.S. enterprise example or multilingual knowledge-base consideration (e.g., how the hybrid architecture handles multilingual document sets) would broaden applicability. #### Inclusive language - Present: language is plain and jargon is explained via analogy; no exclusionary terms identified. - Improve: minimal — content is already accessible in tone. #### Neurodiversity support - Present: content is chunked into a single continuous explanation without headers, chapter markers, or a visible outline (as delivered). - Improve: on-screen chapter markers or a description-box outline would help viewers who benefit from predictable structure. #### Representation - Present: only one speaker (Penn) and one named questioner (Larry); no diversity of voices or perspectives represented in this short-format Q&A. - Improve: not a strong fit for this format — a single-expert Q&A video has structural limits on representation; a companion post citing multiple practitioners' experiences could balance this. #### Psychological safety - Present: tone is calm, direct, and non-alarmist even when discussing a real risk (payroll-data exposure); no fear-based framing. - Improve: none identified — this is a strength of the source. ### ✅ Logical Validity #### Strong reasoning - The library/library-addition analogy (00:00:00–00:01:08) is a strong pedagogical analogy: it maps an abstract technical concept (vector retrieval prioritized over model-internal knowledge) onto a concrete, widely-understood structure, and it's used consistently throughout rather than dropped after one use. - The causal chain at 00:04:00 (poor knowledge management → AI has to search inefficiently → token/cost explosion → potential governance failure) is a clear, well-sequenced causal argument that connects a data-hygiene problem to a security outcome, which is the video's strongest piece of reasoning. - To clarify further: naming even one concrete metric (e.g., approximate token-cost multiplier between well-governed and poorly-governed setups) would convert this from a plausible causal story into a falsifiable claim. #### Fallacies - **Hasty generalization** at 00:06:08 ("You do not need to be fine-tuning models anymore... for most applications, most places, that is not a good use of AI resource") — presented as a broad, near-universal recommendation without citing supporting data, benchmarks, or named exceptions beyond the earlier "very large company budgets" scoping. Fix: qualify with the same enterprise-scale caveat used earlier in the video, and/or name at least one scenario where fine-tuning remains appropriate (e.g., domain-specific behavior/tone shaping vs. knowledge retrieval). - **Incomplete comparison (false dichotomy risk)** at 00:05:44 — contrasts only Anthropic and OpenAI on data-handling approach ("bring your data to us" vs. "bring our models to your infrastructure"), omitting other major enterprise AI vendors (Google, Microsoft Azure OpenAI Service, Meta, on-prem open-weight deployments) that also offer customer-infrastructure deployment options. Fix: name the comparison set explicitly or soften to "some providers" vs. implying a two-vendor landscape. #### Tone of voice - Direct, conversational, and confident throughout — Penn answers "yes" before explaining, and closes with an unhedged imperative ("You do not need to be fine-tuning models anymore"). Tone stays consistent and non-defensive even when naming a real governance failure mode (payroll data), with no escalation into alarmism or hedging. ### 🛡️ DARVO Analysis - Not applicable to this source — this is a single-speaker technical explainer answering a viewer's question, with no accusation, conflict, or defensive posture present. There is no deny/attack/reverse-victim-offender pattern to identify; Penn is not responding to any criticism or allegation in this content. ## 🎯 Conclusion Penn's answer to Larry's question is unambiguous: RAG, combined with a structured database and a knowledge graph, has replaced fine-tuning as the recommended enterprise pattern for connecting AI to proprietary knowledge — because it's cheaper, stays current, and (critically) inherits existing access controls rather than requiring parallel governance. The video's strongest contribution is reframing this as a governance and security decision, not just a retrieval-performance one, using the payroll-data example to make the stakes concrete. Its weakest point is scope: the "you don't need fine-tuning anymore" conclusion is stated more broadly than the "very large company budgets" framing established earlier in the same answer would support. Practical next step for viewers: before building or defending a fine-tuning roadmap, audit whether a hybrid RAG + structured-database + knowledge-graph architecture — wired into existing permission systems — would meet the same knowledge-retrieval need at lower cost and lower governance risk. ## 🔄 Refactor Appendix — You Ask, I Answer: Replacing Fine Tuned Models With RAG? ### 📚 Literature Atoms (H3) #### [H1] Hybrid database beats fine-tuning for enterprise AI > [!quote] ID:rw_zRhIZ_iAlwA_01 > "You do not need to be fine-tuning models anymore. For the most part, for most applications, most places, that is not a good use of AI resource." **Context & use**: - ~00:06:08 in the video, delivered as the answer's summary conclusion. - Central claim of the entire video; frames every other point (library analogy, three-technology breakdown, governance risk) as supporting evidence for this conclusion. **Tags**: #xref **Highlight**: [view highlight](https://www.youtube.com/watch?v=zRhIZ_iAlwA&t=368) <!-- atomicity: 1 idea ✓ --> #### [H2] Three technologies make up the enterprise hybrid database > [!quote] ID:rw_zRhIZ_iAlwA_02 > "The current best practice is a hybrid database of three different technologies in combination." **Context & use**: - ~00:01:44; introduces the vector database + structured database + knowledge graph triad that structures the rest of the explanation. - Directly answers the "how" behind the H1 conclusion — this is the mechanism, not just the recommendation. **Tags**: #xref **Highlight**: [view highlight](https://www.youtube.com/watch?v=zRhIZ_iAlwA&t=104) <!-- atomicity: 1 idea ✓ --> #### [H3] RAG governance risk: ungoverned AI can expose restricted data > [!quote] ID:rw_zRhIZ_iAlwA_03 > "An AI system has gotten access to things that it maybe shouldn't have for everybody, like payroll data." **Context & use**: - ~00:05:14–00:05:24; presented as a real (if unnamed/unsourced) consequence of skipping the hybrid architecture. - The strongest security/governance argument in the video — ties architecture choice directly to a concrete harm. **Tags**: #xref #addendum **Highlight**: [view highlight](https://www.youtube.com/watch?v=zRhIZ_iAlwA&t=314) <!-- atomicity: 1 idea ✓ --> ### 🧠 Permanent Candidates (H4) #### [P1] RAG architecture is a governance control, not just a retrieval optimization **Claim** (one sentence, Courtney's voice): Choosing a hybrid RAG + structured-database + knowledge-graph architecture over fine-tuning is fundamentally a data-governance decision, because inheriting existing permission systems is what prevents an AI system from surfacing restricted data. **Evidence pointer**: [[#H3]] **Scope & conditions**: - Holds when: the enterprise already has mature access-control lists and knowledge-management practices for the underlying data to inherit. - Breaks when: the underlying document/email store has no consistent access controls to begin with — in that case, RAG architecture alone can't manufacture governance that doesn't exist upstream. **Links** (≥1 with annotation): - [[Outpost hosting gotchas]] — because contrasts on scale (Outpost's managed-WP failure modes are small-business infra risk; this note is enterprise-AI-scale data-governance risk — same "architecture choice has security consequences" pattern at different scale) **Optional**: not empirical in the falsifiable-experiment sense as stated by Penn (no cited data); no `proof_state` added. <!-- atomicity: 1 idea ✓ --> ### ✏️ Fleeting Queue - [F1] Verify Penn's Anthropic-vs-OpenAI data-handling claim (00:05:44) against Anthropic's actual enterprise/on-prem deployment documentation before citing it elsewhere — https://www.youtube.com/watch?v=zRhIZ_iAlwA&t=344 #xref - [F2] Look for a follow-up Penn video or post with concrete cost-comparison numbers (fine-tuning vs. hybrid RAG) to firm up H1/H4 in the Hypotheses section above — no direct source yet #articleseed ### → Things 3 (max 2 actions) - [ ] Cross-check whether any Outpost/Airo AI Builder enterprise-AI content should reference the "RAG as governance control" framing from [[#P1]] — content-strategy #things <!-- deep-read:sections end --> 📄 **Full text:** [[Full Document Contents/Articles/You Ask, I Answer Replacing Fine Tuned Models With RAG.md|You Ask, I Answer: Replacing Fine Tuned Models With RAG? by Christopher Penn]] ## Highlights - The current best practice is a hybrid database uh of three different technologies in combination. Again, this is enterprise AI, so you're talking very large data sets, you're talking very large companies, you're talking very large company budgets who can afford to do this sort of thing. And the three technologies are retrieval-augmented-generation, which uses that vector system that I was talking about where you where you basically make a new version of the model in a sense as a an addition on the library. Combined with hybrid structured databases. So, this is something like PostgreSQL or BigQuery or what have you. Any database can handle having a combination of both structured data and vectors. So, imagine that new branch of the library has two different card catalogs. One is your traditional Dewey Decimal, one is your ISBN, right? Your Library of Congress numbers. Um the machines work with ISBN, the humans work with Dewey Decimal. And both index the same data, but they do it in different ways. And so, the machine, because it can remember an ISBN number and and uh Library of Congress numbers, and we can't. We We remember Dewey Decimal much better cuz they're shorter numbers. They do a better job. Each one does a a good job of of getting to the knowledge. And then, the third system, which is sort of a meta-layer on top of this that you see particularly with systems like Microsoft Copilot uh for the enterprise version of Copilot, is um knowledge graphs. So, Microsoft knowledge graph, which runs as part of Copilot Premium, um uh all any of these knowledge graphs, Neo4j and graph networks, etc., that provide a fast way to get through the card catalog. Imagine that there's you know, that's the computer system that searches the card catalog. Those three combinations, so you have vector database, structured database, uh and then knowledge graph that sits on top of it, is how modern enterprises today are handling their data with AI. ([View Highlight](https://read.readwise.io/read/01kxnfkqe45kphnedf2932m2q3)) - Note: The three-technology hybrid database mechanism: vector DB + structured DB + knowledge graph. - Tags: atom/literature/h2 - And if you don't have those systems, it gets very hard to use your data with AI, and the risks of exploding token budgets and exploding costs gets very, very high because think about it. In an enterprise with enterprise AI, in an enterprise, you probably have hundreds of thousands, if not millions of documents. You certainly have millions of emails and other things all over the place. And you have different kinds of access controls and security lists. And if you are if you've done a good job with your knowledge management, and now we're talking about stuff that goes back to like the '90s, um then it's very easy to let AI work with your knowledge management system. That's why Microsoft, for example, has something like knowledge graph because it's a it's a good complement to the data that you already have. If on the other hand, you haven't done a good job with your enterprise knowledge management, then it's going to be very difficult for AI to work with that, and your AI costs will be very, very high as AI tries to, you know, root through and search through all sorts of things that maybe uh aren't the best use of its time, certainly not the best use of its budget, and potentially can lead to situations like has happened at many companies where an AI system has gotten access to things that it maybe shouldn't have for everybody, like payroll data. Good AI governance, knowledge graphs, retrieval augmented generation as part of a hybrid database is the way to handle that because then it adheres to and obeys your existing structure of permissions and access control. You don't have to worry about it having access to data it shouldn't because it's going to use the existing systems as they are. That's one of the things is even from early days Anthropic apart from other providers is most other providers say bring your data to us and we'll process it. Uh Open AI for example is like that. Anthropic says no, we'll bring our models to your infrastructure and work in inside of them and that is a bit more privacy-friendly. ([View Highlight](https://read.readwise.io/read/01kxnfkvk5nexmer21m8bdzsjs)) - Note: Governance risk: ungoverned AI can surface restricted data like payroll records. - Tags: addendum, atom/literature/h3 - But at the end of the day, those are the systems. You do not need to be fine-tuning models anymore. For the most part, for most applications, most places, that is not a good use of AI resource. In the enterprise, most enterprise AI benefits most from the hybrid database, the knowledge graph, um and the vectors and the structured data within the hybrid database combined with a good system like I can't believe I'm going to say this, but combine it with something like SharePoint where there's both human-accessible and machine-accessible data. It's the same data, but it's structured differently. Thanks for the question. Talk to you on the next one. ([View Highlight](https://read.readwise.io/read/01kxnfkh2kza50msw1pbrven00)) - Note: Central claim: fine-tuning no longer a good use of AI resource for most enterprise applications. - Tags: atom/literature/h1