What Is Retrieval-Augmented Generation, aka RAG?  ## Metadata - Author: [[Rick Merritt]] - Full Title: What Is Retrieval-Augmented Generation, aka RAG? - Category: #articles - Document Note: --- tags: - #retrieval-augmented-generation - #RAG - #NVIDIA Author: Rick Merritt (NVIDIA Blog) Source: NVIDIA Blog Reference: https://blogs.nvidia.com/blog/what-is-retrieval-augmented-generation/ (Published: 2025-01-31; originally Nov 15, 2023) date created: 2025-08-16 13:15 --- # Title: What Is Retrieval-Augmented Generation, aka RAG ## Table of Contents: - [📝 Summary](#-summary) - [📖 Main Content](#-main-content) - [🗒️ Notes](#-notes) - [📖 References](#-references) - [🗣️ Paraphrase](#-paraphrase) - [📰 Newsletter](#-newsletter) - [✅ Action Items](#-action-items) - [🧪 Hypotheses](#-hypotheses) - [📊 Methodology](#-methodology) - [❓ Questions](#-questions) - [🔮 Future Research](#-future-research) - [💡 Implications](#-implications) - [🧠 Learning Styles Assessment](#-learning-styles-assessment) - [📊 Accessibility Assessment](#-accessibility-assessment) - [🌍 DEIB (Diversity, Equity, Inclusion, and Belonging) Assessment](#-deib-diversity-equity-inclusion-and-belonging-assessment) - [✅ Logical validity](#-logical-validity) - [🔍 DARVO Analysis](#-darvo-analysis) - [🔍 SEO Evaluation](#-seo-evaluation) - [🎯 Conclusion](#-conclusion) --- ## 📝 Summary: Retrieval-Augmented Generation (RAG) is a powerful technique that enhances large language models (LLMs) by enabling them to fetch relevant, external information—like a courtroom clerk bringing precedents to a judge. This process improves accuracy, grounding responses in verifiable data, and is easier and more efficient than retraining models. NVIDIA facilitates RAG with tools like NeMo Retriever and NIM microservices to build scalable, secure pipelines. --- ## 📖 Main Content: RAG bridges the gap between general LLM knowledge (parameterized during training) and domain-specific, up-to-date content by retrieving relevant information at query time. The process: retrieve, then generate. Mechanism: 1. **Encode** the user query into a vector. 2. **Retrieve** similar content from a knowledge base. 3. **Augment** the LLM’s prompt with retrieved context. 4. **Generate** a grounded response with citations. Benefits: - Enhanced accuracy and reduced hallucinations. - Source transparency via citations. - Faster updates without full retraining. NVIDIA’s ecosystem for RAG: - **NeMo Retriever** for embedding, reranking, ingestion. - **NIM microservices** for scalable deployment. - **Blueprints** for enterprise AI assistants and knowledge systems. --- ### 🗒️ Notes - Term coined by Patrick Lewis (2020). - Semantic retrieval surpasses keyword search for context-awareness. - Applications: medicine, finance, enterprise knowledge bases. - Major driver: combat AI hallucination. - NVIDIA approach includes governance, security, and GPU acceleration. - RAG supports dynamic updates—critical for evolving datasets. --- ## 📖 References: - NVIDIA Blog (Rick Merritt, 2025-01-31) - NVIDIA Glossary: RAG - NVIDIA Developer: RAG Architecture - Wikipedia: Retrieval-augmented generation - Barron’s: Jensen Huang on hallucinations - WSJ Podcast: AI business risk & RAG - Wired: Reducing AI hallucinations - NVIDIA GitHub: RAG Blueprint --- ## 🗣️ Paraphrase: **Original:** “Retrieval-augmented generation is a technique for enhancing the accuracy and reliability of generative AI models with information from specific and relevant data sources.” **Paraphrased:** RAG improves AI accuracy by feeding it targeted, relevant external information. **Original:** “Retrieval-augmented generation gives models sources they can cite, like footnotes in a research paper, so users can check any claims.” **Paraphrased:** RAG equips AI with citation-like references so users can verify the basis of responses. --- ## 📰 Newsletter: > **RAG: The AI Librarian for Your LLM** > Generative AI can be brilliant… or confidently wrong. RAG—Retrieval-Augmented Generation—fixes that by pulling in verified, real-time information before your AI answers. NVIDIA’s NeMo Retriever, NIM microservices, and RAG blueprints make it easy to build chatbots, search tools, or developer docs that cite their sources. Smarter, faster, and more trustworthy—this is how you teach AI to be both a genius and a good librarian. --- ## ✅ Action Items: **Analysis of Alignment with Your Role:** 1. **Open-Source Advocacy:** - RAG workflows benefit from open knowledge repositories and shared vector databases. - Encourages contributions to public domain datasets that can fuel RAG systems in WordPress and beyond. - Showcases how open data and transparent retrieval pipelines foster community trust. 2. **Contributor Engagement:** - Implement RAG-driven search in contributor documentation to surface the most relevant, up-to-date onboarding resources. - Gamify contribution impact by showing when a contributor’s content is retrieved in real-time support scenarios. 3. **Education:** - Create tutorials demonstrating how to build RAG-powered documentation assistants for WordPress devs. - Host workshops on integrating RAG with WordPress’s REST API and plugin ecosystems to surface contextual help. 4. **Sustainable Funding:** - Advocate for funding RAG infrastructure as a shared service within open-source communities, reducing redundant effort. - Position RAG as a public-good infrastructure worthy of sponsorship via The WPCC or corporate partners. 5. **OSPO Leadership:** - Use RAG to catalog and retrieve all corporate open-source contributions, improving transparency reporting. - Offer RAG-powered dashboards for executive visibility into community health and project dependencies. 6. **Transparency & Inclusivity:** - Ensure multilingual datasets are included in RAG pipelines for global WordPress community support. - Source content from diverse contributors, ensuring retrieval does not reinforce dominant-language or region biases. --- ## 🧪 Hypotheses: - Integrating RAG into contributor onboarding will reduce ramp-up time by at least 30%. - RAG-powered documentation assistants will increase self-service support satisfaction scores in open-source communities. --- ### 📊 Methodology: - Pre/post surveys measuring onboarding time before and after RAG implementation. - Usage analytics for RAG-enhanced documentation portals. - Comparative analysis of multilingual coverage in RAG search results. --- ### ❓ Questions: - How can RAG improve issue triage in WordPress open-source governance? - What governance models ensure RAG pipelines remain transparent and bias-mitigated? --- ### 🔮 Future Research: - Developing an open-source RAG WordPress plugin for live documentation retrieval. - Exploring cooperative funding models for RAG infrastructure shared across OSS projects. --- ### 💡 Implications: - Positions WordPress as an innovator in AI-assisted open-source documentation. - Strengthens OSPO metrics by making contribution histories easily retrievable. - Broadens access to relevant knowledge for underrepresented contributors. --- ### 🧠 Learning Styles Assessment: - **Visual:** Add flow diagrams for RAG architecture. - **Aural:** Offer narrated tutorials. - **Verbal:** Strong analogy (judge & clerk) present. - **Kinesthetic:** Include code-lab exercises. - **Logical:** Clear stepwise retrieval process. - **Social:** Add discussion prompts for contributor Slack channels. - **Solitary:** Encourage private experimentation with RAG setups. - **Naturalistic:** Potential analogy: RAG as a pollination network in ecosystems. --- ### 📊 Accessibility Assessment: - Needs alt-text for images in NVIDIA blog. - Confirm semantic HTML for headings. - Optimize keyboard navigation in interactive demos. - Ensure text scaling works without layout breakage. --- ### 🌍 DEIB Assessment: - Could integrate case studies from non-English OSS communities. - Language is already inclusive. - Expand retrieval datasets to represent marginalized contributor voices. --- ### ✅ Logical validity: **Strengths:** - Causal reasoning: RAG reduces hallucination by grounding responses. - Evidence-based: Links to tools and frameworks. - Ethical reasoning: Promotes transparency. ** - Summary: Retrieval-augmented generation (RAG) is a technique for enhancing the accuracy and reliability of generative AI models with facts fetched from external sources. - URL: https://blogs.nvidia.com/blog/what-is-retrieval-augmented-generation/ NVIDIA artist's concept of retrieval-augmented generation aka RAG *Editor’s note: This article, originally published on Nov. 15, 2023, has been updated.* To understand the latest advancements in [generative AI](https://www.nvidia.com/en-us/glossary/data-science/generative-ai/), imagine a courtroom. Judges hear and decide cases based on their general understanding of the law. Sometimes a case — like a malpractice suit or a labor dispute — requires special expertise, so judges send court clerks to a law library, looking for precedents and specific cases they can cite. Like a good judge, large language models ([LLMs](https://www.nvidia.com/en-us/glossary/data-science/large-language-models/)) can respond to a wide variety of human queries. But to deliver authoritative answers — grounded in specific court proceedings or similar ones — the model needs to be provided that information. The court clerk of AI is a process called [retrieval-augmented generation](https://www.nvidia.com/en-us/glossary/retrieval-augmented-generation/), or RAG for short. #### **How It Got Named ‘RAG’** Patrick Lewis, lead author of the [2020 paper that coined the term](https://arxiv.org/pdf/2005.11401.pdf), apologized for the unflattering acronym that now describes a growing family of methods across hundreds of papers and dozens of commercial services he believes represent the future of generative AI. [](https://blogs.nvidia.com/wp-content/uploads/2023/11/Patrick-Lewis-RAG-lead-author.jpg)Patrick Lewis “We definitely would have put more thought into the name had we known our work would become so widespread,” Lewis said in an interview from Singapore, where he was sharing his ideas with a regional conference of database developers. “We always planned to have a nicer sounding name, but when it came time to write the paper, no one had a better idea,” said Lewis, who now leads a RAG team at AI startup Cohere. #### **So, What Is Retrieval-Augmented Generation (RAG)?** [Retrieval-augmented generation](https://www.nvidia.com/en-us/glossary/retrieval-augmented-generation/) is a technique for enhancing the accuracy and reliability of generative AI models with information fetched from specific and relevant data sources. In other words, it fills a gap in how LLMs work. Under the hood, LLMs are neural networks, typically measured by how many parameters they contain. An LLM’s parameters essentially represent the general patterns of how humans use words to form sentences. That deep understanding, sometimes called parameterized knowledge, makes LLMs useful in responding to general prompts. However, it doesn’t serve users who want a deeper dive into a specific type of information. #### **Combining Internal, External Resources** Lewis and colleagues developed retrieval-augmented generation to link generative AI services to external resources, especially ones rich in the latest technical details. The paper, with coauthors from the former Facebook AI Research (now Meta AI), University College London and New York University, called RAG “a general-purpose fine-tuning recipe” because it can be used by nearly any LLM to connect with practically any external resource. #### **Building User Trust** Retrieval-augmented generation gives models sources they can cite, like footnotes in a research paper, so users can check any claims. That builds trust. What’s more, the technique can help models clear up ambiguity in a user query. It also reduces the possibility that a model will give a very plausible but incorrect answer, a phenomenon called hallucination. Another great advantage of RAG is it’s relatively easy. A [blog](https://ai.meta.com/blog/retrieval-augmented-generation-streamlining-the-creation-of-intelligent-natural-language-processing-models/) by Lewis and three of the paper’s coauthors said developers can implement the process with as few as [five lines of code](https://huggingface.co/facebook/rag-token-nq). That makes the method faster and less expensive than retraining a model with additional datasets. And it lets users hot-swap new sources on the fly. #### **How People Are Using RAG** With retrieval-augmented generation, users can essentially have conversations with data repositories, opening up new kinds of experiences. This means the applications for RAG could be multiple times the number of available datasets. For example, a generative AI model supplemented with a medical index could be a great assistant for a doctor or nurse. Financial analysts would benefit from an assistant linked to market data. In fact, almost any business can turn its technical or policy manuals, videos or logs into resources called knowledge bases that can enhance LLMs. These sources can enable use cases such as customer or field support, employee training and developer productivity. The broad potential is why companies including [AWS](https://aws.amazon.com/blogs/machine-learning/simplify-access-to-internal-information-using-retrieval-augmented-generation-and-langchain-agents/), [IBM](https://research.ibm.com/blog/retrieval-augmented-generation-RAG), [Glean](https://www.glean.com/resources/guides/what-is-retrieval-augmented-generation-rag), [Google](https://cloud.google.com/use-cases/retrieval-augmented-generation?hl=en), [Microsoft](https://learn.microsoft.com/en-us/azure/search/retrieval-augmented-generation-overview), NVIDIA, [Oracle](https://www.oracle.com/artificial-intelligence/generative-ai/retrieval-augmented-generation-rag/) and [Pinecone](https://www.pinecone.io/learn/retrieval-augmented-generation/) are adopting RAG. #### **Getting Started With Retrieval-Augmented Generation** The [NVIDIA AI Blueprint for RAG](https://build.nvidia.com/nvidia/build-an-enterprise-rag-pipeline) helps developers build pipelines to connect their AI applications to enterprise data using industry-leading technology. This reference architecture provides developers with a foundation for building scalable and customizable retrieval pipelines that deliver high accuracy and throughput. The blueprint can be used as is, or combined with other [NVIDIA Blueprints](https://build.nvidia.com/blueprints) for advanced use cases including [digital humans](https://developer.nvidia.com/blog/build-a-digital-human-interface-for-ai-apps-with-an-nvidia-nim-agent-blueprint/) and [AI assistants](https://developer.nvidia.com/blog/three-building-blocks-for-creating-ai-virtual-assistants-for-customer-service-with-an-nvidia-nim-agent-blueprint/). For example, the [b](https://build.nvidia.com/nvidia/ai-virtual-assistant-for-customer-service)[lueprint for AI assistants](https://build.nvidia.com/nvidia/ai-virtual-assistant-for-customer-service) empowers organizations to build AI agents that can quickly scale their customer service operations with generative AI and RAG. In addition, developers and IT teams can try the free, hands-on [NVIDIA LaunchPad lab](https://www.nvidia.com/en-us/launchpad/ai/generative-ai-knowledge-base-chatbot/) for building AI chatbots with RAG, enabling fast and accurate responses from enterprise data. All of these resources use [NVIDIA NeMo Retriever](https://www.nvidia.com/en-us/ai-data-science/products/nemo/), which provides leading, large-scale retrieval accuracy and [NVIDIA NIM](https://www.nvidia.com/en-us/ai/) microservices for simplifying secure, high-performance AI deployment across clouds, data centers and workstations. These are offered as part of the [NVIDIA AI Enterprise](https://www.nvidia.com/en-us/data-center/products/ai-enterprise/) software platform for accelerating AI development and deployment. Getting the best performance for RAG workflows requires massive amounts of memory and compute to move and process data. The [NVIDIA GH200 Grace Hopper Superchip](https://nvidianews.nvidia.com/news/gh200-grace-hopper-superchip-with-hbm3e-memory), with its 288GB of fast HBM3e memory and 8 petaflops of compute, is ideal — it can deliver a 150x speedup over using a CPU. Once companies get familiar with RAG, they can combine a variety of off-the-shelf or custom LLMs with internal or external knowledge bases to create a wide range of assistants that help their employees and customers. RAG doesn’t require a data center. [LLMs are debuting on Windows PCs](https://developer.nvidia.com/blog/supercharging-llm-applications-on-windows-pcs-with-nvidia-rtx-systems/), thanks to NVIDIA software that enables all sorts of applications users can access even on their laptops. [](https://blogs.nvidia.com/wp-content/uploads/2023/11/Using-RAG-on-PCs.jpg)An example application for RAG on a PC. PCs equipped with NVIDIA RTX GPUs can now run some AI models locally. By using RAG on a PC, users can link to a private knowledge source – whether that be emails, notes or articles – to improve responses. The user can then feel confident that their data source, prompts and response all remain private and secure. A [recent blog](https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblogs.nvidia.com%2Fblog%2F2023%2F10%2F17%2Ftensorrt-llm-windows-stable-diffusion-rtx%2F&data=05%7C01%7Crmerritt%40nvidia.com%7Cdfd2267cb8344597f73408dbd0e6fc36%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C638333462716560839%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=V5FClNyBybzOnm3%2FKxiTT4i9aoT0GdLWzdLfnF8LBk0%3D&reserved=0) provides an example of RAG accelerated by TensorRT-LLM for Windows to get better results fast. #### **The History of RAG** The roots of the technique go back at least to the early 1970s. That’s when researchers in information retrieval prototyped what they called question-answering systems, apps that use natural language processing ([NLP](https://www.nvidia.com/en-us/glossary/natural-language-processing/)) to access text, initially in narrow topics such as baseball. The concepts behind this kind of text mining have remained fairly constant over the years. But the machine learning engines driving them have grown significantly, increasing their usefulness and popularity. In the mid-1990s, the Ask Jeeves service, now Ask.com, popularized question answering with its mascot of a well-dressed valet. IBM’s Watson became a TV celebrity in 2011 when it handily beat two human champions on the *Jeopardy!* game show. [](https://blogs.nvidia.com/wp-content/uploads/2023/11/Ask-Jeeves-2.jpg)[Picture of Ask Jeeves, an early RAG-like web service](https://blogs.nvidia.com/wp-content/uploads/2023/11/Ask-Jeeves-2.jpg) Today, LLMs are taking question-answering systems to a whole new level. #### **Insights From a London Lab** The seminal 2020 paper arrived as Lewis was pursuing a doctorate in NLP at University College London and working for Meta at a new London AI lab. The team was searching for ways to pack more knowledge into an LLM’s parameters and using a benchmark it developed to measure its progress. Building on earlier methods and inspired by [a paper](https://arxiv.org/pdf/2002.08909.pdf) from Google researchers, the group “had this compelling vision of a trained system that had a retrieval index in the middle of it, so it could learn and generate any text output you wanted,” Lewis recalled. [](https://blogs.nvidia.com/wp-content/uploads/2023/11/IBM-Watson-wins-Jeopardy-YT.jpg)The IBM Watson question-answering system became a celebrity when it won big on the TV game show Jeopardy! When Lewis plugged into the work in progress a promising retrieval system from another Meta team, the first results were unexpectedly impressive. “I showed my supervisor and he said, ‘Whoa, take the win. This sort of thing doesn’t happen very often,’ because these workflows can be hard to set up correctly the first time,” he said. Lewis also credits major contributions from team members Ethan Perez and Douwe Kiela, then of New York University and Facebook AI Research, respectively. When complete, the work, which ran on a cluster of NVIDIA GPUs, showed how to make generative AI models more authoritative and [trustworthy](https://blogs.nvidia.com/blog/what-is-trustworthy-ai/). It’s since been cited by hundreds of papers that amplified and extended the concepts in what continues to be an active area of research. #### **How Retrieval-Augmented Generation Works** At a high level, here’s [how retrieval-augmented generation works](https://developer.nvidia.com/topics/ai/retrieval-augmented-generation). When users ask an LLM a question, the AI model sends the query to another model that converts it into a numeric format so machines can read it. The numeric version of the query is sometimes called an embedding or a vector. In retrieval-augmented generation, LLMs are enhanced with embedding and reranking models, storing knowledge in a vector database for precise query retrieval. The embedding model then compares these numeric values to vectors in a machine-readable index of an available knowledge base. When it finds a match or multiple matches, it retrieves the related data, converts it to human-readable words and passes it back to the LLM. Finally, the LLM combines the retrieved words and its own response to the query into a final answer it presents to the user, potentially citing sources the embedding model found. #### **Keeping Sources Current** In the background, the embedding model continuously creates and updates machine-readable indices, sometimes called vector databases, for new and updated knowledge bases as they become available. [](https://blogs.nvidia.com/wp-content/uploads/2023/11/LangChain-2-LLM-with-a-retriveal-process.jpg)Retrieval-augmented generation combines LLMs with embedding models and vector databases. Many developers find LangChain, an open-source library, can be particularly useful in chaining together LLMs, embedding models and knowledge bases. NVIDIA uses LangChain in its reference architecture for retrieval-augmented generation. The LangChain community provides its own [description of a RAG process](https://blog.langchain.dev/tutorial-chatgpt-over-your-data/). The future of generative AI lies in [agentic AI](https://blogs.nvidia.com/blog/what-is-agentic-ai/) — where LLMs and knowledge bases are dynamically orchestrated to create autonomous assistants. These AI-driven agents can enhance decision-making, adapt to complex tasks and deliver authoritative, verifiable results for users. Some content could not be imported from the original document. [View content ↗](https://disqus.com/embed/comments/?base=default&f=nvidiablog&t_u=https%3A%2F%2Fblogs.nvidia.com%2Fblog%2Fwhat-is-retrieval-augmented-generation%2F&t_d=What%20Is%20Retrieval-Augmented%20Generation%2C%20aka%20RAG%3F&t_t=What%20Is%20Retrieval-Augmented%20Generation%2C%20aka%20RAG%3F&s_o=default#version=9b3ae17cbcf6c4e7b830b077e32e537e), What Is Retrieval-Augmented Generation, aka RAG?, Rick Merritt, 01k2bbfwee83mcf83cfskyeann.