> For the complete documentation index, see [llms.txt](https://www.pranaypourkar.co.in/the-programmers-guide/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://www.pranaypourkar.co.in/the-programmers-guide/ai/generative-ai/large-language-models-llm/prompt-engineering/prompt-engineering-techniques/3.-knowledge-grounded-techniques/retrieval-augmented-generation-rag.md). # Retrieval-Augmented Generation (RAG) ## About Retrieval-Augmented Generation (RAG) is a technique where the model **retrieves relevant information from external data sources** and uses that information to generate responses. Instead of relying only on internal training knowledge, the system: 1. Retrieves relevant documents/data 2. Injects that data into the prompt 3. Generates an answer based on retrieved context Core idea: > Retrieve first, then generate. RAG combines two components: * **Retriever** → Finds relevant information * **Generator (LLM)** → Produces the final response This allows the model to: * Use up-to-date information * Access domain-specific knowledge * Reduce hallucination ## Why RAG Is Critical ? LLMs alone have limitations: * Knowledge is static (based on training cutoff) * Cannot access private/internal data * May hallucinate when unsure RAG solves these problems by: * Providing real-time or updated knowledge * Enabling access to internal systems (docs, APIs, DBs) * Grounding responses in actual data This is critical for: * Enterprise systems * Knowledge base assistants * API documentation querying * Compliance and audit workflows * Support and troubleshooting systems Without RAG, AI remains generic.\ With RAG, AI becomes **context-aware and domain-specific**. ## The Purpose of RAG RAG is designed to: 1. Improve factual accuracy 2. Reduce hallucination 3. Enable dynamic knowledge access 4. Integrate external data sources 5. Support domain-specific use cases It transforms AI from: Static knowledge system → Dynamic knowledge system ## Where RAG Fits in the Prompt Lifecycle ``` User Query ↓ Retrieval (Search relevant documents) ↓ Context Injection (Add retrieved data to prompt) ↓ Reasoning (Model processes query + context) ↓ Output Generation ↓ Validation ``` RAG introduces a **pre-processing layer before generation**. ## Different RAG Technique Types ### 1. Basic RAG * Retrieve top relevant documents * Inject into prompt * Generate response Simple but effective. ### 2. Embedding-Based RAG * Convert documents into vector embeddings * Perform semantic search * Retrieve most relevant chunks Most commonly used approach. ### 3. Hybrid RAG * Combine keyword search + vector search * Improves retrieval accuracy Useful for structured + unstructured data. ### 4. Multi-Step RAG * Retrieve → refine query → retrieve again * Improves precision for complex queries ### 5. Tool-Augmented RAG * Retrieve data via APIs, databases, or services * Use real-time system data Example: * Fetch transaction details * Query logs * Call internal APIs ## Common RAG Mistakes ### 1. Poor Retrieval Quality If retrieval is weak: * Wrong documents are selected * Model generates incorrect answers RAG is only as good as its retriever. ### 2. Injecting Too Much Context Adding large amounts of data: * Increases token usage * Reduces signal-to-noise ratio * Confuses the model Use relevant, concise chunks. ### 3. Not Constraining the Model If not instructed: “Use only provided context” The model may still: * Add external knowledge * Hallucinate beyond retrieved data ### 4. Ignoring Chunking Strategy Large documents must be split into: * Meaningful chunks * Contextually complete units Poor chunking leads to: * Missing information * Broken reasoning ### 5. No Ranking or Filtering If multiple documents are retrieved without ranking: * Less relevant data may dominate * Important information may be ignored Relevance scoring is critical. ## Sample Prompts ### Without RAG ``` Explain the internal API structure of our payment service. ``` Issue: * Model does not know your internal system * Will generate generic or incorrect response ### With RAG (Context Injected) ``` Use the following API documentation to answer the question. Context: --- --- Question: Explain the internal API structure of the payment service. Answer using only the provided context. ``` Benefits: * Domain-specific accuracy * Reduced hallucination * Aligned with actual system --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://www.pranaypourkar.co.in/the-programmers-guide/ai/generative-ai/large-language-models-llm/prompt-engineering/prompt-engineering-techniques/3.-knowledge-grounded-techniques/retrieval-augmented-generation-rag.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.