For the complete documentation index, see llms.txt. This page is also available as Markdown.

Retrieval-Augmented Generation (RAG)

About

Retrieval-Augmented Generation (RAG) is a technique where the model retrieves relevant information from external data sources and uses that information to generate responses.

Instead of relying only on internal training knowledge, the system:

  1. Retrieves relevant documents/data

  2. Injects that data into the prompt

  3. Generates an answer based on retrieved context

Core idea:

Retrieve first, then generate.

RAG combines two components:

  • Retriever → Finds relevant information

  • Generator (LLM) → Produces the final response

This allows the model to:

  • Use up-to-date information

  • Access domain-specific knowledge

  • Reduce hallucination

Why RAG Is Critical ?

LLMs alone have limitations:

  • Knowledge is static (based on training cutoff)

  • Cannot access private/internal data

  • May hallucinate when unsure

RAG solves these problems by:

  • Providing real-time or updated knowledge

  • Enabling access to internal systems (docs, APIs, DBs)

  • Grounding responses in actual data

This is critical for:

  • Enterprise systems

  • Knowledge base assistants

  • API documentation querying

  • Compliance and audit workflows

  • Support and troubleshooting systems

Without RAG, AI remains generic. With RAG, AI becomes context-aware and domain-specific.

The Purpose of RAG

RAG is designed to:

  1. Improve factual accuracy

  2. Reduce hallucination

  3. Enable dynamic knowledge access

  4. Integrate external data sources

  5. Support domain-specific use cases

It transforms AI from:

Static knowledge system → Dynamic knowledge system

Where RAG Fits in the Prompt Lifecycle

RAG introduces a pre-processing layer before generation.

Different RAG Technique Types

1. Basic RAG

  • Retrieve top relevant documents

  • Inject into prompt

  • Generate response

Simple but effective.

2. Embedding-Based RAG

  • Convert documents into vector embeddings

  • Perform semantic search

  • Retrieve most relevant chunks

Most commonly used approach.

3. Hybrid RAG

  • Combine keyword search + vector search

  • Improves retrieval accuracy

Useful for structured + unstructured data.

4. Multi-Step RAG

  • Retrieve → refine query → retrieve again

  • Improves precision for complex queries

5. Tool-Augmented RAG

  • Retrieve data via APIs, databases, or services

  • Use real-time system data

Example:

  • Fetch transaction details

  • Query logs

  • Call internal APIs

Common RAG Mistakes

1. Poor Retrieval Quality

If retrieval is weak:

  • Wrong documents are selected

  • Model generates incorrect answers

RAG is only as good as its retriever.

2. Injecting Too Much Context

Adding large amounts of data:

  • Increases token usage

  • Reduces signal-to-noise ratio

  • Confuses the model

Use relevant, concise chunks.

3. Not Constraining the Model

If not instructed:

“Use only provided context”

The model may still:

  • Add external knowledge

  • Hallucinate beyond retrieved data

4. Ignoring Chunking Strategy

Large documents must be split into:

  • Meaningful chunks

  • Contextually complete units

Poor chunking leads to:

  • Missing information

  • Broken reasoning

5. No Ranking or Filtering

If multiple documents are retrieved without ranking:

  • Less relevant data may dominate

  • Important information may be ignored

Relevance scoring is critical.

Sample Prompts

Without RAG

Issue:

  • Model does not know your internal system

  • Will generate generic or incorrect response

With RAG (Context Injected)

Benefits:

  • Domain-specific accuracy

  • Reduced hallucination

  • Aligned with actual system

Last updated