Retrieval-Augmented Generation (RAG) is often described as a simple pipeline: Query → Retrieve documents → Send context to an LLM → Generate answer In production, however, retrieval is rarely that simple. The retriever can return irrelevant documents. Important information may be buried in the middle of a document. A query may be too vague for semantic search. Retrieved chunks may lose their surrounding context. And sometimes the model does not need retrieval at all. The quality of a RAG system therefore depends heavily on how information is retrieved, filtered, ranked, compressed, and presented to the model . This guide covers nine techniques that address different parts of the RAG pipeline: Reranking Hybrid Search Chunking Strategies Multi-Query Retrieval Parent Document Retrieval Context Compression HyDE Self-RAG CRAG 1. Reranking Retrieve candidates. Reranking finds the best. A vector database may search through hundreds or thousands of documents and return the top 20 candidate chunks. But the first result from vector search is not necessarily the best result. For example, suppose the correct answer is ranked at position #19. If the application only sends the top 3–5 chunks to the LLM, the correct information never reaches the model. 100 Pages ↓ Vector Search ↓ 20 Candidate Chunks │ ┌───────────┴───────────┐ ↓ ↓ Without Reranking With Reranking ↓ ↓ Top 3–5 Chunks Reranker ↓ ↓ │ Top 5 Relevant │ Chunks ↓ ↓ ↓ ↓ LLM LLM Without reranking, the most relevant chunk might be ranked #19 and never reach the model. With reranking, a reranker evaluates the retrieved candidates using both the query and the content of each chunk, promoting the most relevant results to the top. The reranker can move a previously low-ranked but highly relevant chunk to the top. What does a reranker do? A reranker essentially asks: Which of these retrieved chunks actually answers the user's question best? Unlike basic vector similarity, a reranker can inspect the relationship between the entire query and the retrieved document. This helps it: Understand context better Find hidden relevance Filter out noise Improve the quality of the final answer Types of rerankers Common approaches include: Cross-Encoder High accuracy Slower Bi-Encoder + Rerank Model Balanced performance LLM-based Reranker Potentially highest quality More expensive Example Suppose a developer asks: How can I make a Node.js API handle thousands of simultaneous connections? A basic vector search might initially return chunks about HTTP status codes, API authentication, or general Node.js syntax. A reranker can compare each candidate directly against the question and prioritize content discussing connection handling, asynchronous I/O, event loops, connection pooling, and horizontal scaling. Key takeaway Retrieve more. Rerank intelligently. Let the LLM see the best context, not merely the first context. 2. Hybrid Search Meaning + Keywords = Better Retrieval Semantic vector search and keyword search solve different problems. Vector search understands meaning . Keyword search understands exact words . Using only one can cause important documents to be missed. The problem Consider this query: "Redis connection timeout" A semantic search might return: Diagnosing cache connection failures Distributed cache troubleshooting Network latency in application infrastructure These documents may be semantically related, but the exact phrase Redis connection timeout might not appear. A keyword search such as BM25 can find: Redis connection timeout configuration Fixing Redis client timeout errors Redis socket timeout settings But keyword search may fail when the document uses different terminology. Hybrid Search Hybrid search combines both approaches: User Query │ ┌─────────┴─────────┐ ↓ ↓ Vector Search Keyword Search ( Semantic ) ( BM25 ) │ │ └─────────┬─────────┘ ↓ Merge & Rank ↓ Final Results The results from both searches are combined and ranked. Common ranking methods Popular approaches include: Reciprocal Rank Fusion (RRF) Weighted score combination Relative score fusion Rank-based fusion Why hybrid search works Vector search is good at understanding intent: "cache performance troubleshooting" Keyword search is good at exact terms: "Redis MISCONF" A production search system often needs both. When hybrid search is useful Hybrid search is particularly useful for: Technical documentation Spelling variations Abbreviations Exact technical terms Natural language queries Systems requiring both high recall and high precision Key takeaway Don't choose between meaning and keywords. Use both. 3. Chunking Strategies Good chunks → Better Retrieval → Better Answers Chunking is one of the most important decisions in a RAG system. Documents are usually too large to embed and retrieve as a single unit, so they must be divided into smaller pieces. But chunk size matters. Why chunking matters If a chunk is too large: Important information can get buried Retrieval becomes less precise More irrelevant context reaches the LLM If a chunk is too small: Context is lost More noise can be introduced Individual chunks may not contain enough information to answer a question The goal is: Keep chunks as small as possible for precision, but as large as necessary for completeness. 1. Fixed-Size Chunking The document is divided into chunks of a fixed number of tokens. For example: Document ↓ 400 tokens ↓ 400 tokens ↓ 400 tokens ↓ ... An overlap can be added between chunks. Advantages Simple Fast Works reasonably well for general documents Disadvantages Can split sentences or concepts in the middle 2. Sentence-Based Chunking Instead of splitting at arbitrary token boundaries, the system splits around sentence boundaries. Sentence 1 Sentence 2 Sentence 3

Sentence 4 Sentence 5 Sentence 6 Advantages Preserves meaning better More natural and readable Disadvantage Sentence lengths can vary significantly. 3. Semantic Chunking Semantic chunking groups sentences or paragraphs based on their meaning. Conceptually: Topic A ├── Authentication configuration ├── Token validation └── Session management

Topic B ├── Database indexing ├── Query planning └── Connection pooling Advantages High topic relevance Keeps related content together Disadvantages More complicated Requires embeddings 4. Parent Document Chunking Small chunks are used for retrieval, but the larger parent section is returned to the LLM. Large Parent Document │ ┌──────────┼──────────┐ ↓ ↓ ↓ Small Small Small Chunk Chunk Chunk The small chunks provide retrieval precision while the parent document provides context. 5. Sliding Window Chunking A moving window is used to create overlapping chunks. Window 1 ████████

Window 2
████████

    Window 3
    ████████ This preserves more context across chunk boundaries. Advantages Good coverage Maintains context flow Disadvantage More chunks mean more storage and potentially more retrieval cost. 6. Structure-Aware Chunking The document's structure is used to determine chunk boundaries. For example: # Authentication ↓

Chunk 1 ## Token Validation ↓ Chunk 2

  • Access token
  • Refresh token ↓ Chunk 3

Configuration Table ↓ Chunk 4

Code Block ↓ Chunk 5 This works particularly well for: Documentation Code Structured PDFs Knowledge bases How should you choose? There is no universally best chunking strategy. The right strategy depends on the data. A production system may combine multiple approaches: Structure-aware splitting + Semantic grouping + Parent document retrieval You should also experiment with chunk sizes such as: 256 tokens 512 tokens 1024 tokens and measure actual retrieval performance. Key takeaway Good chunks bring the right context. The right context helps the LLM produce the right answer. 4. Multi-Query Retrieval One question. Multiple perspectives. Better results. A single query can fail because documents may describe the same concept using completely different language. Even if query expansion improves the wording, searching in only one direction can still miss relevant documents. The idea Instead of searching once, ask the LLM to generate multiple versions of the query. For example: Original question: How does OAuth token refresh work? The system might generate: What is OAuth token refresh?

How does a refresh token work?

What happens when an access token expires?

How does an application obtain a new access token?

What is the OAuth refresh-token flow? Each query is searched independently. Original Question ↓ Generate Queries ↓ ┌────────┬────────┬────────┐ ↓ ↓ ↓ ↓ Search Search Search Search └────────┴────────┴────────┘ ↓ Merge & Rerank ↓ Final Chunks Why this works Different documents use different terminology. One document might say: OAuth token refresh while another says: renewing an expired access token and another says: obtaining a new bearer token using a refresh credential Multiple queries give the retriever more opportunities to find relevant information. Multi-Query vs Query Expansion These concepts are related but not identical. Feature Query Expansion Multi-Query Main goal Better wording Different viewpoints Queries Similar variations More diverse queries Focus Query improvement Retrieval coverage Recall Good Often higher Typical use General search Production RAG When it works best Multi-query retrieval is particularly useful for: Large knowledge bases Technical documentation Enterprise search Research papers Legal documents Medical documents Production RAG systems Key takeaway Don't ask once. Ask in multiple smart ways. More angles give the retriever more chances to find the right information. 5. Parent Document Retrieval Small chunks = better search. Parent documents = better understanding. Small chunks are useful because they make retrieval precise. But small chunks have a problem: They can lose context. Consider retrieving this chunk: "... it automatically retries failed operations ..." The chunk might be relevant, but by itself it doesn't tell us what "it" refers to. The original section might say: "The job processor automatically retries failed operations when a worker temporarily loses access to the message queue." The parent document provides the missing context. How Parent Document Retrieval works Step 1: Create small chunks Document ↓ Chunk 1 Chunk 2 Chunk 3 Chunk 4 ... Step 2: Search the small chunks The vector database retrieves the most relevant chunks. Top Chunks: 1 2 8 9 10 Step 3: Map chunks to their parent Each chunk stores a reference to its parent section or document. Chunk 8 ↓ Parent Document / Section Step 4: Send the parent context to the LLM Instead of giving the LLM only the tiny chunk, provide the relevant parent section. Small chunks → Search

Parent document → Context This creates a useful separation: Retrieve small. Read big. When to use it Parent document retrieval is useful when: Chunks are very small Documents contain many references Answers require surrounding context Pronouns and references are common The meaning depends on information elsewhere in the section Implementation tip Store a parent_id with each chunk. For example: Chunk: { id : "chunk_123" , parent_id: "section_42" , embedding: [ ...] } After retrieval, use parent_id to fetch the larger context. Key takeaway Chunks help you find information. Parent documents help the model understand it. 6. Context Compression Too much context can be as bad as too little. Imagine a retriever returns 40 chunks, but your LLM can effectively process only 8 useful chunks. Sending all 40 creates several problems: Higher token usage Higher cost More irrelevant information More redundancy Potentially worse answers This is related to the lost-in-the-middle problem: important information can become harder for the model to use when surrounded by large amounts of irrelevant context. How context compression works 40 Retrieved Chunks ↓ Compress ↓ Keep Relevant Information ↓ 8 Clean Chunks ↓ LLM The compressor attempts to remove everything that does not contribute meaningfully to answering the question. What can be compressed? Common targets include: Duplicate chunks Filler sentences Low-relevance information Long-winded explanations Off-topic sections Repeated information Popular compression techniques LLM Summarization Summarize each chunk into a smaller representation. Large chunk ↓ 1–2 sentence summary Keyword / Keyphrase Extraction Keep the most important terms and phrases. Redundancy Removal Remove information that appears repeatedly across retrieved documents. Extractive Compression Keep only the sentences that directly contribute to answering the query. Relevance Scoring Score individual sentences or chunks and keep only high-scoring content. Example Suppose retrieval returns 40 chunks. After compression: 40 chunks ↓ 8 chunks ↓ ~75% token reduction ↓ Better focused context The exact improvement depends on the data and compression method, but the goal is to make the context smaller without losing useful information . Key takeaway More context is not always better. Relevant context is better. 7. HyDE Hypothetical Document Embeddings Think before you search. HyDE stands for Hypothetical Document Embeddings . It addresses a common retrieval problem: The user's query may be too short or vague to produce a strong embedding. For example: "message queue retries" The query contains only a few terms. A better search signal could be a hypothetical answer generated by an LLM. How HyDE works Instead of embedding the original question: User Question ↓ Embedding ↓ Vector Search HyDE introduces an intermediate generation step: User Question ↓ Generate Hypothetical Answer ↓ Embed Hypothetical Answer ↓ Vector Search ↓ Retrieve Documents ↓ LLM For example, the user asks: How does a message queue retry failed jobs? The LLM might generate a hypothetical answer such as: A message processing system can retry a failed job when the worker encounters a temporary error. Retry policies commonly use a maximum attempt count and exponential backoff before moving permanently failed messages to a dead-letter queue. The hypothetical answer contains more meaningful domain terms than the original question. The system embeds that hypothetical answer and uses the embedding to search the knowledge base. Why it can work The generated answer may contain: More domain-specific terminology More context Better representation of the user's intent Terms that are likely to appear in relevant documents This can improve semantic matching. HyDE is not random guessing The hypothetical answer is not used as the final answer. It is primarily a search representation . The actual answer still comes from retrieved documents. Question ↓ Hypothetical Answer ↓ Embedding ↓ Retrieve Real Documents ↓ Generate Final Answer HyDE vs Query Expansion Query expansion usually creates multiple alternative queries. HyDE generates a hypothetical document or answer and embeds that representation. Query Expansion → Multiple queries

HyDE → One hypothetical answer → One embedding When HyDE is useful HyDE can help with: RAG systems Research assistants Code search Legal search Medical document search Enterprise knowledge bases Vague or complex queries Key takeaway HyDE turns a weak question into a stronger search signal. 8. Self-RAG Why search every time? Let the model decide first. Traditional RAG often retrieves documents for every query. But not every question needs external retrieval. For example: What is the square root of 144? Retrieving documents from a vector database would be unnecessary. The problem Always retrieving causes: Extra latency Extra token usage Additional infrastructure cost Unnecessary vector database load Self-RAG introduces a decision step. User Question ↓ Should I retrieve? ↓ ┌────────┴────────┐ NO YES ↓ ↓ Answer Retrieve Directly ↓ Generate How Self-RAG works The model first considers: Do I need external information? Do I already know the answer? Is the question domain-specific? Is the information likely to be recent? Do I need private or internal documents? If the answer is NO The model answers using its internal knowledge. Example: What is 15 × 8? No retrieval is required. If the answer is YES The system retrieves relevant documents. Example: What changed in our company 's API documentation this week? Retrieval is useful because the information is recent and internal. When should Self-RAG retrieve? Typical cases include: Recent information Live information Domain-specific knowledge Private/internal documents Complex multi-hop questions When can it skip retrieval? Typical cases include: General knowledge Simple mathematics Logic questions Common facts Questions where the model is sufficiently confident Benefits Self-RAG can: Reduce unnecessary retrieval Save tokens Reduce cost Reduce vector database load Improve response latency Use retrieval when it actually matters Key takeaway Traditional RAG retrieves every time. Self-RAG decides whether retrieval is needed before acting. 9. CRAG Corrective Retrieval-Augmented Generation Not every retrieved chunk is useful. CRAG checks the quality before trusting it. A retriever is not perfect. It can return: Irrelevant chunks Outdated information Misleading information Incomplete information If the LLM blindly trusts those chunks, it can produce a confident but incorrect answer. CRAG introduces a quality-control step. How CRAG works User Question ↓ Retrieve Documents ↓ Evaluate Retrieved Documents ↓ ┌───────┴───────┐ GOOD BAD ↓ ↓ Use Docs Correct Retrieval ↓ Refine / Re-query ↓ Retrieve Again ↓ Final Context ↓ LLM The retrieved documents are evaluated before they are trusted. What does the evaluator check? Potential criteria include: Relevance Completeness Consistency Whether the documents actually support the question If the documents are good enough, they can be passed to the LLM. If they are poor, the system can attempt corrective actions. Examples include: Refine the query Try another search Expand the query Rerank the results Filter noisy chunks Retrieve from another source Example Suppose the user asks: Which database is a good choice for high-volume event analytics? The retriever returns: 1. Introduction to relational databases 2. Key-value cache configuration 3. Columnar database architecture for analytics 4. Basic SQL CRUD operations The evaluator can determine that only some of these documents directly address the question. The system can then remove weak results and perform additional retrieval if necessary. The goal is: Retrieve ↓ Check ↓ Correct ↓ Answer Benefits CRAG can: Reduce confident wrong answers Improve retrieval quality Reduce hallucinations Handle difficult queries Filter noisy retrieval results Improve the quality of final context When CRAG helps most CRAG is particularly usef