aicitation.hu / Articles / Concept

Citation Pipeline Glossary: Retrieval, Reranking, Grounding

A technical breakdown of the three-stage pipeline that decides which documents an AI model actually sees and cites.

By Zoltán Varga2026-11-056 min read

Query Rewriting and Initial Retrieval

The process begins when a user submits a question. The system often rewrites this query to improve search performance. It expands terms, corrects typos, or splits complex questions into simpler parts. This step prepares the text for the search engine. Without this adjustment, the system might miss relevant documents because the original phrasing was too narrow or ambiguous.

Once the query is ready, the retrieval phase starts. The system searches a database of documents to find potential matches. It uses two primary methods: keyword matching and semantic similarity. Keyword matching looks for exact word overlaps. Semantic matching understands the meaning behind the words. The system runs both methods simultaneously to gather a broad set of candidates.

You need both methods because they solve different problems. Keyword search excels with technical terms, code snippets, or specific names where exact spelling matters. Semantic search handles business questions or conceptual inquiries where the meaning matters more than the specific words. Relying on only one method risks missing critical information. A robust pipeline combines them to ensure high recall.

The initial retrieval outputs a list of candidate chunks. These are small segments of text, not full documents. The system selects the top results from each search method. It does not yet decide which one is the absolute best. It simply gathers a pool of possibilities. This pool serves as the raw material for the next stage of processing.

• Query rewriting adjusts the user input to match database structures.
• Hybrid retrieval combines keyword and semantic search methods.
• Chunking breaks large documents into smaller, searchable segments.
• Initial retrieval prioritizes recall to avoid missing relevant data.

BM25, Dense Vectors, and Hybrid Search

BM25 is a classic algorithm for keyword-based retrieval. It calculates a score based on term frequency and document length. It works well when the user uses precise terminology. If you search for a specific error code or a chemical formula, BM25 finds the exact matches efficiently. It treats the text as a bag of words and ignores deeper context.

Dense retrieval uses vector embeddings to represent meaning. It converts text into long lists of numbers. These numbers capture the semantic relationship between words. Two sentences with different words but similar meanings will have close vector values. This method helps when the user asks a question using synonyms or paraphrases that the document does not use explicitly.

Hybrid search merges these two approaches. It retrieves the top results from the BM25 method and the top results from the dense vector method. It then normalizes the scores from both methods onto a common scale. This allows the system to compare a keyword score with a semantic score fairly. The combined list contains the strongest candidates from both worlds.

Research indicates that hybrid approaches outperform single-method searches in diverse scenarios. Technical queries benefit from the precision of BM25. Business or conceptual queries benefit from the flexibility of dense vectors. By running both, the system mitigates the weaknesses of each. This dual-path strategy increases the likelihood that the correct information appears in the candidate pool.

• BM25 prioritizes exact keyword matches and term frequency.
• Dense retrieval captures semantic meaning through vector embeddings.
• Normalization aligns scores from different algorithms to a single scale.
• Hybrid search reduces the risk of missing relevant context.

Reranking and the Context Window

Reranking refines the list of candidates before the model reads them. A cross-encoder model analyzes the query and each candidate document together. It assigns a new, highly accurate relevance score. This step is computationally expensive but necessary for precision. It filters out false positives that passed the initial retrieval stage.

The context window limits how much text the model can process at once. It is a fixed number of tokens. If the reranked list exceeds this limit, the system truncates the lower-scoring items. Only the top-ranked chunks enter the window. Anything outside this boundary is invisible to the model during generation. The model cannot cite what it cannot see.

Chunk size and overlap affect what fits in the window. Smaller chunks allow more distinct ideas to enter the context. Larger chunks preserve more narrative flow but consume space faster. Overlap ensures that sentences split at the edge of a chunk remain coherent. A 25% overlap is a common setting to maintain continuity between adjacent segments.

Effective reranking ensures the most critical information occupies the limited window space. It pushes irrelevant noise to the bottom of the list. This maximizes the utility of every token available to the model. Poor reranking leads to the model focusing on peripheral details while missing the core answer. The quality of the final output depends entirely on this selection process.

• Cross-encoders provide precise relevance scoring for reranking.
• The context window acts as a strict capacity limit for input data.
• Chunk overlap maintains coherence across segment boundaries.
• Reranking determines which information the model actually accesses.

Grounding and Source Selection

Grounding connects the model's generated text to the retrieved sources. It forces the model to derive answers only from the provided context. Without grounding, the model relies on its internal training data. This internal knowledge can lead to hallucinations or outdated facts. Grounding restricts the model to the verified information in the current session.

A well-grounded system verifies that every claim in the output has a supporting source. It checks if the retrieved text contains the necessary facts. If the context lacks the answer, the system should state that it does not know. It should not invent a plausible-sounding response. This discipline prevents the spread of misinformation.

Grounding also influences how the model handles ambiguity. If the retrieved documents contradict each other, the model must reflect that conflict. It cannot resolve contradictions using external knowledge. It must report the discrepancy based on the provided text. This transparency builds trust in the system's reliability and accuracy.

Failure to ground leads to confident but incorrect statements. The model might blend old training data with new queries. This creates a false sense of authority. Proper grounding ensures the answer reflects the specific documents selected by the retrieval and reranking pipeline. It anchors the output in reality rather than probability.

• Grounding restricts generation to retrieved context only.
• Hallucinations occur when models rely on internal training data.
• Contradictions in sources must be reported, not resolved internally.
• Verification ensures every claim traces back to a source chunk.

Attribution and Post-Hoc Citation

Attribution links the final answer to the specific source chunks used. It provides a transparent trail for the user. A citation points to the exact document or section that supported a statement. This allows users to verify the information independently. It transforms the AI from a black box into a verifiable tool.

Accurate attribution requires tracking which chunks survived the entire pipeline. The system must remember the ID of every piece of text that influenced the answer. It maps the generated sentences back to these IDs. This mapping happens after the text is generated. It is a post-processing step that adds metadata to the output.

Fabricated citations happen when the system guesses a source. The model might invent a title or a URL that looks real. This occurs when the grounding mechanism fails or the attribution logic is weak. Users lose trust immediately upon discovering a fake reference. Robust systems only cite sources that were actually retrieved and processed.

Proper attribution supports accountability. It shows exactly where the information came from. It distinguishes between the model's synthesis and the original author's words. This clarity is essential for professional and academic use. It ensures that the AI acts as a conduit for information, not a generator of fiction.

• Attribution maps generated text to specific source identifiers.
• Fabricated citations result from weak grounding or logic errors.
• Transparent trails allow users to verify claims independently.
• Post-hoc processing adds source metadata to the final output.

AI-assisted: drafted with a language model from the author's research notes, then edited and checked before publication. Author: Zoltán Varga, AI visibility researcher and AI ecosystem architect, Budapest.