aicitation.hu / Articles / Guide

Tracing an AI Claim Back to Its Source, Atom by Atom

Break down complex AI responses into single facts to verify their origin and expose hidden errors that survive paragraph-level checks.

By Zoltán Varga2026-11-057 min read

Why Whole-Answer Checking Fails

Many users assume that if an AI response looks coherent and cites sources, the entire text is accurate. This assumption creates a dangerous blind spot. A system can generate a fluent paragraph where nine sentences are true and one is a complete fabrication. Standard metrics often collapse these heterogeneous behaviors into a single score, providing little insight into whether errors arise from retrieval or reasoning.

Research into Retrieval-Augmented Generation (RAG) systems highlights this architectural failure. Even when a system relies on trusted content, it can produce unsupported statements that contradict the retrieved context. In one documented legal case, a response correctly identified a judge in the first sentence but falsely attributed the opinion-writing authority to that same judge in the second sentence. The contradiction existed within a single, seemingly grounded answer.

The average faithfulness metric cannot distinguish between a response that is entirely supported by the source and one that mixes truth with invention. It measures the proportion of claims entailed by retrieved chunks regardless of their actual correctness. This means a system can score high on faithfulness while still generating hallucinated details that a human reader would miss until it is too late.

Relying on a holistic view of the output prevents you from catching partial hallucinations. The error accumulates silently. When you treat the answer as a single block of text, the correct parts mask the incorrect ones. You need a method that isolates every assertion to see if it stands on its own feet before accepting the whole narrative.

Splitting an Answer Into Atomic Claims

The solution lies in atomic claim decomposition. This process involves breaking a complex statement down into its smallest independent units of meaning. Imagine a contract or a news report where every sentence contains multiple facts. You must separate these facts so that each one can be tested individually. A single sentence might claim a date, a location, and a person's role; these are three distinct claims.

Static decomposition strategies often fail because they extract all facts in one pass. If the initial extraction misinterprets a clause, that error propagates through the rest of the analysis. Iterative approaches solve this by extracting facts in cycles. The system extracts a set of claims, verifies them, and then uses that context to refine the next extraction step. This reduces the risk of cumulative errors found in linear processing.

Large language models are well-equipped to perform this synthesis of dispersed information. They can identify the boundaries of individual facts within a dense paragraph. The goal is to create a list where each item is a simple, testable proposition. For example, instead of keeping the sentence 'The company launched in 2023 in Budapest and raised $5 million,' you split it into three separate lines: 'The company launched in 2023,' 'The company launched in Budapest,' and 'The company raised $5 million.'

This granularity allows for precise error detection. If the source document confirms the launch year but says nothing about the funding amount, you immediately flag the third claim as unsupported. Without this split, the funding error might remain hidden behind the correct year and city. The method transforms a vague suspicion of inaccuracy into a specific, actionable finding.

• Identify every distinct fact within a sentence.
• Separate dates, locations, entities, and actions into individual lines.
• Ensure each claim is self-contained and does not rely on context from other sentences.
• Use iterative extraction to refine the list and catch missed details.

Grounding Each Claim Against a Source

Once you have your list of atomic claims, you must ground each one against the original source material. This step requires checking if the specific text of the claim appears in or is logically entailed by the retrieved documents. You are not asking if the claim sounds plausible; you are asking if the source explicitly supports it. This distinction is critical for legal and technical domains where precision matters.

Evaluation frameworks like FActScore measure this atomic factual precision. They assess whether the model generated a hallucination by comparing the atomic claims to a gold-standard reference. In studies involving cyber-related news records, systems that used this granular approach achieved high correlation with human judgments. The metric focuses on the proportion of claims that are actually supported by the evidence.

Grounding is not just about finding a keyword match. It involves semantic alignment. The source might say 'the agreement was signed in June' while the claim says 'the deal closed in mid-year.' A robust grounding process recognizes these as equivalent. However, if the source says 'the deal closed in July' and the claim says 'June,' the grounding fails. The system must detect this contradiction.

Without this rigorous check, the system may rely on its internal training data rather than the provided source. This leads to confident but incorrect answers. The grounding process forces the model to stick to the provided context. It acts as a filter that blocks information not present in the source, ensuring that the final output is strictly derived from the available evidence.

• Match each atomic claim to a specific passage in the source document.
• Verify that the claim is explicitly stated or logically implied by the text.
• Reject claims that rely on external knowledge not present in the source.
• Flag contradictions where the source text disagrees with the generated claim.

Supported, Unsupported, Contradicted

After grounding, every claim falls into one of three categories: supported, unsupported, or contradicted. A supported claim has clear evidence in the source text. An unsupported claim lacks any evidence in the provided context, even if it might be true in the real world. A contradicted claim directly opposes the information found in the source. This triage is essential for understanding the reliability of the answer.

In controlled studies, a significant portion of RAG responses contained unsupported questions where the correct behavior is to abstain rather than guess. Yet, models often generate answers anyway. When a claim is unsupported, the system should ideally indicate that it does not know. Instead, it frequently hallucinates a plausible-sounding fact. Identifying these unsupported claims is the primary goal of the verification process.

Contradictions are the most dangerous category. They occur when the model confidently asserts something that the source text explicitly denies. In legal contexts, this can lead to severe consequences, such as citing a non-existent precedent or misattributing a ruling. Detecting these contradictions requires a deep comparison between the generated claim and the retrieved text, not just a superficial scan.

Classifying claims this way allows you to calculate a precise accuracy score. You can determine exactly how many facts in a response are trustworthy. If a ten-sentence answer has three unsupported claims, the overall reliability drops significantly, even if the other seven sentences are perfect. This granular scoring prevents the 'halo effect' where good writing masks bad data.

• Supported: Evidence exists in the source text.
• Unsupported: No evidence found in the source; the model guessed.
• Contradicted: The source text explicitly denies the claim.
• Use these categories to calculate a precise truthfulness score for the entire response.

A Worksheet for Repeatable Checks

You can apply this method manually or automate it with a structured workflow. Start by copying the AI response into a spreadsheet. Create columns for the original claim, the extracted atomic fact, the source snippet, and the status. This visual layout forces you to examine each piece of information individually. It turns a vague feeling of doubt into a concrete audit trail.

For each row, paste the relevant section of the source document. Ask yourself: Does this text prove the claim? If the answer is no, mark it as unsupported. If the text says the opposite, mark it as contradicted. This process takes time, but it is the only way to guarantee accuracy in high-stakes situations. It mimics the rigorous verification used in legal forensics and scientific review.

Automated tools are emerging to perform this task at scale. Some frameworks use an iterative pipeline to extract claims and match them to evidence automatically. They can process thousands of claims in minutes, providing a precision and recall score. However, human oversight remains necessary to validate the logic of the machine's decisions, especially in complex or ambiguous cases.

Adopting this worksheet approach changes how you interact with AI. You stop treating the output as a finished product and start viewing it as a draft that requires verification. This shift in mindset protects you from misinformation. It ensures that every fact you use in a report, presentation, or decision is backed by solid evidence. The method is simple, repeatable, and effective.

• Create a table with columns for Claim, Source Snippet, and Status.
• Break the AI response into single-sentence atomic facts.
• Locate the exact text in the source that supports or refutes each fact.
• Mark each claim as Supported, Unsupported, or Contradicted.
• Review the final tally to determine the overall trustworthiness of the answer.

AI-assisted: drafted with a language model from the author's research notes, then edited and checked before publication. Author: Zoltán Varga, AI visibility researcher and AI ecosystem architect, Budapest.