aicitation.hu / Articles / Analysis

Why Nearly Half of AI-Cited Sources Are Fabricated

Empirical testing shows high-salience brands face higher rates of false citations than smaller entities, challenging the assumption that fame ensures accuracy.

By Zoltán Varga2026-11-056 min read

Study Design and Verification Protocol

The research examined 20 Hungarian B2B entities split into two groups: ten high-salience Tier 1 brands and ten low-salience Tier 3 companies. Researchers executed 280 query runs across two major AI platforms using seven distinct query templates focused on compliance, ethics, and factual citation. Each run captured the generated answer, the list of cited sources, and the specific claims attached to those sources. The team then performed non-AI HTTP verification to check if the provided URLs existed and if the content matched the AI's description.

Verification relied on manual inspection rather than automated checks alone. A source URL might return a successful HTTP status code, yet the page content could fail to support the specific claim made by the model. This distinction separates a broken link from a misattributed source. The protocol treated any source that did not explicitly support the generated statement as a fabrication, regardless of whether the domain itself was real.

The dataset included 393 unique source URLs generated across the 280 runs. Two platforms drove the majority of these results, with one generating significantly more citations than the other. The study design intentionally included regulatory queries, such as GDPR compliance and ESG reporting, to test how sensitive topics influence citation behavior. This approach allowed the researchers to measure fabrication rates under different pressure conditions.

Data collection occurred in May 2026, capturing the behavior of specific model versions available at that time. The study excluded platforms known for strict grounding constraints to focus on generative models that attempt to cite sources freely. This scope ensures the findings reflect the risks present in the most common conversational AI interfaces used by professionals today.

Defining a Fabricated Source

A fabricated source is not merely a broken link or a non-existent webpage. It includes URLs that lead to real pages but do not contain the information the AI claims they hold. For example, a model might cite a company's annual report to support a claim about a specific sustainability metric that appears nowhere in that document. This error is a false attribution rather than a simple retrieval failure. The definition hinges on the mismatch between the generated claim and the actual source content.

The study distinguishes between a raw mention and a verified mention. A raw mention occurs whenever the entity name appears in the answer or citation list. A verified mention requires that the attached claim is accurate, supported by the cited source, and contextually faithful. When the source fails to back the claim, the mention becomes a fabrication. This distinction matters because aggregate citation counts often hide these critical errors.

False attribution is a specific failure mode where the model assigns a capability or statement to an entity that does not possess it. This happens frequently with large, familiar brands. The model uses the brand name as a semantic anchor to complete a sentence, even if the underlying facts are invented. This behavior aligns with the concept of the Brand Hallucination Paradox, where familiarity increases the risk of confident falsehoods.

Existing literature warns that even when a source exists, the AI may not represent it accurately. Users must verify the content themselves to confirm the truth of the statement. Relying on the presence of a URL or a citation marker is insufficient for professional or legal decision-making. The burden of proof shifts to the human reader to validate the connection between the claim and the source.

Headline Numbers and Statistical Limits

The full study recorded a 46.22% fabrication rate for Tier 1 high-salience brands across 225 generated sources. In contrast, Tier 3 low-salience entities showed a 36.90% fabrication rate across 168 sources. This difference of 9.32 percentage points indicates that popular brands face a higher risk of false citation than smaller companies. The trend held consistent across multiple pilot studies with smaller sample sizes, suggesting a robust directional pattern.

Statistical analysis yielded a chi-square value of 3.423 with a p-value of approximately 0.064. This result falls just above the conventional threshold of 0.05 required for strict statistical significance. However, the study was designed as an exploratory analysis, and the effect size remains consistent across different scales. A larger sample of 50 entities per tier would likely push the result into the statistically significant range.

The overall fabrication rate across all 393 sources stood at 42.24%. This figure represents the aggregate risk for any entity queried in this specific context. The data suggests that nearly half of the citations provided for top-tier brands in regulatory or compliance contexts are unreliable. Users encountering these citations should treat them as unverified hypotheses rather than established facts.

Platform behavior heavily influenced these numbers. One platform generated 91.6% of the total sources and exhibited a fabrication rate of 44.72%. The other platform generated far fewer citations and showed a much lower fabrication rate of 15.15%. This disparity highlights that the risk of fabrication is not uniform across all AI tools but depends on the specific model's tendency to generate citations.

Fabrication Versus Plain Error

Not every mistake in an AI response constitutes a fabricated source. A plain error might involve a typo, a wrong date, or a misinterpretation of a real document. A fabricated source involves the invention of a reference or the misrepresentation of a real one to support a false claim. The study focuses specifically on the latter, where the source itself is either non-existent or irrelevant to the claim.

Regulatory queries triggered the highest rates of fabrication. Questions regarding GDPR compliance resulted in a 52.94% fabrication rate among the sources generated. Compliance-related questions about ESG reporting followed closely at 48.0%. These topics demand high precision, yet the models produced the most false citations in these areas. This creates a dangerous feedback loop where the need for accuracy coincides with the highest error rate.

The study identified a phenomenon where models refuse to answer for low-salience entities but fabricate answers for high-salience ones. Tier 3 brands often received responses stating the model could not find specific URLs. Tier 1 brands, however, received confident answers with specific, yet often false, citations. This engagement rate paradox means that famous brands are more likely to be cited incorrectly than obscure ones.

Rejection-induced confabulation escalation occurs when a user pushes back on an initial answer. Repeatedly asking for more evidence or better sources can cause the model to increase its fabrication rate. The system attempts to satisfy the user's demand for a source by inventing one rather than admitting ignorance. This mechanism turns quality filters into accelerators for hallucinations in compliance contexts.

Implications for Cited Brands

Brands cannot assume that frequent mentions in AI answers equate to positive visibility. A high volume of citations may mask a high volume of false attributions. A brand might appear in hundreds of answers, yet the majority of those mentions could be factually incorrect or unsupported. This creates a reputational risk where the brand is associated with claims it never made.

The Brand Hallucination Paradox suggests that large, familiar entities are more vulnerable to false attribution than smaller ones. The model's training data contains strong associations with these names, making them easy targets for plausible completions. Smaller entities lack this familiarity, so the model is less likely to invent specific claims about them. This dynamic forces large brands to monitor AI outputs more rigorously.

Verification requires a shift from measuring mention rates to measuring verified mention rates. Raw mention counts are analogous to recall, while verified counts are analogous to precision. A brand with high recall but low precision faces a different strategic challenge than one with low recall but high precision. Entities must calibrate their visibility strategies based on these distinct error profiles.

As of May 2026, no automated tool can fully replace human verification of AI citations. The gap between retrieval-augmented updates and parametric memory updates further complicates the picture. New information may appear in search results while the model's internal knowledge remains outdated. This lag creates a window where the model generates confident but obsolete or incorrect citations for rapidly changing entities.

AI-assisted: drafted with a language model from the author's research notes, then edited and checked before publication. Author: Zoltán Varga, AI visibility researcher and AI ecosystem architect, Budapest.