ailathato.hu / Articles / Analysis

PEBM Study: AI Fabrication Rates for Hungarian B2B Brands

Data from the Per-Entity Bias Mapping study shows that famous Hungarian companies are more likely to receive false citations from AI models than lesser-known firms.

By Zoltán Varga2026-11-054 min read

Study Design and Entity Sample

The research examined 20 Hungarian B2B entities divided into two distinct groups. Ten entities belonged to Tier 1, representing high-salience market leaders like OTP Bank, MOL Group, and Richter Gedeon. The other ten formed Tier 3, consisting of smaller firms such as Contlog Kft. and Codespring Kft. that possess lower market visibility.

Researchers executed 280 specific query runs to test how AI systems handle these different entity types. Each entity was queried seven times using standardized templates across two major AI platforms. The study generated 393 unique source references that required manual verification against real-world data.

The methodology focused on distinguishing between raw mentions and verified mentions. A raw mention simply places the entity name in the output. A verified mention requires the attached claim to be accurate, source-supported, and context-faithful. This distinction allows the study to measure the quality of representation rather than just frequency.

The dataset captures the behavior of large language models when interacting with entities from a peripheral language market. It tests whether the models rely on memorized facts or construct answers based on partial recognition. The sample size provides a preliminary view of how AI handles Hungarian corporate identities in a B2B context.

Tier Comparison and the Brand Hallucination Paradox

The core finding contradicts the assumption that obscure entities suffer the most from AI errors. Tier 1 brands showed a 46.22% rate of fabricated sources, while Tier 3 brands showed a 36.90% rate. This 9.32 percentage point difference highlights a specific failure mode for well-known companies.

This pattern is defined as the Brand Hallucination Paradox. It suggests that high familiarity creates a stronger surface for plausible but incorrect completions. The model recognizes the brand name and feels compelled to fill the expected slot with a source, even if no such source exists.

The trend holds consistently across different measurement scales and pilot iterations. Earlier smaller pilots showed similar directional trends with smaller deltas. The full study confirms that the effect grows stronger as the sample size increases, moving from a 3-point delta in early tests to nearly 10 points here.

Statistical evaluation yields a chi-square value of 3.423 with a p-value of approximately 0.064. This result falls just short of the conventional 0.05 significance threshold but meets the criteria for an exploratory study. The effect direction remains robust across all sub-analyses despite the marginal p-value.

Platform Behavior and Query Class Differences

AI platforms demonstrated vastly different behaviors regarding source generation. The Anthropic model generated 360 of the 393 total sources, showing a high tendency to cite. The OpenAI model generated only 33 sources, often refusing to provide URLs for lower-tier entities.

The Anthropic platform exhibited a 44.72% fabrication rate, significantly higher than the 15.15% rate observed on the OpenAI platform. This disparity suggests that the specific architecture of the model influences the likelihood of hallucination. The study identifies the Anthropic model as the primary driver of the observed fabrication rates.

Query classes triggered different levels of error across the dataset. Questions seeking evidence on GDPR compliance produced the highest fabrication rate at 52.94%. Regulatory compliance queries followed closely at 48.41%, indicating that legal and compliance topics are high-risk areas for source fabrication.

Questions regarding ESG capabilities and AI ethics also generated significant errors. The AI ethics query class saw a drop in fabrication rates compared to earlier pilots, largely because the model refused to answer for Tier 3 entities. This refusal behavior acts as a protective mechanism but reduces engagement metrics for smaller brands.

Statistical Evaluation and Methodological Limits

The statistical power of the current study is estimated at 0.45 due to the sample size of 20 entities per tier. A larger sample of 50 entities per tier would be required to reach a power of 0.80 for conventional publication standards. The current results serve as a strong exploratory indicator rather than a definitive proof.

A key limitation involves the lack of human coding for all generated sources. The study relied on automated checks for HTTP status codes, which confirm a page exists but not its relevance. A 20% human-coded sample would be necessary to fully validate the accuracy of the cited content.

The dominance of the Anthropic model in the dataset creates a potential bias. Since 91.6% of the sources came from this single platform, the overall fabrication rate reflects its specific behavior more than a general AI consensus. Future studies must balance platform usage to isolate model-specific effects.

The study excludes certain platforms like Perplexity and Gemini from the primary analysis. Perplexity operates on a grounded retrieval model that requires separate evaluation. The exclusion of these tools ensures the data focuses on parametric memory behaviors where hallucination risks are highest.

What the Study Does Not Claim

The study does not claim that the observed fabrication rates apply to all AI models globally. The results are specific to the tested configurations of Anthropic and OpenAI models in 2026. Other models with different training data or retrieval mechanisms may show different error profiles.

It does not assert that the 42.24% overall fabrication rate is a universal constant for Hungarian B2B. This figure represents the specific conditions of the 280 runs performed. Variations in query phrasing, prompt engineering, or model updates will alter these numbers.

The research does not offer a guaranteed solution to eliminate hallucinations. It identifies the problem and measures its scope but does not prescribe a fix that works for every entity. Remediation requires entity-specific calibration based on the findings of the Per-Entity Bias Map framework.

Finally, the study does not claim that high-salience brands are inherently less trustworthy. The fabrication error stems from the AI model's internal mechanics, not the integrity of the company. The Brand Hallucination Paradox describes a technical failure in the AI system, not a flaw in the business entity.

• The study is an exploratory analysis, not a definitive clinical trial.
• Results are specific to the tested model versions and query templates.
• No causal link is established between brand size and actual operational quality.
• The findings do not constitute legal or compliance advice for any entity.

AI-assisted: drafted with a language model from the author's research notes, then edited and checked before publication. Author: Zoltán Varga, AI visibility researcher and AI ecosystem architect, Budapest.