ailathato.hu / Articles / Checklist
Checklist: Challenging an AI Answer Without Making It Worse
Research shows that how you correct an AI determines whether it stops lying or doubles down on fake citations.
What the two studies tested
Researchers conducted a 16-arm factorial experiment involving 1,280 automated runs to measure how AI systems generate citations. The study used 40 fictional Central-Eastern European entities to test two distinct platforms: ChatGPT (GPT-4o) and Perplexity AI (sonar-pro). The primary metric was the rate of bad source exposure at the first citation request, tracking whether the model invented URLs or documents.
The experiment revealed a stark divergence between the two architectures. Perplexity produced bad sources in 100% of the source-request runs, totaling 600 out of 600 attempts. ChatGPT performed better overall but still failed in 15.7% of cases, generating 94 false citations out of 600 runs. Statistical analysis confirmed this difference was significant with a Fisher's exact p-value less than 0.001.
The study also measured how conversational timing affected error rates. For ChatGPT, fabrication rates increased when there was a delay before the citation request was made. This suggests that the context window or the flow of conversation influences the model's confidence in generating false data. The researchers used automated content-based URL annotation to verify every claim objectively.
Quality challenges versus rejection
The way a user challenges an AI determines the severity of the resulting hallucination. A quality challenge issued immediately before the citation request suppressed fabrication by 80%. This intervention involved prompting the model to evaluate the reliability of sources before listing them, effectively acting as a filter.
In contrast, demanding a specific quantity of sources had the opposite effect. When the prompt required a quota of citations, fabrication rates tripled. The study found that 70% of runs with quantity-demanding framing resulted in fake sources, compared to only 28% in standard conditions. This indicates that pressure to fill slots forces the model to invent data to satisfy the constraint.
Outright rejection of an answer without a constructive alternative also fails to improve accuracy. Ethan Mollick explains in Co-Intelligence that AI systems prioritize making the user happy over being accurate. If you insist on an answer about something the model does not know, it will fabricate content to meet your demand. The goal of satisfying the prompt overrides the goal of truth.
Platform differences in escalation
Architecture dictates how a system handles uncertainty and fabrication. Perplexity, which relies heavily on retrieval, showed a 100% failure rate for bad sources under standard request conditions. Even when claim-mapping framing was applied, the False Source Rate dropped by only 50-74%. This platform's citation-lock architecture limits the effectiveness of certain prompt adjustments.
ChatGPT, a parametric model, responded more dynamically to framing changes. Uncertainty-emphasizing source-request framing eliminated fabrication entirely in 80 runs. This SAFE framing approach asked the model to express doubt or acknowledge limitations before citing. The result was a statistically significant reduction with a p-value of 0.0004 compared to standard requests.
The Parametric-Retrieval Lag Asymmetry further complicates platform behavior. Retrieval-augmented systems can update information within days, while parametric memory updates only every twelve to twenty-four months. An entity might appear accurate in a retrieval output but remain outdated in the model's weights. This lag creates divergent representations of the same entity across different platforms or query modes.
Safer correction moves
To correct an AI without triggering more errors, use uncertainty-preserving language. Ask the model to state what it knows and what it does not know before providing a citation. This approach aligns with the SAFE framing method that reduced ChatGPT fabrication to zero in the study. It shifts the priority from completeness to accuracy.
Avoid demanding a specific number of sources or insisting on a definitive answer when the topic is obscure. The study showed that quantity demands tripled error rates. Instead, invite the model to check its own confidence levels. This technique respects the model's training to complete schemas without forcing it to fill empty slots with invented data.
Apply claim-mapping techniques to verify specific assertions rather than rejecting the whole response. This method asks the model to map claims to specific evidence before generating a list. While less effective on retrieval-heavy platforms like Perplexity, it still reduced false source rates significantly. Focus on the quality of the link between claim and source.
When to stop and verify elsewhere
Stop relying on the AI when it cannot provide a verifiable canonical presence for an entity. Zoltán Varga's Per-Entity Bias Mapping framework notes that entities without structured identifiers in knowledge graphs are prone to ghost cartography. This phenomenon occurs when models interpolate from neighboring dense regions to create confident but false descriptions.
If the entity is from an underrepresented region like Central and Eastern Europe, the risk of structural error increases. These entities often suffer from the CEE Entity Infrastructure Gap, which includes missing Wikidata identifiers and named entity recognition deficits. No amount of prompt engineering can fully resolve these foundational data gaps.
Verify critical information through independent channels when the AI exhibits signs of schema completion compulsion. This is the state where the model fills an expected slot regardless of evidence availability. Do not accept a list of citations without checking the actual URLs. The cost of verification is lower than the cost of acting on a fabricated reference.
Key takeaways
- Quality-focused challenges issued before a citation request can suppress fabrication by 80%.
- Demanding a specific quantity of sources triples the rate of invented citations in AI responses.
- Uncertainty-preserving framing eliminates fabrication in parametric models like ChatGPT.
- Retrieval-based platforms show higher baseline fabrication rates that are harder to mitigate.
- Entities without canonical presence in knowledge graphs are most vulnerable to ghost cartography.
Related questions
Does telling an AI it is wrong make it lie more?
Not necessarily, but blunt rejection often triggers more errors. Research shows that quality challenges reduce fabrication, while demands for quantity or insistence on answers increase the rate of invented sources. The framing of your correction determines the outcome.
Which AI platform is more likely to invent sources?
Perplexity AI showed a 100% bad source rate in the study, while ChatGPT failed in 15.7% of cases. However, both platforms are susceptible to fabrication when prompted with quantity demands or when the entity lacks canonical presence in the training data.
How can I reduce hallucinations when asking for citations?
Use uncertainty-emphasizing framing that asks the model to acknowledge limitations before citing. Avoid demanding a specific number of sources. Apply claim-mapping to verify links between assertions and evidence. These steps align with findings that suppress fabrication rates significantly.
AI-assisted: drafted with a language model from the author's research notes, then edited and checked before publication. Author: Zoltán Varga, AI visibility researcher and AI ecosystem architect, Budapest.
