I thought GPT5 wasn’t supposed to hallucinate as much?

Understanding AI Hallucinations: A Case Study of Recent GPT-5 Behavior

In the rapidly evolving landscape of artificial intelligence, large language models like GPT-5 are heralded for their impressive capabilities. However, recent observations suggest that these systems are not immune to a phenomenon known as “hallucination,” where the AI generates plausible-sounding but incorrect or fabricated information. This article examines a recent example illustrating this issue, discusses its implications, and highlights the importance of critical evaluation when working with AI-generated content.

The Incident: Challenging AI Verification of Historical Claims

In a recent interaction, a user prompted GPT-5 to verify the authenticity of a quote attributed to Pope Leo. Surprisingly, GPT-5 responded by asserting that the quote was fabricated, citing the absence of a Pope Leo XIV as the reason. This was peculiar because the source relied upon was deemed reputable, and the claim that Pope Leo XIV did not exist contradicted well-known historical facts.

When the user prompted GPT-5 to recheck the information with a confirmation that “Pope Leo is real,” the AI did not issue the usual correction or acknowledgment of its prior mistake. Instead, it provided a detailed explanation: there is no credible evidence for a Pope Leo XIV, and claims suggesting his existence are unfounded or part of a hoax. The AI went further to detail its research process, explaining how it examined trusted sources—including Vatican records, Catholic news outlets, and recognized historical directories—and found no verifiable evidence supporting the existence of such a pope.

This behavior underscores a critical aspect of AI operation: despite expectations of increased accuracy and reduced hallucinations, large language models can still produce confident but incorrect statements. In this case, GPT-5 demonstrated an understanding of the topic but also a tendency to erroneously dismiss unverified claims based on limited data, emphasizing the importance of human oversight.

Analyzing the Roots of the Misinformation

The specific claim about Pope Leo XIV appears to originate from speculative or fringe sources online, including satirical sites and unverified social media posts. Such sources often lack credible backing and are not corroborated by official Vatican documentation or reputable historical records. The proliferation of this misinformation exemplifies how the internet can amplify false narratives, especially when they are presented in a seemingly authoritative manner.

Additionally, the AI’s reasoning highlights a common issue with language models: they generate responses based on patterns in training data but may lack the contextual awareness to distinguish between verified facts and rumors. When confronted with conflicting information, the