What is wrong with GPT-5-Thinking? What is this? It never used to be this inaccurate.

Title: Assessing the Reliability of AI-Generated Content: A Case Study with GPT-5 Thinking

Introduction

Artificial Intelligence (AI), particularly large language models like GPT-5, has revolutionized the way we access and synthesize information. However, recent experiences highlight that AI’s accuracy can sometimes fall short, especially in complex or nuanced tasks. This article examines a specific case involving AI’s responses to fantasy football queries, illustrating the challenges and limitations of current AI reasoning capabilities.

Exploring the Case: AI’s Performance in Fantasy Football Analysis

In a recent interaction, a user tasked GPT-5 with researching specific player performances, leading to several inaccuracies and cognitive inconsistencies that merit closer inspection.

Initial Response and Misattribution

The AI initially provided detailed statistics about Christian Kirk, claiming he missed Weeks 1-2 due to hamstring injuries and then had a “spike-week” performance in Week 3, with a line of “7 for 104.” The user correctly identified that this line was more consistent with Nico Collins’s Week 3 performance, not Kirk’s. The AI subsequently acknowledged the mistake, clarifying that Collins indeed posted 8 receptions for 104 yards and a touchdown in that week, while Christian Kirk’s actual stats were different.

Errors in Data Consistency and Source Attribution

Further scrutiny revealed that the AI’s summary contained inaccuracies. It cited Kirk’s season statistics as “6-45-0” and listed links to credible sources such as NFL.com, CBS Sports, and ESPN to substantiate its claims. Yet, the core data was inconsistent, highlighting a disconnect between AI-generated information and verified facts.

Analyzing the Reasoning: Herd Behavior and Contextual Factors

The AI explained the recent surge in Kirk’s popularity as driven by “herd behavior” and timing—suggesting that after a notable performance (which was actually Collins’s), players like Kirk become popular waiver additions. While this reasoning is plausible from a behavioral standpoint, the AI’s inability to correctly parse and differentiate between player performances underscores limitations in its contextual understanding.

Reflections on AI’s Extended Thinking and Reliability

The key concern raised by the user relates to the phenomenon of “extended thinking,” where the AI’s reasoning process becomes less reliable over extended or complex exchanges. The AI’s tendency to mix up player statistics, misattribute performance lines, and produce inconsistent explanations suggests that its internal reasoning may lack robustness, especially when handling overlapping or similar data points.

Conclusion

This case exemplifies the current challenges