Understanding Why Large Language Models Struggle to Count the ‘R’s in “Strawberry” (Variation 159)

Understanding Why Large Language Models Struggle to Count Letters in Words: The Case of “Strawberry”

In the realm of artificial intelligence, large language models (LLMs) often encounter surprising limitations—one common example being their difficulty in accurately counting specific letters within a word, such as the number of “R”s in “Strawberry.” This has led to some jesting about their seemingly simple counting errors. But what underlying factors contribute to this challenge?

At their core, LLMs process text by segmenting input into smaller units known as “tokens.” These tokens could be words, subwords, or characters, depending on the tokenization method used. Following this, the model transforms these tokens into mathematical representations called “vectors,” which serve as the foundation for subsequent processing layers.

However, because these models are not expressly trained to memorize or count individual characters, their internal representations—these vectors—do not retain explicit, character-level details. As a result, they lack a precise conceptual memory of the specific composition of words, such as the exact number of certain letters within them. This is why, despite the simplicity of the task, LLMs can often miscount or misunderstand the letter composition of words like “Strawberry.”

For a more detailed explanation, including visual diagrams, you can explore this resource: Why LLMs Can’t Count Letters.

Understanding these fundamental limitations helps us better interpret what LLMs can and cannot do, especially as their applications continue to expand across various domains.