Generative AI chatbots like ChatGPT are limited to the creativity level of an amateur

Generative AI chatbots like ChatGPT are limited to the creativity level of an amateur

Many are concerned about the impact of generative AI on creative professions. Rest assured, it’s now mathematically proven that the creativity of generative AI chatbots like ChatGPT, Gemini, Claude, DeepSeek, etc., will never match human creativity, and especially those remarkable “eureka moments.”

The study “The Cat Sat on the…? Why Generative AI Has Limited Creativity,” published in “The Journal of Creative Behavior,” mathematically demonstrates the creative limitations of chatbots based on a Large Language Models (LLMs), which we will call genAI chatbots.

David H. Cropley of the School of Engineering, University of South Australia, has shown that while genAI chatbots are often more creative than the average untrained human (amateur creative), their creativity plateaus, and highly creative and educated humans (professional creatives) demonstrate greater creativity than genAI chatbots. This limit, shared by all genAI chatbots, corresponds to the boundary between amateur and professional creativity.

This study confirms what many have observed: that genAI chatbots possess a creative capacity superior to that of the average human, but inferior to that of a highly creative and well-educated individual. For example, consider the experiment where participants are asked for the different possible uses of a brick or a paperclip.

According to the scientific consensus (Runco and Jaeger 2012), creativity (C) is the ability to produce or create something new and original (N), while adapting it to the specific problem or context, which is called efficiency (E). This corresponds to the equation: C = N x E

Generative AI plateaus at the level of amateur creativity

Cropley’s article demonstrates that the creativity of generative AI based on large language models is mathematically bounded at an upper limit (C=0.25), which corresponds to the boundary between amateur and professional creativity.

The core of Cropley’s demonstration is the algorithmic inability of large language models to achieve high-level creativity, which simultaneously requires high efficiency and high novelty. Large language models generate word segments (tokens) based on their semantic probability in a given context. Efficiency (E) corresponds to the probability of the generated symbol, and novelty (N) is inversely correlated with efficiency, modeled as N = 1 – E. By simple algebraic substitution of N, which equals 1 – E, in the definition of creativity C = N x E, we obtain C = (1 – E) x E or C = E – E². The creativity of a LLM response is constrained by this function. It can easily be calculated that this function reaches a maximum of 0.25 (on a scale of 0 to 1).

This ceiling means that LLMs cannot simultaneously achieve a high level of efficiency and novelty, unlike humans.

Furthermore, this creativity ceiling of 0.25 corresponds to the boundary between so-called “little-c” (amateur) and “pro-c” (professional) creativity in the Kaufman and Beghetto 2009 model. Numerous empirical studies cited in Cropley’s article confirm that the creativity of large language models (LLMs) is at the level of average human creative performance, that is, around the boundary between “little-c” and “pro-c”.

The low creativity of LLMs is not due to poor training or a lack of data, but is a fundamental consequence of their design: predicting the next word segment (token) based on probability. LLMs are extraordinary imitators of human creativity, but they are not its source. Exceeding this limit would require a fundamental redesign of the architecture of large language models.

Limitations of the study

Cropley’s view is simplistic (binary), while the GenAI chatbot chooses between banality (correct) and risky creativity (surprise).

The study is based on a simplified mathematical model of creativity. To make the equation easily solvable, the study uses a linear approximation where novelty N is the exact inverse of effectiveness E, i.e., N = 1 − E. The author himself admits that this formula is a simplification of information theory where “surprise” is calculated non-linearly (−log p). That said, this linear approximation captures the general trend (the more probable a word, the less surprising it is) but does not perfectly reflect the mathematical complexity of statistical surprise.

The study does not take into account the autoregressive nature of GenAI chatbots, which leads to user guidance. The study applies to a classic GenAI chatbot in the context of a “zero-shot” use case (a single request). It doesn’t take into account any potential dialogue with the user that could guide the process and add their own creativity. That said, a user with amateur-level creativity cannot add much creativity unless by sheer chance. The situation is different if the user is highly creative, but in this case, it’s the human who brings their creativity, not the robot.

Also, injecting randomness into the output of a GenAI chatbot (softmax function) creates variance (i.e., it increases the temperature variable), but this is just statistical noise, not genuine intellectual creativity.

Since the space of contexts subjected to the GenAI chatbot is practically infinite, a GenAI chatbot cannot simply regurgitate something by heart. It applies learned knowledge to a new situation, which could be considered generalization. To be precise, it’s not so much “true” generalization but rather interpolation within the distribution of the training data. If a large language model (LLM) has only seen additions, it’s highly doubtful that it could invent multiplication. This question is still debated by AI experts.

Now, let’s use Daniel Kahneman’s System 1 (fast and intuitive), System 2 (slow and logical) formulation. Without symbolic reasoning capabilities (System 2), a LLM remains trapped by its training data (System 1).

The most recent systems, which incorporate “heuristic search” (like Monte Carlo tree search) with LLM (OpenAI o1 and above) and the integration of agents that simulate symbolic reasoning mechanisms, point towards a primitive System 2.

Finally, Cropley’s study in no way foreshadows future AI systems based on other principles such as “world models” or neurosymbolism. Even if “superintelligence” were to emerge, human beings would not abandon their creativity, nor their ability to solve problems or write texts themselves. It’s much like we don’t give up playing chess, lifting weights, or running just because machines can do it better.

“Eureka moments”

This highlights the importance of “eureka moments” in creativity. A “eureka moment” is also called an “aha! moment”. A “eureka moment” is the instant when an idea spontaneously and involuntarily emerges from our thoughts, often as we fall asleep (hypnagogia), during a nap, while dreaming, or during quiet activities like walking, playing music, or taking a shower.

A “eureka moment” doesn’t come out of nowhere. It most often results from a long period of reflection and sometimes even a genuine obsession with a problem, with periods of focused work, daydreaming, and contemplation where inspiration is nurtured and solutions are found. Generally, a “eureka moment” occurs during a quiet moment, after intense and focused analytical work.

A “eureka moment” is not simply divergent or convergent creativity, but rather a combination of both. It’s a solution that’s both “ready-made” and “tailored” to the problem, not simply a “strange idea,” an “unusual combination,” or what’s called a bisociation—it’s THE SOLUTION.

The Creative Nap

The “Creative Nap” is a nearly forgotten creativity technique that harnesses “eureka moments,” used by historical geniuses like the surrealist painter Salvador Dalí, the prolific inventor Thomas Edison, and probably the great physicist Albert Einstein.

Their trick was to take a nap with an object (for example, a key) in their hand. As they drifted off to sleep, the object would slip from their grasp, and the sound of it hitting the floor would wake them. Sometimes a brilliant idea would emerge, a “eureka moment” that they would quickly jot down.

A creative technique validated by neuroscience

This creativity technique has recently been validated by neuroscience. The study “Sleep onset is a creative sweet spot“, published in Science Advances by a team of French neuroscientists, demonstrates the importance of these moments of rest in the creative process, thus offering a structured method for accessing inspiration and innovation. The French study showed a 300% increase in the number of people able to find a solution to a mathematical problem after a short creative nap.

In neuroscience, it has been established that the hypnagogic state, characterized by a semi-conscious state between wakefulness and sleep, is particularly conducive to creativity. It allows for a free flow of thoughts and their associations, similar to dreaming, while maintaining enough awareness to remember one’s thoughts. A “creative nap” of just a few seconds, halfway between wakefulness and sleep, therefore has the capacity to stimulate creativity. However, we mustn’t forget our brilliant idea while trying to write it down… Hence the idea of ​​dictating it and having it transcribed by a software assistant with minimal effort.

The “EurekIA™ Idea Catcher”

EurekIA™ technologies is offering top-level creatives like you the “EurekIA™ Idea Catcher”, which stimulates your creativity and collects your “eureka moments” confidentially using AI and neuroscience, while remaining human-centered.

Specifically, the “EurekIA™ Idea Catcher” includes a mobile app and the “EurekIA™ egg” (inspired by Columbus’s egg), which you hold in your hand and which falls to the ground as you fall asleep, emitting a tinkling sound. Thanks to an AI algorithm based on convolutional neural networks, the mobile app recognizes this distinctive sound, wakes you with a ringtone, and then immediately records your spoken ideas. Finally, your ideas are automatically transcribed into text by another AI model (in this case, a large language model), with minimal friction and complete confidentiality, as everything runs locally on your phone without any internet connection. The process of a creative nap.

Ultra-confidential and environmentally friendly

A key innovation of the “EurekIA™ Idea Catcher” is the local execution of AI models on the phone, ensuring complete confidentiality without relying on an external web service (brilliant ideas aren’t shared with the tech giants). This is also an environmental benefit because edge AI doesn’t require large, remote data centers since everything runs on the phone. The app can also be used to take confidential notes.

Everyone’s brain is unique! Even if you don’t have the brain of Dalí, Edison, or Einstein, thanks to the “EurekIA™ Idea Catcher” you can replicate their creative process and multiply your own “eureka moments!” ensuring you never again lose the brilliant ideas that spring from your mind.

To learn more, visit eurekia.com

 

Statement: As a matter of principle and consistency, EurekIA™ Technologies does not use generative AI to produce images, videos, podcasts, or texts. Our sole use of AI is for assisted proofreading or translation tools like Corrector 101 or Google Translate before professional review. Furthermore, EurekIA™ Technologies does not claim to have a monopoly on creativity, good taste, or good ideas. We ask for your understanding. However, someone with a creative mind can use the “EurekIA™ Idea Catcher” and do much better! We welcome your creative suggestions. As scientists and simply honest people, we apply the great Newton’s maxim: “Standing on the shoulders of giants.” This is why we place great importance on truth, fact-checking, and the attribution of ideas. That said, we may sometimes exaggerate on social media to grab attention, but the facts will be presented in a more nuanced and scientific way on our website. Finally, our communications are full of hyperlinks. If you find any unfounded claims, please contact us. We endorse the Montréal Declaration for a Responsible Development of AI.
Share the Post: