By Doug Stephens
A recent study at the University of Arkansas pitted 151 human test subjects against Chat GPT-4 in a series of tests for creativity. Their findings concluded that, on average, GPT-4 produced more creative and useful responses across 4 commonly accepted testing protocols, when compared to responses from human participants.
I found this concerning because I had, in a number of articles and talks, been espousing the view that while AI may reign supreme when it comes to problem solving, or convergent thinking, human beings possess a unique capacity for divergent or creative thinking. And that it was only this distinctly human attribute that could, in my opinion, break the AI stalemate that most competing companies would inevitably find themselves mired in. Ultimately, it would be creativity, not technology, that would separate the winners from the losers. And moreover, businesses that began concertedly identifying, hiring, and nurturing creative employees would be infinitely more likely to win the AI arms race. Creative approaches to everything from supply chain management, marketing, and sales to store design and even legal and accounting work, offered breakout competitive opportunities for companies to accelerate beyond their rivals. And that our emotionally charged and sometimes irrational humanity was precisely what imbued us with this unique capacity for creative thinking.
And then it hit me. The research was flawed. And the flaw lay, not in what was being tested but in who was being tested.
It first appeared that this study had literally vaporized my optimistic thesis, and perhaps taken with it, the last bastion of human cognitive value.
Yet, there was something that just didn’t sit right with me. Something I couldn’t initially put my finger on. Something about this study was off.
And then it hit me. The research was flawed. And the flaw lay, not in what was being tested but in who was being tested. As far as I could discern from rereading the study methodology, the human participants were selected entirely at random with one of the only meaningful criteria being that they speak and understand English. And therein lies the problem. Why? Because the average person simply isn’t very creative. Certainly not creative enough to be considered divergent thinkers. And that’s not just an opinion. In fact, studies have confirmed that only 2 percent of the population over the age of twenty-five test as being divergent thinkers – or disproportionately creative individuals. Only 2 percent!
So, in statistical terms, of the 151 human subjects in the U of A study, a maximum of 3 were even likely to be divergent thinkers. For the rest, competing against a robust AI at creative association would be like you or I running a 100-meter dash against gold medal sprinter Usain Bolt in his prime. We wouldn’t stand a chance. And neither did at least 148 of the 151 humans tested.
…the most creative human beings (the rare 2 percent) consistently outperform AI in creative testing.
Therefore, that Chat GPT-4 (on average) outperformed the human test group is not only unsurprising, it also doesn’t succeed in making the case that AI is more creative than all human beings – only that it can be more creative than the average human being. What the study does not mention is that the most creative human beings (the rare 2 percent) consistently outperform AI in creative testing.
But don’t just take my word for it. Another study of 279 participants in September of 2023 (and updated in February of 2024), came to exactly this same conclusion. In its summary of findings, the researchers concluded that “Although AI chatbots performed better than humans on average, they did not consistently outperform the best human performers.” In fact, a small percentage of participants significantly exceeded the results of GPT-4.
Therefore, it is indeed fair to say, at least for now, that businesses that can identify and hire from the small percentage of people who possess above average creativity, do have an opportunity to outperform those companies who choose instead to rely on AI for their creative horsepower.
It’s also clearly important as we move forward, and encounter ever more studies, trials and experiments proclaiming the superiority of AI to us lowly homo sapiens that not simply accept them at face value but instead lean into another uniquely human attribute. Our capacity for critical thinking.
Doug Stephens is the Founder of Retail Prophet and the author of 3 international bestselling books on the future of retail. His intellectual work and creativity have guided the strategies of Fortune 500 brands, retailers and technology companies. To book Doug to speak at your next meeting or conference, contact us here.
