Redefining Prediction: The Essential Dynamic of Creativity in Forecaster–AI Collaboration
Photo by koto_feja/Getty Images
By Karen Hagar, RAND Adjunct, Superforecaster
As Artificial Intelligence (AI) takes shape in society, the curious anticipate; the cautious speculate. Civilization may be changed by a transformative tsunami—rivaling humanity’s most dynamic breakthroughs such as the discovery of fire and the seismic shift of the Industrial Revolution. Some indicators reveal the AI revolution may soon surpass the pace of advancement of the last century.
As AI alters sectors of the labor market, it will not be task “complexity” or “routine” that will determine job replacement by AI. The converse is true. Only a few occupations will be fully automatable. Jobs that require higher skill levels are the most susceptible to AI’s Goldilocks zone. These are roles such as managerial oversight, analysis, and making decisions based on predictable information and rules. The jobs least likely to be automated are physical/manual labor work and non-routine interactive jobs that require creativity, judgment, empathy, and social skills. Given this backdrop, could AI put professional human forecasters out of business?
AI has recently been showcased in geopolitical forecasting, and it was more accurate than a few human forecasters. However, experts note that the most advanced large language models (LLMs) are still about 30 percent less accurate than top elite human forecasters (i.e., “superforecasters”). Notwithstanding, the framework of the geopolitical forecasting culture is likely to morph as LLMs continue to predict with greater accuracy.
Many superforecasters were first exposed to artificial intelligence during IARPA’s Hybrid Forecasting Competition, which concluded with superforecasters outperforming AI in accuracy scores by approximately 21 percent. This trend points to a future in which professional forecasters and LLMs may work in tandem, creating a powerful form of hybrid forecasting or a “super-brain.”
Yet, there is speculation that integration of AI into forecasting may be a double-edged sword due to its inability to understand human nature. In an interview, UCSD Professor Julia Mossbridge said, “[We use] emotions and intuition in every decision. Sometimes analytical methods inform decision-making, but whether you are conscious or not of them, your emotions and intuition always influence decisions.” So, what is the AI experience? Her research asserts that certain aspects of AI can outperform humans by orders of magnitude—mostly in terms of performing tasks much faster than we can—yet they rarely surpass humans by the same margin in domains like intuition, creativity, and emotion (so-called “right-hemispheric” skills).
Many assume that AI “thinks” in straightforward, predictable ways like simple machines while humans think in more complicated, mysterious ways. But Mossbridge states this is not true. AI has complex, hidden processes that are just as opaque and unpredictable as human thought; and it is reasonable to say that some aspects of the AI experience are somewhat like human emotion or intuition.
However, AI does not have the same life experiences that shape humans—growing up, interacting with the world, forming relationships, aging, and eventually facing mortality. Until AI can live through these kinds of experiences in human-like bodies that grow, age, and die like ours, our intuitions will be superior to anything AI can produce. Mossbridge wrapped up by sharing that “in general, the reading, processing, and writing speed advantages of GPTs can help with forecasting—and I see humans as the key driver of the human-relevant intuition and emotional substrates for forecasting. We are both needed.”
It is worth noting that AI comprises multiple model types, and LLMs should not be conflated with chatbots. In practice, geopolitical forecasters are most likely to work with LLMs, which generate text-based analyses that can inform and support forecasting. Representative examples include GPT-4/5, Claude, and LLaMA.
The newest LLM models can generate ideas and predict what [a forecaster] might want or think next. At the same time, it could negatively influence thinking before we even start reasoning or using intuition. It can decide what information to view first and hide other information; this can consequently influence gut reactions and careful thinking without user realization. Thus, AI anticipates and shapes information and can affect both fast, instinctive thinking (System 1) and slow, deliberate thinking (System 2), with a newly described system referred to by some researchers as “System 0.”
Humans have experience in leveraging external supportive tools such as books, experts, thesauruses, etc., or forms of hybrid thinking that extend beyond the brain. But System 0 can reorganize how thinking works. Luckily for humans, while AI is excellent at pattern recognition, it might stumble at cause-and-effect relationships because this skill may require intuitive perception; this will continue to necessitate a human in the loop to ensure the validity and contextual accuracy of decisions.
While AI is good at correlation, it has difficulty thinking in terms of counterfactuals such as “what would happen if …” scenarios. Ongoing research suggests providing AI with high quality data sets and advanced frameworks may improve its causal reasoning. However, achieving this remains complex, and AI capabilities are currently limited in this domain of intelligence. Therefore, careful consideration and reasoned judgment are required regarding how and when to rely on its conclusions.
AI is not automatically better than humans yet is a tool that can assist humans in overcoming aspects of cognitive bias. It also works well for creativity because it can assist in generating new ideas, combinations, or solutions that humans might not think of on their own, unlike straightforward decision-making, which often relies on following clear rules. Creative tasks benefit from generating lots of possibilities, exploring unusual connections, or imagining alternatives.
Here is the crux: The most proficient human forecaster/AI fusion teams of the future may be those who work collaboratively to spark new creative ideas and then build upon them to generate even more insights.
Much like during the Renaissance era, a time of heightened creativity and innovation, the world’s greatest minds in the future may be guided by imagination and ingenuity. In this future, top forecasters would rely less on logic—since AI provides that—and instead excel through creative foresight and visionary thinking. They would thrive in a trusting, symbiotic partnership with AI, predicting future outcomes with human judgment at the core.
Karen Hagar is one of the six Pro Forecasters selected to become RFI Adjuncts within RAND’s Defense and Political Science Department. Hagar is a social scientist with 14 years of forecasting experience, earning recognition as the #2 forecaster in the 2011-13 IARPA Ace Competition and being designated a Superforecaster® since 2014. Her expertise spans anthropology, archaeology, human behavior, and cultural analysis.