Summary: New research presents an automated mind-mapping framework that pairs the raw power of large-scale linguistic models (LLMs) with intuitive behavioral statistics. By using LLM to read, interpret, and categorize thousands of free-text personal reasons written by human participants during high-level gambling tasks, the team successfully proved that human cognition is a highly reliable and statistically valid source of data, showing that the core logic people rely on shifts systematically depending on the problem.
Important Facts
- Gambling Text Test: During simulated gambling rounds involving dynamic risk parameters, participants were prevented from simply clicking on a choice; they had to actively explain their internal thought processes and rationales in their own words after every single round.
- Algorithmic Codebook: Using decades of established behavioral finance and decision-making theory, researchers have developed a broad taxonomy of possible human heuristics, ranging from the intense focus on the best outcome (“maximax heuristics”) to the extreme avoidance of destruction (“minimax loss aversion”).
- LLMs as Quality Auditors: Rather than relying on human research assistants to manually read and mark thousands of individual journal papers, the team produced fine-tuned LLMs. The models acted as quality auditors, reading the free-text data at scale and quickly marking the specific psychological reasons driving each entry.
- Validating Mathematical Choices: To ensure that the classification of the LLM text was not correlated or inappropriate, the researchers referred to the text tags of the model as opposed to the objective statistical modeling of the participants' actual choices. Choice ratings verified text profiles with incredible accuracy, confirming that people said they fit well with their way of life he did.
- Dynamic Strategic Shifting: The collected data have clearly shown that decision-making strategies are not permanent personality traits. Instead, people are highly flexible: they dynamically and systematically change their thinking profiles from round to round based on how the problem is presented.
- New Toolkit for Public Policy: Dr. Fuławka emphasizes that this automated analytical framework opens up vast horizons for studying human behavior within complex, real-world ecosystems. By allowing researchers to dissect large amounts of freely written public feedback, policymakers can better understand how societies interpret and facilitate public health trade-offs, economic planning, and technological adaptation.
Source: TUD
Lead author Dr. Kamil Fuławka, a researcher at SynoSys says: “Our understanding of human behavior, including decision-making, can be deepened by asking people to describe their thought processes in detail. “However, systematic analysis of such free-text data requires balanced and robust analytical frameworks – an effort that cannot now be supported by LLMs”
In the study, participants took part in gambling and had to describe each decision in their own words. To analyze these explanations, researchers draw on existing theories and models of decision making to develop a larger set of possible reasons for a decision, such as focusing on the best outcome or avoiding the greatest loss. Large-scale linguistic models (LLMs) identified which of these reasons emerged from participants' free-text explanations, while statistical modeling of human preferences provided confirmation.
A combination of verbal reports, LLMs, and rigorous statistical modeling clearly demonstrated that demographics are a valuable source of data. It also showed that the reasons people rely on are not fixed, but are shifted systematically and structurally in the decision problem.
“Many important decisions—from financial planning and medical choices to social issues, technology use, and public policy—involve complex trade-offs that cannot be fully understood by looking at choices alone,” said Kamil Fuławka, emphasizing the importance of the research findings: “In those settings, people's explanations may be particularly useful for revealing specific problems, focusing on simple decisions, and facilitating adaptation.” The framework presented in the study shows how LLMs can help researchers analyze these definitions at a higher level, opening up new opportunities to study human decisions in real and complex environments.”
The framework presented in the study also shows how LLMs can help researchers to analyze these explanations on a large scale, thus opening up new opportunities to study human decision-making processes in real and complex environments.
Important Questions Answered:
A: Looking only at a person's final choice is like trying to understand a complex murder mystery by looking only at the last page of a book. Traditional behavioral economics tracks which button someone clicks or which product they buy, but remains completely blind to the hidden thought processes, doubts, and mental shortcuts that led to that action. While researchers have long sought to read people's written descriptions to understand their true motivations, hand reading is a huge burden. Using the LLM as a high-speed, automated reader, scientists can now analyze thousands of detailed personal journals quickly, giving them an unprecedented look inside the human mind on a massive scale.
A: This was the smartest and most important layer of SynoSys testing. To ensure that the LLM accurately documents the science of human psychology instead of generating “false hypotheses,” the team used randomization statistics as a strong validation safeguard. They took the psychological reasons the AI found in the participant's text, such as “trying to avoid a catastrophic loss”, and connected that reason to a separate statistical program that tracked the participant's actual physical gambling behavior. The figures and the text are perfectly aligned: the decisions made by humans correspond precisely to the written stimuli of the AI extracted from their journals, proving that the LLM scale is very accurate and scientifically sound.
A: Many important choices in life, such as planning for retirement, choosing cancer treatment, or adapting to new technology, involve messy trade-offs that do not fall into a simple multiple-choice question. Historically, governments and banks have struggled to analyze many public surveys because coding open-ended responses is very slow. This new framework allows public policy analysts to use LLMs to quickly read and measure thousands of complex, written responses from real citizens. This reveals exactly how society simplifies complex problems, what specific pieces of information they focus on, and how to design safer, clearer, and more supportive systems that resemble real human behavior.
Editor's Notes:
- This article was edited by a Neuroscience News editor.
- The journal paper is fully revised.
- Additional content added by our staff.
About this AI and decision making research issues
Author: Benjamin Griebe
Source: TUD
Contact person: Benjamin Griebe – TUD
Image: Image posted in Neuroscience News
Actual research: Open access.
“Large-scale linguistic models accurately identify decision reasons in verbal reports” by Dirk U. Wulff, Kamil Fuławka, Ralph Hertwig. PNAS
DOI:10.1073/pnas.2526798123
Abstract
Large language models accurately identify decision reasons in verbal reports
Understanding the reasons behind human choice under risk is a primary goal of decision scientists, but conventional approaches that rely on behavioral data are limited by strong assumptions of invariance. We present an analytical framework using large-scale linguistic models (LLMs) to analyze verbal reports and identify stated reasons for choosing between lottery tickets.
The validated LLM accurately identified the pre-defined decision reasons in the participants' free-text reports, matching their actual choices in 95% of trials. Our analysis reveals that the reasons that lead to people's decisions differ systematically and are driven more by the structure of the choice problem than by individual differences.
Most importantly, the reasons identified in the oral reports present more objective and informative representations of the decision processes compared to those derived from the selection alone; in addition, problem-specific causal profiles achieve out-of-sample predictive accuracy that rivals established computer models.
This work demonstrates that verbal reports are a rich data source and our analytical framework can unlock their potential, yield results that challenge the field's basic invariance assumptions and pave the way for context-sensitive and more interpretable models of human decision-making.
