Summary: Developing a novel automated evaluation framework called CASPER, the researchers analyzed thousands of human-written and machine-generated stories across eight different axes of literary theory. The findings clearly show that AI models systematically strip away one of the defining characteristics of memorable fiction: mystery.
While human writers tend to embrace narrative ambiguity, leave deeper questions unanswered, and allow characters to constantly debate each other, AI models similarly “play it safe.” They rely too much on flat, predictable archetypes and force storylines into artificial, well-structured resolutions.
Important Facts
- Safe Resolution Bias: Lead author Anneliese Brei notes that AI systems have an inherent statistical bias to wrap up issues neatly. They violently resolve internal conflicts, answer every mystery, and make sure the characters fit right into their chosen story arcs on the final page.
- Illusion of Scale: A critical revelation of the study is that increasing the parameter size does not solve the problem. Larger, more sophisticated LLMs produced characters as flat and archetypal as those produced by smaller, less complex models. The deficit is based on how the models understand the conversation, not the processing power.
- Unresolved Acceptance: When analyzing human authors, the CASPER framework reveals a high level of comfort with chaos. Fiction often leaves characters unresolved, morally gray, or fundamentally open to interpretation, a structural ambiguity that makes the story stick with the reader.
- Exploring Character Evolution: The study systematically mapped the character's behavior against the eight main dimensions of writing theory, analyzing the precise transition from exaggerated caricatures to real people, along with tracking whether the characters really change or just follow the script.
- CASPER Benchmark: Beyond exposing creative limitations, CASPER serves as an important, standardized measurement framework. It enables AI developers and creative studios to test whether future, next-generation models are truly developing narrative depth and character complexity rather than just grammar.
- Takeaways for the Writing Community: For writers using AI as a brainstorming assistant or co-author, the UNC study offers a clear warning: letting a machine manage character development risks complicating the narrative, making the human touch essential to reintroducing conflict, changing expectations, and intentionally introducing uncertainty.
Source: UNC Chapel Hill
Researchers at the University of North Carolina at Chapel Hill have found that while artificial intelligence can spin compelling stories, its characters can still lack one of the qualities that make human fiction memorable: mystery.
As AI writing tools become more common in publishing and entertainment, Carolina researchers want to understand if the characters created by these programs are as varied and diverse as those created by human writers. Their findings suggest that, despite advances in technology, AI still tends to rely on familiar patterns.
The study examined how characters in AI-generated stories compare to those written by humans. Using ideas from literary theory, the researchers analyzed eight different aspects of character portrayal, including whether characters seem real or exaggerated, whether they change over time and whether they remain mysterious or fully understood at the end of the story.
To do this, the team developed CASPER, an automated framework that evaluated thousands of stories and measured character traits in ways that have never been systematically used in AI-generated fiction.
“We found that AI models tend to 'play it safe' with their characters, in the way that they wrap the headlines neatly,” said Anneliese Brei, a computer science graduate student at UNC-Chapel Hill and lead author of the study.
“On the other hand, human writers like to leave questions unanswered and let characters remain ambiguous. That distinction is important because ambiguity is what makes a story stick with the reader.”
The research comes at a time when AI tools specifically designed for creative writing are gaining traction. Platforms like Sudowrite and Squibler can help with writing novels, while AI is increasingly being used in film and television to generate script outlines and dialogue. The survey also showed that many fiction writers are now incorporating AI into some part of their creative process.
Their analysis revealed that AI-generated characters tend to rely heavily on visual archetypes and tend to come to systematic decisions at the end of the story. Human authors, in contrast, appeared more comfortable allowing characters to remain unresolved, contradictory or open to interpretation.
“One of the most surprising findings is that large and powerful AI models do not create different characters than small ones,” said Nicholas Sanaie, a computer science graduate student at Carolina and co-author of the study.
“That tells us that the challenge is not just about scale. It's about how these models understand how to tell the story themselves.”
CASPER gives researchers, developers and creative professionals a way to measure whether new AI programs are actually improving the expression of complex characters rather than just being fluent writers. It may also guide the development of future storytelling tools that better support creativity and narrative depth.
“As more and more people interact with AI to write novels, screenplays and other creative works, we need ways to understand both what these systems do well and where they fall short,” said Snigdha Chaturvedi, associate professor of computer science at UNC-Chapel Hill and senior author of the study.
“CASPER gives us a lens to explore the depth and diversity of characters, which can ultimately help developers create conversational programs that better reflect the complexity of the human experience.”
For writers experimenting with AI, the findings offer practical implications: AI may be a capable creative partner, but compelling stories may still require a clear human willingness to accept uncertainty, to contend with characters who don't fit well into conventional boxes.
Important Questions Answered:
A: This preference for “purity” is built right into the way large language models are trained. AI models are trained on statistical probabilities to predict the next most satisfying, meaningful word based on vast mountains of internet data. When constructing a story, the model naturally prepares for the most probable paths, which means that it relies on predictable structures and systematic decisions. It is designed to provide answers, not to dwell on discomfort. Human life, however, is full of loose ends and contradictions, qualities that human authors deliberately capture, but AI views as mathematical anomalies to be resolved.
A: CASPER is an automated computational linguistics framework designed by the UNC team to transform abstract text theory into measurable data. It analyzes thousands of blocks of text, tracking how characters are described and how their actions unfold from the beginning to the end of the narrative across eight specific dimensions. Measuring “mystery” or ambiguity, CASPER examines if the character's inner motives are completely explained by the narrator at the climax, or if their behavior remains poorly measured, contradictory, and open to multiple interpretations, measuring the invisible factors that distinguish flat caricatures from unforgettable literary images.
A: Not at all. Researchers view AI as an incredibly capable creative partner, but one that needs a firm, human hand at the wheel. AI is great at helping writers think through plot points, cure blank-page syndrome, or accomplish simple background explanations quickly. The real lesson here is that you cannot delegate the soul of character development in the machine. If a novelist lets an AI write their main characters unchecked, those characters will turn into flat, safe clichés. The real magic of storytelling still requires a human writer willing to go in and deliberately screw up the code by adding intractable errors, messy contradictions, and a healthy dose of genuine mystery.
Editor's Notes:
- This article was edited by a Neuroscience News editor.
- The journal paper is fully revised.
- Additional content added by our staff.
About this AI and innovation research issues
Author: Gabriella Neyman
Source: University of North Carolina at Chapel Hill
Contact person: Gabriella Neyman – University of North Carolina at Chapel Hill
Image: Image posted in Neuroscience News
