news

293 authors tested GPT-4 to write short stories and found that AI had almost no effect on the writing of expert authors

2024-07-15

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina


Generative AI models have made it easier and faster to generate text, images, video, and audio. Text and media that might have taken humans years to create can now be generated in seconds.

While AI outputs appear creative, can these models actually enhance human creativity?

A new paper published in Science Advances by two researchers explores this question, examining how people write short stories using OpenAI’s large language model, GPT-4.

The conclusion was that the model was helpful, but only to a point. They found that while the AI ​​improved the quality of work by writers with poor ideas, it struggled to help writers who already had good ideas and had little impact on the quality of the stories they produced.

At the same time, stories involving artificial intelligence are more similar than stories imagined entirely by humans.

The study joins a growing body of research exploring how generative AI might affect jobs that require creativity, and suggests that while AI tools can improve individual creativity, they can reduce it overall.


(Source: AI generated)

To understand the impact of generative AI on human creativity, we first need to determine how to measure creativity. This study used two metrics: novelty and usefulness.

Novelty refers to the originality of the story, while usefulness, in this case, reflects the likelihood that each short story will develop into a book or other publishable work.

First, the authors recruited 293 people through the research platform Prolific to complete a task designed to measure their inner creativity. Participants were asked to come up with 10 words that were as different as possible.

Next, participants were asked to write an eight-sentence story for a young person on one of three themes: a jungle adventure, an ocean adventure, or an adventure on another planet.

They were then randomly divided into three groups. The first group could only rely on their own ideas, while the second group could choose to get one story inspiration from GPT-4, and the third group could choose to get up to five story inspirations from the AI ​​model.

Among participants who had the option of AI assistance, the vast majority (88.4%) took advantage of it.

Before another group of participants reviewed their work, they were asked to rate the creativity of their stories. Each reviewer read six stories and was asked to give feedback on the stories’ stylistic features, novelty, and usefulness.

The researchers found that writers who used the AI ​​model the most were considered the most creative. Writers who scored lower on the first test benefited the most.

However, stories written by already creative writers did not receive the same boost.

"We saw this 'levelling effect' where the least creative writers got the biggest gains, but we didn't see any benefit for already creative people," said Anil Doshi, an assistant professor at the University College London School of Management and co-author of the paper.

Tuhin Chakrabarty, a computer science researcher at Columbia University who studies artificial intelligence and creativity but was not involved in the study, said:

He said the findings make sense given that already creative people don’t need to use AI to be creative.

There are some potential downsides to using AI models for help. Chakraborty said AI-generated stories are similar in both semantics and content, and AI-generated content is easy to spot, such as long sentences full of stereotypes.

“These characteristics may also reduce overall creativity,” he said. “Good writing is about showing, not telling, and AI is always telling.”

Because the stories generated by AI models can only be extracted from the data on which these models were trained, the stories produced in the study are less unique than if human participants had come up with their own ideas.

If the publishing industry adopts generative AI on a large scale, the books we read may become monotonous because they are all generated by models trained on the same corpus.

Oliver Hauser, a professor at the University of Exeter Business School in the United Kingdom, is another co-author of the study.

He said that as we grapple with what rapidly evolving technology means for society and the economy, there is even more reason to study what AI models can and cannot do.

“Just because technology can be transformative doesn’t mean it will be transformative in the future,” he said.

Support: Ren

Operation/Layout: He Chenlong

01/ The Hong Kong City University team developed a new nano-layered membrane that can be used for freshwater treatment in special scenarios, finding a breakthrough in the application of two-dimensional materials

02/ A credible answer to a decades-old chemical problem: Scientists propose a new microscopic mechanism for the dissolution of hydrogen chloride to form hydrochloric acid, which will promote the development of multiple disciplines

03/ Scientists create a new method for quantum sensing and manipulation that can accurately detect weak signals and can be used to detect and manipulate single nuclear spins

04/ The new batch of "35 Tech Innovators Under 35" selected by MIT Technology Review in China is officially released! Witness the innovative power of young scientists in Shanghai

05/ With a dynamic strength of 14GPa, the Peking University team successfully developed ultra-strong carbon nanotube fibers that can be used as lightweight, high-performance structural and protective materials