AI-generated scientific research pollutes online academic information ecosystem, worrying report finds published at Harvard Kennedy School Review of misinformation.
A team of researchers studied the prevalence of research articles containing evidence of artificially generated text on Google Scholaran academic search engine that makes it easy to find research published historically in a multitude of academic journals.
The team specifically questioned the misuse of pre-trained generative transformers (or GPTs), a type of large language model (LLM) that includes now-familiar software such as OpenAI’s ChatGPT. These models are able to quickly interpret text inputs and quickly generate responses, in the form of figures, images and long lines of text.
As part of the research, the team analyzed a sample of scientific articles found on Google Scholar showing signs of GPT use. The selected articles contained one or two common expressions that chatbots (usually, chatbots) supported by the use of LLMs. The researchers then studied the extent to which these questionable articles were distributed and hosted on the Internet.
“The risk of what we call ‘evidence hacking’ increases significantly when AI-generated research is disseminated in search engines,” said Björn Ekström, a researcher at the Swedish School of Library and Science. information and co-author of the article. a university in Boras release. “This can have tangible consequences, because incorrect results can permeate further into society and perhaps also into more and more areas. »
According to the recent team, the way Google Scholar retrieves Internet searches does not eliminate articles whose authors have no scientific affiliation or peer review; the engine will extract academic bycatch (student papers, reports, preprints, etc.) as well as research that has passed more scrutiny.
The team found that two-thirds of the articles studied were at least partly produced through undisclosed use of GPTs. Of the articles made by GPT, researchers found that 14.5% were about health, 19.5% were about the environment, and 23% were about IT.
“Most of these GPT-fabricated articles were found in unindexed journals and working papers, but some cases included research published in mainstream scientific journals and conference proceedings,” the team wrote.
The researchers highlighted two main risks induced by this development. “First, the abundance of fabricated studies infiltrating all areas of the research infrastructure threatens to overwhelm the scientific communication system and jeopardize the integrity of the scientific record,” wrote the group. “A second risk is the increased possibility that compelling scientific-appearing content has in fact been deceptively created with AI tools and is also optimized for retrieval by publicly available academic search engines, in especially Google Scholar.”
Because Google Scholar is not an academic database, it is easy for the public to use when searching for scientific literature. It’s good. Unfortunately, it is more difficult for the general public to separate the wheat from the chaff when it comes to reputable journals; even the difference between peer-reviewed research and a working paper can be confusing. Additionally, AI-generated text was found in some peer-reviewed works as well as less-reviewed articles, indicating that GPT-manufactured work is muddying the waters throughout the academic information system online, and not just in work that exists outside of most official channels.
“If we can’t be sure that the research we read is authentic, we risk making decisions based on incorrect information,” said Jutta Haider, co-author of the study and also a researcher at the Swedish School of libraries and information sciences, in the same press release. “But as much as this is a question of scientific misconduct, it is also a question of media and information education. »
In recent years, editors have failed to filter out a handful of scientific articles that were actually completely absurd. In 2021, Springer Nature was forced to retract more than 40 articles in the Arab Journal of Geoscienceswhich, despite the title of the journal, covered varied subjects, including sport, air pollution and children’s medicine. In addition to being off-topic, the articles were poorly written – to the point of being meaningless – and the sentences often lacked a compelling line of thought.
Artificial intelligence is exacerbating the problem. Last February, the publisher Frontiers took the flak for having published an article in his journal Cell And Developmental biology which included images generated by AI software Midjourney; specifically, very Anatomically incorrect images of rat signaling pathways and genitalia. Borders I retracted the paper a few days after its publication.
AI models can be a boon to science; systems can decode fragile texts of the Roman Empire, find Previously unknown Nazca LinesAnd reveal hidden details in dinosaur fossils. But the impact of AI can be as positive or negative as the human using it.
Peer-reviewed journals – and perhaps hosts and search engines for academic writing – need guardrails to ensure that technology works in support of scientific discovery, not against it.