Science has an AI credibility problem

Artificial intelligence is making scientific research easier to produce, but not necessarily easier to trust.

An October 2026 study in the peer-reviewed journal Scientometrics examined more than 234,000 research papers and estimated that AI-assisted writing rose from 9.8% in 2024 to 23.5% in 2025. While such assistance is not inherently problematic, AI can also fabricate citations, scientific images and experimental data, producing convincing research without genuine evidence.

“AI has mostly made a lot of existing problems much worse,” Ivan Oransky, a science journalist and co-founder of the research-integrity watchdog Retraction Watch, told The Beiruter. 

It's absolutely flooding the scientific literature, and journals and publishers are struggling to keep up.

The stakes are particularly high when unreliable findings enter the evidence used to guide medical treatments, inform government policy, or support subsequent scientific discoveries. Once published, questionable research can acquire an authority that proves difficult to reverse, even when its underlying evidence is eventually discredited.

When scientific evidence becomes artificial

Scientific images and experimental datasets once carried telltale signs of manipulation. Generative AI is making those signs considerably harder to recognize.

“The more consequential development is the growing sophistication of fabricated evidence,” Elisabeth Bik, a microbiologist and research-integrity investigator, told The Beiruter. 

AI-generated research has become so sophisticated that we can no longer reliably detect it. We don't know what we don't know, making it impossible to determine the true scale of the problem.

Unlike earlier forms of manipulation, generative AI can produce realistic scientific images and statistically convincing synthetic datasets.

“If a human tries to fabricate data, they generally won't do a very good job statistically,” Bik explained. 

But generative AI can produce very large datasets that look remarkably realistic, making them extremely difficult to detect.

While fabricated experimental data can be exceptionally difficult to uncover, inconsistencies in scientific references can leave more readily identifiable traces.

Michal Wojcik, a researcher investigating problematic scientific references, told The Beiruter he had flagged approximately 2,100 articles with reference problems over the course of a year.

“There are certain combinations of mismatched references that lead me to believe an article is AI-generated, or that its references were modified using a large language model,” Wojcik said.

If even nonexistent sources can survive editorial scrutiny, more sophisticated forms of fabricated evidence may prove considerably harder to identify.

The economics of publishing more

Academic incentives have long rewarded publication volume, with hiring, promotion, and research funding often tied to publication records and citation counts. These pressures have encouraged paper mills, commercial operations that manufacture scientific manuscripts and sell authorship.

Commercial publishing introduces another set of incentives. Many open-access journals charge authors publication fees, creating financial incentives to accept more manuscripts, potentially at the expense of editorial scrutiny.

“Publishers are essentially following the money,” Oransky said. 

They're responding to, and I would argue aiding and abetting, academia's publish-or-perish incentives. A “publish or perish” mentality was the kindling for the problems in scientific literature, and now AI is the accelerant.

The scale of the problem became apparent in October 2026, when the Institute of Electrical and Electronics Engineers, or IEEE, retracted 1,417 conference papers in a single day following an investigation into suspected paper-mill activity.

An analysis published in Nature identified 2,062 suspect papers across 366 IEEE conference proceedings from 2021 to 2026, of which 1,622 had subsequently been retracted.

The consequences are particularly significant for international researchers whose careers depend on publishing in recognized journals. Publication requirements can influence hiring and promotion across universities in Asia, Africa, Europe and the Middle East, while publication fees impose unequal burdens on institutions with vastly different financial resources.

Wojcik argues that publishers are failing to make use of existing safeguards.

“We would benefit far more from implementing the tools we already have, such as checking for duplicated images, verifying that cited studies actually exist, and identifying conflicts of interest,”  he said.

Such checks cannot establish whether an experiment genuinely occurred, but they can identify verifiable errors before publication. The concern is that detecting these problems requires resources and editorial attention at precisely the moment publishers face pressure to process more manuscripts.

Can artificial intelligence police itself?

As generative AI complicates the verification of scientific research, publishers are turning to the same technology to strengthen their review processes.

In October 2026, Springer Nature partnered with Google DeepMind to develop an AI-assisted peer-review system using Gemini models to examine manuscripts, figures and potential areas of concern. Such tools could ease the burden of peer review but introduce new uncertainties about the reliability of automated assessments.

AI has already proved useful in certain areas of research-integrity investigations. Bik explained she uses tools including ImageTwin and Proofig to detect duplicated or manipulated scientific images, even when figures have been rotated or altered.

“There is another side to the technology,” Bik said. 

AI has also been very effective in helping detect certain types of fraud.

Yet software designed to identify duplicated images may fail to detect entirely fabricated scientific figures.

Wojcik also cautioned that while automated systems can detect inconsistencies, assessing experimental methods and distinguishing credible findings from pseudoscience still require human judgment.

Bik suggested that stronger authentication could offer an alternative to relying primarily on detection. Researchers might be required to provide additional documentation confirming that studies were conducted, patients existed and experimental protocols were registered before results were obtained.

Such requirements could, however, place additional burdens on smaller research institutions without necessarily preventing deliberate misconduct. Even if new technologies allow publishers to identify suspicious material more effectively, the capacity to investigate allegations, correct published research and withdraw unreliable findings remains dependent on human expertise and institutional resources.

The credibility of the scientific record

The consequences of unreliable research do not necessarily end when a paper is withdrawn. Published findings can circulate through academic citations, clinical discussions and policy recommendations long before their validity is questioned, leaving errors embedded in subsequent research.

Retractions can correct the scientific record, but their effectiveness depends on timely investigations and clear communication from journals.

Yet the growing visibility of scientific misconduct presents another difficulty. Publicizing unreliable research can strengthen accountability, but it can also provide ammunition to those seeking to discredit scientific institutions altogether. For Oransky, acknowledging mistakes, investigating misconduct and explaining corrections are essential to demonstrating that scientific knowledge remains subject to scrutiny.

“Whenever people try to hold larger organizations or institutions accountable, others with their own motivations for denigrating those institutions are going to latch onto that work,” Oransky said.

But those of us working in scientific reform and integrity have been arguing that we should be talking about these problems more, not less. We need more people holding institutions accountable.

Scientific credibility has never depended on the complete absence of error. It depends on the capacity to distinguish credible evidence from unreliable claims and to correct the record when that distinction fails. As AI accelerates the production of scientific literature, maintaining that capacity may require changes not only to the technologies used to review research, but also to the institutions and incentives governing its publication.