IEEE Access Rejects Two AI-Generated Papers Before Peer Review

IEEE Access rejected two AI-generated scientific papers submitted by writer Benjamin Skuse before either reached peer review, according to his two-round test conducted with IEEE’s cooperation. Skuse submitted the papers under his own name to test whether the journal’s safeguards could identify research produced with consumer chatbots and an autonomous research system.

The first submission was stopped after automated integrity checks exposed widespread reference problems. The second used real references but relied heavily on non-peer-reviewed preprints, which the journal’s editors said was enough to reject it. Manual review also found nonsensical details in figures that the automated tools had missed.

For the first paper, Skuse used free versions of ChatGPT and Gemini to create a study about optical solitons, his former Ph.D. field. He prompted the systems section by section, switched between them when one refused or stalled, and had them generate the text, figures, captions, citations and IEEE Access formatting. The resulting paper described research that had never taken place.

IEEE Access managing editor Jenny Mahoney ran the submission through the STM Integrity Hub, which identified 12 references that could not be verified. A separate checker from MolecularConnections could not validate 24 references, including one attributed to a sole author named “John Smith.” Mahoney, an experimental physicist working outside the paper’s subject area, said nothing seemed majorly scientifically wrong to her, but the volume of hallucinated references cast doubt on the entire submission.

Skuse raised the difficulty in the second round by using FARS, or Fully Automated Research System, developed by Singapore-based startup Analemma. FARS produced a research proposal in 21 minutes, ran a computational experiment and generated a full paper about two hours after Skuse selected the option to write it. He removed the system’s disclosure notice and replaced it with his name before submitting the paper.

Unlike the first submission, the FARS paper did not contain fabricated references. Mahoney said IEEE Access’s team would have rejected it because a large share of its citations pointed to preprints that had not undergone peer review. The journal’s automated integrity tools did not flag problems in the figures, but Mahoney found a technical diagram with a line labeled “lhewl” and another illustration of an AI brain containing random dots.

Both papers were rejected before peer review, and their scientific claims were not independently validated.