Times Higher Education reveals the machines are moving beyond physical work and simple sorting, and can now fool academic gatekeepers with their composition skills… in article reviews, at least:
Using automatic text generation software, computer scientists at Italy’s University of Trieste created a series of fake peer reviews of genuine journal papers and asked academics of different levels of seniority to say whether they agreed with their recommendations to accept for publication or not.
In a quarter of cases, academics said they agreed with the fake review’s conclusions, even though they were entirely made up of computer-generated gobbledegook – or, rather, sentences picked at random from a selection of peer reviews taken from subjects as diverse as brain science, ecology and ornithology.
“Sentences like ‘it would be good if you can also talk about the importance of establishing some good shared benchmarks’ or ‘it would be useful to identify key assumptions in the modelling’ are probably well suited to almost any review,” explained Eric Medvet, assistant professor at Trieste’s department of engineering and architecture, who conducted the experiment with colleagues at his university’s Machine Learning Lab.
“If, by chance, a generated review combines sentences which are not too specific, but credible, the review itself may appear as written by a real, human reviewer even to the eyes of an experienced reader,” added Dr Medvet, whose paper, “Your Paper has been Accepted, Rejected, or Whatever: Automatic Generation of Scientific Paper Reviews”, was published in the journal Lecture Notes in Computer Science last month.
Mixing the fake reviews with real reviews was also likely to distort decisions made by academics by making weak papers appear far stronger thanks to a series of glowing reviews, the paper found.
…
With nearly 1,000 so-called “predatory publishers” seeking pay-to-publish journal papers, automatically generated reviews may make it easier for bogus papers to gain credibility, he added.