Will making ‘replication studies’ easier to find help science self-correct?

A group of researchers is working to make replication studies more accessible by posting them on the post-publication peer-review platform PubPeer, alongside links to the original studies. This move aims to improve the visibility of such studies, which are crucial for ensuring the credibility of scientific findings. The initiative comes amid growing concerns about the difficulty of locating replication efforts, which can lead to wasted research time and resources. Over the past decade, numerous attempts have been made to replicate scientific experiments, yet many of these have failed to produce the same results as the original studies. Without easy access to these replication studies, scientists risk citing original research without realizing its limitations. According to Don Moore, a social psychologist at the University of California, Berkeley, increasing awareness of both successful and unsuccessful replications could accelerate scientific progress through self-correction. A 2012 study by Moore on overconfidence was recently successfully replicated, highlighting the value of such efforts. The project involves creating discussion threads on PubPeer for approximately 2,400 replication studies, grouped into batches of about 100 each. These studies are indexed in the FORRT Library of Reproduction and Replication Attempts (FLoRA), part of the Framework for Open and Reproducible Research Training (FORRT). FORRT, a community-driven initiative, seeks to educate researchers on open science practices and reproducibility. In 2024, FORRT launched a database to better organize replication studies, though challenges remain in linking these studies to their originals in existing databases. Josefina Weinerova, a psychologist at Birkbeck, University of London, and co-leader of the PubPeer project, noted that replication studies are generally less cited and less visible compared to original studies. Out of the roughly 2,400 studies cataloged, nearly 1,000 have been successfully replicated independently, while 865 replication attempts failed. For the remaining studies, researchers were able to replicate some of the outcomes. Weinerova and her colleagues intend to notify the authors of all the studies for which they have created threads on PubPeer. She mentioned that her team has already contacted the authors of the first batch of studies, although most did not respond. Of those who did, none requested that comments not be posted. PubPeer is frequently used to highlight potential issues or errors in published papers, leading some researchers to complain when their work is mentioned without any identified errors. However, Weinerova emphasized that researchers should welcome attention given to studies that confirm the validity of their work. “Successful replication is still valuable information for other authors,” she stated. While there may be a perception of stigma associated with comments on PubPeer, many of the discussions are not necessarily focused on identifying major flaws in papers. The initiative underscores the ongoing debate within the scientific community regarding transparency and accountability. By making replication studies more accessible, the hope is that researchers will be better informed about the reliability of previous findings, potentially reducing the likelihood of future misinterpretations or misuse of data. As the project progresses, it remains to be seen how effectively it will address the challenges of visibility and citation of replication studies.