Irreproducible Examples

Notes on examples of irreproducible research that has caused significant issues. Some of these will become case studies during the course. Initially, these are all from RetractionWatch.

Papers

• Editorial Expression of Concern: Non-adaptive origins of interactome complexity

• Retraction Watch Univ.: No misconduct, but “poor research practice” in mgt prof’s work now subject to 7 retractions

  • "The investigation was severely hindered by the near total unavailability of raw data files and statistical output files for the seven papers under study."

• Retraction Watch Data questions prompt retraction of PLOS ONE cardiovascular paper

  • The authors have indicated that the raw data for Figures 3, 5 and 6 are not available to allow for a full evaluation of the concerns noted.

• MS paper second to fall following University of Queensland investigation

  • "no primary data can be located, and no evidence has been found that the study described in the article was conducted"

• Doing the right thing: Psychology researchers retract after realizing data “were not analyzed properly”

  • "The errors were detected when other lab members reanalyzed the data for another purpose. At that point, it turned out that the original data analyses by the first author included several operations which are hard to replicate and which do not fit fully with the methods as agreed upon with the co-authors and as described in the paper."

• Seizure study retracted after authors realize data got “terribly mixed”

• Diabetes researchers retract, correct and republish study on mortality rates - maybe an example of how things can be fixed.

• Bad spreadsheet merge kills depression paper, quick fix resurrects it Fix appears to invert the conclusion.

• Spreadsheet error forces loss of Appetite paper on obesity and parenting styles

  • "tracked down the source of the error to a column switching mistake in copying data from one spreadsheet to another by a research assistant"

• “Extensive” errors force retraction of lymphoma radiation paper

  • "They could not rerun and correct the analyses because the raw data was held elsewhere and there was no funding to pay for a statistician anyway."

Related to Software

Related to poor software installation:

• Does “the computer ate my homework” explain retraction of higher ed paper?
  • "the author discovered that there were installation errors with the software used to run the factor analysis in the paper."

Related to software upgrade:

• Two detailed retraction notices correct the cardiology record
  • "Excel software was used to calculate the p-values. On recalculation using a newer version of the programme, the values are coming back different: Prevalence of MVO comparing adenosine to non-adenosine is now 0.15, therefore non-significant. Also the P-value for the mass of MVO in adenosine versus non-adenosine is 0.34, again non-significant."

Examples of retractions that are made possible by good reproducibility and retention practices

• mBio retracts anthrax paper whose authors say they misinterpreted findings
• Authors retract PNAS brain genetics paper for statistical issues

Retraction Watch Categories of interest

• Retraction Watch Archive for the ‘not reproducible’ Category
• Retraction Watch Archive for the ‘erroneous data’ Category
• Retraction Watch Archive for the ‘unreliable findings’ Category
• Retraction Watch Archive for the ‘investigator error’ Category
• Retraction Watch Archive for the ‘studies about retractions’ Category