Skip to Main Content
It looks like you're using Internet Explorer 11 or older. This website works best with modern browsers such as the latest versions of Chrome, Firefox, Safari, and Edge. If you continue with this browser, you may see unexpected results.

ACRL/EBSS Psychology Committee

Definitions

Reproducibility

"The ability of a researcher to duplicate the results of a prior study using the same materials and procedures as were used by the original investigator. So in an attempt to reproduce a published statistical analysis, a second researcher might use the same raw data to build the same analysis files and implement the same statistical analysis to determine whether they yield the same results. " (Bollen et al., 2015)

Replicability

“The ability of a researcher to duplicate the results of a prior study if the same procedures are followed but new data are collected.” (Bollen et al., 2015)

Reference

Bollen, K., Cacioppo, J. T., Kaplan, R. M., Krosnick, J. A., Olds, J. L., & Dean, H. (2015). Social, behavioral, and economic sciences perspectives on robust and reliable science. Report of the Subcommittee on Replicability in Science Advisory Committee to the National Science Foundation Directorate for Social, Behavioral, and Economic Sciences. https://nsf.gov/sbe/AC_Materials/SBE_Robust_and_Reliable_Research_Report.pdf 

Questionable Practices and Guidelines for Best Practices

Common Questionable Practices
  • Failure to report all dependent measures
  • Collecting more data after seeing whether results were significant
  • Failing to report all conditions
  • Stopping data collection after achieving the desired results
  • Rounding down p-values
  • Selectively reporting studies “that worked”
  • Excluding data after looking at the impact of doing so
  • Claiming to have predicted an unexpected findings
  • Falsely claiming that results are unaffected by demographics
  • Falsifying data

Reference: 

John, L. K., Loewenstein, G., & Prelec, D. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23(5), 524–532. https://doi.org/10.1177/0956797611430953 

Best Practice Guidelines

Supporting Reproducibility

What Librarians Can Do to Support Reproducibility