What meta-analyses reveal about the replicability of psychological research.

Item request has been placed! ×
Item request cannot be made. ×
loading   Processing Request
Read More Add to Saved list
  • Additional Information
    • Source:
      Publisher: American Psychological Association Country of Publication: United States NLM ID: 0376473 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1939-1455 (Electronic) Linking ISSN: 00332909 NLM ISO Abbreviation: Psychol Bull Subsets: MEDLINE
    • Publication Information:
      Original Publication: Washington [etc.] : American Psychological Association
    • Subject Terms:
      Data Interpretation, Statistical* ; Meta-Analysis as Topic* ; Publication Bias* ; Reproducibility of Results*; Behavioral Research/*standards ; Psychology/*standards ; Research Design/*standards; Behavioral Research/statistics & numerical data ; Humans ; Psychology/statistics & numerical data ; Research Design/statistics & numerical data
    • Abstract:
      Can recent failures to replicate psychological research be explained by typical magnitudes of statistical power, bias or heterogeneity? A large survey of 12,065 estimated effect sizes from 200 meta-analyses and nearly 8,000 papers is used to assess these key dimensions of replicability. First, our survey finds that psychological research is, on average, afflicted with low statistical power. The median of median power across these 200 areas of research is about 36%, and only about 8% of studies have adequate power (using Cohen's 80% convention). Second, the median proportion of the observed variation among reported effect sizes attributed to heterogeneity is 74% (I2). Heterogeneity of this magnitude makes it unlikely that the typical psychological study can be closely replicated when replication is defined as study-level null hypothesis significance testing. Third, the good news is that we find only a small amount of average residual reporting bias, allaying some of the often-expressed concerns about the reach of publication bias and questionable research practices. Nonetheless, the low power and high heterogeneity that our survey finds fully explain recent difficulties to replicate highly regarded psychological studies and reveal challenges for scientific progress in psychology. (PsycINFO Database Record (c) 2018 APA, all rights reserved).
    • Comments:
      Erratum in: Psychol Bull. 2019 Jul;145(7):764. (PMID: 31192651)
    • Publication Date:
      Date Created: 20181016 Date Completed: 20191014 Latest Revision: 20191014
    • Publication Date:
      20221213
    • Accession Number:
      10.1037/bul0000169
    • Accession Number:
      30321017