Research Synthesis Methods
Meta-analysis multiplicity, the concept of conducting multiple tests of statistical significance within one study, is an underdeveloped literature (Tendal, Nüesch, Higgins, Jüni, & Gøtzsche, 2011). We address this issue by considering how Type I errors can impact meta-analytic results, suggest how statistical power may be affected through the use of multiplicity corrections, and propose how meta-analysts should analyze multiple tests of statistical significance. The context for this study is a meta-review of meta-analyses published in two leading review journals in education and psychology. Our review of 130 meta-analyses revealed a strong reliance on statistical significance testing without considering of Type I errors or the use of multiplicity corrections. In order to provide valid conclusions, meta-analysts must consider these issues prior to conducting the study.
Pigott, Terri D. and Polanin, Joshua R.. The Use of Meta-Analytic Statistical Significance Testing. Research Synthesis Methods, 6, 1: 63-73, 2015. Retrieved from Loyola eCommons, Education: School of Education Faculty Publications and Other Works, http://dx.doi.org/10.1002/jrsm.1124
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 License.
© 2014 John Wiley & Sons, Ltd.