Reliability Generalization Meta-analyses in Mathematics Education Research: A Research Synthesis

Abstract

The purpose of this systematic review was to characterize the implementation of reliability generalization meta-analytic (RGM) practices within mathematics education-related empirical research. RGM studies are used to investigate and generalize the reliability of a measure across various studies. An exhaustive literature search was conducted to locate studies related to mathematics education, including RGM studies of psychological tests. The literature search included articles as well as grey literature (e.g., conference proceedings, dissertations, theses). Of the 27 RGM studies examined, five were on scales that related to mathematics education research, five were on scales related to motivation and/or learning, four related to self-esteem, self-concept, and/or self-efficacy, six related to perceptions, well-being, and/or anxiety, and seven related to personality or behavior. Of the mathematics education-related RGM studies, 85.5% (N=9,184) of the articles examined across studies had no mention of reliability or fell into the convention of citing previously reported reliabilities. Increasing awareness of RGM studies could lead to an increase in RGM studies conducted on mathematics education research scales, leading to increased understanding of mathematics education scales. This paper contributes to the literature on the practical and empirical importance of RGM for mathematics and STEM education praxis.