A Rasch Analysis of the Self-determination, Purpose, Identity, and Engagement in Science (SPIRES) Survey: Instrument Validation and Recommendations

Abstract

The purpose of the present study was to perform a cross-validation of an existing measure of science students’ motivational traits using the Rasch modeling approach. The validity of the Self-determination, Purpose, Identity, and Engagement in Science (SPIRES) survey was originally investigated using factor analysis, but a secondary validation of this instrument has not yet been published. This is a recommended practice when using a psychometric instrument within a new context or with a different student population. In this validity study, we took a Rasch modeling approach instead of factor analysis because, unlike factor analysis, Rasch modeling is sample-independent. The original factor analysis validation of the SPIRES suggested the survey is composed of three larger ideas or constructs, while our Rasch modeling results suggest there are four constructs. Since our Rasch analyses were sample-independent, we conclude that the SPIRES survey is a four construct survey and may be treated as such across educational contexts without further need to validate using factor analysis, which could continuously produce inconsistent results. These results provide the basis for a validity argument for researchers using the SPIRES in their work. Our work also demonstrates an advantage to using a Rasch modeling approach over factor analysis for instrument validation.