Gain Scores, ANCOVA, and Propensity Matching Procedures for Evaluating Treatments in Education
Open Education Studies
First page number:
Last page number:
Researchers have several options available to analyze data from interventions when participants have not been randomly allocated into conditions. Among these are the gain score, ANCOVA, and propensity matching procedures. Each of these attempts to account for pre-treatment differences among the conditions, but they do so differently. These procedures are reviewed and methods for estimating them in R are shown. The choice of which of these procedures to use can be diffcult. Different situations are shown where they perform differently. The primary conclusion of this paper is that models should be hypothesized for how the data may arise, data simulated for these models, and the properties of statistical procedures evaluated. A goal of this paper is to show these procedures without extensive mathematics in order to allow a broad readership to use these methods in their situations.
Propensity matching; ANCOVA; Gain scores; Lord's paradox; Causality; Graphs; Simulation
Education | Educational Assessment, Evaluation, and Research
Wright, D. B.
Gain Scores, ANCOVA, and Propensity Matching Procedures for Evaluating Treatments in Education.
Open Education Studies, 2(1),