Gain Scores, ANCOVA, and Propensity Matching Procedures for Evaluating Treatments in Education

Document Type

Article

Publication Date

7-4-2020

Publication Title

Open Education Studies

Volume

2

Issue

1

First page number:

45

Last page number:

65

Abstract

Researchers have several options available to analyze data from interventions when participants have not been randomly allocated into conditions. Among these are the gain score, ANCOVA, and propensity matching procedures. Each of these attempts to account for pre-treatment differences among the conditions, but they do so differently. These procedures are reviewed and methods for estimating them in R are shown. The choice of which of these procedures to use can be diffcult. Different situations are shown where they perform differently. The primary conclusion of this paper is that models should be hypothesized for how the data may arise, data simulated for these models, and the properties of statistical procedures evaluated. A goal of this paper is to show these procedures without extensive mathematics in order to allow a broad readership to use these methods in their situations.

Keywords

Propensity matching; ANCOVA; Gain scores; Lord's paradox; Causality; Graphs; Simulation

Disciplines

Education | Educational Assessment, Evaluation, and Research

Language

English

UNLV article access

Search your library

Share

COinS