The “Pliability” of Criminological Analyses: Assessing Bias in Regression Estimates Using Monte Carlo Simulations

Document Type

Article

Publication Date

11-14-2018

Publication Title

Journal of Quantitative Criminology

First page number:

1

Last page number:

24

Abstract

Objectives When biased coefficient and standard error estimates are published, they can result in inaccurate findings which might motivate ineffective—or harmful—policy choices and reduce the legitimacy of social scientific research. In this paper, we demonstrate how Monte Carlo simulations (MCS) can be used to evaluate potential bias in estimates. Methods We define estimation bias and provide an overview of MCS, which involves three steps. First, data are generated according to model parameters and assumptions. Second, the data are analyzed and estimates are saved. Third, these two steps are repeated 2500 times to yield a distribution of estimates to compare with the original estimates. In our first example of using MCS to evaluate potential bias, we use data from a previous project to estimate an OLS regression model and then assess the consistency of estimates. In our second example, we evaluate published regression model estimates. In the third example, we employ experimental methodology with MCS to show how a correlation estimate would vary if there was a moderating effect of a third variable. Results Standard error estimates in the first example exhibited severe bias due to violated assumptions. The second example showed model estimates were largely unbiased. The third example showed that the strength of a moderating effect is positively related to correlation estimate bias. Conclusions Although MCS have been increasingly used by criminologists, they could be used by a broader body of researchers, instructors, and policymakers to assess and ensure the reliability of reported findings.

Keywords

Monte Carlo simulation; Estimation bias; Regression; Replication

Disciplines

Criminology and Criminal Justice

Language

English

UNLV article access

Search your library

Share

COinS