Treating Rapid Responses as Incorrect for Non-Timed Formative Tests

Document Type

Article

Publication Date

1-1-2019

Publication Title

Open Education Studies

Volume

1

Issue

1

First page number:

56

Last page number:

72

Abstract

When students respond rapidly to an item during an assessment, it suggests that they may have guessed. Guessing adds error to ability estimates. Treating rapid responses as incorrect answers increases the accuracy of ability estimates for timed high-stakes summative tests like the ACT. There are fewer reasons to guess rapidly in non-timed formative tests, like those used as part of many personalized learning systems. Data from approximately 75 thousand formative assessments, from 777 students at two northern California charter high schools, were analyzed. The accuracy of ability estimates is only slightly improved by treating responses made in less than five seconds as incorrect responses. Simulations show that the advantage is related to: whether guesses are made rapidly, the amount of time required for thoughtful responses, the number of response alternatives, and the preponderance of guessing. An R function is presented to implement this procedure. Consequences of using this procedure are discussed.

Keywords

Response times; Ability; Personalized learning

Disciplines

Education | Educational Psychology

Language

English

UNLV article access

Search your library

Share

COinS