Internal Consistency and Power When Comparing Total Scores from Two Groups

Document Type

Article

Publication Date

1-1-2016

Publication Title

Multivariate Behavioral Research

Volume

51

Issue

4

First page number:

482

Last page number:

494

Abstract

Researchers now know that when theoretical reliability increases, power can increase, decrease, or stay the same. However, no analytic research has examined the relationship of power to the most commonly used type of reliability—internal consistency—and the most commonly used measures of internal consistency, coefficient alpha and ICC(A,k). We examine the relationship between the power of independent samples t tests and internal consistency. We explicate the mathematical model upon which researchers usually calculate internal consistency, one in which total scores are calculated as the sum of observed scores on K measures. Using this model, we derive a new formula for effect size to show that power and internal consistency are influenced by many of the same parameters but not always in the same direction. Changing an experiment in one way (e.g., lengthening the measure) is likely to influence multiple parameters simultaneously; thus, there are no simple relationships between such changes and internal consistency or power. If researchers revise measures to increase internal consistency, this might not increase power. To increase power, researchers should increase sample size, select measures that assess areas where group differences are largest, and use more powerful statistical procedures (e.g., ANCOVA). © 2016 Taylor & Francis Group, LLC.

Keywords

Coefficient alpha; internal consistency; intraclass correlation; power; reliability

Language

English

Find It

UNLV article access

Search your library

Share

COinS