Effects of aggregation on implicit bias measurement

Scholars debate the extent to which implicit bias is a stable individual attitude versus a feature of social contexts. The primary evidence for the social context view is that group averages are more stable, and more strongly associated with disparate outcomes, than individual scores. Group averages, however, depend on aggregating many observations, raising the question of whether their apparently superior reliability and validity may be a statistical artefact of aggregation. Would individual difference correlations be as large as context-based correlations if we only aggregated more measures per person? We report two studies testing the effects of aggregating repeated implicit tests. We find that aggregating up to six tests increases test-retest reliability somewhat, but increases validity correlations only slightly, and has no benefit after three tests. Results suggest that large correlations at the level of contexts cannot be reduced to the statistical effects of aggregating multiple tests.