Evaluating Models of Visual Working Memory in Change Detection: Discrete-slots or Non-Diagnostic Data?

A long-standing debate in the visual working memory (VWM) literature is whether limits arise from ‘slots’, or a continuous resource. In a seminal study, Rouder and colleagues (2008) tested between these two views using Receiver Operating Characteristics analysis in a change detection task. The authors reported evidence for the discrete-slot model of VWM. These findings are widely cited in support of item-based theories. They have also had a major, long-standing influence on VWM measurement because they are one of the few pieces of formal evidence suggesting that the discrete-slot measure of capacity, ‘K’, is a valid measure. In the current work, we ask: Could these researchers recover the resource model if it were the true data-generating model? Using model recovery analysis, we demonstrate that the answer is ‘probably not.’ We find that metrics of model fit were biased towards the discrete-slot model, which had fewer parameters. To address this, we evaluated a broader scope of models, which were matched on their number of parameters and theoretical assumptions. With this reanalysis, we found that discrete-slot and resource models accommodate the Rouder et al., data equally well, indicating that these data are not diagnostic for discriminating between these models. We applied the same reanalysis to a set of replication studies of Rouder et al. (Donkin et al., 2014) and found comparable support for both models in one study and evidence for the resource model of VWM in a second study. Collectively, these re-analyses indicate that claims of support for discrete-slot models from the Rouder et al., (2008) paper are not justified, nor is the use of ‘K’ in change detection. Furthermore, our work points to the broad importance of formally evaluating the diagnosticity of data and metrics of model fit, as well as matching theoretical assumptions of models when conducting model comparison.