Learning to Integrate Parts for Whole Through Correlated Neural Variability

Neural activity in the cortex exhibits a wide range of firing variability and rich correlation structures. Studies on neural coding indicate that correlated neural variability can influence the quality of neural codes, either beneficially or adversely. However, the mechanisms by which correlated neural variability is transformed and processed across neural populations to achieve meaningful computation remain largely unclear. Here we propose a theory of covariance computation with spiking neurons which offers a unifying perspective on neural representation and computation with correlated noise. We employ a recently proposed computational framework known as the moment neural network to resolve the nonlinear coupling of correlated neural variability with a task-driven approach to constructing neural network models for performing covariance-based perceptual tasks. In particular, we demonstrate how perceptual information initially encoded entirely within the covariance of upstream neurons’ spiking activity can be passed, in a near-lossless manner, to the mean firing rate of downstream neurons, which in turn can be used to inform inference. The proposed theory of covariance computation addresses an important question of how the brain extracts perceptual information from noisy sensory stimuli to generate a stable perceptual whole and indicates a more direct role that correlated variability plays in cortical information processing.