Accurate brain age prediction with lightweight deep neural networks

•A lightweight deep learning model, Simple Fully Convolutional Network (SFCN), is presented, achieving state-of-the-art brain age prediction and sex classification performance in UK Biobank MRI brain imaging data.•Even with limited number of training subjects (e.g., 50), SFCN performs better than widely-used regression models.•A semi-multimodal ensemble strategy is proposed and achieved first place in the PAC 2019 brain age prediction challenge.•Linear regression can remove brain age prediction bias (even on unlabelled data) while maintaining state-of-the-art performance.