Transformer neural network to predict and interpret pregnancy loss from activity data in Holstein dairy cows

Predicting/detecting pregnancy loss of dairy cows offers the opportunity to shorten the time interval between artificial inseminations. Although several methods of pregnancy detection are being practiced, models with accurate, timely and interpretable detection of pregnancy are still lacking. This study proposed a transformer neural network to predict the probability of pregnancy loss based on continuous activity data, which were collected from activity-monitoring tags attached to 185 Holstein cows from a commercial dairy farm in Cayuga County, NY, USA. Our best model achieved an average accuracy of 0.87, F1 score of 0.87, recall of 0.87 and specificity of 0.90 using 14-day time-series activity windows (90% overlap) using 5-fold cross-validation, out-performing commonly used classic statistical learning and deep learning models for time-series data. The results indicated that our predictive model gave high probabilities of correctly detecting pregnancy loss prior to the increased activities and veterinary confirmation by transrectal ultrasound. In addition, our model interpretation aligned with the changes in the temporal activity levels, revealing that drastic fluctuations in time-series activity data contributed heavily to the final prediction. To the best of our knowledge, this is the first work on developing transformer models for the prediction of pregnancy loss in dairy cows. In addition to facilitating the development of future precision management on modern farms, our work potentiates an increase in the reproductive efficiency and profitability of dairy farms.