A Machine Learning Augmented Data Assimilation Method for High-Resolution Observations

The accuracy of initial conditions is an important driver of the forecast skill of numerical weather prediction models. Increases in the quantity of available measurements, particularly high-resolution remote sensing observational data products from satellites, are valuable inputs for improving those initial condition estimates. However, the traditional data assimilation methods for integrating observations into forecast models are computationally expensive. This makes incorporating dense observations into operational forecast systems challenging, and it is often prohibitively time-consuming. Additionally, high-resolution observations often have correlated observation errors which are difficult to estimate and create problems for assimilation systems. As a result, large quantities of data are discarded and not used for state initialization. Using the Lorenz-96 system for testing, we demonstrate that a simple machine learning method can be trained to assimilate high-resolution data. Using it to do so improves both initial conditions and forecast accuracy. Compared to using the Ensemble Kalman Filter with high-resolution observations ignored, our augmented method has an average root-mean-squared error reduced by 37%. Ensemble forecasts using initial conditions generated by the augmented method are more accurate and reliable at up to 10 days of forecast lead time. Weather forecasts are highly sensitive to the estimate of the current state of the atmosphere, known as initial conditions. The atmosphere is chaotic, meaning that small errors in this estimate can grow quickly as the forecast model predicts events further into the future. The satellite era has contributed to large improvements in weather forecasts by providing additional data that allow for more accurate estimates of initial conditions. However, current methods for generating initial conditions are computationally time-consuming, and as a result, large fractions of available measurements are not used for this purpose. In a proof-of-concept study using a simplified representation of the atmosphere for testing, we train a machine learning (ML) method to replicate the results of a traditional method. Once trained, ML models are usually very fast. Applying the trained model exclusively to measurements that would otherwise be too time-consuming to use produces better initial conditions and more accurate forecasts. Machine Learning (ML) augmented data assimilation (DA) of high-resolution observations improves the analysis in a nonlinear dynamical modelExplainable Artificial Intelligence identifies system covariances to guide neural network training for analysis state reproductionShort-term forecasts from the analysis generated by the ML augmented DA are more accurate and more reliable