Simulation-based validation of a method to detect changes in SARS-CoV-2 reinfection risk

Background Given the high global seroprevalence of SARS-CoV-2, understanding the risk of reinfection becomes increasingly important. Models developed to track trends in reinfection risk should be robust against possible biases arising from imperfect data observation processes. Objectives We performed simulation-based validation of an existing catalytic model designed to detect changes in the risk of reinfection by SARS-CoV-2. Methods The catalytic model assumes the risk of reinfection is proportional to observed infections. Validation involved using simulated primary infections, consistent with the number of observed infections in South Africa. We then simulated reinfection datasets that incorporated different processes that may bias inference, including imperfect observation and mortality, to assess the performance of the catalytic model. A Bayesian approach was used to fit the model to simulated data, assuming a negative binomial distribution around the expected number of reinfections, and model projections were compared to the simulated data generated using different magnitudes of change in reinfection risk. We assessed the approach’s ability to accurately detect changes in reinfection risk when included in the simulations, as well as the occurrence of false positives when reinfection risk remained constant. Key Findings The model parameters converged in most scenarios leading to model outputs aligning with anticipated outcomes. The model successfully detected changes in the risk of reinfection when such a change was introduced to the data. Low observation probabilities (10%) of both primary- and re-infections resulted in low numbers of observed cases from the simulated data and poor convergence. Limitations The model’s performance was assessed on simulated data representative of the South African SARS-CoV-2 epidemic, reflecting its timing of waves and outbreak magnitude. Model performance under similar scenarios may be different in settings with smaller epidemics (and therefore smaller numbers of reinfections). Conclusions Ensuring model parameter convergence is essential to avoid false-positive detection of shifts in reinfection risk. While the model is robust in most scenarios of imperfect observation and mortality, further simulation-based validation for regions experiencing smaller outbreaks is recommended. Caution must be exercised in directly extrapolating results across different epidemiological contexts without additional validation efforts. ### Competing Interest Statement The authors have declared no competing interest. ### Funding Statement J.R.C.P. and C.v.S. are supported by the South African Department of Science and Innovation and the National Research Foundation. Any opinion, finding, and conclusion or recommendation expressed in this material is that of the authors, and the NRF does not accept any liability in this regard. This work was also supported by the Wellcome Trust (grant no. 221003/Z/20/Z) in collaboration with the Foreign, Commonwealth and Development Office, United Kingdom. Writing assistance was provided by Yuri Munsamy, PhD of SACEMA, South Africa. This assistance was funded by Stellenbosch University. ### Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable. Yes All data produced are available online at and (supplemented by )