How to mitigate selection bias in COVID-19 surveys: evidence from five national cohorts

Background Non-response is a common problem, and even more so during the COVID-19 pandemic where social distancing measures challenged data collections. As non-response is often systematic, meaning that respondents are usually healthier and from a better socioeconomic background, this potentially introduces serious bias in research findings based on COVID-19 survey data. The goal of the current study was to see if we can reduce bias and restore sample representativeness despite systematic non-response in the COVID-19 surveys embedded within five UK cohort studies using the rich data available from previous time points. Methods A series of three surveys was conducted during the pandemic across five UK cohorts: National Survey of Health and Development (NSHD, born 1946), 1958 National Child Development Study (NCDS), 1970 British Cohort Study (BCS70), Next Steps (born 1989-90) and Millennium Cohort Study (MCS, born 2000-02). We applied non-response weights and utilised multiple imputation, making use of covariates from previous waves which have been commonly identified as predictors of non-response, to attempt to reduce bias and restore sample representativeness. Results Response rates in the COVID-19 surveys were lower compared to previous cohort waves, especially in the younger cohorts. We identified bias due to systematic non-response in the distributions of variables including parental social class and childhood cognitive ability. In each cohort, respondents of the COVID-19 survey had a higher percentage of parents in the most advantaged social class, and a higher mean of childhood cognitive ability, compared to the original (full) cohort sample. The application of non-response weights and multiple imputation was successful in reducing bias in parental social class and childhood cognitive ability, nearly eliminating it for the former. Conclusions The current paper demonstrates that it is possible to reduce bias from non-response and to a large degree restore sample representativeness in multiple waves of a COVID-19 survey embedded within long running longitudinal cohort studies through application of non-response weights or multiple imputation. Such embedded COVID-19 surveys therefore have an advantage over cross-sectional COVID-19 surveys, where non-response bias cannot be handled by leveraging previously observed information on non-respondents. Our findings suggest that, if non-response is appropriately handled, analyses based on the COVID-19 surveys within these five cohorts can contribute significantly to COVID-19 research, including studying the medium and long-term effects of the pandemic. ### Competing Interest Statement The authors have declared no competing interest. ### Funding Statement This research was supported by the Centre for Longitudinal Studies Resource Centre Award: ESRC grant 2022-25 (ES/W013142/1). ### Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes The details of the IRB/oversight body that provided approval or exemption for the research described are given below: NCDS, BCS, Next Steps and MCS are publicly available on UK Data Service, see links below. NSHD data are available on request to the NSHD Data Sharing Committee. Interested researchers can apply to access the NSHD data via a standard application procedure. As part of a joint COVID survey project between NSHD, NCDS, BCS, Next Steps and MCS we were provided with a basic set of NSHD variables which could be used for Covid survey related research (such as the present paper). I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable. Yes Data from NCDS, BCS, Next Steps and MCS are available through the UK Data Service (https://ukdataservice.ac.uk/). NSHD data are available on request to the NSHD Data Sharing Committee. Interested researchers can apply to access the NSHD data via a standard application procedure. Data requests should be submitted to mrclha.swiftinfo{at}ucl.ac.uk; further details can be found at http://www.nshd.mrc.ac.uk/data.aspx