Does Death and Disability Matter?

Schwindt and colleagues present a retrospective analytical study of 13,950 term births with a historical control group and find that neonatal simulation training decreases the incidence of chest compressions in term newborns.1.Schwindt E, Stockenhuber R, Kainz T, et al. Resuscitation Neonatal simulation training decreases the incidence of chest compressions in term newborns.Google Scholar The intervention in the study was a neonatal simulation-based education programme introduced into a single centre level 2 neonatal service in Austria in January 2015. This was designed to include two complimentary training components: neonatal resuscitation training and neonatal simulation training. Resuscitation training was lower fidelity and focused on the NLS (Newborn Life Support) algorithm, individual and team skills, whilst the simulation training considered crisis resource management, team culture and working in teams within the available infrastructure. Both types of training have been shown to improve skills and change culture.2.Curran V. Fleet L. White S. et al.A randomized controlled study of manikin simulator fidelity on neonatal resuscitation program learning outcomes.Adv Health Sci Educ. 2015; 20: 205-218https://doi.org/10.1007/s10459-014-9522-8Crossref PubMed Scopus (39) Google Scholar, 3.Nimbalkar A. Patel D. Kungwani A. Phatak A. Vasa R. Nimbalkar S. Randomized control trial of high fidelity vs low fidelity simulation for training undergraduate students in neonatal resuscitation.Bmc Res Notes. 2015; 8: 636https://doi.org/10.1186/s13104-015-1623-9Crossref PubMed Scopus (39) Google Scholar Both training modes involved a theoretical lecture and a hands-on component. There were 2 sessions per training day and up to 8 people in a session, slightly higher than the 6 that a recent randomised control trial suggested was optimal for resuscitation training.4.Nabecker S. Huwendiek S. Theiler L. Huber M. Petrowski K. Greif R. The effective group size for teaching cardiopulmonary resuscitation skills – a randomized controlled simulation trial.Resuscitation. 2021; 165: 77-82https://doi.org/10.1016/j.resuscitation.2021.05.034Abstract Full Text Full Text PDF PubMed Scopus (4) Google Scholar Session content was well described but it is not clear how the 11 training sessions delivered training to the 350 training recipients reported. Data from 2012–2014 (pre-training) were compared to 2015–2019 (post-training). Training sessions were not mandatory and were offered to midwives, nurses (neonatal and anaesthetic), obstetricians, anaesthetists, paediatricians and neonatologists. Study outcomes were mortality and intervention level at birth. In addition to looking at mortality Schwindt et al. chose shorter term clinical outcomes such as intervention level at birth e.g., short or longer term invasive ventilation or chest compressions (CC). They found a non-significant rise in ventilatory support and a significant decrease in CC. These outcomes measure ‘received’ rather than ‘required’ interventions. It is worth noting that the single centre involved had a relatively high pre-intervention CC rate of 0.91 % with rates of 0.2–0.3 % being more commonly cited in guidelines.5.Curran V. Fleet L. White S. et al.Epidemiology and outcomes of infants after cardiopulmonary resuscitation in the neonatal or pediatric intensive care unit from a national registry.Resuscitation. 2021; 165: 14-22Abstract Full Text Full Text PDF PubMed Scopus (4) Google Scholar, 6.Perlman J.M. Risser R. Cardiopulmonary resuscitation in the delivery room: associated clinical events.Arch Pediatr Adolesc Med. 1995; 149: 20-25https://doi.org/10.1001/archpedi.1995.02170130022005Crossref PubMed Scopus (300) Google Scholar This does not change their finding of a significant reduction in received CC, but a higher CC incidence rate would reduce the number of recruits needed to show a difference and does make it harder to apply these results to other centres. A potential confounder was the 2015 ERC NLS algorithm7.Wyllie J. Bruinenberg J. Roehr C.C. Rüdiger M. Trevisanuto D. Urlesberger B. European resuscitation council guidelines for resuscitation 2015: Section 7. Resuscitation and support of transition of babies at birth.Resuscitation. 2015; 95 (Epub 2015 Oct 15. PMID: 26477415): 249-263https://doi.org/10.1016/j.resuscitation.2015.07.029Abstract Full Text Full Text PDF PubMed Scopus (347) Google Scholar change to include 30 seconds of ventilation breaths prior to commencing CC for bradycardia. Clarification with the authors found that this algorithm change was not implemented in this study centre during the study period. Other issues, acknowledged by the authors, are the introduction of ECG leads to measure a heart rate, which may give an earlier heart rate reading potentially affecting whether CC was given. The study centre also changed the face masks they used during the study period which has the potential to alter mask leak and the efficacy of ventilation which could impact their short and longer term respiratory support and receipt of CC outcomes. The impact of centralisation of neonatal services over time was queried with the authors who felt this did not impact their study population or findings. The retrospective nature of the study prevented tracking of which staff had completed training. However, training was multi-disciplinary and staff with regular attendance at deliveries were prioritised. Although the purpose of simulation training is to improve patient outcomes, assessing the impact of a training intervention on a clinical outcome is difficult. This is demonstrated by a recent systematic review by Linhard et al.8.Lindhard M.S. Thim S. Laursen H.S. Schram A.W. Paltved C. Henriksen T.B. Simulation-based neonatal resuscitation team training: a systematic review.Pediatrics. 2021; 147https://doi.org/10.1542/peds.2020-042010Crossref PubMed Scopus (6) Google Scholar in which only 2 of 24 studies considered a patient outcome (neonatal mortality) and both were at high risk of bias. The authors of the systematic review suggested researchers should include patient outcomes or clinical proxies of treatment quality whenever possible. They also found meta-analysis was impossible because of heterogenous interventions and outcomes. Although rare, there are examples of simulation training as part of a wider quality improvement programme that have led to a change in patient outcomes e.g., a reduction hypoxic-ischaemic encephalopathy rates.9.Draycott T. Sibanda T. Owen L. et al.Does training in obstetric emergencies improve neonatal outcome?.BJOG. 2006; 113: 177-182https://doi.org/10.1111/j.1471-0528.2006.00800.xCrossref PubMed Scopus (411) Google Scholar Mortality as an outcome in resuscitation studies has been translated into neonatal studies from the adult resuscitation literature. The statistical problem this presents is that whilst mortality is fortunately low in neonatal cardiac arrest, it is high in adults with a cardiac arrest.10.National Cardiac Arrest Audit (UK), provided by the Intensive Care National Audit and Research Centre. Available online at https://www.icnarc.org/Our-Audit/Audits/Ncaa/Reports/Key-Statistics [accessed 7th July 2022].Google Scholar As neonatal mortality is a low incidence event large numbers of patients are required to show any difference. Schwindt et al. demonstrate this in their retrospective calculation that they would have needed 90,000 recruits to show a difference in mortality but only 2000 to show a difference in receipt of chest compressions. They note the Bhatia study,11.Bhatia M. Stewart A.E. Wallace A. Kumar A. Malhotra A. Evaluation of an in-situ neonatal resuscitation simulation program using the new world Kirkpatrick model.Clin Simul Nurs. 2021; 50: 27-37https://doi.org/10.1016/j.ecns.2020.09.006Abstract Full Text Full Text PDF Scopus (4) Google Scholar which evaluated a training curriculum and found reduced receipt of chest compressions and reduced neonatal mortality. In that study of 40,000 births, 38,000 cases were needed to detect a mortality change but only 7500 to find a change in chest compressions. The commonly used outcome of neurodevelopmental delay is also challenging as it requires longer follow up and is influenced by multiple factors. Outcomes in simulation studies are often measures of participant confidence, perceived value of the training and sometimes change in behaviour at individual or institutional levels. These outcomes are described within the 4 level Kirkpatrick framework12.Kirkpatrick D.L. Evaluating training programs: the four levels. Berrett-Koehler; Publishers Group West [distributor], San Francisco: Emeryville, CA1994Google Scholar, 13.Kirkpatrick D.L. Kirkpatrick J.D. Evaluating training programs: the four levels.3rd ed. Berrett-Koehler Publishers Inc., San Francisco, CA2006Google Scholar with level 4 considering patient outcomes. In contrast, GRADE certainty of evidence classifications of clinical outcomes uses a numeric 1–9 range, in relation to their importance to decision making for neonatal resuscitation guidelines, with 1–3 low importance, 4–6 important but not critical and 7–9 critical.14.Strand M.L. Simon W.M. Wyllie J. Wyckoff M.H. Weiner G. Consensus outcome rating for international neonatal resuscitation guidelines.Arch Dis Child Fetal Neonatal Ed. 2020; 105 (Epub 2019 Mar 29. PMID: 30926715): 328-330https://doi.org/10.1136/archdischild-2019-316942Crossref PubMed Scopus (12) Google Scholar We suggest that this approach could be an extension of the Kirkpatrick approach commonly applied to simulation research and might help select which patient outcomes or clinical proxies might help establish the impact of a training intervention. Heterogenous interventions and outcomes are problematic and perhaps the simulation literature could align with standard outcome sets such as the Utstein criteria.15.Nolan J.P. Berg R.A. Andersen L.W. et al.Cardiac arrest and cardiopulmonary resuscitation outcome reports: update of the Utstein resuscitation registry template for in-hospital cardiac arrest: a consensus report from a task force of the International Liaison Committee on Resuscitation (American Heart Association, European Resuscitation Council, Australian and New Zealand Council on Resuscitation, Heart and Stroke Foundation of Canada, InterAmerican Heart Foundation, Resuscitation Council of Southern Africa, Resuscitation Council of Asia).Resuscitation. 2019; 144: 166-177https://doi.org/10.1016/j.resuscitation.2019.08.021Abstract Full Text Full Text PDF PubMed Scopus (45) Google Scholar The main strength of this paper is the innovative use of shorter-term clinical outcomes to show an impact of training. Other strengths of this paper are the numbers included, the description of training and the significant effect of training shown in their multi-factorial, generalised linear mixed effects model. We applaud this attempt to integrate simulation training and clinical outcomes in this excellent paper and that the authors recognise the challenge in doing this and propose an innovative approach. Simulation enthusiasts are ‘sure it works’ and David Gaba, one of the pioneers of simulation in healthcare said, “… no industry in which human lives depend on the skilled performance of responsible operators has waited for unequivocal proof of the benefits of simulation before embracing it.”.16.Gaba D.M. Improving anesthesiologists' performance by simulating reality.Anesthesiology. 1992; 76: 491-494Crossref PubMed Scopus (216) Google Scholar However, this does not mean that we should not look for evidence in a way that gives a reasonable chance of finding a meaningful clinical outcome. This maximises effective use of educator's time and focuses on interventions that are most likely to improve patient outcomes. Death and disability do matter but they may not be the best outcomes measurements to show whether training improves clinical outcomes. Neither author has a conflict of interest to declare. Both authors contributed equally to this editorial manuscript.