Effect of Digital Early Warning Scores on hospital protocol adherence: stepped-wedge evaluation (Preprint)

BACKGROUND Early Warning Scores (EWS) are commonly used in hospitals to assess a patient’s risk of deterioration. EWS are traditionally recorded on paper observation charts but are increasingly recorded digitally. In either case, evidence for the clinical effectiveness of such scores is mixed. OBJECTIVE We examine whether the introduction of a digital Early Warning Score (EWS) system was associated with more frequent observation of patients with abnormal vital signs, a precursor to earlier clinical intervention. METHODS We conducted a two-armed stepped-wedge study from February 2015-December 2016, over four hospitals in one UK hospital trust. In the control arm, vital signs were recorded using paper observation charts. In the intervention arm, a digital EWS system was used. The primary outcome measure was Time To Next Observation (TTNO), defined as the time between a patient’s first elevated EWS (EWS ≥ 3) and subsequent observations set. Secondary outcomes were time to death in hospital, length of stay, and time to unplanned ICU admission. Differences between the two arms were analysed using a mixed effects Cox model. RESULTS The median TTNO in the control and intervention arms were 128 minutes (IQR: 73-218) and 131 minutes (IQR: 73-223), respectively. The corresponding hazard ratio for TTNO was 0.99 (95% CI: 0.91 - 1.07, P = .73). CONCLUSIONS There were no significant difference in the primary and secondary outcomes, despite evidence of strong clinical engagement with the system. One possible explanation is that the system did not provide sufficient or timely enough feedback to effect changes to clinical decision making. Our findings contrast with previous claims that digital EWS systems are associated with improvement in clinical outcomes. Future research should investigate the causal mechanisms by which such systems alter staff behaviours and patient outcomes.