Machine learning algorithm 'more accurate' in predicting ICU readmission

New research shows that a machine learning approach to predicting ICU readmission was significantly more accurate than previously published algorithms or prediction tools. Implementation of this approach could target patients who may benefit from additional time in the ICU or more frequent monitoring after transfer to the hospital ward, according to the research published online in Annals of the American Thoracic Society.

This new risk score includes variables available in real-time in the EHR. "Our algorithm could be inserted into an electronic health record [EHR], a dashboard, or even a mobile app that could be used by both ICU teams and rapid response teams to design targeted interventions aimed at reducing morbidity and mortality for patient transferred out of the ICU," write Juan C. Rojas, MD, Department of Medicine, University of Chicago, and co-authors.  

Patients transferred from the intensive care unit (ICU) to the wards who are later readmitted to the ICU have increased length of stay, healthcare expenditure, and mortality compared to those who are never readmitted.  Previous studies that performed retrospective reviews of ICU readmissions suggest that over 10 percent of these readmissions are potentially preventable. For these reasons, improving risk-stratification for patients at risk of clinical deterioration on the hospital ward after ICU discharge could have important benefits for critically ill hospitalised patients.

The aim of this study was to develop a novel prediction tool based upon a machine learning algorithm called gradient boosted machine (GBM) using variables that are available in real-time in the EHR, and to compare the derived model to previously published risk scores. This observational cohort study was conducted at a U.S. academic hospital with approximately 600 inpatient beds. A total of 24,885 ICU transfers to the wards were included, with 14,962 transfers (60%) in the training cohort and 9,923 transfers (40%) in the internal validation cohort. 

Patient characteristics, nursing assessments, ICD-9 codes from prior admissions, medications, ICU interventions, diagnostic tests, vital signs, and laboratory results were extracted from the EHR and used as predictor variables in a gradient boosted machine model. Accuracy for predicting ICU readmission was compared to the Stability and Workload Index for Transfer (SWIFT) score and Modified Early Warning Score (MEWS) in the internal validation cohort and also externally using the Medical Information Mart for Intensive Care (MIMIC-III) database (n=42,303 ICU transfers).

Eleven percent (2,834) of discharges to the wards were later readmitted to the ICU. The machine-learning derived model had significantly better performance (AUC 0.76) than either the SWIFT score (AUC 0.65), or MEWS (AUC 0.58); p value < 0.0001 for all comparisons, according to the researchers. At a specificity of 95%, the derived model had a sensitivity of 28% compared to 15% for SWIFT score and 7% for the MEWS. Accuracy improvements with the derived model over MEWS and SWIFT were similar in the MIMIC III cohort, the researchers noted.

"To date, a machine learning model has not been developed to predict ICU readmission. However, machine learning models have been shown to be more accurate than prior models using logistic regression in predicting hospital readmission in patients with congestive heart failure," write Dr. Rojas et al. 

However, as this new risk score was developed using patient data from a single institution, the authors say the generalisability of their findings will need further validation. Also, important factors such as ICU bed availability and hospital census can impact ICU readmissions and, therefore, the prediction of these events. "We did not have these important hospital level variables in our model," the authors point out.

Source: Annals of the American Thoracic Society

Image Credit: PublicDomainPictures.net