Journal of Investigative Medicine; Jun 2019; vol. 67 (no. 5); p. 929
- A prospective study of 937 ICU discharges (University of Chicago, IL, USA) found that a machine learning model was more accurate than clinician intuition for predicting ICU readmission, suggesting that a machine learning model using real-time patient data could provide clinicians with additional information to guide decision-making regarding the timing of ICU discharge.
Abstract
Objective: Patients who experience unplanned readmission to the intensive care unit (ICU) have increased hospital lengths of stay, costs, and mortality compared to patients discharged from the ICU but not readmitted. We have previously shown that a machine learning approach to predicting ICU readmission is more accurate than existing risk scores. However, whether a machine learning model is more accurate than clinician intuition for predicting ICU readmission is unknown. Therefore, we aimed to compare the accuracy of clinician intuition to our recently validated machine learning algorithm.
Method: We conducted a prospective study in the medical ICU of an academic hospital from October 2015 to September 2017. Clinicians (nurses, residents, fellows, attending physicians) were voluntarily surveyed once per day after rounds about the likelihood of ICU readmission for patients being discharged on a 1-10 scale. Survey data was linked with electronic health records to determine readmission outcomes, and only surveys collected within 36 hours of ICU discharge were included. The machine learning model was run on clinical data available at the time of ICU discharge to predict the probability of a future ICU readmission during the same hospitalization. Areas under the receiver operating characteristic curves (AUCs) were calculated and compared between the median clinician intuition score, the machine learning algorithm, and a combined model for predicting ICU readmission.
Results: A total of 2832 surveys from 937 unique ICU discharges were included, of which 114 patients (12%) were readmitted to the ICU during the hospitalization. The median clinician score was 3 (IQR 2-4). The combined model had the highest AUC for predicting those patients ever readmitted (AUC 0.79 [95% CI 0.74-0.83]), followed by the machine learning model (AUC 0.78 [95% CI 0.74-0.83]), and median clinician intuition (AUC 0.71 [95% CI 0.66-0.75]) (figure 1). Both the combined model and the machine-learning model were more accurate than clinician intuition alone (p<0.01 for both comparisons). AUC results were similar for predicting readmission within 48 hours.
Conclusion: A machine learning model was more accurate than clinician intuition for predicting ICU readmission. Our results demonstrate that a machine learning model using real-time patient data would provide clinicians with additional information to guide decision-making regarding the timing of ICU discharge. Further research is needed to determine if the use of our machine learning model to target interventions for high risk patients would help improve outcomes for patients transferred out of the ICU.
