PMID- 32332242 OWN - NLM STAT- MEDLINE DCOM- 20210525 LR - 20221005 IS - 1528-1132 (Electronic) IS - 0009-921X (Print) IS - 0009-921X (Linking) VI - 478 IP - 10 DP - 2020 Oct TI - What Is the Accuracy of Three Different Machine Learning Techniques to Predict Clinical Outcomes After Shoulder Arthroplasty? PG - 2351-2363 LID - 10.1097/CORR.0000000000001263 [doi] LID - 2351-2363 AB - BACKGROUND: Machine learning techniques can identify complex relationships in large healthcare datasets and build prediction models that better inform physicians in ways that can assist in patient treatment decision-making. In the domain of shoulder arthroplasty, machine learning appears to have the potential to anticipate patients' results after surgery, but this has not been well explored. QUESTIONS/PURPOSES: (1) What is the accuracy of machine learning to predict the American Shoulder and Elbow Surgery (ASES), University of California Los Angeles (UCLA), Constant, global shoulder function, and VAS pain scores, as well as active abduction, forward flexion, and external rotation at 1 year, 2 to 3 years, 3 to 5 years, and more than 5 years after anatomic total shoulder arthroplasty (aTSA) or reverse total shoulder arthroplasty (rTSA)? (2) What is the accuracy of machine learning to identify whether a patient will achieve clinical improvement that exceeds the minimum clinically important difference (MCID) threshold for each outcome measure? (3) What is the accuracy of machine learning to identify whether a patient will achieve clinical improvement that exceeds the substantial clinical benefit threshold for each outcome measure? METHODS: A machine learning analysis was conducted on a database of 7811 patients undergoing shoulder arthroplasty of one prosthesis design to create predictive models for multiple clinical outcome measures. Excluding patients with revisions, fracture indications, and hemiarthroplasty resulted in 6210 eligible primary aTSA and rTSA patients, of whom 4782 patients with 11,198 postoperative follow-up visits had sufficient preoperative, intraoperative, and postoperative data to train and test the predictive models. Preoperative clinical data from 1895 primary aTSA patients and 2887 primary rTSA patients were analyzed using three commercially available supervised machine learning techniques: linear regression, XGBoost, and Wide and Deep, to train and test predictive models for the ASES, UCLA, Constant, global shoulder function, and VAS pain scores, as well as active abduction, forward flexion, and external rotation. Our primary study goal was to quantify the accuracy of three machine learning techniques to predict each outcome measure at multiple postoperative timepoints after aTSA and rTSA using the mean absolute error between the actual and predicted values. Our secondary study goals were to identify whether a patient would experience clinical improvement greater than the MCID and substantial clinical benefit anchor-based thresholds of patient satisfaction for each outcome measure as quantified by the model classification parameters of precision, recall, accuracy, and area under the receiver operating curve. RESULTS: Each machine learning technique demonstrated similar accuracy to predict each outcome measure at each postoperative point for both aTSA and rTSA, though small differences in prediction accuracy were observed between techniques. Across all postsurgical timepoints, the Wide and Deep technique was associated with the smallest mean absolute error and predicted the postoperative ASES score to +/- 10.1 to 11.3 points, the UCLA score to +/- 2.5 to 3.4, the Constant score to +/- 7.3 to 7.9, the global shoulder function score to +/- 1.0 to 1.4, the VAS pain score to +/- 1.2 to 1.4, active abduction to +/- 18 to 21 degrees , forward elevation to +/- 15 to 17 degrees , and external rotation to +/- 10 to 12 degrees . These models also accurately identified the patients who did and did not achieve clinical improvement that exceeded the MCID (93% to 99% accuracy for patient-reported outcome measures (PROMs) and 85% to 94% for pain, function, and ROM measures) and substantial clinical benefit (82% to 93% accuracy for PROMs and 78% to 90% for pain, function, and ROM measures) thresholds. CONCLUSIONS: Machine learning techniques can use preoperative data to accurately predict clinical outcomes at multiple postoperative points after shoulder arthroplasty and accurately risk-stratify patients by preoperatively identifying who may and who may not achieve MCID and substantial clinical benefit improvement thresholds for each outcome measure. CLINICAL RELEVANCE: Three different commercially available machine learning techniques were used to train and test models that predicted clinical outcomes after aTSA and rTSA; this device-type comparison was performed to demonstrate how predictive modeling techniques can be used in the near future to help answer unsolved clinical questions and augment decision-making to improve outcomes after shoulder arthroplasty. FAU - Kumar, Vikas AU - Kumar V AD - V. Kumar, S. Overman, A. Teredesai, KenSci, Seattle, WA, USA. FAU - Roche, Christopher AU - Roche C AD - C. Roche, Exactech, Gainesville, FL, USA. FAU - Overman, Steven AU - Overman S AD - V. Kumar, S. Overman, A. Teredesai, KenSci, Seattle, WA, USA. AD - S. Overman, University of Washington School of Medicine, Seattle, WA, USA. FAU - Simovitch, Ryan AU - Simovitch R AD - R. Simovitch, Hospital For Special Surgery - FL, West Palm Beach, FL 33401, USA. FAU - Flurin, Pierre-Henri AU - Flurin PH AD - P-H. Flurin, Bordeaux-Merignac Sport Clinic, Merignac, France. FAU - Wright, Thomas AU - Wright T AD - T. Wright, University of Florida Department of Orthopaedic Surgery, Gainesville, FL, USA. FAU - Zuckerman, Joseph AU - Zuckerman J AD - J. Zuckerman, Department of Orthopedic Surgery, NYU Langone Orthopedic Hospital, New York, NY, USA. FAU - Routman, Howard AU - Routman H AD - H. Routman, Atlantis Orthopedics, Palm Beach Gardens, FL, USA. FAU - Teredesai, Ankur AU - Teredesai A AD - V. Kumar, S. Overman, A. Teredesai, KenSci, Seattle, WA, USA. LA - eng PT - Journal Article PT - Multicenter Study PL - United States TA - Clin Orthop Relat Res JT - Clinical orthopaedics and related research JID - 0075674 SB - IM CIN - Clin Orthop Relat Res. 2020 Oct;478(10):2364-2366. PMID: 32511144 MH - Aged MH - *Arthroplasty, Replacement, Shoulder MH - Female MH - Humans MH - Machine Learning/*standards MH - Male MH - Middle Aged MH - Minimal Clinically Important Difference MH - Pain Measurement MH - Predictive Value of Tests MH - Range of Motion, Articular MH - Treatment Outcome PMC - PMC7491877 COIS- All ICMJE Conflict of Interest Forms for authors and Clinical Orthopaedics and Related Research(R) editors and board members are on file with the publication and can be viewed on request. EDAT- 2020/04/26 06:00 MHDA- 2021/05/26 06:00 PMCR- 2021/10/01 CRDT- 2020/04/26 06:00 PHST- 2020/04/26 06:00 [pubmed] PHST- 2021/05/26 06:00 [medline] PHST- 2020/04/26 06:00 [entrez] PHST- 2021/10/01 00:00 [pmc-release] AID - 00003086-202010000-00026 [pii] AID - CORR-D-20-00154 [pii] AID - 10.1097/CORR.0000000000001263 [doi] PST - ppublish SO - Clin Orthop Relat Res. 2020 Oct;478(10):2351-2363. doi: 10.1097/CORR.0000000000001263.