PMID- 37703989 OWN - NLM STAT- MEDLINE DCOM- 20240318 LR - 20240318 IS - 1532-6500 (Electronic) IS - 1058-2746 (Linking) VI - 33 IP - 4 DP - 2024 Apr TI - Evaluating the fairness and accuracy of machine learning-based predictions of clinical outcomes after anatomic and reverse total shoulder arthroplasty. PG - 888-899 LID - S1058-2746(23)00651-1 [pii] LID - 10.1016/j.jse.2023.08.005 [doi] AB - BACKGROUND: Machine learning (ML)-based clinical decision support tools (CDSTs) make personalized predictions for different treatments; by comparing predictions of multiple treatments, these tools can be used to optimize decision making for a particular patient. However, CDST prediction accuracy varies for different patients and also for different treatment options. If these differences are sufficiently large and consistent for a particular subcohort of patients, then that bias may result in those patients not receiving a particular treatment. Such level of bias would deem the CDST "unfair." The purpose of this study is to evaluate the "fairness" of ML CDST-based clinical outcomes predictions after anatomic (aTSA) and reverse total shoulder arthroplasty (rTSA) for patients of different demographic attributes. METHODS: Clinical data from 8280 shoulder arthroplasty patients with 19,249 postoperative visits was used to evaluate the prediction fairness and accuracy associated with the following patient demographic attributes: ethnicity, sex, and age at the time of surgery. Performance of clinical outcome and range of motion regression predictions were quantified by the mean absolute error (MAE) and performance of minimal clinically important difference (MCID) and substantial clinical benefit classification predictions were quantified by accuracy, sensitivity, and the F1 score. Fairness of classification predictions leveraged the "four-fifths" legal guideline from the US Equal Employment Opportunity Commission and fairness of regression predictions leveraged established MCID thresholds associated with each outcome measure. RESULTS: For both aTSA and rTSA clinical outcome predictions, only minor differences in MAE were observed between patients of different ethnicity, sex, and age. Evaluation of prediction fairness demonstrated that 0 of 486 MCID (0%) and only 3 of 486 substantial clinical benefit (0.6%) classification predictions were outside the 20% fairness boundary and only 14 of 972 (1.4%) regression predictions were outside of the MCID fairness boundary. Hispanic and Black patients were more likely to have ML predictions out of fairness tolerance for aTSA and rTSA. Additionally, patients <60 years old were more likely to have ML predictions out of fairness tolerance for rTSA. No disparate predictions were identified for sex and no disparate regression predictions were observed for forward elevation, internal rotation score, American Shoulder and Elbow Surgeons Standardized Shoulder Assessment Form score, or global shoulder function. CONCLUSION: The ML algorithms analyzed in this study accurately predict clinical outcomes after aTSA and rTSA for patients of different ethnicity, sex, and age, where only 1.4% of regression predictions and only 0.3% of classification predictions were out of fairness tolerance using the proposed fairness evaluation method and acceptance criteria. Future work is required to externally validate these ML algorithms to ensure they are equally accurate for all legally protected patient groups. CI - Copyright (c) 2023 Journal of Shoulder and Elbow Surgery Board of Trustees. Published by Elsevier Inc. All rights reserved. FAU - Allen, Christine AU - Allen C AD - Exactech, Inc. Gainesville, FL, USA. FAU - Kumar, Vikas AU - Kumar V AD - Exactech, Inc. Gainesville, FL, USA. FAU - Elwell, Josie AU - Elwell J AD - Exactech, Inc. Gainesville, FL, USA. FAU - Overman, Steven AU - Overman S AD - Overman Consulting PLLC. Seattle, WA, USA. FAU - Schoch, Bradley S AU - Schoch BS AD - Mayo Clinic Jacksonville, Jacksonville, FL, USA. FAU - Aibinder, William AU - Aibinder W AD - University of Michigan, Ann Arbor, MI, USA. FAU - Parsons, Moby AU - Parsons M AD - King and Parsons Orthopedic Center, Portsmouth, NH, USA. FAU - Watling, Jonathan AU - Watling J AD - Maine Health, Portland, ME, USA. FAU - Ko, Jiawei Kevin AU - Ko JK AD - Swedish Orthopedic Institute, Seattle, WA, USA. FAU - Gobbato, Bruno AU - Gobbato B AD - IDOMED, Nova Brasilia, Jaragua do Sul, Brazil. FAU - Throckmorton, Thomas AU - Throckmorton T AD - University of Tennessee-Campbell Clinic, Memphis, TN, USA. FAU - Routman, Howard AU - Routman H AD - Atlantis Orthopedics, Palm Beach Gardens, FL, USA. FAU - Roche, Christopher P AU - Roche CP AD - Exactech, Inc. Gainesville, FL, USA. Electronic address: Chris.Roche@exac.com. LA - eng PT - Journal Article DEP - 20230912 PL - United States TA - J Shoulder Elbow Surg JT - Journal of shoulder and elbow surgery JID - 9206499 SB - IM MH - Humans MH - Middle Aged MH - *Arthroplasty, Replacement, Shoulder/adverse effects MH - *Shoulder Joint/surgery MH - Treatment Outcome MH - Retrospective Studies MH - Range of Motion, Articular OTO - NOTNLM OT - Machine learning OT - accuracy evaluation OT - anatomic total shoulder arthroplasty OT - clinical outcomes OT - fairness and equity OT - reverse total shoulder arthroplasty EDAT- 2023/09/14 00:42 MHDA- 2024/03/18 06:44 CRDT- 2023/09/13 19:13 PHST- 2023/04/08 00:00 [received] PHST- 2023/07/31 00:00 [revised] PHST- 2023/08/06 00:00 [accepted] PHST- 2024/03/18 06:44 [medline] PHST- 2023/09/14 00:42 [pubmed] PHST- 2023/09/13 19:13 [entrez] AID - S1058-2746(23)00651-1 [pii] AID - 10.1016/j.jse.2023.08.005 [doi] PST - ppublish SO - J Shoulder Elbow Surg. 2024 Apr;33(4):888-899. doi: 10.1016/j.jse.2023.08.005. Epub 2023 Sep 12.