PMID- 37283449 OWN - NLM STAT- MEDLINE DCOM- 20230608 LR - 20230713 IS - 1092-0684 (Electronic) IS - 1092-0684 (Linking) VI - 54 IP - 6 DP - 2023 Jun TI - Which supervised machine learning algorithm can best predict achievement of minimum clinically important difference in neck pain after surgery in patients with cervical myelopathy? A QOD study. PG - E5 LID - 10.3171/2023.3.FOCUS2372 [doi] AB - OBJECTIVE: The purpose of this study was to evaluate the performance of different supervised machine learning algorithms to predict achievement of minimum clinically important difference (MCID) in neck pain after surgery in patients with cervical spondylotic myelopathy (CSM). METHODS: This was a retrospective analysis of the prospective Quality Outcomes Database CSM cohort. The data set was divided into an 80% training and a 20% test set. Various supervised learning algorithms (including logistic regression, support vector machine, decision tree, random forest, extra trees, gaussian naive Bayes, k-nearest neighbors, multilayer perceptron, and extreme gradient boosted trees) were evaluated on their performance to predict achievement of MCID in neck pain at 3 and 24 months after surgery, given a set of predicting baseline features. Model performance was assessed with accuracy, F1 score, area under the receiver operating characteristic curve, precision, recall/sensitivity, and specificity. RESULTS: In total, 535 patients (46.9%) achieved MCID for neck pain at 3 months and 569 patients (49.9%) achieved it at 24 months. In each follow-up cohort, 501 patients (93.6%) were satisfied at 3 months after surgery and 569 patients (100%) were satisfied at 24 months after surgery. Of the supervised machine learning algorithms tested, logistic regression demonstrated the best accuracy (3 months: 0.76 +/- 0.031, 24 months: 0.773 +/- 0.044), followed by F1 score (3 months: 0.759 +/- 0.019, 24 months: 0.777 +/- 0.039) and area under the receiver operating characteristic curve (3 months: 0.762 +/- 0.027, 24 months: 0.773 +/- 0.043) at predicting achievement of MCID for neck pain at both follow-up time points, with fair performance. The best precision was also demonstrated by logistic regression at 3 (0.724 +/- 0.058) and 24 (0.780 +/- 0.097) months. The best recall/sensitivity was demonstrated by multilayer perceptron at 3 months (0.841 +/- 0.094) and by extra trees at 24 months (0.817 +/- 0.115). Highest specificity was shown by support vector machine at 3 months (0.952 +/- 0.013) and by logistic regression at 24 months (0.747 +/- 0.18). CONCLUSIONS: Appropriate selection of models for studies should be based on the strengths of each model and the aims of the studies. For maximally predicting true achievement of MCID in neck pain, of all the predictions in this balanced data set the appropriate metric for the authors' study was precision. For both short- and long-term follow-ups, logistic regression demonstrated the highest precision of all models tested. Logistic regression performed consistently the best of all models tested and remains a powerful model for clinical classification tasks. FAU - Park, Christine AU - Park C AD - 1Department of Neurosurgery, Duke University, Durham, North Carolina. FAU - Mummaneni, Praveen V AU - Mummaneni PV AD - 2Department of Neurosurgery, University of California, San Francisco, California. FAU - Gottfried, Oren N AU - Gottfried ON AD - 1Department of Neurosurgery, Duke University, Durham, North Carolina. FAU - Shaffrey, Christopher I AU - Shaffrey CI AD - 1Department of Neurosurgery, Duke University, Durham, North Carolina. FAU - Tang, Anthony J AU - Tang AJ AD - 3Department of Neurological Surgery, Columbia University Vagelos College of Physicians and Surgeons, The Och Spine Hospital at NewYork-Presbyterian, New York, New York. FAU - Bisson, Erica F AU - Bisson EF AD - 4Department of Neurosurgery, University of Utah, Salt Lake City, Utah. FAU - Asher, Anthony L AU - Asher AL AD - 5Neuroscience Institute, Carolinas Healthcare System and Carolina Neurosurgery & Spine Associates, Charlotte, North Carolina. FAU - Coric, Domagoj AU - Coric D AD - 5Neuroscience Institute, Carolinas Healthcare System and Carolina Neurosurgery & Spine Associates, Charlotte, North Carolina. FAU - Potts, Eric A AU - Potts EA AD - 6Goodman Campbell Brain and Spine, Indianapolis, Indiana. FAU - Foley, Kevin T AU - Foley KT AD - 7Department of Neurosurgery, University of Tennessee, Semmes-Murphey Neurologic and Spine Institute, Memphis, Tennessee. FAU - Wang, Michael Y AU - Wang MY AD - 8Department of Neurosurgery, University of Miami, Florida. FAU - Fu, Kai-Ming AU - Fu KM AD - 9Department of Neurosurgery, Weill Cornell Medical Center, New York, New York. FAU - Virk, Michael S AU - Virk MS AD - 9Department of Neurosurgery, Weill Cornell Medical Center, New York, New York. FAU - Knightly, John J AU - Knightly JJ AD - 10Atlantic Neurosurgical Specialists, Morristown, New Jersey. FAU - Meyer, Scott AU - Meyer S AD - 10Atlantic Neurosurgical Specialists, Morristown, New Jersey. FAU - Park, Paul AU - Park P AD - 11Department of Neurosurgery, University of Michigan, Ann Arbor, Michigan. FAU - Upadhyaya, Cheerag AU - Upadhyaya C AD - 12Marion Bloch Neuroscience Institute, Saint Luke's Health System, Kansas City, Missouri. FAU - Shaffrey, Mark E AU - Shaffrey ME AD - 13Department of Neurosurgery, University of Virginia, Charlottesville, Virginia. FAU - Buchholz, Avery L AU - Buchholz AL AD - 13Department of Neurosurgery, University of Virginia, Charlottesville, Virginia. FAU - Tumialan, Luis M AU - Tumialan LM AD - 14Barrow Neurological Institute, Phoenix, Arizona. FAU - Turner, Jay D AU - Turner JD AD - 14Barrow Neurological Institute, Phoenix, Arizona. FAU - Sherrod, Brandon A AU - Sherrod BA AD - 4Department of Neurosurgery, University of Utah, Salt Lake City, Utah. FAU - Agarwal, Nitin AU - Agarwal N AD - 15Department of Neurosurgery, Washington University in St. Louis, Missouri. FAU - Chou, Dean AU - Chou D AD - 3Department of Neurological Surgery, Columbia University Vagelos College of Physicians and Surgeons, The Och Spine Hospital at NewYork-Presbyterian, New York, New York. FAU - Haid, Regis W AU - Haid RW AD - 16Atlanta Brain and Spine Care, Atlanta, Georgia; and. FAU - Bydon, Mohamad AU - Bydon M AD - 17Department of Neurologic Surgery, Mayo Clinic, Rochester, Minnesota. FAU - Chan, Andrew K AU - Chan AK AD - 3Department of Neurological Surgery, Columbia University Vagelos College of Physicians and Surgeons, The Och Spine Hospital at NewYork-Presbyterian, New York, New York. LA - eng PT - Journal Article PT - Research Support, Non-U.S. Gov't PL - United States TA - Neurosurg Focus JT - Neurosurgical focus JID - 100896471 SB - IM MH - Humans MH - Retrospective Studies MH - Prospective Studies MH - *Neck Pain/diagnosis/surgery MH - Bayes Theorem MH - Supervised Machine Learning MH - Algorithms MH - *Spinal Cord Diseases/surgery OTO - NOTNLM OT - Quality Outcomes Database OT - cervical spondylotic myelopathy OT - machine learning OT - neck pain OT - patient satisfaction OT - patient-reported outcomes EDAT- 2023/06/07 13:10 MHDA- 2023/06/08 06:42 CRDT- 2023/06/07 07:53 PHST- 2023/01/31 00:00 [received] PHST- 2023/03/22 00:00 [accepted] PHST- 2023/06/08 06:42 [medline] PHST- 2023/06/07 13:10 [pubmed] PHST- 2023/06/07 07:53 [entrez] AID - 10.3171/2023.3.FOCUS2372 [doi] PST - ppublish SO - Neurosurg Focus. 2023 Jun;54(6):E5. doi: 10.3171/2023.3.FOCUS2372.