PMID- 34292166 OWN - NLM STAT- MEDLINE DCOM- 20211025 LR - 20211025 IS - 1438-8871 (Electronic) IS - 1439-4456 (Print) IS - 1438-8871 (Linking) VI - 23 IP - 7 DP - 2021 Jul 22 TI - Effective Treatment Recommendations for Type 2 Diabetes Management Using Reinforcement Learning: Treatment Recommendation Model Development and Validation. PG - e27858 LID - 10.2196/27858 [doi] LID - e27858 AB - BACKGROUND: Type 2 diabetes mellitus (T2DM) and its related complications represent a growing economic burden for many countries and health systems. Diabetes complications can be prevented through better disease control, but there is a large gap between the recommended treatment and the treatment that patients actually receive. The treatment of T2DM can be challenging because of different comprehensive therapeutic targets and individual variability of the patients, leading to the need for precise, personalized treatment. OBJECTIVE: The aim of this study was to develop treatment recommendation models for T2DM based on deep reinforcement learning. A retrospective analysis was then performed to evaluate the reliability and effectiveness of the models. METHODS: The data used in our study were collected from the Singapore Health Services Diabetes Registry, encompassing 189,520 patients with T2DM, including 6,407,958 outpatient visits from 2013 to 2018. The treatment recommendation model was built based on 80% of the dataset and its effectiveness was evaluated with the remaining 20% of data. Three treatment recommendation models were developed for antiglycemic, antihypertensive, and lipid-lowering treatments by combining a knowledge-driven model and a data-driven model. The knowledge-driven model, based on clinical guidelines and expert experiences, was first applied to select the candidate medications. The data-driven model, based on deep reinforcement learning, was used to rank the candidates according to the expected clinical outcomes. To evaluate the models, short-term outcomes were compared between the model-concordant treatments and the model-nonconcordant treatments with confounder adjustment by stratification, propensity score weighting, and multivariate regression. For long-term outcomes, model-concordant rates were included as independent variables to evaluate if the combined antiglycemic, antihypertensive, and lipid-lowering treatments had a positive impact on reduction of long-term complication occurrence or death at the patient level via multivariate logistic regression. RESULTS: The test data consisted of 36,993 patients for evaluating the effectiveness of the three treatment recommendation models. In 43.3% of patient visits, the antiglycemic medications recommended by the model were concordant with the actual prescriptions of the physicians. The concordant rates for antihypertensive medications and lipid-lowering medications were 51.3% and 58.9%, respectively. The evaluation results also showed that model-concordant treatments were associated with better glycemic control (odds ratio [OR] 1.73, 95% CI 1.69-1.76), blood pressure control (OR 1.26, 95% CI, 1.23-1.29), and blood lipids control (OR 1.28, 95% CI 1.22-1.35). We also found that patients with more model-concordant treatments were associated with a lower risk of diabetes complications (including 3 macrovascular and 2 microvascular complications) and death, suggesting that the models have the potential of achieving better outcomes in the long term. CONCLUSIONS: Comprehensive management by combining knowledge-driven and data-driven models has good potential to help physicians improve the clinical outcomes of patients with T2DM; achieving good control on blood glucose, blood pressure, and blood lipids; and reducing the risk of diabetes complications in the long term. CI - (c)Xingzhi Sun, Yong Mong Bee, Shao Wei Lam, Zhuo Liu, Wei Zhao, Sing Yi Chia, Hanis Abdul Kadir, Jun Tian Wu, Boon Yew Ang, Nan Liu, Zuo Lei, Zhuoyang Xu, Tingting Zhao, Gang Hu, Guotong Xie. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 22.07.2021. FAU - Sun, Xingzhi AU - Sun X AUID- ORCID: 0000-0002-6519-0197 AD - Ping An Healthcare Technology, Beijing, China. FAU - Bee, Yong Mong AU - Bee YM AUID- ORCID: 0000-0002-5482-2646 AD - Department of Endocrinology, Singapore General Hospital, Singapore, Singapore. AD - SingHealth Duke-NUS Diabetes Centre, Singapore Health Services, Singapore, Singapore. FAU - Lam, Shao Wei AU - Lam SW AUID- ORCID: 0000-0002-0898-5858 AD - Health Services Research Centre, Singapore Health Services, Singapore, Singapore. AD - Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore. FAU - Liu, Zhuo AU - Liu Z AUID- ORCID: 0000-0001-9343-2969 AD - Ping An Healthcare Technology, Beijing, China. FAU - Zhao, Wei AU - Zhao W AUID- ORCID: 0000-0002-9998-1544 AD - Ping An Healthcare Technology, Beijing, China. FAU - Chia, Sing Yi AU - Chia SY AUID- ORCID: 0000-0002-7591-3141 AD - Health Services Research Unit, Singapore General Hospital, Singapore, Singapore. FAU - Abdul Kadir, Hanis AU - Abdul Kadir H AUID- ORCID: 0000-0001-6742-7189 AD - Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore. AD - Health Services Research Unit, Singapore General Hospital, Singapore, Singapore. FAU - Wu, Jun Tian AU - Wu JT AUID- ORCID: 0000-0002-9401-4715 AD - Health Services Research Centre, Singapore Health Services, Singapore, Singapore. FAU - Ang, Boon Yew AU - Ang BY AUID- ORCID: 0000-0002-2099-8969 AD - Health Services Research Centre, Singapore Health Services, Singapore, Singapore. FAU - Liu, Nan AU - Liu N AUID- ORCID: 0000-0003-3610-4883 AD - Health Services Research Centre, Singapore Health Services, Singapore, Singapore. AD - Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore. FAU - Lei, Zuo AU - Lei Z AUID- ORCID: 0000-0002-8582-3676 AD - Ping An Healthcare Technology, Beijing, China. FAU - Xu, Zhuoyang AU - Xu Z AUID- ORCID: 0000-0003-0344-3424 AD - Ping An Healthcare Technology, Beijing, China. FAU - Zhao, Tingting AU - Zhao T AUID- ORCID: 0000-0002-2069-5538 AD - Ping An Healthcare Technology, Beijing, China. FAU - Hu, Gang AU - Hu G AUID- ORCID: 0000-0002-3085-1135 AD - Ping An Healthcare Technology, Beijing, China. FAU - Xie, Guotong AU - Xie G AUID- ORCID: 0000-0002-2772-8961 AD - Ping An Healthcare Technology, Beijing, China. AD - Ping An Healthcare and Technology Co, Ltd, Shanghai, China. AD - Ping An International Smart City Technology Co, Ltd, Shenzhen, China. LA - eng PT - Journal Article PT - Research Support, Non-U.S. Gov't DEP - 20210722 PL - Canada TA - J Med Internet Res JT - Journal of medical Internet research JID - 100959882 RN - 0 (Blood Glucose) SB - IM MH - Blood Glucose MH - *Diabetes Mellitus, Type 2/drug therapy MH - Humans MH - Reproducibility of Results MH - Retrospective Studies MH - Treatment Outcome PMC - PMC8367185 OTO - NOTNLM OT - long-term outcome OT - model concordance OT - reinforcement learning OT - short-term outcome OT - type 2 diabetes COIS- Conflicts of Interest: None declared. EDAT- 2021/07/23 06:00 MHDA- 2021/10/26 06:00 PMCR- 2021/07/22 CRDT- 2021/07/22 12:22 PHST- 2021/02/09 00:00 [received] PHST- 2021/05/06 00:00 [accepted] PHST- 2021/03/30 00:00 [revised] PHST- 2021/07/22 12:22 [entrez] PHST- 2021/07/23 06:00 [pubmed] PHST- 2021/10/26 06:00 [medline] PHST- 2021/07/22 00:00 [pmc-release] AID - v23i7e27858 [pii] AID - 10.2196/27858 [doi] PST - epublish SO - J Med Internet Res. 2021 Jul 22;23(7):e27858. doi: 10.2196/27858.