PMID- 35916701
OWN - NLM
STAT- PubMed-not-MEDLINE
LR  - 20220930
IS  - 2291-9694 (Print)
IS  - 2291-9694 (Electronic)
VI  - 10
IP  - 8
DP  - 2022 Aug 2
TI  - Uncertainty Estimation in Medical Image Classification: Systematic Review.
PG  - e36427
LID - 10.2196/36427 [doi]
LID - e36427
AB  - BACKGROUND: Deep neural networks are showing impressive results in different 
      medical image classification tasks. However, for real-world applications, there 
      is a need to estimate the network's uncertainty together with its prediction. 
      OBJECTIVE: In this review, we investigate in what form uncertainty estimation has 
      been applied to the task of medical image classification. We also investigate 
      which metrics are used to describe the effectiveness of the applied uncertainty 
      estimation. METHODS: Google Scholar, PubMed, IEEE Xplore, and ScienceDirect were 
      screened for peer-reviewed studies, published between 2016 and 2021, that deal 
      with uncertainty estimation in medical image classification. The search terms 
      "uncertainty," "uncertainty estimation," "network calibration," and 
      "out-of-distribution detection" were used in combination with the terms "medical 
      images," "medical image analysis," and "medical image classification." RESULTS: A 
      total of 22 papers were chosen for detailed analysis through the systematic 
      review process. This paper provides a table for a systematic comparison of the 
      included works with respect to the applied method for estimating the uncertainty. 
      CONCLUSIONS: The applied methods for estimating uncertainties are diverse, but 
      the sampling-based methods Monte-Carlo Dropout and Deep Ensembles are used most 
      frequently. We concluded that future works can investigate the benefits of 
      uncertainty estimation in collaborative settings of artificial intelligence 
      systems and human experts. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): 
      RR2-10.2196/11936.
CI  - (c)Alexander Kurz, Katja Hauser, Hendrik Alexander Mehrtens, Eva Krieghoff-Henning, 
      Achim Hekler, Jakob Nikolas Kather, Stefan Frohling, Christof von Kalle, Titus 
      Josef Brinker. Originally published in JMIR Medical Informatics 
      (https://medinform.jmir.org), 02.08.2022.
FAU - Kurz, Alexander
AU  - Kurz A
AUID- ORCID: 0000-0001-6175-9203
AD  - Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), 
      Heidelberg, Germany.
FAU - Hauser, Katja
AU  - Hauser K
AUID- ORCID: 0000-0001-9390-3505
AD  - Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), 
      Heidelberg, Germany.
FAU - Mehrtens, Hendrik Alexander
AU  - Mehrtens HA
AUID- ORCID: 0000-0003-1234-5041
AD  - Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), 
      Heidelberg, Germany.
FAU - Krieghoff-Henning, Eva
AU  - Krieghoff-Henning E
AUID- ORCID: 0000-0001-8381-3100
AD  - Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), 
      Heidelberg, Germany.
FAU - Hekler, Achim
AU  - Hekler A
AUID- ORCID: 0000-0002-4974-2457
AD  - Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), 
      Heidelberg, Germany.
FAU - Kather, Jakob Nikolas
AU  - Kather JN
AUID- ORCID: 0000-0002-3730-5348
AD  - Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany.
FAU - Frohling, Stefan
AU  - Frohling S
AUID- ORCID: 0000-0001-7907-4595
AD  - Department of Translational Medical Oncology, National Center for Tumor Diseases 
      (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany.
FAU - von Kalle, Christof
AU  - von Kalle C
AUID- ORCID: 0000-0001-9221-3297
AD  - Department of Clinical-Translational Sciences, Berlin Institute of Health (BIH), 
      Berlin, Germany.
FAU - Brinker, Titus Josef
AU  - Brinker TJ
AUID- ORCID: 0000-0002-3620-5919
AD  - Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), 
      Heidelberg, Germany.
LA  - eng
PT  - Journal Article
PT  - Review
DEP - 20220802
PL  - Canada
TA  - JMIR Med Inform
JT  - JMIR medical informatics
JID - 101645109
PMC - PMC9382553
OTO - NOTNLM
OT  - deep learning
OT  - medical image classification
OT  - medical imaging
OT  - network calibration
OT  - out-of-distribution detection
OT  - uncertainty estimation
COIS- Conflicts of Interest: TJB is the owner of Smart Health Heidelberg GmbH 
      (Handschuhsheimer Landstr. 9/1, 69120 Heidelberg, Germany, 
      https://smarthealth.de) which develops telemedicine mobile apps (such as AppDoc; 
      https://online-hautarzt.net and Intimarzt; https://intimarzt.de), outside of the 
      submitted work.
EDAT- 2022/08/03 06:00
MHDA- 2022/08/03 06:01
PMCR- 2022/08/02
CRDT- 2022/08/02 10:13
PHST- 2022/01/14 00:00 [received]
PHST- 2022/06/04 00:00 [accepted]
PHST- 2022/04/11 00:00 [revised]
PHST- 2022/08/02 10:13 [entrez]
PHST- 2022/08/03 06:00 [pubmed]
PHST- 2022/08/03 06:01 [medline]
PHST- 2022/08/02 00:00 [pmc-release]
AID - v10i8e36427 [pii]
AID - 10.2196/36427 [doi]
PST - epublish
SO  - JMIR Med Inform. 2022 Aug 2;10(8):e36427. doi: 10.2196/36427.