PMID- 38030552
OWN - NLM
STAT- MEDLINE
DCOM- 20240105
LR  - 20240105
IS  - 1471-6771 (Electronic)
IS  - 0007-0912 (Linking)
VI  - 132
IP  - 1
DP  - 2024 Jan
TI  - Statistically significant differences versus convincing evidence of real 
      treatment effects: an analysis of the false positive risk for single-centre 
      trials in anaesthesia.
PG  - 116-123
LID - S0007-0912(23)00616-5 [pii]
LID - 10.1016/j.bja.2023.10.036 [doi]
AB  - BACKGROUND: The American Statistical Association has highlighted problems with 
      null hypothesis significance testing and outlined alternative approaches that may 
      'supplement or even replace P-values'. One alternative is to report the false 
      positive risk (FPR), which quantifies the chance the null hypothesis is true when 
      the result is statistically significant. METHODS: We reviewed single-centre, 
      randomised trials in 10 anaesthesia journals over 6 yr where differences in a 
      primary binary outcome were statistically significant. We calculated a Bayes 
      factor by two methods (Gunel, Kass). From the Bayes factor we calculated the FPR 
      for different prior beliefs for a real treatment effect. Prior beliefs were 
      quantified by assigning pretest probabilities to the null and alternative 
      hypotheses. RESULTS: For equal pretest probabilities of 0.5, the median 
      (inter-quartile range [IQR]) FPR was 6% (1-22%) by the Gunel method and 6% 
      (1-19%) by the Kass method. One in five trials had an FPR >/=20%. For trials 
      reporting P-values 0.01-0.05, the median (IQR) FPR was 25% (16-30%) by the Gunel 
      method and 20% (16-25%) by the Kass method. More than 90% of trials reporting 
      P-values 0.01-0.05 required a pretest probability >0.5 to achieve an FPR of 5%. 
      The median (IQR) difference in the FPR calculated by the two methods was 0% 
      (0-2%). CONCLUSIONS: Our findings suggest that a substantial proportion of 
      single-centre trials in anaesthesia reporting statistically significant 
      differences provide limited evidence of real treatment effects, or, 
      alternatively, required an implausibly high prior belief in a real treatment 
      effect. CLINICAL TRIAL REGISTRATION: PROSPERO (CRD42023350783).
CI  - Copyright (c) 2023 The Author(s). Published by Elsevier Ltd.. All rights reserved.
FAU - Sidebotham, David
AU  - Sidebotham D
AD  - Department of Cardiothoracic and ORL Anaesthesia, Auckland City Hospital, 
      Auckland, New Zealand; Cardiothoracic and Vascular Intensive Care Unit, Auckland 
      City Hospital, Auckland, New Zealand; Department of Anaesthesiology, Faculty of 
      Health Sciences, University of Auckland, New Zealand. Electronic address: 
      dsidebotham@adhb.govt.nz.
FAU - Dominick, Felicity
AU  - Dominick F
AD  - Department of Cardiothoracic and ORL Anaesthesia, Auckland City Hospital, 
      Auckland, New Zealand.
FAU - Deng, Carolyn
AU  - Deng C
AD  - Department of Anaesthesiology, Faculty of Health Sciences, University of 
      Auckland, New Zealand; Department of Anaesthesia and Perioperative Medicine, 
      Auckland City Hospital, Auckland, New Zealand.
FAU - Barlow, Jake
AU  - Barlow J
AD  - Department of Cardiothoracic and ORL Anaesthesia, Auckland City Hospital, 
      Auckland, New Zealand; Cardiothoracic and Vascular Intensive Care Unit, Auckland 
      City Hospital, Auckland, New Zealand.
FAU - Jones, Philip M
AU  - Jones PM
AD  - Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, 
      Jacksonville, FL, USA.
LA  - eng
PT  - Journal Article
PT  - Review
DEP - 20231128
PL  - England
TA  - Br J Anaesth
JT  - British journal of anaesthesia
JID - 0372541
SB  - IM
MH  - Humans
MH  - Bayes Theorem
MH  - Data Interpretation, Statistical
MH  - Research Design
MH  - *Anesthesia
MH  - *Anesthesiology
OTO - NOTNLM
OT  - Bayes factor
OT  - Bayes' theorem
OT  - anaesthesia
OT  - false positive risk
OT  - research design
OT  - sample size
OT  - significance testing
EDAT- 2023/11/30 00:42
MHDA- 2024/01/05 06:43
CRDT- 2023/11/29 22:06
PHST- 2023/07/19 00:00 [received]
PHST- 2023/10/29 00:00 [revised]
PHST- 2023/10/31 00:00 [accepted]
PHST- 2024/01/05 06:43 [medline]
PHST- 2023/11/30 00:42 [pubmed]
PHST- 2023/11/29 22:06 [entrez]
AID - S0007-0912(23)00616-5 [pii]
AID - 10.1016/j.bja.2023.10.036 [doi]
PST - ppublish
SO  - Br J Anaesth. 2024 Jan;132(1):116-123. doi: 10.1016/j.bja.2023.10.036. Epub 2023 
      Nov 28.